[FR] Support forced tool calling

[lfkellogg]

Is your feature request related to a problem? Please describe. The Gemini API supports setting a "mode" for function calling: https://ai.google.dev/gemini-api/docs/function-calling#function_calling_mode

For example this allows the developer to choose whether the LLM output should only contain function calls, or if the LLM should decide whether to return text or a function call.

The googleai plugin currently does not support tool_config. This should be added to the custom model config.

Describe the solution you'd like The mode could be specified during the generate() call.

Describe alternatives you've considered N/A

Additional context N/A

[MichaelDoyle]

I'm wondering if we should elevate this to a top level generate/model config. @mbleigh @pavelgj

Open AI, Anthropic, et al support the same:

https://platform.openai.com/docs/api-reference/chat/create#chat-create-tool_choice https://docs.anthropic.com/en/docs/tool-use#forcing-tool-use

Vertex AI:

https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling#tool-config

[mbleigh]

👍 anything shared across 3+ providers feels like it ought to be a built-in. Let's add it to the standard model config and add implementations in our plugins.

[Yorkemartin]

I've experienced a problem where tools that i want to be called at least once on every flow are not called for no clear reason. i found it unable to be solved and it forced me to redo many of my tools as more traditional processes. This seems like it would help this problem significantly.

[mbleigh]

This is supported in both Google AI and Vertex AI now:

ai.generate({
  config: {functionCallingConfig: {mode: "ANY"}}
})

See Vertex and Google AI implementations.