ai icon indicating copy to clipboard operation
ai copied to clipboard

support sequential tool calls and fix mistral

Open deathbyknowledge opened this issue 9 months ago • 2 comments

  • Mistral errors if an assistant message includes both content and tool_calls so this adds a check for it.
  • Mistral also errors if a tool call does not include tool_call_id, so this includes it.
  • Add a model setting sequentialCalls, so a model is allowed to chain tool calls together, instead of only when the last message is a user message. This makes all responses be a simulated stream.

Usage:

const workersAI = createWorkersAI({ binding: env.AI });
const model = workersAI(Model.Mistral24B, { sequentialCalls: true })

deathbyknowledge avatar Apr 03 '25 15:04 deathbyknowledge

Hey @deathbyknowledge, thanks for this! I've got a couple of questions.

  • If this is fixing something that breaks normal usage, then is it the right way to go to hide the functionality behind a model setting? I'm worried here that no-one will discover this setting and just assume that the model is broken.

  • Would you be able to add some example JSON objects to the description to explain exactly what the issue is here, because I'm interested to see if there's a general solution.

  • Is this an issue that is just for Mistral? And if so, is it the job of the workers-ai-provider to fix it, or is it a model output issue?

I think generally I'm missing some context that defines the exact problem that we have here. And it would be good to detail it for posterity.

andyjessop avatar Apr 07 '25 11:04 andyjessop

revisiting this, I'd like to land support for mistral, can we address feedback and move forward here?

threepointone avatar May 28 '25 04:05 threepointone