predibase
feat: Function calling with output schema enforcement
Introduces function-calling capabilities with JSON schema enforcement on output.
Example using Mistral-7B-Instruct-v0.3:
curl 127.0.0.1:8080/generate -X POST -d '{
"inputs": "WHat is the current temperature of New York, San Francisco and Chicago?",
"parameters": {
"max_new_tokens": 512,
"tools": [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": [
"celsius",
"fahrenheit"
]
}
}
}
}
}
]
}
}' -H 'Content-Type: application/json'
{"generated_text":"[{ \"id\": \"00000173499999999683\", \"type\": \"function\", \"function\": { \"name\": \"get_current_weather\", \"arguments\": { \"location\": \"New York\" } } },{ \"id\": \"00000173499999999684\", \"type\": \"function\", \"function\": { \"name\": \"get_current_weather\", \"arguments\": { \"location\": \"San Francisco\" } } },{ \"id\": \"00000173499999999685\", \"type\": \"function\", \"function\": { \"name\": \"get_current_weather\", \"arguments\": { \"location\": \"Chicago\" } } }]"}
@tgaddair @magdyksaleh
Waiting for this merge to try out Function Calling in Lorax
@magdyksaleh Still waiting for this feature. I want to deploy Mistral Nemo and Llama 3.1 models with function calling
I have been waiting for function calling feature for long time and got exited for this PR. But this PR is still not merged.
@tgaddair @jeffreyftang
@llama-shepard Sorry, I've been preoccupied with other obligations. What's remaining on this PR is to verify that pulling in chat templates will provide support for models other than mistral 3 (which is already working).
This would be extremely useful for me. When can we expect this to be merged?
