ollama
Decorator to generate schemas for tools
I would like to know if there is any intention to add some sort of basic decorator to automatically generate schemas for functions we provide to the model. Examples? Maybe something like langchain has.
As you see there in the doc we can easily generate the description in JSON with that feature:
@tool(parse_docstring=True)
def foo(bar: str, baz: int) -> str:
"""The foo.
Args:
bar: The bar.
baz: The baz.
"""
return bar
foo.args_schema.schema()
{
"title": "fooSchema",
"description": "The foo.",
"type": "object",
"properties": {
"bar": {
"title": "Bar",
"description": "The bar.",
"type": "string"
},
"baz": {
"title": "Baz",
"description": "The baz.",
"type": "integer"
}
},
"required": [
"bar",
"baz"
]
}
I think this is a basic feature to ask. It is very practical and useful, think about that.
Thank you.
Man before you opened that issue I didn't knew I could use tools 💀
Maybe I am the one skipping docs, however I would like to know where/how did you find out that.
Having an actual documentation would be really good ðŸ˜
Maybe I am the one skipping docs, however I would like to know where/how did you find out that.
Yesterday ollama maintainers announced a new release that provided support for tools. They gave one example available in this repository. That is why I came here to check if there are some new features related to it. The ollama release was the reason I asked for auto generated schemas for tools, so I don't have to manually write things like:
tools = [
{
"type": "function",
"function": {
"name": "get_flight_times",
"description": "Get the flight times between two cities",
"parameters": {
"type": "object",
"properties": {
"departure": {
"type": "string",
"description": "The departure city (airport code)",
},
"arrival": {
"type": "string",
"description": "The arrival city (airport code)",
},
},
"required": ["departure", "arrival"],
},
},
},
]
With the feature I asked you would just:
@tool
def get_flight_times(departure: str, arrival: str) -> str:
"""
write basic docstring here
...
"""
And you are good to go...
Ok, however I wrote some code to see how it could be done, there are still some edge cases to handle and I wrote it kind of in a rush, here it is:
import json
import inspect
def foo(bar: int, par: int = 2):
"""
Computes par * bar
Basic stuff
:param bar: bar description
:param par: par description
"""
return par * bar
# --- Get Parameters Information
sig = inspect.signature(foo)
required = []
properties = {}
return_type = sig.return_annotation.__name__
for name, param in sig.parameters.items():
properties[name] = {'type': param.annotation.__name__}
if param.default == inspect.Parameter.empty:
required.append(name)
# get parameter description
param_lines = [line.strip() for line in foo.__doc__.split('\n') if line.strip().startswith(':param')]
param_lines = [line[line[1:].find(':')+2:].strip() for line in param_lines]
for prop, desc in zip(properties.keys(), param_lines):
properties[prop]['description'] = desc
# get docs
doc_lines = [line.strip() for line in foo.__doc__.split('\n') if not line.strip().startswith(':param')]
# --- Create docs
tool_doc = {
'type': type(foo).__name__,
'function': {
'name': foo.__name__,
'description': '\n'.join(doc_lines),
'parameters': {
'type': return_type,
'properties': properties,
'required': required
}
}
}
json_str = json.dumps(tool_doc, indent=4)
print(json_str)
Output:
{
"type": "function",
"function": {
"name": "foo",
"description": "\nComputes par * bar\nBasic stuff\n",
"parameters": {
"type": "_empty",
"properties": {
"bar": {
"type": "int",
"description": "bar description"
},
"par": {
"type": "int",
"description": "par description"
}
},
"required": [
"bar"
]
}
}
}
For some reason type is
_empty
For some reason type is _empty
In open webui tools.py file they just write "object":
"description": function_doc.get("description", function_name),
"parameters": {
"type": "object",
"properties": {
Maybe it is for OpenAI API compatibility.
In open webui tools.py file they just write "object":
I misinterpreted its meaning.
As shown it is simple to implement, I would enjoy working on it myself, however there are no contribution guidelines; so I think that's on the maintainers now.
In open webui tools.py file they just write "object":
I misinterpreted its meaning.
As shown it is simple to implement, I would enjoy working on it myself, however there are no contribution guidelines; so I think that's on the maintainers now.
Yes, this is very simple to implement, and there are some implementations out there that the maintainer can look into if he has any doubts, like also lanchain implementation.
Lets see what those maintainers say about that feature, if it is good or not to include something like this in this library. For me, as they are already integrating some "tools" support in this library, would be good to step ahead and add something useful like that. In my opinion download a whole framework for something so basic like that (langchain) is not interesting.
In my opinion download a whole framework for something so basic like that (langchain) is not interesting.
I agree, I rarely use langchain and I do not feel comfortable using It; however It objectively provides some nice to have features (under and absurd/buggy level of abstractions)
Seems like someone was also interested in something related to it and provided a PR here. But one user thinks that the best approach is a third-party library.
I think I will close this issue. Thanks for your contributions @antoninoLorenzo, @synacktraa.
Seems like someone was also interested in something related to it and provided a PR here. But one user thinks that the best approach is a third-party library.
I think I will close this issue. Thanks for your contributions @antoninoLorenzo, @synacktraa.
I've nearly completed converting the PR into a third-party library. The library will support the registration of functions—both synchronous and asynchronous—as well as pydantic models, TypedDicts, namedtuples, and dataclasses as tools. Additionally, there will be an infer method that leverages litellm to interact with LLM APIs. A key advantage is that pydantic and litellm won't be bundled with the package, keeping it lightweight. Users can choose to install those dependencies separately if they need those features.
In open webui tools.py file they just write "object":
I misinterpreted its meaning.
As shown it is simple to implement, I would enjoy working on it myself, however there are no contribution guidelines; so I think that's on the maintainers now.
Yes, this is very simple to implement, and there are some implementations out there that the maintainer can look into if he has any doubts, like also lanchain implementation.
Lets see what those maintainers say about that feature, if it is good or not to include something like this in this library. For me, as they are already integrating some "tools" support in this library, would be good to step ahead and add something useful like that. In my opinion download a whole framework for something so basic like that (langchain) is not interesting.
That's correct; many people find this feature unnecessary because frameworks like Langchain already exist. However, the issue is that if someone only wants to use the tool-calling feature, these frameworks are too heavy for such a simple task.