WIP: Fix/update deps
WIP
This PR will remove most of the unnecessary dependencies:
- anthropic
- vertexai
- transformers
- langchain (once we have class in community)
- aiohttp
@diogoazevedo15 i've updated the VertexAI provider to not be dependant of the genai huge package.
Some breaking changes and errors I've noticed:
-
normal chat is working but functions stopped working, need your help to fix this
-
I noticed that we weren't sending the chats properly when there's context, as well as when you have system messages. On LLMstudio's side we need to accept OpenAI's standard but on the VertexAI provider we need to format it like this:
curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-pro-latest:generateContent?key=$GOOGLE_API_KEY" \
-H 'Content-Type: application/json' \
-d '{ "system_instruction": {
"parts":
{ "text": "You are a cat. Your name is Neko."}},
"contents": {
"parts": {
"text": "Hello there"}}}'
curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-pro-latest:generateContent?key=$GOOGLE_API_KEY" \
-H 'Content-Type: application/json' \
-d '{ "system_instruction": {
"parts":
{ "text": "You are a cat. Your name is Neko."}},
"contents":[
{"role":"user",
"parts":[{
"text": "Hello"}]},
{"role": "model",
"parts":[{
"text": "Great to meet you. What would you like to know?"}]},
{"role":"user",
"parts":[{
"text": "I have two dogs in my house. How many paws are in my house?"}]},
]}}'
https://ai.google.dev/api/generate-content#chat
In the meanwhile, let's not add function calling to anthropic, before we change it to the API request
Anthropic added without function call
Changes:
- Changed vertex.py to vertexai.py.
- Added tool calling to vertex models.
Tool calling works as follows:
- Tools can be in the OpenAI or Vertex Format.
- Messages can be in string or OpenAI format.
- Tool call mode is set to 'Auto'.
- Tool calls (name and parameters) are added to the chat message, so chains can work.
- Tool responses are added to the system message, so the model understands what it has called and what responses it has so far.