LLMstudio icon indicating copy to clipboard operation
LLMstudio copied to clipboard

WIP: Fix/update deps

Open claudiolemos opened this issue 1 year ago • 3 comments

WIP

This PR will remove most of the unnecessary dependencies:

  • anthropic
  • vertexai
  • transformers
  • langchain (once we have class in community)
  • aiohttp

claudiolemos avatar Aug 08 '24 13:08 claudiolemos

@diogoazevedo15 i've updated the VertexAI provider to not be dependant of the genai huge package.

Some breaking changes and errors I've noticed:

  • normal chat is working but functions stopped working, need your help to fix this

  • I noticed that we weren't sending the chats properly when there's context, as well as when you have system messages. On LLMstudio's side we need to accept OpenAI's standard but on the VertexAI provider we need to format it like this:

curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-pro-latest:generateContent?key=$GOOGLE_API_KEY" \
-H 'Content-Type: application/json' \
-d '{ "system_instruction": {
    "parts":
      { "text": "You are a cat. Your name is Neko."}},
    "contents": {
      "parts": {
        "text": "Hello there"}}}'
curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-pro-latest:generateContent?key=$GOOGLE_API_KEY" \
-H 'Content-Type: application/json' \
-d '{ "system_instruction": {
    "parts":
      { "text": "You are a cat. Your name is Neko."}},
    "contents":[
        {"role":"user",
         "parts":[{
           "text": "Hello"}]},
        {"role": "model",
         "parts":[{
           "text": "Great to meet you. What would you like to know?"}]},
        {"role":"user",
         "parts":[{
           "text": "I have two dogs in my house. How many paws are in my house?"}]},
      ]}}'

https://ai.google.dev/api/generate-content#chat

claudiolemos avatar Aug 08 '24 13:08 claudiolemos

In the meanwhile, let's not add function calling to anthropic, before we change it to the API request

claudiolemos avatar Aug 08 '24 13:08 claudiolemos

Anthropic added without function call

claudiolemos avatar Aug 08 '24 16:08 claudiolemos

Changes:

  1. Changed vertex.py to vertexai.py.
  2. Added tool calling to vertex models.

Tool calling works as follows:

  1. Tools can be in the OpenAI or Vertex Format.
  2. Messages can be in string or OpenAI format.
  3. Tool call mode is set to 'Auto'.
  4. Tool calls (name and parameters) are added to the chat message, so chains can work.
  5. Tool responses are added to the system message, so the model understands what it has called and what responses it has so far.

diogoazevedo15 avatar Aug 16 '24 15:08 diogoazevedo15