GPT Vision new model issue

[Gisman4ik]

If you want to send img to gpt vision, you have to pass it in payload like this, where "content" - array

payload = {
  "model": "gpt-4-vision-preview",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What’s in this image?"
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "data:image/jpeg;base64,{base64_image}"
          }
        }
      ]
    }
  ],
  "max_tokens": 300
}

With new models in library i didn't find how to combine content parts into one array Now library can send only one type of content part. And this is an error

{
    "model": "gpt-4-vision-preview",
    "temperature": 0,
    "messages": [
        {
            "role": "system",
            "content": "Be helpful"
        },
        {
            "role": "user",
            "content":  {
                    "type": "image_url",
                    "image_url": {
                        "url": "\{base64}",
                        "detail": "auto"
                    }
             }
    ],
    "max_tokens": 300
}

Is it bug or i just don't see it?

Also is't possible to send imageUrl as base64 string, only as URL

[kalafus]

does this pull address the issue?

[Gisman4ik]

does this pull address the issue?

Yes