Merlin von Trott
Merlin von Trott
have you tried setting GROQ_API_KEY locally and leaving it out of the arguments?
I think it would be great if it could also pull whether the model supports vision. Can probably be done by checking if ("modality": "multimodal") Example (Gemini): "architecture": { "modality":...
I created a pull request with some additional models. It is not the implementation in this issue, but it would be nice in the meantime: https://github.com/BerriAI/litellm/pull/3545
qwen2.5-vl-7B works for me. But qwen2.5-vl-72B does not. All I do is change the model name and I get. ``` 2025-04-28T15:30:09.125558018Z engine.py :115 2025-04-28 15:30:09,124 Error initializing vLLM engine: The...
They are both model type `qwen2_5_vl` right ? Does qwen2.5-vl-72B need a different transformers version ?
Okay, it just did not have enough resources. That was not really clear from the error message. Here is what worked for me (In Runpod). Use 8 x 48GB GPUs...
Seeing a Google Docs / Drive Connector is the only thing truly missing from AnythingLLM. I would so love to see this.