mu1ReX comments

Results 6 comments of


                                             mu1ReX

Attachments in canvas are deleted by mistake

Same issue.

Multi-GPU inference Error: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:5 and cuda:6!

> > 使用的就是basic_demo/cli_demo_multi_gpus.py,还是一样的报错 > > 我也遇到了同样的错误，请问解决了吗我也是可以参考： # https://github.com/THUDM/CogVLM/issues/256 # https://huggingface.co/THUDM/cogagent-chat-hf/tree/main

ValueError: The model's max seq len (4096) is larger than the maximum number of tokens that can be stored in KV cache (3664). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.`

same error.

ValueError: The model's max seq len (4096) is larger than the maximum number of tokens that can be stored in KV cache (3664). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.`

> Same exception with `ValueError: The model's max seq len (2048) is larger than the maximum number of tokens that can be stored in KV cache (176). Try increasing `gpu_memory_utilization`or...

请问还更新吗？看有很多合并请求，但是release的版本还是一年前的

推荐用这个插件[Diagrams.Net](https://github.com/jensmtg/obsidian-diagrams-net)

[Usage]: run gguf model need template，how to write？

> Hi same issue for me. I am trying vllm with facebook/opt-125m using openai template, can someone help ? ValueError: As of transformers v4.44, default chat template is no longer...