zhouyuustc comments

Repositories
Issues
Comments

Results 2 comments of


                                            zhouyuustc

希望能使用vllm加速llm reranker

可否分享一下示例代码 +1

Error in Gemma 2 using model_worker (probably an error in conversation.py)

When I tested gemma-2-9b-it using modelw_worker, what I got was: { "object": "error", "message": "**NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE.**\n\n(probability tensor contains either `inf`,...