replicate-examples icon indicating copy to clipboard operation
replicate-examples copied to clipboard

vLLM examples aren't working

Open mattt opened this issue 1 year ago • 1 comments

I've gotten a report about Cog not working with some of the language models in this repo. In particular, llama-2-70b-chat-awq and mixtral-instruct.

Could you please consider removing these examples and/or linking instead to the new, official Cog project for vLLM?

mattt avatar Jul 01 '24 13:07 mattt

Hey @mattt! :)

Ah yea, sorry. Lotta sketchiness in here. Down to rm/point to official project! if it's getting annoying for you, a PR would be much appreciated 🙏 - else I'll get to it when I find a chance :)

nateraw avatar Jul 11 '24 17:07 nateraw