replicate-examples
replicate-examples copied to clipboard
vLLM examples aren't working
I've gotten a report about Cog not working with some of the language models in this repo. In particular, llama-2-70b-chat-awq and mixtral-instruct.
Could you please consider removing these examples and/or linking instead to the new, official Cog project for vLLM?
Hey @mattt! :)
Ah yea, sorry. Lotta sketchiness in here. Down to rm/point to official project! if it's getting annoying for you, a PR would be much appreciated 🙏 - else I'll get to it when I find a chance :)