Alpay Ariyak comments

Results 32 comments of


                                            Alpay Ariyak

OpenAI API: API errors have wrong HTTP code

Are you using the endpoint regularly or through openai compatibility?

OpenAI API: API errors have wrong HTTP code

@lucasavila00

OpenAI API: API errors have wrong HTTP code

Will fix the issue this week, thank you!

How can i update to vLLM v0.4.1 for llama3 support ?

Hi all, thank you for raising this issue! I have just merged the vLLM 0.4.2 update into main, you can use it by changing your Docker image in your endpoint...

[Model Request] OpenChat 3.5 -

Have you uploaded it somewhere on HuggingFace? Seems like an eos token issue, might be fixable with a change to the config file

create test.py

I ran it and got results, some examples of which are below. What could be the issue? I already installed the PEFT and Transformer versions that were mentioned. Sample 2:...

create test.py

You're right, thank you!

Add runpod/cuda image (12.4, 12.3, 12.2, 12.1, 11.8)

@yocontra thank you for the suggestion, added those versions in #63.

Errors cause the instance to run indefinitely

Are you still facing this issue currently?

Errors cause the instance to run indefinitely

@bartlettD Could you provide an example model and GPU model please?