Uzzimauzer

Hello, You may download model and all needed files from [Hugging Face](https://huggingface.co/openchat/openchat_3.5), and in the --model flag, specify the path to the folder with model, like: `python3 -m ochat.serving.openai_api_server --model...

> So, based on my tests so far, it looks like it's simply enough to install `triton==3.2.0` on top of the existing installation. Either run `uv pip install triton==3.2.0` if...

> ``` > FROM docker.io/vllm/vllm-openai:latest > > RUN --mount=type=cache,target=/root/.cache/uv \ > uv pip install --system triton==3.2.0 > ``` This worked, ran successfully, but the '-O 3' parameter problem remained