rvsh2

``` rvsh@bob:~$ docker run --gpus all --shm-size 64G -p 6006:6006 -p 8888:8888 -it ghcr.io/collabora/whisperfusion-3090:latest ========== == CUDA == ========== CUDA Version 12.2.2 Container image Copyright (c) 2016-2023, NVIDIA CORPORATION &...

Thanks for update. I tried this but got some errors: ```rvsh@bob:~$ docker run --gpus all --shm-size 64G -p 8010:80 -it ghcr.io/collabora/whisperfusion:latest s6-rc: info: service s6rc-oneshot-runner: starting s6-rc: info: service s6rc-oneshot-runner...

I've modified the code in chat.py to show the messages generated in those two cases: ``` if stream: print(chat_create_kwargs["messages"]) streaming_response = await client.chat.completions.create( **chat_create_kwargs, stream=True, ) self.append(*messages) finish_reason, function_call_request, tool_arguments...

vllm gives this output: ``` functionary | Future exception was never retrieved functionary | future: functionary | Traceback (most recent call last): functionary | File "/workspace/functionary/functionary/vllm_monkey_patch/async_llm_engine.py", line 42, in _raise_exception_on_finish...

If i disable grammar sampling I've got this in vllm: ``` functionary | ERROR: Exception in ASGI application functionary | Traceback (most recent call last): functionary | File "/usr/local/lib/python3.10/dist-packages/uvicorn/protocols/http/httptools_impl.py", line...

Hi, I manage to solve the issue but I don't know why it works as it should. In chat.py I replaced lines in submit function to: ``` if finish_reason ==...

I use local llm based on ollama. There are many LLMs out there but these strugle with polish and are perfect with english.