Rob Cheung comments

Results 17 comments of


                                            Rob Cheung

Update dependencies to current environment

Hi - yeah I can probably get to it some time this week. Stay tuned...

Update dependencies to current environment

got around to bringing this project back up to speed with most dependencies today

[Serve] Exited containers not cleaned up

yeah we are passing `--rm` in manually now and it is much better

Add example of serving LLM with Ray Serve and vLLM

Am I right in assuming that the prometheus metrics that vllm exports will just automatically get propagated to the ray serve metrics endpoint? (provided i use the `log_stats` when starting...

Chromadb fails to install on MacOS 13.2.1 (i9) with latest XCode

same as well

Chromadb fails to install on MacOS 13.2.1 (i9) with latest XCode

I am not on intel, fwiw -

Unable to install on Windows

@dcosson also getting this on OS X, using rbenv w/ ruby 2.2.4 vagrant 1.8.5

Error on RollbarReactNative.h file imports

+1 same issue. is there a reason for the dynamic import check? seems like the qualified one is always better

Logits Processors `Guide` integration will be buggy when `len(tokens) > 1` in a `Write` instruction

also curious about the state of things here. @simon-mo @rlouf do you know the latest on this?

No such file or directory: "VILA1.5-13b-AWQ/llm/model-00001-of-00006.safetensors"

I got a little farther by specifying the actual `.pt` file: ``` (base) ray@6c663dea2a49:~/llm-awq/tinychat$ CUDA_VISIBLE_DEVICES=0 python vlm_demo_new.py --model-path VILA1.5-13b-AWQ/ --quant-path VILA1.5-13b-AWQ/llm/vila-1.5-13b-w4-g128-awq-v2.pt --image-file https://media.substrate.run/docs-fuji-red.jpg /home/ray/anaconda3/lib/python3.10/site-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use...