Junlin Chen comments

Results 5 comments of


                                            Junlin Chen

[Bug]: TypeError in XFormersMetadata

I have ran into the same problem too. This is my environment ``` Collecting environment information... PyTorch version: 2.2.1+cu121 Is debug build: False CUDA used to build PyTorch: 12.1 ROCM...

[RFC] Initial Support for CPUs

could you give me a cpu inference example? I try ``` //start python3 -m vllm.entrypoints.openai.api_server \ --device cpu //input curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{ "model": "facebook/opt-125m", "messages": [...

Failed to convert 65B llama to hf weights

I also encountered this problem and successfully solved it. This is mainly due to insufficient hard disk space.

[Bug]: Query to the openapi server with cpu backend is throwing error

I also encountered the same error.

Failed to build

> if you build with `cmake [...] -D CMAKE_C_FLAGS="-D_POSIX_C_SOURCE=199309L" [...]` or `make config [...] CONFIG_FLAGS="-D CMAKE_C_FLAGS=-D_POSIX_C_SOURCE=199309L"` > > you should be able to build without problem, > > Its a...