Blue2Giant

Results 1 comments of Blue2Giant

vllm deployment model get stuck when processing api calls

But it not really work, I still get the "**AssertionError: Expected a cached item for mm_hash='f7c6d95dd966f5129dd6e0569b09af98dbcbab54613007ab7bc3aec6dd8376ea'** " even though set --enable-prefix-caching False, and --mm-processor-cache-gb 0