Simon
Simon
Use a normal, temporary solution after passing the litellm proxy.
> 我也只是改了CUDA_VISIBLE_DEVICES=0,1，然后告诉我cuda满了，我盯着gpu看的，是突然一下就爆了。单卡训练反倒没问题 我也是这样的问题，多卡直接爆掉。
Milvus Lite`https://github.com/milvus-io/milvus-lite` ,完全兼容Milvus ， 可以嵌入到 Python 应用程序。`pip install milvus` `https://pypi.org/project/milvus/` 方便未来生成环境使用Milvus，可盐可甜。
Can't run it or it still says ‘RuntimeError: CUDA error: no kernel image is available for execution on the device’， Using the latest image of alpindale/aphrodite-engine
Thank you very much, I recompiled the image according to your patch and now it is ready for model inference. docker image address :`` nodecloud/aphrodite-engine ``
Me too!
> 通过运行此命令降级为我解决了这个问题。 > > ``` > pip install tensorrt==10.1.0 tensorrt-cu12==10.1.0 tensorrt-cu12-bindings==10.1.0 tensorrt-cu12-libs==10.1.0 --force-reinstall > ``` Solved my problem, thanks.