Doxie
Doxie
I have the same question. Does anybody have an idea of this?
> I had a problem using BackgroundGenerator with PyTorch Distributed Data Parallel. > When I turn DDP and BackgroundGenerator both on and iterate dataloader, processes that are not in rank...
同求，opt系列的这类支持也很好，希望chatglm可以跟上
Does anyone have any ideas about this?
>  my tested results. fyi @yananchen1989 Hi, may I know which model does this result come from?
has been stucked here for sometime, please help
@piotrm-nvidia Thank you for your reply. I could install nvidia-pytriton==0.2.5 on my docker image, but 0.2.5 only support python3.8, while my program need to run with python 3.10+. Begin with...
@piotrm-nvidia Thanks for your help! I followed your instructions to build the wheel，but It failed at the last step, and I didn't get the wheel file. `Error response from daemon:...
How about launch a separate vllm server for doing this? It would be much easier and flexible I think.