Alok Kumar Sahu comments

Results 4 comments of


                                            Alok Kumar Sahu

"peer access is not supported between these two devices" when using multiple GPUs

> Append "--use_custom_all_reduce disable" to trtllm-build command can fix it. This is working for me on 8x4090 Thank you. It worked for me as well. Thanks a lot!!

[tensorrt-llm backend] A question about launch_triton_server.py

Any clue how to resolve this issue, please let me know?

[tensorrt-llm backend] A question about launch_triton_server.py

I used world size 4 and it worked ________________________________ From: dwq370 ***@***.***> Sent: Friday, July 5, 2024 7:24:18 AM To: triton-inference-server/tensorrtllm_backend ***@***.***> Cc: Alok Kumar Sahu ***@***.***>; Comment ***@***.***> Subject:...

[tensorrt-llm backend] A question about launch_triton_server.py

Okay ________________________________ From: dwq370 ***@***.***> Sent: Friday, July 5, 2024 8:18:55 AM To: triton-inference-server/tensorrtllm_backend ***@***.***> Cc: Alok Kumar Sahu ***@***.***>; Comment ***@***.***> Subject: Re: [triton-inference-server/tensorrtllm_backend] [tensorrt-llm backend] A question about...