dduck999 comments

Repositories
Issues
Comments

Results 3 comments of


                                            dduck999

TensorRT-LLM Backend with multiple LLMs (not to be confused with multi-model)

Hi @adityarajsahu , have you built the folder structure along with the template files as in the example? Then have you run the script to write into these files?

TensorRT-LLM Backend with multiple LLMs (not to be confused with multi-model)

> Hey [@dduck999](https://github.com/dduck999), just found the issue, thanks. @adityarajsahu great! If you ever find a way to host more than **one** model behind **one** Triton TRT-LLM endpoint, please share 😊

How to serve multiple TensorRT-LLM models in the same process / server?

> Yes, see the link to the documentation in my April 16 message. This link shows multi-models (models containing text, vision, etc.). But does Triton + TRT-LLM backend support serving...