dduck999

TensorRT-LLM Backend with multiple LLMs (not to be confused with multi-model)

5

I followed this guide: https://github.com/triton-inference-server/tensorrtllm_backend?tab=readme-ov-file#quick-start I got this working but this only serves one model. Not only that, it breaks my understanding of the one-model equals one-folder structure of the...