enochlev

Fix DIET Classifier crash on empty input

2

**Simple Fix to prevent DIET classifier from crashing on empty string**: - From what I understand all other components don't crash when input string is empty except for the DIET...

cuDDN minor version diffrences (TensorRT was linked against cuDNN 8.9.X but loaded cuDNN 8.9.X)

2

### System Info I got this warning when trying to compile the model and I can imagine it may reduce performance TensorRT was linked against cuDNN 8.9.6 but loaded cuDNN...

Enable Custom Models

3

Problem statment: There are alot of powerfull LLMs backend with terrible UI and alot of nice front end LLMw with bad/unscalable backend. This PR will allow to use custom models...

Fixed Whitespace Error in Streaming mode

2

When a TensorRTLLM is deployed with streaming mode tokens have no white spaces in between streaming chunks. [Link to Issue](https://github.com/triton-inference-server/tensorrtllm_backend/issues/332#issuecomment-2063243340 ) This is because calling tokenizer.decode does a whitespace strip...

I deployed raw weights from llama3.1 8b instruct model and the model seems to ignore eos token when I invoke it. Here is how I deployed it and invoke it....