Harshini Komali

Results 10 comments of Harshini Komali

Does the Triton models directory support the subdirectory Model?

cc @tanmayv25

TensorRT model low throughput

@jbkyang-nvi @tanmayv25 Any thoughts?

TensorRT model low throughput

@rs-ixz The team is looking into it.

vLLM/OpenAI Compatible Endpoint

@Elsayed91 I filed a feature request to the team. DLIS-6323

Tests for Top Level Request Caching for Ensemble Models

working on refactoring tests according to offline discussion with @oandreeva-nv

`unable to create cuda shared memory handle` when using multiprocessing to send multiple requests

Hi @justanhduc I tried to reproduce the issue and created a repro script. I used Triton 24.03. With p =1, I did not get any error and the script ran...

Unable to load/unload models through SageMaker + Triton

cc @dyastremsky @rmccorm4

how to set `model_transaction_policy`

@JonSmall Can you please describe the models (framework, inputs, outputs), ideally include the model configuration file (if using an ensemble include the model configuration file for that as well).

Triton server memory accumulation problem

@krishung5 Do you have any idea or insight on this?

I got the same problem on 21.11 and 21.12, it works with the single model or a couple of models, but triton never releases them.

cc @GuanLuo @rmccorm4 @jbkyang-nvi