Niloofar Zarif

DLRM Inference fails:KeyError

1

Hi, I am using the Terabyte dataset and the 12GB pre-trained model you published. I am running inference locally so I used `run_local.sh` script. I first ran the script to...

Phi-3-Vision Batch Inference Prompt format

1

### This issue is for a: (mark with an `x`) ``` - [ ] bug report -> please search issues before submitting - [x] feature request - [x] documentation issue...

Hi, I am following the [this doc](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/tensorrtllm_backend/docs/build.html) and used this command to build Triton image with trt_llm backend from source: `DOCKER_BUILDKIT=1 docker build -t triton_trt_llm -f dockerfile/Dockerfile.trt_llm_backend . ` I...

Hi Team, I'm encountering an issue with dynamic batching performance in a Triton Inference Server ensemble for a Vision-Language Model (VLM). **My Setup:** **Ensemble Structure:** **preprocessing step:** Python backend **vision-encoder...

Multimodal support for OpenAI-Compatible frontend

3

**Is your feature request related to a problem? Please describe.** I am trying to profile a multimodal model with `genai-perf`. My model is not OpenAI-compatible by default so I am...