TensorRT-LLM
TensorRT-LLM copied to clipboard
NVIDIA
can i add images embedding to llm input? How can i do it？
such as，i want to use a Visual Pretrained Language Models to take the image embedding,and add it to llm input to get the output
We support several popular multimodal models in examples/multimodal/.
For these models, we pass image embedding input to LLM via prompt_table argument (this extends the embedding table of LLM) and modify input_ids with indices into prompt_table.
You can check tensorrt_llm/runtime/multimodal_model_runner.py for how this mechanism is used for different models.
