[Encoder-Decoder] LoRA - BART not working - LoraParams and input dims don't match, lora tokens 1 input tokens 0

[thanhlt998]

• I am following the example of BART LoRA at here. I cannot convert the engines without flag --context_fmha disable. I change test_remove_padding = False then running run.py raises the following error:

terminate called after throwing an instance of 'tensorrt_llm::common::TllmException'                                                                                                                          
  what():  [TensorRT-LLM][ERROR] Assertion failed: LoraParams and input dims don't match, lora tokens 1 input tokens 0 (/app/tensorrt_llm/cpp/tensorrt_llm/plugins/loraPlugin/loraPlugin.cpp:283)             
1       0x7f7d8dec64cb /usr/local/lib/python3.10/dist-packages/tensorrt_llm/libs/libnvinfer_plugin_tensorrt_llm.so(+0x544cb) [0x7f7d8dec64cb]                                                                 
2       0x7f7d8decbff0 /usr/local/lib/python3.10/dist-packages/tensorrt_llm/libs/libnvinfer_plugin_tensorrt_llm.so(+0x59ff0) [0x7f7d8decbff0]                                                                 
3       0x7f7e9088e20c /usr/local/tensorrt/lib/libnvinfer.so.10(+0x113b20c) [0x7f7e9088e20c]                                                                                                                  
4       0x7f7e9083bb67 /usr/local/tensorrt/lib/libnvinfer.so.10(+0x10e8b67) [0x7f7e9083bb67]                                                                                                                  
5       0x7f7e9083d671 /usr/local/tensorrt/lib/libnvinfer.so.10(+0x10ea671) [0x7f7e9083d671]                                                                                                                  
6       0x7f7e9e8a61c0 /usr/local/lib/python3.10/dist-packages/tensorrt/tensorrt.so(+0xa61c0) [0x7f7e9e8a61c0]                                                                                                
7       0x7f7e9e846123 /usr/local/lib/python3.10/dist-packages/tensorrt/tensorrt.so(+0x46123) [0x7f7e9e846123]                                                                                                
8       0x55925f134c9e python(+0x15ac9e) [0x55925f134c9e]                                                                                                                                                     
9       0x55925f12b3cb _PyObject_MakeTpCall + 603                                                                                                                                                             
10      0x55925f1433eb python(+0x1693eb) [0x55925f1433eb]                                                                                                                                                     
11      0x55925f12359a _PyEval_EvalFrameDefault + 25674                                                                                                                                                       
12      0x55925f13559c _PyFunction_Vectorcall + 124                                                                                                                                                           
13      0x55925f11d96e _PyEval_EvalFrameDefault + 2078                                                                                                                                                        
14      0x55925f14325e python(+0x16925e) [0x55925f14325e]                                                                                                                                                     
15      0x55925f11fa9d _PyEval_EvalFrameDefault + 10573                                                                                                                                                       
16      0x55925f14325e python(+0x16925e) [0x55925f14325e]                                              
17      0x55925f11fa9d _PyEval_EvalFrameDefault + 10573                                                
18      0x55925f13559c _PyFunction_Vectorcall + 124                                                    
19      0x55925f143db2 PyObject_Call + 290
20      0x55925f11fa9d _PyEval_EvalFrameDefault + 10573                                                
21      0x55925f143111 python(+0x169111) [0x55925f143111]                                              
22      0x55925f11eb77 _PyEval_EvalFrameDefault + 6695                                                 
23      0x55925f143111 python(+0x169111) [0x55925f143111]                                              
24      0x55925f11eb77 _PyEval_EvalFrameDefault + 6695                                                 
25      0x55925f119f96 python(+0x13ff96) [0x55925f119f96]                                              
26      0x55925f20fc66 PyEval_EvalCode + 134
27      0x55925f23ab38 python(+0x260b38) [0x55925f23ab38]                                              
28      0x55925f2343fb python(+0x25a3fb) [0x55925f2343fb]                                              
29      0x55925f23a885 python(+0x260885) [0x55925f23a885]                                              
30      0x55925f239d68 _PyRun_SimpleFileObject + 424                                                   
31      0x55925f2399b3 _PyRun_AnyFileObject + 67
32      0x55925f22c45e Py_RunMain + 702
33      0x55925f202a3d Py_BytesMain + 45
34      0x7f7fca132d90 /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f7fca132d90]                      
35      0x7f7fca132e40 __libc_start_main + 128
36      0x55925f202935 _start + 37
[3fded6951faf:06562] *** Process received signal ***

• My system information: GPU: RTX2080Ti