[Bug] lora微调internvl2.58B并合并后，无法推理，报错

[fighterzzzh]

Checklist

• [x] 1. I have searched related issues but cannot get the expected help.
• [ ] 2. The bug has not been fixed in the latest version.
• [ ] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

The argument trust_remote_code is to be used with Auto classes. It has no effect here and is ignored. You are using a model of type internvl_chat to instantiate a model of type . This is not supported for all configurations of models and can yield errors. The argument trust_remote_code is to be used with Auto classes. It has no effect here and is ignored. You are using a model of type internvl_chat to instantiate a model of type . This is not supported for all configurations of models and can yield errors. Traceback (most recent call last): File "/home/gpuserver/Downloads/zzh/weitiao/test_lmdeploy.py", line 8, in pipe = pipeline(model,chat_template_config=ChatTemplateConfig(model_name='internvl2_5')) File "/home/gpuserver/Downloads/zzh/myenvinfer/lib/python3.10/site-packages/lmdeploy/api.py", line 79, in pipeline backend_config = autoget_backend_config(model_path, backend_config) File "/home/gpuserver/Downloads/zzh/myenvinfer/lib/python3.10/site-packages/lmdeploy/archs.py", line 85, in autoget_backend_config backend = autoget_backend(model_path) File "/home/gpuserver/Downloads/zzh/myenvinfer/lib/python3.10/site-packages/lmdeploy/archs.py", line 40, in autoget_backend turbomind_has = is_supported_turbomind(model_path) File "/home/gpuserver/Downloads/zzh/myenvinfer/lib/python3.10/site-packages/lmdeploy/turbomind/supported_models.py", line 111, in is_supported llm_arch = cfg.llm_config.architectures[0] AttributeError: 'dict' object has no attribute 'architectures'

Reproduction

from lmdeploy import pipeline from lmdeploy.vl import load_image from lmdeploy import pipeline, ChatTemplateConfig

#model = '/home/gpuserver/OpenGVLab/InternVL2_5-4B'

model="/home/gpuserver/Downloads/zzh/weitiao/InternVL/internvl_chat/work_dirs/internvl_chat_v2_5/merge_internvl2_5_4b_dynamic_res_2nd_finetune_lora" pipe = pipeline(model,chat_template_config=ChatTemplateConfig(model_name='internvl2_5'))

image = load_image('/home/gpuserver/Downloads/zzh/weitiao/clot/CLoT_cn_2000/ex_images/004atEXYgy1gpb3tsdolwj60y219fwkp02.jpg') response = pipe(('请你根据这张图片，讲一个脑洞大开的梗', image)) print(response.text)

Environment

lmdeploy                  0.8.0

Error traceback

[lvhan028]

从堆栈信息看起来是config.json中的结构和官方的internvl2.5-8b不一样。 建议加断点 debug 试试

[github-actions[bot]]

This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.

[github-actions[bot]]

This issue is closed because it has been stale for 5 days. Please open a new issue if you have similar issues or you have any new updates now.

[jimmysue]

@fighterzzzh 最后问题解决了吗？