Li delun

Seems like I can simply change the 'model.config.seq_length = 4096', and the output works fine.

with `CUDA_VISIBLE_DEVICES=0,1 swift infer --ckpt_dir ckp_output/internlm-xcomposer2-7b-chat/v10-20240502-202001/checkpoint-60/` it works, but the VMem usage goes from 18GB to 48+GB with only 4 questions(6pics+text) and lead to OOM? Is this a bug?

Change `model.generation_config.max_new_tokens` solved the problem. But I'm still confused about the difference of these params, can you help to explain it? ``` form https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-xcomposer2-7b/files model.config.max_length model.generation_config.max_length model.generation_config.max_length model.generation_config.max_new_tokens ```

Thanks!

@ChonghaoSima I got the same error, Caused by the merge pull request #60.

@ChonghaoSima Change the Pool num to 1 still leads to the error.

@DevLinyan I run the gpt_eval.py and got `You tried to access openai.ChatCompletion, but this is no longer supported in openai>=1.0.0 - see the README at https://github.com/openai/openai-python for the API.`, I...

Changing call_chatgpt in gpt_eval.py: def call_chatgpt(self, chatgpt_messages, max_tokens=40, model="gpt-3.5-turbo"): response = openai.chat.completions.create( model=model, messages=chatgpt_messages, temperature=0.6, max_tokens=max_tokens ) reply = response.choices[0].message.content total_tokens = response.usage.total_tokens return reply, total_tokens And change prompts to:...

@DevLinyan Thanks for the reply. I still have doubts about the cause of this error. According to your reply, this error is not due to a model inference error, but...

We checked again and find the error caused by the aborted answer of the LLM: ``` python answer= "There is a black sedan to the front of the ego vehicle,...