Use vllm engin for eval

[TulipDu]

Hi,How to use vllm engin for eval?

Here say add --vllm，but I get : auto-round: error: unrecognized arguments: --vllm) so How to use vllm engin for eval? auto_round Version: 0.9.0 vllm Version: 0.11.0

[TulipDu]

This is my shell

# pip install auto-round 
export AR_USE_MODELSCOPE=true
    # --device_map  auto \
auto-round \
    --model /mnt/afs/models/DeepSeek-V2-Lite-Chat \
    --scheme W8A16 \
    --iters 0 \
    --format auto_round \
    --group_size 128 \
    --scale_dtype bfloat16 \
    --enable_torch_compile \
    --output_dir /mnt/afs/models/DeepSeek-V2-Lite-Chat-AutoRound-INT8\
    --tasks ifeval,mmlu\
    --limit 0.1 \
    --vllm

[xin3he]

Hi @TulipDu The usage is updated from --vllm to --eval_backend vllm

[xin3he]

@TulipDu @wenhuach21 I checked the code, this usage is not supported. tune+eval is enabled in a separate workflow which means two eval flow contained in AutoRound. --eval_backend is not supported in this case. We have plan to fix it recently.