opencompass icon indicating copy to clipboard operation
opencompass copied to clipboard

[Feature] 请问主观评测脚本支持用本地模型作为judge模型吗?

Open KAKSIS opened this issue 7 months ago • 0 comments

Describe the feature

examples/eval_subjective.py 在这个文件中,我把judge_models改为了vllmwithchattemplate的形式,似乎并不能正常评测,alpaca eval的最终输出结果为空。 请问主观评测脚本支持用本地模型作为judge模型吗?

Will you implement it?

  • [ ] I would like to implement this feature and create a PR!

KAKSIS avatar Jun 18 '25 09:06 KAKSIS