opencompass
opencompass copied to clipboard
[Feature] 请问主观评测脚本支持用本地模型作为judge模型吗?
Describe the feature
examples/eval_subjective.py 在这个文件中,我把judge_models改为了vllmwithchattemplate的形式,似乎并不能正常评测,alpaca eval的最终输出结果为空。 请问主观评测脚本支持用本地模型作为judge模型吗?
Will you implement it?
- [ ] I would like to implement this feature and create a PR!