kkwhale7
kkwhale7
same problem!
@tonysy @lvhan028 @so2liu @cdpath i need your help!!
You haven't implemented the evaluation logic for subjective questions, why are the values displayed on the official website different from ours 
> We only include the objective questions of Gaokao in OpenCompass but your score in your website 18.9 in GAOKAO  we cant reproduce it!
in my way, I only calculate the objective score 15.13 
> The MCQ problems select one answer from 4 choices. The result is meaningless when smaller than 25%. I got it.So do you directly ignore the scores of multiple topic...
I have discovered a new phenomenon where the predictions generated by the gen task in GAOKAO are also inconsistent, somehow in ZeroRetriever, all below are with llama2-7b models  
thank you for your patience. I know your score calculation method now. But why are the two predictions different when I have the same config, This postprocess method only answers...
> I have discovered a new phenomenon where the predictions generated by the gen task in GAOKAO are also inconsistent, somehow in ZeroRetriever, all below are with llama2-7b models ...