Luna_yyq

icon indicating an email [email protected]

icon company The University of Texas at Austin icon location Austin, TX icon location PhD_student@UT-Austin

Results 1 issues of Luna_yyq

Significant Performance Drop in GSM8k Evaluation with Updated SFT ckpt

3 comment

Hi, Thank you for your work. We're re-evaluating experiments using an updated SFT ckpt from https://huggingface.co/alignment-handbook/zephyr-7b-sft-full and using lm-evaluation-harness v0.4.0 for evaluation. We've noticed a significant performance drop in GSM8k....