LLMforScience

Results 7 comments of LLMforScience

Significant Performance Drop in GSM8k Evaluation with Updated SFT ckpt

Hi @yinyueqin Have you reproduced the preformance? I also can not reproduce the preformance

Looking forward for your help and Question

Thank you for your reply! Do you rember how many molecular graphs (precentage) are not correct in Molecule3D?

Weird logits and model starts degeneration while training DPO

Hi, Did you solve the problem?

Cannot reproduce generated samples in UCLA-AGI/SPIN_iter0

I also have the same problem. I can not reproduce the generated samples.

Too much GPU memory consume

Thank you for your response. I have another question about the mismatch of results. Why are results reported in your paper different from open LLM leadbord?

Too much GPU memory consume

Thanks for your reply. How did you test open LLM leadbord? Did you submit your model to https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard or use https://github.com/EleutherAI/lm-evaluation-harness by yourself?

Too much GPU memory consume

@xiamengzhou Thanks for your reply! Could you please tell me which version of lm-evaluation-harness did you use?