Mughaira

Hi, Did you solve this? And were you able to get the positive and negative labels from the websites mentioned? like Digsee? Thank you

Hello, Math task also has subtasks like : math_algebra_hard,math_counting_and_prob_hard,math_geometry_hard etc. But this page on [normalization](https://huggingface.co/docs/leaderboards/open_llm_leaderboard/normalization) does not account for that. Do we just average the individual results? and what about...

Thank you. But with the new update of lm_eval, we do not get scores like `"leaderboard_math_hard": { "exact_match,none": 0.08383685800604229, "exact_match_stderr,none": 0.007327815605050628, "alias": " - leaderboard_math_hard" }`, just the individual ones...

Got it. Thanks :)