Young-Jun Lee

Super hero!

@AllenShow Hi, did you resolve this issue?

I have the same problem too. Did you solve it?

@clefourrier Yes, sure. Below is the code ```python math_500_pass_k_at_4 = LightevalTaskConfig( name="math_500_pass_k_at_4", suite=["custom"], prompt_function=math_500_prompt_fn, hf_repo="HuggingFaceH4/MATH-500", hf_subset="default", hf_avail_splits=["test"], evaluation_splits=["test"], few_shots_split=None, few_shots_select=None, generation_size=32768, metrics=[ Metrics.pass_at_k_math(sample_params={"k": 1, "n": 4}), Metrics.pass_at_k_math(sample_params={"k": 2, "n": 4}),...

@tomtyiu I still couldn't see all the results I need - specifically pass@1, pass@2, and pass@4. I'm planning to create a trend plot showing performance across different k values. Do...

@clefourrier Even after installing the latest version of lighteval and running the same code, I'm still getting the identical results shown in the figure. I'm wondering if this might be...

@clefourrier Then, how should I handle this? How to specify K and N, except the below way that I used? ```python math_500_pass_k_at_4 = LightevalTaskConfig( name="math_500_pass_k_at_4", suite=["custom"], prompt_function=math_500_prompt_fn, hf_repo="HuggingFaceH4/MATH-500", hf_subset="default", hf_avail_splits=["test"],...