shaojie zhang

icon location Syracuse icon location A Data Science Grad student who's from Bio-Medical Physics undergrad.

Results 3 comments of shaojie zhang

Getting connection refused

Ops, so..... no more data?

Qwen2.5-Coder-instruct-3b only achieves 45.12 pass@1 on humaneval using Opencompass

I noticed a similar issue. For the 1.5B base model, the reported pass@1 rate was 43.9%, but the actual pass@1 rate without using post-processing other than truncation was 5.49%.

Qwen2.5-Coder-instruct-3b only achieves 45.12 pass@1 on humaneval using Opencompass

> Additionally, I noticed that Qwen2.5-Coder applies various dataset-specific processing steps. Could this be the reason why the results in the technical report are significantly better? > > See the...