Zichun Yu
Zichun Yu
Hi, @LeoXinhaoLee I am also curious about it. Are there any conclusions?
Hi @tchaton is there a way to set DATA_OPTIMIZER_CACHE_FOLDER in the python script rather than as an environment variable? I didn't find such an interface. Thank you!
Hi @manishshettym, thanks for your insightful comments. We will try to parse the code better / examine better prompts. As we evaluate the code generation in a zero-shot fashion, we...
Perfect! Thanks, @Naman-ntc I will try it.
Hi @DeepCreeper, Yes, we need to add both system and user prompts when we call the API for optimal performance. You can find more information in this [repo](https://github.com/theoxo/self-repair) (and also...
Same issue. I use exactly the weights they provided in `pile_doremi_r1_120M_ref:pile_baseline_50kvocab_nopack_120M.json`. I found squad acc drops a lot in the main model when training goes on. BTW, I only run...
@kiseliu Thanks for your information. I would like to discuss the experimental configurations in more detail. Could you see the email I sent you (gmail)?
Sure. This is my wandb report: https://api.wandb.ai/links/zhiyuan-chenyan-zhenghao-group/cfo97a5p The squad is the most unstable one.
Also, it seems that the avg_acc of the baseline model easily achieves over 6 in both my and @kiseliu 's experiments while in @sangmichaelxie 's report, the highest checkpoint is...
Thank you so much! I will try it out.