Ang Lv

icon location A Ph.D. student in RUC, Beijing, China.

Results 1 issues of Ang Lv

expert_bias is updated during training but saved checkpoint contains only zero values

1 comment

### Bug description Hi Torchtitan team, I've encountered an issue while training LLaMA 4. During training, I observed that the expert_bias parameters are being updated as expected — I can...