Shibo Hao

icon indicating a website link https://ber666.github.io/

icon company UC San Diego icon location San Diego icon location Ph.D. student at UC San Diego; B.S. in Computer Science at Peking University

Results 5 comments of Shibo Hao

Which random seed are you using?

My result of bert-base is similar to yours.

map::at errors

Have you solved the problem?

weird outputs of 13B for unconditional generation

> I had similar generations for multi-GPU runs. Setting random seeds made them coherent for me. Thank you so much! That solved the bug for me!

Do these Metrics really need to be scaled?

Agree

Gradient Accumulation Error with Ulysses Sequence Parallel Causes Inconsistent Loss

Hi, thanks for sharing your experiment results! We are also running SFT with sp and gradient accumulation, and the trained model shows some strange behaviors. Have you figured out the...