Srinivasan Nandakumar

icon company NUS icon location Singapore icon location AI Research engineer. My research focus involves working with large language models for low resource languages and coding tasks

Results 2 issues of Srinivasan Nandakumar

Change bf16 to amp_bf16

3 comment

Hi, I tried replicating the pretraining bert script and when I ran it with the yaml script I got the following error: Value bf16 is not available in Precision. I...

Tinyllama issues

3 comment

Hi, I am finetuning tiny llama on T4 with FP16. When I use packing the loss seems to be okay. But when I set it to false, the grad_norm goes...

currently fixing