Sachin Gururangan
Results
13
issues of
Sachin Gururangan
This PR adds instruction tuning to openLM. Currently a work in progress, but got some pretty good initial results with openLM 1B, doing a small amount of finetuning on the...
@sagadre has done a big grid search of HPs, lets update the names (ie potato_neox -> open_lm_410m) and add jsons with optimal HPs
Would be great to benchmark tokens/sec of OpenLM, comparing to other libraries like Mosaic, Metaseq, etc.