Jordan Clive
Jordan Clive
@Muennighoff, [GEM/web_nlg](https://github.com/bigscience-workshop/promptsource/blob/eval-hackathon/promptsource/templates/GEM/web_nlg/en/templates.yaml) and [`GEM/wiki_auto_asset_turk`](https://github.com/bigscience-workshop/promptsource/blob/eval-hackathon/promptsource/templates/GEM/wiki_auto_asset_turk/templates.yaml) are examples of multiple references. For example GEM/wiki_auto_asset_turk/test_asset has 10 references. Yes, the reasoning is because in NLG, having one reference is quite often unreliable, so...
@smeyerhot This code is currently just for model training and evaluation. But should be trivial to load it for inference, it uses the same HF generate method.
@andreaskoepf I am going to run a 30B LoRA model just on the sft datasets and will post the sampling report.
Also exact lr schedule, as not clear the learning rates during linear warm up and cosine annealing and after what no. of batches these changes occur
Yes, I can make that available if you still want.
Looks good! I don't think we should replace the grammar. Maybe just sample \n\n, \n and space. I mean when I use GPT I usually use new lines.
@CloseChoice Is there a dataset config I can specify to test out the control system prompts in #2708 ?
I can contribute here.
@theophilegervet. Yes that is strange, I didn't encounter this error when training 7B `decapoda-research/llama-7b-hf` or 13b instead of `openlm-research/open_llama_13b` with fp16. If you set eval_steps to 1 and change the...
@KennethEnevoldsen Is there anything meaningful new contributors can help with?