ghtaro

How to prepare 2023-02-12_oasst_prod.jsonl

3

Hi, I am trying to run sample scripts in https://github.com/LAION-AI/Open-Assistant/tree/main/model. I would to run RL training script, but I could not prepare oasst dataset specified in config yml.  It...

Large loss jump in the beginning of second epoch in training

3

Hi, I have run SFT using my own checkpoint of GPTJ-6B model (by setting "input model" widget box to the place where the model and tokenizer are located). I trained...

[QUESTION] Megatron-LM installation with CUDA 11.6

1

Hi, I am using single-node multi-GPU cluster (A100 * 6) and would like to use Megatron-LM to learn llama2 model on that. On the computational environment, I am not able...