ghtaro comments

Results 7 comments of


                                            ghtaro

data_utils.py List index out of range

Hi @ratishsp , Thank you very much for sharing the code and for answering many questions for people trying to replicate your result which is very helpful to me as...

data_utils.py List index out of range

Hi @ratishsp , Thank you very much for your prompt reply. >I am not sure about the root cause of the issue you are facing. I was able to setup...

How to prepare 2023-02-12_oasst_prod.jsonl

@andreaskoepf Thank you very much for your reply. I managed to run RL training with WebGPT, but will definitely try en_100_tree and visit OA discord!

Large loss jump in the beginning of second epoch in training

@srowen @matthayes thanks. Let me rerun the training with lower LR (5e-7 would be fine?) and will check the quality of inference on test dataset. I am concerned with the...

OverflowError: out of range integral type conversion attempted while running python qlora.py

Hi, I changed to `huggyllama/llama-7b` and applied the chanige #20. I avoided the above errors and now below* ``` Traceback (most recent call last): File "/Workspace/Repos/[email protected]/qlora/qlora.py", line 853, in train()...

Cannot resume from checkpoint because it is not detected as valid

same for me

multi-gpu get error:Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:7

@2018211801 do you have any update on the issue? The same error happens to me.