Jipeng Zhang comments

Repositories
Issues
Comments

Results 3 comments of


                                            Jipeng Zhang

No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1'

You may need to downgrade the deepspeed version to 0.8.2 or 0.8.0. pip install deepspeed==0.8.2. Also, if this can not work, you can try to switch your pip mirror by...

Fine-Tuning Crashes for no reason when Eight GPU cards are used.

I suggest to switch `lm-eval` package's version back to 0.4.0. Just like: ``` git clone -b v0.0.4 https://github.com/EleutherAI/lm-evaluation-harness.git cd lm-evaluation-harness pip install -e . ``` Also, if there any further...

Question about running stage 3 script

BTW, I met a similar issue on Adam version: ``` IsADirectoryError: [Errno 21] Is a directory: '../grads/llama2-7b-p0.05-lora-seed3/flan_v2-ckpt422-adam' ```