CaTTail

CUDA out of memory

5

**Describe** CUDA out of memory. I'm fine-tuning the llama-2-70B using 3 sets of machines containing 8\*A100s (40GB)=24\*A100(40GB), and this error reported at first seemed like it should be an out-of-memory...

Appreciate your great work! Is it possible to fine tune the llama-2-70B for a 3\*8*A100 (40G) configuration, thanks!

Thanks for your amazing work! I'm experiencing out-of-memory problems when using wizardmath's fine-tuning code to do Supervised fine-tuning for 70B (Llama-2-13B doesn't have this problem), using a configuration of 3...

About llama-2-70B fine-tuning

2

Appreciate your great work! Is it possible to fine tune the llama-2-70B for a 3\*8*A100 (40G) configuration, thanks!