get state dict OOM

[WallE-Chang]

I train llama 13 in 8 3090 with lora. Model can be forwarded and backwarded. But when model get state dict, gpu is OOM.

[edwardjhu]

Seems like the trainer is saving the entire model as opposed to saving just the LoRA modules. I suspect that doing the latter will resolve this.