bitsandbytes icon indicating copy to clipboard operation
bitsandbytes copied to clipboard

chore: delete useless buffered activation

Open Ther-nullptr opened this issue 1 year ago • 0 comments

For QLoRA models, we do not need to update the $\mathbf{W}$, so the buffered activation of $\mathbf{A}$ is useless. It is suggested not to save $\mathbf{A}$ in ctx to save the memory.

Ther-nullptr avatar Jul 05 '24 11:07 Ther-nullptr