Ashvini Jindal

Training Time for 1x 4090 GPU

5

Hi there, I am running on 1x 4090 GPU and using pytorch 2.0 nighty build. Running `python train.py` as suggested with default parameters: > out_dir = 'out' eval_interval = 2000...

How to enable training on competition only along with packing = True

4

Hi unsloth team, I am wondering how to enable `packing = True` when I need to only train on `output` tokens for a `' text pair eg: ''. This is...

Code for attentive pooling for CNN

1

Dear Authors, Thanks for your work. It was really interesting to learn something new :) I have a doubt regarding attentive pooling for CNN. There are two classes `class Attention`...

Compare performance with office C implementation of Glove

1

Hi, Could you compare your code performance/loss with official C implementation of Stanford?

Zero Loss: The algorithm failed to converge because the input matrix is ill-conditioned or has too many repeated singular values

1

Hi GaLore Team, congratulations for the interesting work! I am trying to fine-tune llama-3 8B model using GaLore but getting this error: `torch._C._LinAlgError: linalg.svd: The algorithm failed to converge because...