Gabriel Mongaras

Small edit to scripts/prepare_alpaca.py to fix codec errors I ran into when finetuning for Falcon.

I was able to finetune a model on a custom dataset. However, I've been running into quite a few issues when trying to generate a sample from the generation script...

Balancer for different model outputs

1

Added balancer support for different outputs from the model. For example, if a model outputted two terms, x1 and x2, and the losses are: losses['loss_a'] = compute_loss_a(x1, y1) losses['loss_b'] =...

p_mean_variance mean calculation

2

I was looking through the code to see how the paper was implemented, but I ran into an issue when looking at the part of the paper measuring the KL...

Thanks for making this repo! Really helpful for a project I'm working on. However, when generating in a batch, there are a couple of isses. The first is the missing...

The llava model requires the modalities parameter to be broadcasted to the batch size, otherwise the zip statement on line 442 in llava/model/llava_arch.py reduces the batch size to 1 (the...

[Bug-App]: Gradient logging issues when using torch.amp.GradScaler

5

### Describe the bug I am using wandb 0.19 on an Ubuntu machine. It looks like wandb logs the scaled gradients when using a gradient scalar as described in this...