Claudiu Daniel Hromei

[Kosmos-2] Fine-tune your checkpoint model on my downstream task

22

Hello everyone, thank you very much for your contribution. I appreciate the effort and consistency in uploading the code for such many models and maintaining this repository. I saw Kosmos-2...

[Usage] Cannot load the tuned projector weights

20

### Describe the issue Issue: I want to fine-tune a multi-modal LLM on a downstream task that uses both images and text. This is what I've done: 1. I tried...

ValueError: Trying to set a tensor of shape torch.Size([128257, 4096]) in "weight" (which has shape torch.Size([128256, 4096])), this look incorrect.

3

Hello everyone, thank you for the great job! I am trying to further fine-tune the LLaVA architecture using your implementation with LLaMA 3 Instruct 8B. I can already fine-tune the...