shidingz

minigemini_instruction.json包含了预训练的LLaVA Images，但是在readme中没有写到微调用到了这部分数据

1

minigemini_instruction.json中有条目的image为llava/LLaVA-Pretrain/images/00013/000133305.jpg，有很多llava的图片，但是readme中没有提到要用llava的图片，是你们疏忽了吗

运行代码报错AttributeError: 'list' object has no attribute 'to'， image_aux_features_raw = self.get_model().get_vision_tower_aux()(images_aux).to(dtype=image_features.dtype, device=image_features.device)

2

Traceback (most recent call last): File "/checkpoint/binary/train_package/minigemini/train/train_mem.py", line 14, in train(attn_implementation="flash_attention_2") File "/checkpoint/binary/train_package/minigemini/train/train.py", line 1262, in train trainer.train() File "/root/.local/lib/python3.8/site-packages/transformers/trainer.py", line 1624, in train return inner_training_loop( File "/root/.local/lib/python3.8/site-packages/transformers/trainer.py", line 1961,...

请问为什么在训练llama的脚本中，预训练和微调所使用的conv不一样

1

https://github.com/dvlab-research/MGM/blob/main/scripts/llama/train/stage_1_2_full_v7b_336_hr_768.sh 在这个脚本中pretrain用的--version plain 而finetune用的是--version v1 前后不一致模型不会混乱吗

Strange Model Loading Issue: Inconsistency with Vision Tower Parameters

2

I found something strange when loading the model. It seems that you have released the vision_tower during training, but when loading the vision_tower, you did not load the gradient-updating parameters,...

I get this error: WARNING: tokenization mismatch: 156 vs. 161. (ignored) when I finetune llama3

1

When I run this script-scripts/llama3/train/stage_2_full_v8b_672_hr_1536.sh, I encounter this error- WARNING: tokenization mismatch: 156 vs. 161. (ignored)