Yujie Li comments

Results 9 comments of


                                            Yujie Li

[Question] pretrain_mm_mlp_adapter of llava v1.6 7B not found

@YQYI I guess you can just use the mlp_adapter of llava v1.5 7B as the blog says ![image](https://github.com/haotian-liu/LLaVA/assets/94554614/1228cab1-d715-47e5-ba61-8a1ef6f54985)

metrics about region captioning

@Hoteryoung I also met this problem and the metric fell even lower after stage2 finetune😂

ValueError: The checkpoint you are trying to load has model type `llava_llama` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

version of transformers not correct

ValueError: The checkpoint you are trying to load has model type `llava_llama` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

try 4.36.2?It runs well on VILA1.5. @gehong-coder

ValueError: The checkpoint you are trying to load has model type `llava_llama` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

@gehong-coder oh, maybe the repository has been upgraded. You can see the original version of guide: ``` conda create -n vila python=3.10 -y conda activate vila pip install --upgrade pip...

ValueError: The checkpoint you are trying to load has model type `llava_llama` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

and pyproject.toml [build-system] requires = ["setuptools>=61.0"] build-backend = "setuptools.build_meta" [project] name = "vila" version = "1.0.0" description = "VILA: On Pre-training for Visual Language Models" readme = "README.md" requires-python =...

ValueError: The checkpoint you are trying to load has model type `llava_llama` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

@gehong-coder this is from network error, maybe shutting down your proxy would work?

[Usage] Visual instruction tuning for LLaVa 1.6

use zero2.json may help? But I don't know why, do you have any idea?

[Usage] Visual instruction tuning for LLaVa 1.6

under directory scripts