mirrorboat

我在微调ShareGPT4V时也遇到类似问题 ## 问题描述 对预训练模型进行Instruction Tuning，执行train_mem.py时传入的本地LLM&projector和Visual Encoder模型权重路径均正确，但是代码无法正确找到Visual Encoder的本地权重 ## 原因分析——微调模型背后的代码逻辑 - 运行train_mem.py - train_mem.py调用train.py - train_mem.py在第938行附近调用`share4v_arch.py`的函数（如下） ``` python model.get_model().initialize_vision_modules( model_args=model_args, fsdp=training_args.fsdp ) ``` - share4v_arch.py在第36行附近调用`builder.py`的函数`build_vision_tower(model_args)`，该函数的定义中包含如下代码 ```python vision_tower = getattr(vision_tower_cfg, 'mm_vision_tower', getattr(vision_tower_cfg,...

> config.json的'mm_vision_tower'` > > > 我在分享GPT4V时也遇到类似问题 > > ## 问题描述 > > 对预训练模型进行指令调优，执行train_mem.py时确定本地LLM&projector和Visual Encoder模型权重路径均正确，但代码无法找到正确Visual Encoder的本地权重 > > ## 原因分析——控制器模型背后的代码逻辑 > > > > * 运行train_mem.py > > * train_mem.py调用train.py >...

I wrote my code based on @Rocketknight1 's. I am a transformers beginner and I hope that there isn't any bug in my code. **Code:** ```python import torch from transformers...

> Thank you for your detailed explanation @Rocketknight1 . I have started using the vllm method, which enables efficient inference. But I'll try to use the model.generate() method for batch...

https://github.com/vllm-project/vllm/issues/4180#issuecomment-2066004748 https://github.com/vllm-project/vllm/issues/4180#issuecomment-2074017550 Here @code-isnot-cold ```python from vllm import SamplingParams, LLM model_path = "/path/to/Meta-Llama-3-8B-Instruct" model = LLM( model=model_path, trust_remote_code=True, tensor_parallel_size=1, ) tokenizer = model.get_tokenizer() myinput=[ [{"role": "user", "content": "1 + 1...