X-D-Lab-Shaoziyao

支持自定义视觉编码器么（llava-llama3）?

3

支持自定义视觉编码器么（llava-llama3）? 例如将clip换成siglip? 该如何实现？哪些代码需要修改？

xtuner mmbench on xtuner/ lava-Llama 3-8b-v1_1-hf

1

I have a model of lava-Llama 3-8b-v1_1-hf format How do I run xtuner mmbench on a model in xtuner/ lava-Llama 3-8b-v1_1-hf format? Now it can only accept models in the...

Do you have any plans to evaluate more multimodal medical models? E.g. BiomedCLIP, PubMedCLIP, etc.?

Training code

4

Hello! Your work is excellent and I am also very interested, I wonder when you can open source the training code or give some examples, thanks!

RuntimeError: expected mat1 and mat2 to have the same dtype, but got: float != c 10::BFloat16

2

 How do I solve this problem? The error is as above, and the config is attached below # Copyright (c) OpenMMLab. All rights reserved. import torch from mmengine.hooks import...

llava训练，一条训练输入中包含多张图片。

2

请问xtuner适配一条数据中多张图片训练么？因为我看输入图片路径只有一个，可以是个列表么？ 例如输入文本为：以下哪一张图片更符合香蕉的定义？A: B: 输入图片路径为['image1.jpg','image2.jpg']