XHB

Results 8 comments of XHB

Can't load tokenizer for 't5-base'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 't5-base' is the correct path to a directory containing all relevant files for a T5Tokenizer tokenizer.

Excuse me, have you solved this problem? I encountered the same problem as you

Can't load tokenizer for 't5-base'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 't5-base' is the correct path to a directory containing all relevant files for a T5Tokenizer tokenizer.

> al Thank you very much. May I ask if you want to load a tokenizer separately, or if you want to put all the files of t5-base together with...

Can't load tokenizer for 't5-base'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 't5-base' is the correct path to a directory containing all relevant files for a T5Tokenizer tokenizer.

> I seem to have put the downloaded models in a separate folder thanks，thanks

Can't load tokenizer for 't5-base'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 't5-base' is the correct path to a directory containing all relevant files for a T5Tokenizer tokenizer.

![Snipaste_2024-08-30_20-27-18](https://github.com/user-attachments/assets/04603cc2-5602-4578-b2b6-2c37ec78f8bf) Hello, is my directory structure correct?

[BUG] <title>自己准备的数据集，在加载的时候，出错：image start token != image end tokens，data fetch error

![image](https://github.com/user-attachments/assets/1e15660f-fd27-4cb8-9367-8ce6de0f2dc8)

[BUG] <title>自己准备的数据集，在加载的时候，出错：image start token != image end tokens，data fetch error

用的模型是MiniCPM-Llama3-V-2_5，LLM_TYPE="llama3" ，2张卡一起跑，lora微调，MODEL_MAX_Length=4096，

[BUG] <title>自己准备的数据集，在加载的时候，出错：image start token != image end tokens，data fetch error

> 请问你是什么卡，a100么？ A800 80G

[BUG] <title>自己准备的数据集，在加载的时候，出错：image start token != image end tokens，data fetch error

> ![image](https://private-user-images.githubusercontent.com/47373076/363548165-7bc90ee4-037b-45d0-bee1-e027bc1ad12b.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjUyNDY1MzEsIm5iZiI6MTcyNTI0NjIzMSwicGF0aCI6Ii80NzM3MzA3Ni8zNjM1NDgxNjUtN2JjOTBlZTQtMDM3Yi00NWQwLWJlZTEtZTAyN2JjMWFkMTJiLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MDIlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTAyVDAzMDM1MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTJmNTJmNGNiOGUxMDkyYjliZDM4OTZmMDEwMjUzY2ZhZjRlZDc4NWNmZmE0MjNiNDQ0ZTJlNzUyZTU5ZDUzYWMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.aORBX96thVd2Cw4M8UuRb5NZ1eOSjUjZDZgU3jePaC0) > > 根据我和另外一位用户的测试，增加这个代码后，就不会出现这个问题，希望能够帮到你 好的，感谢感谢