随时学丫

how to infer or predict my own trained model

4

I have trained model according to the tutorial you provided, but I can't find the predict model code, like predict.py, can you guide me?

训练对齐器，显存溢出

1

看了训练流程图，我理解这个对齐器是不是在全参微调，我跑百川7b的模型，4090 24G的显卡，跑不起来，显存满了，只能换更大的显存吗？多大的显存合适？ 

没看懂这个key

[Bug] Lora finetuning memory keeps rising until it is Out Of Memory

1

[Bug] Lora微调内存一直上涨直到Out Of Memory 中文：其他项目中都会增加参数`overwrite_cache`去释放暴涨的显存，这个项目我加这个参数没用，请问有可能解决吗？ english: in other projects will add parameter `overwrite_cache` to release the skyrocketing video memory. In this project, it is useless for me to add this...

After AWQ quantization, the output cannot be stopped normally

2

model: qwen1.5-7b Python 3.10.8 cuda 12.1 torch 2.4.0 transformers 4.41.2 **error case**  **tokenizer_config.json**  **train code** ``` import datasets import json from awq import AutoAWQForCausalLM from transformers import AwqConfig,...