Lian Junhong
Lian Junhong
> @olihough86我基本上只是做了什么@collant上面建议。我使用该`lengths.ipnb`文件生成训练数据的片段（我只是从每个文件中随机抽取了大约 1600 个字符片段）。然后我将该数据集加载到`finetune.py`文件中并运行训练（并修改它`generate_prompt`以仅返回文本本身而没有所有`### Input`等等）。 > > @T-Atlas我在 3090 和旧的 Tesla P40（比您的 V100 更旧；P40 的计算能力只有 6.1，而 V100 的计算能力为 7.0）上运行了微调，但只有 256 个上下文长度。P40 在 256 上下文中占用了大约 18-20GB。由于 VRAM 限制，我无法在 P40 上运行 512...
> @collant can you help me understand how can I load the Lora model trained with the 52k dataset and use it to train on another data.json? > > In...
I would like to ask, I found that the speed of inference using the model is much slower than training the model. Is this normal? Are there any tricks to...
How to load a model pre-trained on a 52k dataset and continue fine-tuning with another dataset.json?
> Thank you very much for your reply! Is the pre trained LoRA weight in the` './lora-alpaca' `path this file? https://huggingface.co/tloen/alpaca-lora-7b And should I use `trainer.train(resume_from_checkpoint=xx)`?
How to load a model pre-trained on a 52k dataset and continue fine-tuning with another dataset.json?
> Did u guys find a solution? I also want to finetune the model on a specific topic, starting from the checkpoint of alpaca cleaned and finetune with a new...
How to load a model pre-trained on a 52k dataset and continue fine-tuning with another dataset.json?
> Thank you for your message. For your concern, I believe that fine-tuning an already fine-tuned model with new datasets can be an effective way to adapt the model to...
How to load a model pre-trained on a 52k dataset and continue fine-tuning with another dataset.json?
> @T-Atlas Yeah, I believe that it's possible and will be effective to train already fine tuned model to do a new task. What I was thinking of "can I...
How to load a model pre-trained on a 52k dataset and continue fine-tuning with another dataset.json?
> @T-Atlas do you have any idea of how to do this with alpaca? Not yet. I plan to think again at work tomorrow.
How to load a model pre-trained on a 52k dataset and continue fine-tuning with another dataset.json?
> Hi @T-Atlas, this can be a silly answer, but did you try using this? > > ```python > trainer.train(resume_from_checkpoint=checkpoint_local_path) > ``` > > Where the `checkpoint_local_path` variable is the...
How to load a model pre-trained on a 52k dataset and continue fine-tuning with another dataset.json?
> @T-Atlas If the `./hf_ckpt` are containing checkpoints of past training, yes. > > But it seems easier to take LoRA-trained parameters, merges them with the base model (for example...