nkjulia
nkjulia
> Hi @xinfu607, > > The framework is using `torch.cuda.amp.autocast` during training, and LLaMA is loaded (by default) from a pretrained checkpoint using `torch.float16` (please see Line 97 in `mini_gpt4.py`)....
> A Photo of a sunset can you share some infomation about your training dataset ？
> The reason why it generated "### instruction" is because your fine-tuning is inefficient. In this case, we put a eos_token_id=2 into the tensor for each instance before fine-tune, at...
i met the same issue...no solutions found yet...
> @lywinged @AngainorDev @nkjulia 请问下各位大佬，如果想要finetune一个模型，让它更关注于某个领域的话，该怎么调整训练方案比较合适？ 理论上拿领域内的数据进行finetune就好了，基于llama我这边还没调通，训练了几个版本都不符合预期
> It's hard to say anything without more context. pretrained llama model : decapoda-research/llama-7b-hf data formate： raw prompts for text-image tasks，didn't use the templates in alpaca.json training set size： 60000...
Finally i figure out my problem where inference! the saved model file "adapter_model.bin" is not valid,with small size,i replace it with other bin file in the last checkpint directory,the inference...
> > Finally i figure out my problem where inference! the saved model file "adapter_model.bin" is not valid,with small size,i replace it with other bin file in the last checkpint...
> > Hi, Could you please share ur training code? For me it's a paraphrasing task. Input needs to be paraphrased. But I am getting poor results. 60k datapoints are...