tofu icon indicating copy to clipboard operation
tofu copied to clipboard
verified publisher icon locuslab

Unable to train fintuned LoRA on forget

Open shaswati1 opened this issue 1 year ago • 0 comments

Getting below error while trying to train the finetuned (LoRA llama2) on forget set: ValueError: Target module Dropout(p=0.05, inplace=False) is not supported. Currently, only the following modules are supported: torch.nn.Linear, torch.nn.Embedding, torch.nn.Conv2d, transformers.pytorch_utils.Conv1D.

can you please help me with this?

shaswati1 avatar Mar 07 '24 20:03 shaswati1