detach

Open SmileTAT opened this issue 1 year ago • 1 comments

why use detach() for promptB？Is it the solution for "Parameter at index 1 with name xxx has been marked as ready twice"?

Sep 11 '24 03:09 SmileTAT

Hi @SmileTAT detach() means setting the gradients to zero for the specified parameter. Here, we want to optimize only one prompt, namely PromptA, so we detach PromptB. TBH, we haven't tried not detaching PromptB; they might have a similar performance. You could give it a try.

Sep 11 '24 05:09 zengyh1900