wannabeOG
Issue related to frozen layers
Hello, I have a question regarding the freeze layers here. I notice that the frozen layers are set to require gradient, in contrast, the not frozen layers don't require gradient. I'm not sure if I understand correctly, but this looks like we actually train frozen layers and ignore the not frozen layers, which is a little unnatural to me. Besides, we still initialize omega for not frozen layers here. Do I miss something? Thanks in advance for your help!
I convert the setting of required grads for frozen layers and not frozen layers, keep all the default arguments and run main.py. Here is the results I got Testing the model now Testing the model on task 1 The forgetting undergone on task 1 is -0.0032000000000000015 Testing the model on task 2 The forgetting undergone on task 2 is -0.0011999999999999927 Testing the model on task 3 The forgetting undergone on task 3 is -0.004400000000000001 Testing the model on task 4 The forgetting undergone on task 4 is 0.0