Can the model be splited

[dcssi]

When I finetune the model, I'm i changing all the weight, or just some layers ? if its the later, can the model be splited so several fine tuned parts are loaded in the gpu along with the base ?

[yukiarimo]

Yes