Mkmmd loss

[sanaAyrml]

PR Type

[Feature ]

Short Description

This is an updated version of MK-MMD loss implementation with additions to Fenda, MOON and Ditto models.

• Fenda: Two mk-mmd losses are implemented, one to maximize the distance between local features and the aggregated model and the other to minimize the distance between global features and the aggregated model. In this setting, betas are only updated at the beginning of each step.

• MOON: Two mk-mmd loss is implemented one to minimize distance between local features and aggregated model and other to maximize distance between local features and old-local features model. In this setting betas are updated even during training after a certain number of step based on beta_update_interval. Also in order to prevent from feature value boost a l2-norm regularizer has been added which can be controlled via feature_l2_norm parameter.

• Ditto: One mk-mmd loss is implemented one to minimize distance between local features and aggregated global model. In this setting betas are updated even during training after a certain number of step based on beta_update_interval. Also in order to prevent from feature value boost a l2-norm regularizer has been added which can be controlled via feature_l2_norm parameter.

Also I reorganized the implementation of contrastive loss as a separate file in fl4health/losses folder. And generalized loss reporting function in fl4health/utils/losses as it was failing to report correctly due to addition of new keys during further steps in MOON setup.

Flamby MK-MMD loss experimental setup has also been added to the research/flamby/fedisic2019 folder.

Test

MK-MMD and contrastive loss test added. Also fixed some client's tests to ensure the state of the model during validation and training loss computation.