disent icon indicating copy to clipboard operation
disent copied to clipboard

Standardise Loss Reduction Modes

Open nmichlo opened this issue 4 years ago • 1 comments

Removed reduction="sum" in #12, but considerable effort is still required to maintain and verify models (#3, #4, #5)

Two options exist:

  1. Allow models to specify their allowed modes: "mean" or "mean_sum"
  2. Only use "mean_sum"

nmichlo avatar Jul 28 '21 12:07 nmichlo

"Loss reduction mode" should maybe be called "beta scaling" and be a boolean value?

  • This takes inspiration from the original BetaVAE paper if I am not mistaken.

The problem with the loss reduction modes is that not all the frameworks supports them.

  • Could this automatically be handled by changing how the loss is aggregated?

nmichlo avatar Feb 05 '22 14:02 nmichlo