disent
disent copied to clipboard
Standardise Loss Reduction Modes
Removed reduction="sum" in #12, but considerable effort is still required to maintain and verify models (#3, #4, #5)
Two options exist:
- Allow models to specify their allowed modes:
"mean"or"mean_sum" - Only use
"mean_sum"
"Loss reduction mode" should maybe be called "beta scaling" and be a boolean value?
- This takes inspiration from the original BetaVAE paper if I am not mistaken.
The problem with the loss reduction modes is that not all the frameworks supports them.
- Could this automatically be handled by changing how the loss is aggregated?