[Bug]: Softmax2D activation does not seem suitable for HN training

[orcinus]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

When used for hypernetwork training, softmax2D either does nothing, or immediately diverges. From my understanding of how softmax2D works, i don't even think it's applicable to hypernetwork training.

It might be a good idea to remove it?

Steps to reproduce the problem

Train HN using softmax2D activation with or without normalization. Change learning rate in small increments. Observe no learning rate actually works - either there's nearly no change from epoch to epoch, or it immediately spins out of control in a couple of iterations.

What should have happened?

N/A

Commit where the problem happens

804d9fb83d0c63ca3acd36378707ce47b8f12599

What platforms do you use to access UI ?

Windows, Linux, Other/Cloud

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

No response

Additional information, context and logs

No response