Dance Diffusion in FP16

[patrickvonplaten]

At the moment dance diffusion in FP16 seems to perform relatively bad even though it was trained in FP16 with AMP autocast -> the reason might just be because the layernorms and/or attention scores have to be done in fp32.

[patrickvonplaten]

To check with original training code: https://github.com/Harmonai-org/sample-generator/blob/main/train_uncond.py

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.