Dmitry Nikolaev

New test addad. It compares NHWC MIOpen batchnorm with NHWC CPU batchnorm PYTORCH_MIOPEN_SUGGEST_NHWC=1 python test/test_nn.py -v TestNN.test_batchnorm_miopen_nhwc_compare_cpu

test rewritten to use native nhwc batchnorm to verify miopen non static PYTORCH_MIOPEN_SUGGEST_NHWC variable to be able to change it in runtime @jithunnair-amd @jeffdaily

Demo of enabling and disabling nhwc batchnorm on MIOpen in runtime by modifying `PYTORCH_MIOPEN_SUGGEST_NHWC` environment variable in runtime. Rename attached file to `batchnorm_nhwc_enable_miopen_in_runtime.py` and run it `python -u batchnorm_nhwc_enable_miopen_in_runtime.py` `MIOpen(HIP):...

`test_nn.py` enables to use MIOpen for NHWC batchnorms for whole file by defining `PYTORCH_MIOPEN_SUGGEST_NHWC=1` on startup. `test_batchnorm_nhwc_cuda` used for demo as simplest nhwc batchnorm test. `MIOPEN_ENABLE_LOGGING_CMD=1` enables miopen logging. String...