Nathan Luehr comments

Results 5 comments of


                                            Nathan Luehr

Update NVIDIA libs used for OSS builds

I have run the pip and nonpip tests locally as described in [tf_sig_build_dockerfiles/README.md](https://github.com/tensorflow/build/blob/master/tf_sig_build_dockerfiles/README.md) on V100 and A100. The only test failures were addressed in TF PRs [56821](https://github.com/tensorflow/tensorflow/pull/56821) and [56823](https://github.com/tensorflow/tensorflow/pull/56823). All...

Tensorflow ist not using full gpu memory

@panagiotisanagnostou what value did you end up setting TF_DEVICE_MIN_SYS_MEMORY_IN_MB to?

Perform gradient clipping on global batch when using gradient accumulation

@zhangqiaorjc is there a reason this has been approved by not merged yet?

Mixed precision doesn't work properly on NVIDIA A10G GPUs

Reproduced on a GTX 3090 (compute capability 8.6, 24GB of memory) With CUDNN 8.2.0 (as provided in the referenced TRT container) I see the OOM for mixed_float16 while float32 runs...

Mixed precision doesn't work properly on NVIDIA A10G GPUs

You can upgrade cudnn to newer minor versions (e.g., 8.3.0 over 8.2.x) without rebuilding TensorFlow. I did not update CUDA in my tests. It is generally safe to use a...