seansong

Also, Is there a way to get the token per second during training on each GPU ? Thanks

@wukaixingxp I was able to run the [official FlopTensorDispatchMode example](https://pytorch.org/tnt/stable/utils/generated/torchtnt.utils.flops.FlopTensorDispatchMode.html#torchtnt.utils.flops.FlopTensorDispatchMode). However, I encountered the same issue when using pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.1. Interestingly, everything works fine without...

Facing the same issue.

Here are some thoughts that come to mind: Average TFLOP/s per GPU Average Tokens/s per GPU Average Samples/s GPU Utilization (% TFLOP/s per GPU)