yxd886
Results
3
comments of
yxd886
Hello, Can JAX support multi-host GPUs? If so, and how to do the configuration?
> I believe there is (very preliminary, untested, subject to change) multi-host Cloud TPU support, and what you describe is maybe one or two steps away from a recipe that...
Dear hawkinsp, Thanks for your prompt reply. I still think that the overlapping of allreduce communication and computation is very important. It is true that for tiny models such as...