ffcv icon indicating copy to clipboard operation
ffcv copied to clipboard
verified publisher icon libffcv

Distributed support for QUASI_RANDOM

Open wouterzwerink opened this issue 2 years ago • 1 comments

Supports distributed training with QUASI_RANDOM traversal order. Relies on the same code as non-distributed to generate the order. Then, divides this into num_ranks contiguous chunks. The last two ranks can have some overlap to ensure that all ranks have the same amount of data.

wouterzwerink avatar Jul 10 '23 20:07 wouterzwerink

Thanks so much for this PR, sorry for the late reply! Did you try training anything with this sampler/were the results okay? If so, I'll merge it in to v1.1.0

andrewilyas avatar Aug 24 '23 15:08 andrewilyas