ffcv icon indicating copy to clipboard operation
ffcv copied to clipboard

ffcv loader and batch_size=1

Open crocis opened this issue 2 years ago • 1 comments

I have a problem related to the ffcv loader. In my code, when I set the batch_size equal to one, the ffcv loader does not work, i.e., when I try to iterate through the dataset, the loader does not return any element. While when I set the batch_size to a different number than one, the loader does work.

      loader = ffcv.loader.Loader(
          str(filename),
          batch_size=1, 
          seed=1,
          order=OrderOption.QUASI_RANDOM,
          pipelines=pipelines,
          drop_last=False,
          indices=indices,
          distributed=False,
          num_workers=3,
          os_cache=True,
      )

crocis avatar Feb 10 '23 05:02 crocis

hi @crocis ! My first guess is that this is an issue with QUASI_RANDOM loading, if you change QUASI_RANDOM to SEQUENTIAL does the issue persist?

andrewilyas avatar Feb 28 '23 22:02 andrewilyas