avalanche icon indicating copy to clipboard operation
avalanche copied to clipboard

Unreasonable default for validation stream creation

Open AlbinSou opened this issue 2 years ago • 0 comments

benchmark = SplitCifar100(20, shuffle=True)
benchmark = benchmark_with_validation(benchmark, val_size=0.05)
print(benchmark.valid_stream[0].classes_in_this_experience) # Shows 125 samples of the same class

The default value to create the validation stream is shuffle=False. This could be reasonable choice if the benchmark dataset is already shuffled (no need to shuffle again in that case). However, when creating it that way, it seems like it's not shuffled.

AlbinSou avatar Jun 27 '23 14:06 AlbinSou