Dragonfire3900

Results 1 issues of Dragonfire3900

Minimizing memory usage with a large custom dataset (possible memory leak with first epoch)

2 comment

I've written a custom dataset with the tfds cli (a GeneratorBasedBuilder without Beam). Overall the dataset is ~60 GB and is sourced from manually downloaded hdf5 files with mostly float32s...

help