H5Fopen(): unable to open file: bad superblock version number

[RookieJunChen]

Hi, when I run the train, the code failed: RuntimeError: DF dataset error: Hdf5ErrorDetail { source: H5Fopen(): unable to open file: bad superblock version number, msg: "Error during Fil e::open of dataset /dockerdata/thujunchen/cjcode/ft_local/DeepFilterNet/DNS16kdataset/VALID_SET_SPEECH.hdf5" }

There is no error reported at df/prepare_data.py.

I have tried the cargo test, which reports that: running 24 tests test reexport_dataset_modules::util::test_find_max_abs ... ok test tests::test_erb_inout ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_07 - should panic ... FAILED test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_10 ... FAILED test reexport_dataset_modules::dataset::tests::test_hdf5_read_vorbis ... FAILED test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_05 - should panic ... FAILED test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_01 ... FAILED test reexport_dataset_modules::dataset::tests::test_hdf5_read_pcm ... FAILED test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_02 ... FAILED test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_04 - should panic ... FAILED test reexport_dataset_modules::dataset::tests::test_hdf5_read_flac ... FAILED test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_06 - should panic ... FAILED test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_09 ... FAILED test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_08 ... FAILED test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_03 ... FAILED test reexport_dataset_modules::dataloader::tests::test_fft_dataset ... FAILED test reexport_dataset_modules::dataset::tests::test_cached_valid_dataset ... FAILED test reexport_dataset_modules::augmentations::tests::test_filters ... ok test reexport_dataset_modules::augmentations::tests::test_gen_noise ... ok test reexport_dataset_modules::augmentations::tests::test_clipping ... ok test reexport_dataset_modules::augmentations::tests::test_rand_resample ... ok test reexport_dataset_modules::augmentations::tests::test_low_pass ... ok test reexport_dataset_modules::dataset::tests::test_mix_audio_signal ... ok test reexport_dataset_modules::augmentations::tests::test_reverb ... ok

failures:

---- reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_06 stdout ---- -------------- TEST START -------------- thread 'reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_06' panicked at 'called Result::unwrap() on an Err value: Hdf5Error Detail { source: H5Fopen(): unable to open file: bad superblock version number, msg: "Error during File::open of dataset ../assets/noise_flac.h df5" }', libDF/src/dataset.rs:1956:41 note: panic did not contain expected string panic message: "called Result::unwrap()on anErr value: Hdf5ErrorDetail { source: H5Fopen(): unable to open file: bad superblock ve rsion number, msg: \"Error during File::open of dataset ../assets/noise_flac.hdf5\" }", expected substring: "Slice end" ---- reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_09 stdout ---- -------------- TEST START -------------- thread 'reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_09' panicked at 'called Result::unwrap() on an Err value: Hdf5Error Detail { source: H5Fopen(): unable to open file: bad superblock version number, msg: "Error during File::open of dataset ../assets/noise_flac.h df5" }', libDF/src/dataset.rs:1956:41

---- reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_08 stdout ---- -------------- TEST START -------------- thread 'reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_08' panicked at 'called Result::unwrap() on an Err value: Hdf5Error Detail { source: H5Fopen(): unable to open file: bad superblock version number, msg: "Error during File::open of dataset ../assets/noise.hdf5" }', libDF/src/dataset.rs:1956:41

---- reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_03 stdout ---- -------------- TEST START -------------- thread 'reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_03' panicked at 'called Result::unwrap() on an Err value: Hdf5Error Detail { source: H5Fopen(): unable to open file: bad superblock version number, msg: "Error during File::open of dataset ../assets/noise_vorbis .hdf5" }', libDF/src/dataset.rs:1956:41

---- reexport_dataset_modules::dataloader::tests::test_fft_dataset stdout ---- ******** Start test_data_loader() ******** Error: DatasetError(Hdf5ErrorDetail { source: H5Fopen(): unable to open file: bad superblock version number, msg: "Error during File::open of d ataset ../assets/clean.hdf5" }) thread 'reexport_dataset_modules::dataloader::tests::test_fft_dataset' panicked at 'assertion failed: (left == right) left: 1, right: 0: the test returned a termination value with a non-zero status code (1) which indicates a failure', /rustc/fe5b13d681f25ee6474be29d7 48c65adcd91f69e/library/test/src/lib.rs:186:5

---- reexport_dataset_modules::dataset::tests::test_cached_valid_dataset stdout ---- Error: Hdf5ErrorDetail { source: H5Fopen(): unable to open file: bad superblock version number, msg: "Error during File::open of dataset ../ass ets/clean.hdf5" } thread 'reexport_dataset_modules::dataset::tests::test_cached_valid_dataset' panicked at 'assertion failed: (left == right) left: 1, right: 0: the test returned a termination value with a non-zero status code (1) which indicates a failure', /rustc/fe5b13d681f25ee6474be29d7 48c65adcd91f69e/library/test/src/lib.rs:186:5

failures: reexport_dataset_modules::dataloader::tests::test_fft_dataset reexport_dataset_modules::dataset::tests::test_cached_valid_dataset reexport_dataset_modules::dataset::tests::test_hdf5_read_flac reexport_dataset_modules::dataset::tests::test_hdf5_read_pcm reexport_dataset_modules::dataset::tests::test_hdf5_read_vorbis reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_01 reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_02 reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_03 reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_04 reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_05 reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_06 reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_07 reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_08 reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_09 reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_10

test result: FAILED. 9 passed; 15 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.67s

error: test failed, to rerun pass '-p deep_filter --lib'

I tried to update hdf5 to 1.10.1 as stated in https://stackoverflow.com/questions/49386121/python-h5py-file-read-oserror-unable-to-open-file-bad-superblock-version-numb, but it also did not work.

[Rikorose]

Are you able to open the datasets with h5py? E.g. using https://github.com/Rikorose/DeepFilterNet/blob/main/DeepFilterNet/df/scripts/sample_from_hdf5.py

[RookieJunChen]

I can open the datasets with h5py by your code.

However, when it comes to training, the code still reports the error:

File "DeepFilterNet/df/train.py", line 631, in main() └ <function main at 0x7f6c6aba5a60>

File "DeepFilterNet/df/train.py", line 140, in main dataloader = DataLoader( └ <class 'libdfdata.torch_dataloader.PytorchDataLoader'>

File "/dockerdata/thujunchen/cjcode/ft_local/DeepFilterNet/pyDF-data/libdfdata/torch_dataloader.py", line 99, in init self.loader = _FdDataLoader( │ └ <class 'builtins._FdDataLoader'> └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7f6c634145b0>

RuntimeError: DF dataset error: Hdf5ErrorDetail { source: H5Fopen(): unable to open file: bad superblock version number, msg: "Error during File::open of dataset /dockerdata/thujunchen/cjcode/ft_local/DeepFilterNet/DNS16kdataset/VALID_SET_SPEECH.hdf5" }

[RookieJunChen]

I also tried the solution in the previous issue by updating hdf5 to 1.12.1, but this also did not work.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 7 days.