Support for single-file multithreading

[mbauman]

#230 implemented thread-safety at a file level. However, if I try to read from a single file from multiple threads, I hit segfaults. I understand that multithreaded writing to a single file is fraught, but would it be a major challenge to support multithreaded reads?

[JonasIsensee]

Hi @mbauman,

this really should be possible. First, the reason that it doesn't work: JLD2 (globally) keeps track of all files that are being opened. This is to ensure that you e.g. don't accidentally open a single file in both read and write mode at the same time.

• If you disable that logic, threaded reading should work. see
• https://github.com/JuliaIO/JLD2.jl/blob/e746c89202e6e831fc89363640edc5d4af66cb1c/src/JLD2.jl#L272

With the current logic it doesn't work since the same file handle is being used to process reads. And also the JLDFile object has plenty internal caching that could hit race-conditions.

• This could in principle be fixed by introducing locks around all operations that change the cache or really read from the file.