deepdish icon indicating copy to clipboard operation
deepdish copied to clipboard

allow passing pytables options on load

Open m4ce opened this issue 6 years ago • 4 comments

This is useful for those cases where one wants to load things directly from memory (e.g. BytesIO) without having to write it out to a file first and load it from there.

bytes_io = io.BytesIO()
dd.io.load("in-memory-sample.h5", driver="H5FD_CORE", driver_core_image=bytes_io.read(), driver_core_backing_store=0)

m4ce avatar Nov 19 '19 17:11 m4ce

Coverage Status

Coverage decreased (-0.6%) to 72.143% when pulling d45d90e718a391e91082f354eadb3f9b98ef3d6a on m4ce:feat/tables_opts into 01af93621fe082a3972fe53ba7375388c02b0085 on uchicago-cs:master.

coveralls avatar Nov 19 '19 17:11 coveralls

Alternatively, load can take a path or a buffer. The signature could be changed to something like:

def load(path_or_buf, group=None, sel=None, unpack=False):

and then internally check whether we are dealing with a string path or a buffer and call tables.open_file with the right params?

m4ce avatar Nov 19 '19 17:11 m4ce

Yes, this would be useful! I think I like the path_or_buf option better. If not, perhaps an option that's called pytables_kwargs. By adding *args, **kwargs, it can swallow any typo without complaining (well, pytables might complain, but that's a bit out of our hands).

gustavla avatar Nov 20 '19 00:11 gustavla

@gustavla, I made the changes and updated the docstring. I opted for pytables_kwargs for simplicity. I am not really sure why the tests fail?

m4ce avatar Nov 20 '19 18:11 m4ce