python-blosc2 icon indicating copy to clipboard operation
python-blosc2 copied to clipboard

vlmeta parameter in kwargs?

Open aleixalcacer opened this issue 2 years ago • 1 comments

Hello again, it could be interesting to pass the vlmeta parameter in the kwargs (passing the meta parameter in kwargs is already supported)?

I'm interested because I'm defining a codec in which I need to read/write information that could be stored in the vlmeta parameter.

My use case :

# encoder and decoder is already defined and registered

cparams = {
    'codec': codec_id,
    'codec_meta': 0,
    'nthreads': 1,
    'filters': [],
    'splitmode': blosc2.SplitMode.NEVER_SPLIT,
}

dparams = {
    'nthreads': 1,
}

 c_img = blosc2.asarray(img, chunks=chunks, blocks=blocks, cparams=cparams, dparams=dparams)

It would be nice if I could pass the vlmeta as kwargs and have it serialized before compression. I think, it could be something like:

# encoder and decoder is already defined and registered

cparams = {
    'codec': codec_id,
    'codec_meta': 0,
    'nthreads': 1,
    'filters': [],
    'splitmode': blosc2.SplitMode.NEVER_SPLIT,
}

dparams = {
    'nthreads': 1,
}

vlmeta = {
  'my_codec': my_info
}

 c_img = blosc2.asarray(img, chunks=chunks, blocks=blocks, cparams=cparams, dparams=dparams, vlmeta=vlmeta)

I know it's not ideal in terms of performance (as they will have to be rewritten every time you compress a chunk). But since it's optional, it shouldn't affect users who care about performance.

If it seems good to you and you don't see any inconvenience, can I start with it and create a PR?

Aleix :)

aleixalcacer avatar May 10 '23 18:05 aleixalcacer

Sorry for the late reply. I like your proposal, so if you are still interested, please go ahead with a PR.

FrancescAlted avatar Mar 19 '24 07:03 FrancescAlted