hdmf icon indicating copy to clipboard operation
hdmf copied to clipboard

propose alternative chunk shape algorithm

Open bendichter opened this issue 2 years ago • 3 comments

Motivation

Fix #995

How to test the behavior?

Show how to reproduce the new behavior (can be a bug fix or a new feature)

Checklist

  • [ ] Did you update CHANGELOG.md with your changes?
  • [ ] Have you checked our Contributing document?
  • [ ] Have you ensured the PR clearly describes the problem and the solution?
  • [ ] Is your contribution compliant with our coding style? This can be checked running ruff from the source directory.
  • [ ] Have you checked to ensure that there aren't other open Pull Requests for the same change?
  • [ ] Have you included the relevant issue number using "Fix #XXX" notation where XXX is the issue number? By including "Fix #XXX" you allow GitHub to close issue #XXX when the PR is merged.

bendichter avatar Nov 08 '23 18:11 bendichter

In general, I think this is reasonable. @CodyCBakerPhD can you also take a look.

oruebel avatar Nov 08 '23 19:11 oruebel

Another approach could be to work with the prime factorization of the residual product of the size. That would get us closer to the desired chunk size but we could end up with very uneven shapes

bendichter avatar Nov 08 '23 19:11 bendichter

Another approach could be to work with the prime factorization of the residual product of the size. That would get us closer to the desired chunk size but we could end up with very uneven shapes

Uneven shape is not necessarily bad, but I think you probably want the ratio to be informed by the overall size of the dataset. I.e., for (time, electrodes) where time is much longer than electrodes, then you'd also want the chunks to be larger in the time dimension.

oruebel avatar Nov 08 '23 20:11 oruebel