[WIP] OpenPMD frontend update

[henrynjones]

This work-in-progress PR replaces the now defunct Pull request #4848 (thank you @neutrinoceros ) and addresses issue #4757 by using the backend-agnostic openPMD-api as the IO module instead of the previous h5py.

TODO

• Add sorted particles to the particle grid in which they are located, for the short term this will only work for ADIOS2, file-based iteration encoded datasets with the particlePatches.

  • For HDF5 files, record_component.available_chunks() only contains one chunk and all levels cover the entire domain. We might have to trim/mask higher level data so that only level 0 data exists on the entire domain, and each subsequent level covers only the refined region.

• Still haven't looked at string handing for file path patterns. Would be great to load a dataset series from file-based iteration-encoded diagnostics. Right now yt uses [0-9] or dd???/dd??? or diag_name_??? while openpmd-api uses %T or %06T for example.

• More information to come in the next few days. For now I have attached a notebook demonstrating some improvements and outstanding issues here: https://github.com/henrynjones/yt_supplemental/blob/main/PR_update.ipynb

• This is a messy version full of comments and print statements for debugging. I mostly wanted to get my branch back up to speed.

PR Checklist

• [ ] New features are documented, with docstrings and narrative docs
  • Modern mesh refinement tools similar to the boxlib frontend for ADIOS2 files.
  • More to add here
• [ ] Adds a test for any bugs fixed. Adds tests for new features.

[BenWibking]

I am very interested in using this frontend. I just wanted to ask about the current status. Is it ready for use? Are there still plans to merge it into main?

[neutrinoceros]

Speaking for myself, I avoid reviewing PRs marked as "WIP" as a way to manage the limited time I can spend on reviews. I'm happy to dive in when @henrynjones marks this as ready.

[henrynjones]

Thank you for your interest and patience @BenWibking. I have moved groups and projects since I started this work but I would like to get this frontend makeover mergeable shortly. My last remaining obvious problem is seen in the demonstration notebook, where calling annotate_particles() after annotate_grids() causes the plot window to zoom out for an unknown reason. As it turns out, I have also been able to reproduce this error with the amrex frontend, so that seems like a separate issue. @neutrinoceros, have you seen this occur with other frontends? Interestingly calling annotate_particles before annotate_grids causes no issues. I will merge again shortly so that you may try it out. After that I will reach out to @ax3l to see if he encounters any problems and then clean things up and try to write a few tests.

[neutrinoceros]

have you seen this occur with other frontends?

I don't think I have. Please report this issue separately so we don't loose track of it !

[pgrete]

What's the status of this frontend? Parthenon based codes are also now writing more and more data in OpenPMD format.