podaac
Cloud Optimized Examples refer to tools and services pending deprecation
The PO.DAAC Cookbook has a section on Cloud Optimized Examples. These examples show the usage of zarr-eosdis-store and the Harmony NetCDF-to-Zarr conversion service.
zarr-eosdis-store has functional overlap with VirtualiZarr, and so we're working through deprecating zarr-eosdis-store in favour of VirtualiZarr instead.
The Harmony NetCDF-to-Zarr service is in the process of being removed from Harmony due to low adoption, and some functional limitations. An alternative tool for ad hoc Zarr store generation is pangeo-forge.
I'm happy to open a PR removing the items referring to these tools, if that sounds good.
Hi @owenlittlejohns when you say "removing the items" are you referring to cookbook notebooks? I think there should only be one or so, in which case you can remove them. Would you please list the notebooks you proposed to remove on this issue before removing them?
Hi @DeanHenze!
Sorry for not including links to the specific parts of the cookbook. Looking back at this, I think there were three things I was looking at:
- Zarr access for netCDF4 files, which uses this source notebook. Maybe this could be replaced with an alternative VirtualiZarr example?
- zarr-eosdis-store-example, which uses this source notebook. This packages is no longer maintained, so maybe this documentation should be outright removed.
- COF Zarr Access via Reformat, which uses this source notebook. This service is no longer available via Harmony, so I think this example should also be removed.
Maybe the first thing to do is remove the zarr-eosdis-store-example and COF Zarr Access via Reformat sections? Then the other notebook could either be updated or swapped out for one that uses VirtualiZarr or an alternative package to do the same thing?
@owenlittlejohns
Oh man looking into this package more I can't believe I wasn't aware of it! Sounds like it was ahead of it's time a little?
- For that first one
Zarr access for netCDF4 files, we could replace the link with this Virtual Data Set Starter Notebook, which uses this source notebook. It's not a perfect 1-1 because this notebook demonstrates access to virtual datasets (VDS's) that we pre-create and host on PO.DAAC, rather than using the DMR++ files to create a Zarr-like store on the fly. But it is the method that we are gravitating toward at PO.DAAC (pre-creation of VDS's). We could include @ayushnag 's DMR++ functionality from earthaccess in this notebook ultimately, but I don't know when I'll get around to it. - This notebook from @ayushnag talks about DMR++ toward the end but I think it's out of date.
- I agree you can remove the zarr-eosdis-store-example and COF Zarr Access via Reformat sections.