openeo-python-client icon indicating copy to clipboard operation
openeo-python-client copied to clipboard

Reduce_temporal throws MetadataException after load_stac

Open VictorVerhaert opened this issue 1 year ago • 6 comments

When using load_stac, the python client tries to read metadata from the provided stac. The only metadata currently being read is bands, so no spatial or temporal metadata.

When the python client fails to read metadata from the stac, metadata is simply set to None. These lines check is metadata is present and have default behaviour otherwise: https://github.com/Open-EO/openeo-python-client/blob/36ecc2bdb0feaf4b7876819946866bd651a18189/openeo/rest/datacube.py#L1350-L1359 and https://github.com/Open-EO/openeo-python-client/blob/36ecc2bdb0feaf4b7876819946866bd651a18189/openeo/rest/datacube.py#L1216

The problem arises however when metadata IS being read from the STAC. As only bands are read cube.metadata is not None, but it does not contain a temporal dimension resulting in a MetadataException originating for example from: https://github.com/Open-EO/openeo-python-client/blob/36ecc2bdb0feaf4b7876819946866bd651a18189/openeo/metadata.py#L271-L275

The solution is threefold I think:

  1. Not only check if the metadata is not None but also whether it contains a temporal dimension
  2. Implement reading a temporal dimension (and spatial) from stac metadata
  3. Catch metadata related errors and throw them as warnings (Discussion) as this blocks the execution of a batch job while it would run fine.

VictorVerhaert avatar Jun 04 '24 09:06 VictorVerhaert

as discussed, a workaround would be to keep the openeo package below 0.29 for now

soxofaan avatar Jun 04 '24 11:06 soxofaan

Another workaround is to add the following: cube.metadata = cube.metadata.add_dimension("t", label=None, type="temporal")

VictorVerhaert avatar Jun 04 '24 11:06 VictorVerhaert

Started with a PR at #568

soxofaan avatar Jun 05 '24 16:06 soxofaan

I already merged an initial fix that avoids throwing the MetadataException, so at least that can already be included in the next release

next steps are properly detecting real properties of the temporal dimension

soxofaan avatar Jun 14 '24 15:06 soxofaan

@soxofaan @VictorVerhaert this method parses already correctly the dimensions, why not re using that? It seems we are trying to re-implement something that already exists: https://github.com/Open-EO/openeo-python-client/blob/3c6045b6f0215455609599a868340cc52cddf86c/openeo/metadata.py#L413

clausmichele avatar Jul 17 '24 08:07 clausmichele

@soxofaan I created a PR here: https://github.com/Open-EO/openeo-python-client/pull/591 . Currently it solves the issue I have, but there are probably other scenarios that could be covered (currently it makes only a difference for STAC Collections having cube:dimensions available.

clausmichele avatar Jul 17 '24 09:07 clausmichele

I think this can now be closed with #591 being merged

soxofaan avatar Aug 21 '24 08:08 soxofaan