iris
iris copied to clipboard
Bugfixes + changes wanted in "combine_regions"
As identified in work on #4845
Follow-on fixes/changes :
- the call to "fix_dask_settings" in the setup_cache call is not necessarily affecting what happens in the main tests, as we had thought. * The "setup" call should be doing that first, instead of last, so that it can affect how test cubes are saved/reloaded.
- the basic "combine_regions" operation ought to be enforcing full-dim chunks in the mesh dimension (or it may not work). Never tested yet as mesh-dim < default chunksize. This can be fixed by rechunking (this was tested).
- we should ensure that the memory cost of this operation does not increase with scaling on an outer dimension -- i.e. it can repeat over timesteps et al without a further cost multiplier
Re: (2) : enforcing full-mesh-dim chunks...
Suggestion captured:
Following creation of result_array, with mesh in the last dim, add lines ...
# rechunk result if needed, to ensure that mesh dim is NOT split
if result_array.chunksize[-1] != result_array.shape[-1]:
chunks = list(result_array.chunksize)
chunks[-1] = result_array.shape[-1]
result_array = result_array.rechunk(chunks)