satpy icon indicating copy to clipboard operation
satpy copied to clipboard

satpy_cf_nc Reader Fails to Read Data Written by cf Writer

Open robertdemariacira opened this issue 3 years ago • 0 comments

Describe the bug In the documentation, the satpy_cf_nc reader is listed as the reader to use for files written by the cf writer. When reading a file written by the cf writer, the code throws an exception while looking for an attribute named "instrument" that does not appear to be written by the cf writer.

To Reproduce Create a Scene from a file (a GOES-16 file was used here), then write the scene using the cf writer and attempt to load it.

in_file = "OR_ABI-L1b-RadF-M6C15_G16_s20222210610205_e20222210619519_c20222210619585.nc"
scn = satpy.Scene(reader="abi_l1b", filenames=[in_file])
scn.load(["C15"])
out_filename = "g16-abi-20220809061020-20220809061951.nc"
scn.save_datasets(writer="cf",
                             filename=out_filename,
                             datasets=["C15"],
                             include_lonlats=False)

read_scn = satpy.Scene(reader="satpy_cf_nc", filenames=[out_filename])
read_scn.load(["C15"])

Expected behavior The load function should successfully load the written data.

Actual results Text output of actual results or error messages including full tracebacks if applicable.

[DEBUG: 2022-08-10 21:07:48 : satpy.readers.yaml_reader] Reading ('/local/home/rdemaria/miniconda3/envs/g16_mod_env/lib/python3.8/site-packages/satpy/etc/readers/abi_l1b.yaml',) [DEBUG: 2022-08-10 21:07:48 : satpy.readers.yaml_reader] Assigning to abi_l1b: ['/mnt/grb/goes16/2022/2022_08_09_221/abi/L1b/RadF/OR_ABI-L1b-RadF-M6C15_G16_s20222210610205_e20222210619519_c20222210619585.nc'] [DEBUG: 2022-08-10 21:07:48 : satpy.composites.config_loader] Looking for composites config file abi.yaml [DEBUG: 2022-08-10 21:07:48 : satpy.composites.config_loader] Looking for composites config file visir.yaml [DEBUG: 2022-08-10 21:07:48 : satpy.readers.abi_l1b] Reading in get_dataset C15. [DEBUG: 2022-08-10 21:07:48 : satpy.writers] Reading ['/local/home/rdemaria/miniconda3/envs/g16_mod_env/lib/python3.8/site-packages/satpy/etc/writers/cf.yaml'] [INFO: 2022-08-10 21:07:48 : satpy.writers.cf_writer] Saving datasets to NetCDF4/CF. /home/rdemaria/miniconda3/envs/g16_mod_env/lib/python3.8/site-packages/satpy/writers/cf_writer.py:571: FutureWarning: The default behaviour of the CF writer will soon change to not compress data by default. warnings.warn("The default behaviour of the CF writer will soon change to not compress data by default.", /home/rdemaria/miniconda3/envs/g16_mod_env/lib/python3.8/site-packages/satpy/writers/cf_writer.py:197: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. if LooseVersion(pyproj.version) < LooseVersion('2.4.1'): [WARNING: 2022-08-10 21:07:48 : satpy.writers.cf_writer] No time dimension in datasets, skipping time bounds creation. [DEBUG: 2022-08-10 21:07:49 : satpy.readers.yaml_reader] Reading ('/local/home/rdemaria/miniconda3/envs/g16_mod_env/lib/python3.8/site-packages/satpy/etc/readers/satpy_cf_nc.yaml',) [DEBUG: 2022-08-10 21:07:49 : satpy.readers.yaml_reader] Assigning to satpy_cf_nc: ['exp/g16-abi-20220809061020-20220809061951.nc'] /home/rdemaria/miniconda3/envs/g16_mod_env/lib/python3.8/site-packages/satpy/readers/satpy_cf_nc.py:240: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use array.size > 0 to check that an array is not empty. if 'modifiers' in ds_info and not ds_info['modifiers']: Traceback (most recent call last): File "exp/test_load.py", line 25, in main() File "exp/test_load.py", line 21, in main scn.load(["C15"]) File "/home/rdemaria/miniconda3/envs/g16_mod_env/lib/python3.8/site-packages/satpy/scene.py", line 1346, in load self._update_dependency_tree(needed_datasets, query) File "/home/rdemaria/miniconda3/envs/g16_mod_env/lib/python3.8/site-packages/satpy/scene.py", line 1356, in _update_dependency_tree comps, mods = load_compositor_configs_for_sensors(self.sensor_names) File "/home/rdemaria/miniconda3/envs/g16_mod_env/lib/python3.8/site-packages/satpy/scene.py", line 171, in sensor_names reader_sensor_names = set([sensor for reader_instance in self._readers.values() File "/home/rdemaria/miniconda3/envs/g16_mod_env/lib/python3.8/site-packages/satpy/scene.py", line 172, in for sensor in reader_instance.sensor_names]) File "/home/rdemaria/miniconda3/envs/g16_mod_env/lib/python3.8/site-packages/satpy/readers/yaml_reader.py", line 383, in sensor_names sensor_names.update(fh.sensor_names) File "/home/rdemaria/miniconda3/envs/g16_mod_env/lib/python3.8/site-packages/satpy/readers/satpy_cf_nc.py", line 222, in sensor_names return {self.sensor} File "/home/rdemaria/miniconda3/envs/g16_mod_env/lib/python3.8/site-packages/satpy/readers/satpy_cf_nc.py", line 217, in sensor return nc.attrs['instrument'].replace('/', '-').lower() KeyError: 'instrument'

Environment Info:

  • OS: Linux
  • Satpy Version: 0.37.0
  • PyResample Version: 1.25.1
  • Readers and writers dependencies (when relevant): [run from satpy.utils import check_satpy; check_satpy()] satpy_cf_nc: ok cf: ok

Additional context I encountered this problem while attempting to write/restore some modified GOES scenes to/from disk.

robertdemariacira avatar Aug 10 '22 21:08 robertdemariacira