Need to document PrePARE's checks on variable attributes

[taylor13]

I need to document in https://goo.gl/NmuENr all the PrePARE's checks on variable attributes. Could someone please review the coding starting at line 548 of https://github.com/PCMDI/cmor/blob/master/LibCV/PrePARE/PrePARE.py and tell me for CMIP6:

1. for which variable attributes is an error raised if a required attribute isn't in the file?
2. for which variable attributes is an error raised if the attribute value in the file is incorrect (inconsistent with the data specs)?

thanks.

[mauzey1]

@taylor13 Why is the goo.gl link going to a site called m.yaaya.video?

[taylor13]

my mistake. I've now corrected the link (above).

[mauzey1]

Here are the variables that need to be in the file and have values consistent with the data spec:

• dimensions
• valid_min
• standard_name
• out_name
• units
• valid_max
• frequency
• modeling_realm
• cell_methods
• cell_measures
• type
• positive
• ok_min_mean_abs
• ok_max_mean_abs
• flag_values
• flag_meanings

"long_name" and "comment" are ignored when PrePARE checks a file.

There seems to be an issue with how PrePARE reads the "cell_measures" attribute. I do not see any values for "cell_measures" that follow the pattern "<param>: <val1> OR <val2>". https://github.com/PCMDI/cmor/blob/d6feec00fb113e21d2c90042f4434a48fdbcd504/LibCV/PrePARE/PrePARE.py#L587 There are mostly values of "area: areacella" with some "area: areacello volume: volcello".

@dnadeau4 @doutriaux1 Should this be changed?

[taylor13]

I have reviewed the code with @dnadeau4 and find the following regarding the checking of variable attributes:

In cmor_variables_c:

lines 1153-1335 Using cmor_set_variable_attibute_internal, CMOR identifies the following as being variable attributes (nb. not all the entries under a variable in a CMOR table define attributes):

• standard_name
• flag_values (if defined)
• flag_meanings (if defined)
• units
• cell_methods
• cell_measures
• positive (if defined)
• missing_value
• _FillValue
• long_name (but this is skipped by PrePARE -- see below)
• comment (but this isn't found in CMIP6 cmor tables, so is skipped by PrePARE -- see below)

The following would also be identified as variable attributes if they were defined in the cmor table, but they never are for CMIP6:

• original_name
• history

In PrePARE.py

PrePARE cycles through attributes from CMOR table and

lines 560-564 PrePARE skips attributes:

• long_name
• comment

lines 568- for other table entries identified as variable attributes, PrePARE will issue an error if the file value doesn't match the cmor table value (but for some attributes -- I think just cell_methods and cell_measures -- there is some leniency):

• standard_name
• flag_values (if defined)
• flag_meanings (if defined)
• units
• cell_methods
• cell_measures
• positive (if defined)
• missing_value
• _FillValue

@dnadeau4 thought the coordinates attribute would also be checked by PrePARE (and I thought so too), but this doesn't seem to be the case; @mauzey1 ran a test where a file had a different coordinates dimension from what was specified in the CMOR table, and PrePARE didn't object.

No other checks are performed on variables.

[durack1]

@taylor13 this is great to have handy, do we document anywhere (in docs not here in this repo) that these checks are being made? Would the CMIP6 Global Attributes, DRS, Filenames, Directory Structure, and CV’s document be a good place to include a summary, along with a PrePARE/CMOR hash/checksum/release tag?

[taylor13]

This is probably an action item for me. see also https://github.com/PCMDI/cmor/issues/434

[durack1]

@taylor13 another very old issue - will close, please comment and reopen if required

[taylor13]

Would be good to document what the next version of PrePARE checks.

[durack1]

Same with this one - if it's useful to keep open, 4.0/Future milestone tag and reopen