cis icon indicating copy to clipboard operation
cis copied to clipboard

Data collocated even when no data for variable is not available

Open tommibergman opened this issue 6 years ago • 7 comments

With AERONET data there some stations that do not have any data for a given variable and year (there are other variables but not this one). Now I have an automatic script which collocates model data to AERONET. With earlier versions (at least 1.4) collocating to empty data would result in empty collocated data as well. But with v1.7.1 I am getting values for collocated data even when AERONET has no data. Of course I can do a work around, but I feel this is a reduction in usability.

tommibergman avatar Jul 04 '19 07:07 tommibergman

Could you provide a minimal example of the collocation you're running? I haven't encountered this problem but have a suspicion what it might be.

adamcpovey avatar Jul 04 '19 08:07 adamcpovey

This can be controlled using the 'missing_data_for_missing_sample' keyword argument, are you using the Python or command line interface?

duncanwp avatar Jul 04 '19 08:07 duncanwp

This can be controlled using the 'missing_data_for_missing_sample' keyword argument, are you using the Python or command line interface?

I have some older scripts which still use command line interface. Does this mean that I would get more control if I would use the Python?

tommibergman avatar Jul 04 '19 09:07 tommibergman

Yes, I tend to only use the Python interface now, it's not as well documented yet but it gives you much more flexibility. See this example notebook to get you going :-)

duncanwp avatar Jul 04 '19 09:07 duncanwp

Could you provide a minimal example of the collocation you're running? I haven't encountered this problem but have a suspicion what it might be.

I made a small test set with one month in my google drive https://drive.google.com/open?id=1sQftoLxgBbRkrpn0-x6DbaIGlOfyFccV .

running test from there will produce a subset of banizombou station and the colocate my data on it. And the collocated data has data while the subset does not.

tommibergman avatar Jul 04 '19 09:07 tommibergman

I was also wondering why do they get different missing value? AERONET has -999 and model data NaN.

tommibergman avatar Jul 04 '19 09:07 tommibergman

I was also wondering why do they get different missing value? AERONET has -999 and model data NaN.

Sorry, my mistake. This is just a difference between an old and a new version.

tommibergman avatar Jul 04 '19 12:07 tommibergman