cdsapi icon indicating copy to clipboard operation
cdsapi copied to clipboard

ERA5 river discharge `zip` or `tar.gz` download are invalid

Open samburu opened this issue 6 years ago • 3 comments

Attempt to extract file downloaded by the script below i get An error occurred while loading the archive

import cdsapi

c = cdsapi.Client()

c.retrieve(
    'cems-glofas-historical',
    {
        'format':'zip',
        'variable':'River discharge',
        'dataset':'Consolidated reanalysis',
        'version':'2.1',
        'year':'2019',
        'month':'01',
        'day':'29'
    },
    'download.zip')

samburu avatar Nov 27 '19 14:11 samburu

Changing the output file extension from zip to nc works.

samburu avatar Nov 28 '19 08:11 samburu

@samburu Ist the nc file a valid zip? Which program could I use to access the file?

simpsus avatar Jun 02 '20 12:06 simpsus

@simpsus I use Xarray or netcdf

import cdsapi
import xarray as xr
import zipfile

c = cdsapi.Client()

c.retrieve(
    'cems-glofas-historical',
    {
        'format':'zip',
        'variable':'River discharge',
        'dataset':'Consolidated reanalysis',
        'version':'2.1',
        'year':'2019',
        'month':'01',
        'day':'29'
    },
    'download.zip')

zip_ref = zipfile.ZipFile('download.zip')
zip_ref.extractall()
ds = xr.open_dataset('CEMS_ECMWF_dis24_20190129_glofas_v2.1.nc')

samburu avatar Jun 02 '20 13:06 samburu

Hi @samburu , This was an error in the sample code provided on the download page. For GLOFAS datasets, when the format requested is netCDF it is actaully a zip folder containing netCDF files. This means your request should have 'format': 'netcdf' and the target download file set to 'download.zip'. That said, we improved some things in the configuration and the sample code on the download pages such that users should no longer have this issue.

Additionally, my preferred method of downloading data is to use the .download() method which uses the filename on the CDS, thus ensures the correct extension. That is:

c.retrieve(
    'cems-glofas-historical',
    {
        'format':'netcdf4.zip',
        'variable':'River discharge',
        'dataset':'Consolidated reanalysis',
        'version':'2.1',
        'year':'2019',
        'month':'01',
        'day':'29'
    }
).download()

EddyCMWF avatar Feb 28 '23 09:02 EddyCMWF

Also to add, with issues and assistance regarding the data you have downloaded you will generally get a much more prompt response from the C3S user support channels: https://cds.climate.copernicus.eu/cdsapp#!/usersupport

EddyCMWF avatar Feb 28 '23 09:02 EddyCMWF