meteostat-python icon indicating copy to clipboard operation
meteostat-python copied to clipboard

Data from ISD datasource no longer included?

Open bram-tv opened this issue 2 years ago • 7 comments

(This might not be the best repository to report the issue but I couldn't find a better one..)

It appears that data from Global dataset (ISD) is no longer included since 12 Augustus 2023?

Looking at the Belgium weather stations:

  • On 11 Augustus 2023: 25 stations that had data from ISD
  • On 13 Augustus 2023: 0 stations that had data from ISD

Looking at one station in detail: station 06447 (Uccle, BE): and looking at the source map file of the hourly data:

2023-08-11,00,B,B,B,E,,B,B,,B,,E
...
2023-08-12,00,B,B,B,E,,B,B,,B,,E
...
2023-08-13,00,E,E,E,E,,E,E,,E,,E
...
2023-08-14,00,E,E,E,E,,E,E,,E,,E
...
...
2023-10-11,00,E,E,E,E,,E,E,,E,,E

From 13 Augustus only modeled data is available and the actual measurements are missing..

The ISD database does appear to have data for that station.

bram-tv avatar Oct 11 '23 08:10 bram-tv

You're correct @bram-tv. Currently, no data is coming in from ISD. I just learned that the ISD FTP server was shut down. I will fix this issue ASAP. Just need to migrate to the HTTP endpoints.

clampr avatar Oct 16 '23 12:10 clampr

ISD data should be coming in again.

clampr avatar Oct 26 '23 10:10 clampr

ISD data should be coming in again.

This does not appear to be the case? Again doing a check on Belgium weather stations and looking for the latest timestamp with a "B" measurement:

06400.map.csv : 2023-10-13,21,B,B,B,E,,B,B,,B,,E
06407.map.csv : 2023-10-13,21,B,B,B,E,,B,B,E,B,,E
06414.map.csv : 2023-10-13,21,B,B,B,E,,B,B,,B,,E
06418.map.csv : 2023-10-13,21,B,B,B,E,,B,B,,B,,E
06428.map.csv : 2023-10-13,21,B,B,B,E,,B,B,,B,,E
06431.map.csv :
06432.map.csv : 2023-10-13,21,B,B,B,E,,B,B,,B,,E
06434.map.csv : 2023-10-13,21,B,B,B,E,,B,B,,B,,E
06447.map.csv : 2023-10-13,21,B,B,B,E,,B,B,,B,,E
06449.map.csv : 2023-10-13,21,B,B,B,E,,B,B,E,B,,E
06450.map.csv : 2023-10-13,21,B,B,B,E,,B,B,E,B,,E
06451.map.csv : 2023-10-13,21,B,B,B,E,,B,B,E,B,,E
06455.map.csv : 2023-10-13,21,B,B,B,E,,B,B,,B,,E
06456.map.csv : 2023-10-13,21,B,B,B,E,,B,B,E,B,,E
06458.map.csv : 2023-10-13,21,B,B,B,E,,B,B,E,B,,E
06464.map.csv : 2023-10-13,21,B,B,B,E,,B,B,,B,,E
06465.map.csv : 2023-10-13,21,B,B,B,E,,B,B,,B,,E
06472.map.csv : 2023-10-13,21,B,B,B,E,,B,B,,B,,E
06476.map.csv : 2023-10-13,21,B,B,B,E,,B,B,E,B,,E
06477.map.csv : 2023-10-13,18,B,B,B,E,,B,B,,B,,E
06478.map.csv : 2023-10-13,21,B,B,B,E,,B,B,E,B,,E
06479.map.csv : 2023-10-13,21,B,B,B,E,,B,B,E,B,,E
06484.map.csv : 2023-10-13,21,B,B,B,E,,B,B,,B,,E
06490.map.csv : 2023-10-13,21,B,B,B,E,,B,B,E,B,,E
06494.map.csv : 2023-10-13,21,B,B,B,E,,B,B,,B,,E
06496.map.csv : 2023-10-13,21,B,B,B,E,,B,B,E,B,,E
EBSH0.map.csv : 2023-09-23,21,B,B,B,E,,E,B,,E,,E

The most recent timestamp is 13th October..

bram-tv avatar Oct 28 '23 10:10 bram-tv

Can you confirm you've purged your cache? For example, 06400 seems to have data coverage up until the day before yesterday: https://meteostat.net/en/station/06400?t=2023-10-28/2023-10-28

clampr avatar Oct 30 '23 14:10 clampr

Can you confirm you've purged your cache? For example, 06400 seems to have data coverage up until the day before yesterday: https://meteostat.net/en/station/06400?t=2023-10-28/2023-10-28

Yes. There is no caching involved. I downloaded the files directly from https://bulk.meteostat.net/ If you look at the map file of 06400 then you can see that the recent data is coming from data source 'D' which is the METAR source and not the ISD source.

It might be better to take a look at the 06447 station since that is only receiving data via ISD and not via METAR.

bram-tv avatar Oct 30 '23 14:10 bram-tv

My bad. You're right @bram-tv. Seems like there was an issue with the job. Need to wait to confirm it's working again. I'll let you know when it's resolved.

clampr avatar Nov 06 '23 09:11 clampr

Any update on this?

bram-tv avatar Dec 19 '23 09:12 bram-tv