weather-tools
weather-tools copied to clipboard
google
Tools to make weather data accessible and useful.
Running `weather-mv` pipelines is difficult because the dependencies are hard to set up. A client-side Docker image for submitting the jobs can make life easier.
Currently, in order to include the MetView dependency on Dataflow, we require users to build their own custom container image and pass specific dataflow arguments in the `weather-mv` command. In...
This issue is used for tracking the missing/incorrect info in the run & build instructions of `weather-mv`'s md files: For less proficient user with GCP: - [ ] instructions for...
PyPi distribution seems to not understand `output_path` in regrid and the extra is not provided. The package requires `output_table` for `rg`, which shouldn't be the case (`output_path` is enough). ##...
The target disk space to use should be relative to the input weather data file size.
Running the following command in a python 3.8 environment created with `pip` from `main` branch fails: ```bash weather-mv rg --uris "gs://$BUCKET/download/*.nc" --output_path "gs://$BUCKET/download-rg-$(date +%s)/" --dry-run ``` error: ```raw [...] /venv38/lib/python3.8/site-packages/xarray/backends/cfgrib_.py:29:...
Local `weather-mv` runs for both `bq` and `rg` are failing with `AttributeError` exceptions, when following `weather_mv`'s README instructions. The same commands seem to be working in Dataflow (with the correct...
Let's distinguish between at least two types of errors in the `weather-dl` pipeline: The first are pipeline execution errors. For example, the network could go down, or a shard of...