Satip

Satip copied to clipboard

[pre-commit.ci] pre-commit autoupdate

1

updates: - [github.com/pre-commit/pre-commit-hooks: v4.3.0 → v4.4.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.3.0...v4.4.0) - [github.com/PyCQA/flake8: 4.0.1 → 6.0.0](https://github.com/PyCQA/flake8/compare/4.0.1...6.0.0) - [github.com/psf/black: 22.6.0 → 22.12.0](https://github.com/psf/black/compare/22.6.0...22.12.0) - [github.com/pre-commit/mirrors-prettier: v2.7.1 → v3.0.0-alpha.4](https://github.com/pre-commit/mirrors-prettier/compare/v2.7.1...v3.0.0-alpha.4)

pre-commit-ci[bot]

Add diagram of how the app works

2

## Detailed Description Would be good to add a flow diagram of how the app is working, what data it is saving and where ## Context easy to see what...

peterdudfield

ENV PYTHON_VERSION=3.10

1

# Pull Request ## Description update to docker 3.10 ## How Has This Been Tested? normal unittest - [ ] Yes ## Checklist: - [ ] My code follows [OCF's...

peterdudfield

## Detailed Description Fix linting erros - see https://github.com/openclimatefix/Satip/actions/runs/3563545670 ## Context Good to keep linting clean ## Possible Implementation - run ` flake8 --max-line-length 100 --extend-ignore=E203 --per-file-ignores="__init__.py:F401" satip` - isort...

peterdudfield

## Detailed Description https://eumetsatspace.atlassian.net/wiki/spaces/EUM/blog/2022/10/24/1908965377/Data+Store+APIs+1.0.0+released ## Context Satip needs to be up to date to keep working with the API. And there might be some changes to how to delete customizations...

jacobbieker

Add pipeline for pulling Indian Ocean Satellite Imagery

1

## Detailed Description Use stuff built for #133 to also start collecting Indian Ocean 15 minutely data. Would be really great if we could try pushing it onto HF or...

jacobbieker

Add automated pipeline for pulling, processing, and saving to HF/GCP/Wherever

1

## Detailed Description Instead of relying on someone to manually run the data pipeline to get more satellite data, we should try automating it, so the dataset just grows on...

jacobbieker

Updates to the way satip uses JPEG XL compression?

6

# Motivation & aims * Make the satellite Zarr easy to sample from on-the-fly during ML training (without having to pre-prepare batches or pre-load a subset of the training data...

JackKelly

Experimenting with chunk size in new Zarr

1

We are switching to a newer, compressed Zarr storage for the satellite data. Currently, the data is collected as individual files for each timestep, and so we are working on...

jacobbieker

Monthly Zarr processing script

1

# Pull Request ## Description _Please delete the italicised instruction text! Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context....

jacobbieker