WIP: Feature/binary wheel

[iainrussell]

This Pull Request, when ready, will change the result of 'pip install eccodes' so that it will now include the ecCodes binary library automatically. This will be built for various versions of Python for Linux and MacOS (X64 and Arm64) via GitHub Actions.

By default, this bundled library will be the one used by the Python bindings. In order to use an externally-installed ecCodes binary library, set an environment variable (name of env var subject to discussion).

Current status and how to test

Example release on testpypi: https://test.pypi.org/project/eccodes/1.7.9/#files

The new wheels can be tested like this inside a virtualenv or similar:

pip install eccodes && pip uninstall eccodes # to install the dependencies from pip
pip install -i https://test.pypi.org/simple/ eccodes
# OR, to get a pure Python wheel:
pip install -i https://test.pypi.org/simple/ eccodes --no-binary eccodes

At runtime, set this environment variable to ignore the binary wheel and use external ecCodes binary libs:

ECCODES_PYTHON_USE_SEPARATE_BINARIES=1

Before import eccodes, set this environment variable to get a debug trace of how it is trying to find the binary libraries:

ECCODES_PYTHON_TRACE_LIB_SEARCH=1

[shahramn]

Re: The name of the environment variable We already have ECCODES_PYTHON_ENABLE_TYPE_CHECKS So I prefer to start new env vars with the prefix "ECCODES_PYTHON_"

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 0% with 61 lines in your changes missing coverage. Please review.

Please upload report for BASE (master@841f94f). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
gribapi/bindings.py	0.00%	43 Missing :warning:
setup.py	0.00%	16 Missing :warning:
gribapi/__init__.py	0.00%	2 Missing :warning:

Additional details and impacted files

@@            Coverage Diff            @@
##             master      #89   +/-   ##
=========================================
  Coverage          ?   32.90%           
=========================================
  Files             ?       16           
  Lines             ?     2547           
  Branches          ?      274           
=========================================
  Hits              ?      838           
  Misses            ?     1697           
  Partials          ?       12           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[iainrussell]

Current names of environment variables:

ECCODES_PYTHON_USE_SEPARATE_BINARIES=1 # ignore the binary wheel and use external ecCodes binary libs
ECCODES_PYTHON_TRACE_LIB_SEARCH=1 # to get debug trace of how it is trying to find the binary libraries

[jacopo-exact]

Can I ask the status of this PR? I'm working under contract in an EU project that would greatly benefit from this PR being merged.

[iainrussell]

Hi @jacopo-exact, it just needs a little clean-up (which I intend to do today), then we need to warn users of the upcoming change before we release it.

[jacopo-exact]

Not a great contribution but, as a sort of field-testing, we've been developing an application using the binary release on testpipy. It works spot-on, also in conjunction with other libraries such as cfgrib that rely on eccodes, and it is a great relief that we do not need to mess with compiling the library or load it from module or spack.

[iainrussell]

Hi @jacopo-exact, that's great feedback - thanks!