Wheels for WASM
I did some attempt at creating wheels for WASM, and had some (mixed) experiences with Pyodide and scikit-build-core:
-
Pyodide needs version 0.23.4 (latest 0.24.1 does not work: https://github.com/pyodide/pyodide/issues/4287)
-
CMake config in python-blosc2 needs a bit of modification in order to run (see attached patch).
-
The included zlib-ng codec has issues with WASM compiler, so it has been removed from the list of codecs here.
-
Some packages (notably py-cpuinfo) are not available in Pyodide repos.
-
Some tests are passing, but others do not. E.g.:
$ python -m pytest tests/test_compress2.py
=============================================================== test session starts ================================================================
platform emscripten -- Python 3.11.2, pytest-7.4.4, pluggy-1.3.0
rootdir: /home/faltet/blosc/python-blosc2
configfile: pytest.ini
collected 24 items
tests/test_compress2.py ..............FF........ [100%]
===================================================================== FAILURES =====================================================================
__________________________________________________ test_compress2[641091-cparams1-dparams1-True] ___________________________________________________
nbytes = 641091, cparams = {'typesize': 1}, dparams = {'nthreads': 4}, gil = True
@pytest.mark.parametrize("gil", [True, False])
@pytest.mark.parametrize(
"nbytes, cparams, dparams",
[
(7, {"codec": blosc2.Codec.LZ4, "clevel": 6, "typesize": 1}, {}),
(641091, {"typesize": 1}, {"nthreads": 4}),
(136, {"typesize": 1}, {}),
(1231, {"typesize": 4}, blosc2.dparams_dflts),
],
)
def test_compress2(nbytes, cparams, dparams, gil):
blosc2.set_releasegil(gil)
bytes_obj = b" " * nbytes
c = blosc2.compress2(bytes_obj, **cparams)
dest = bytearray(bytes_obj)
blosc2.decompress2(c, dst=dest, **dparams)
assert dest == bytes_obj
dest2 = blosc2.decompress2(c, **dparams)
> assert dest2 == bytes_obj
E AssertionError: assert b'\x00\x00\x0...0\x00\x00\x00' == b' ... '
E At index 0 diff: b'\x00' != b' '
E Use -v to get more diff
tests/test_compress2.py:91: AssertionError
__________________________________________________ test_compress2[641091-cparams1-dparams1-False] __________________________________________________
nbytes = 641091, cparams = {'typesize': 1}, dparams = {'nthreads': 4}, gil = False
@pytest.mark.parametrize("gil", [True, False])
@pytest.mark.parametrize(
"nbytes, cparams, dparams",
[
(7, {"codec": blosc2.Codec.LZ4, "clevel": 6, "typesize": 1}, {}),
(641091, {"typesize": 1}, {"nthreads": 4}),
(136, {"typesize": 1}, {}),
(1231, {"typesize": 4}, blosc2.dparams_dflts),
],
)
def test_compress2(nbytes, cparams, dparams, gil):
blosc2.set_releasegil(gil)
bytes_obj = b" " * nbytes
c = blosc2.compress2(bytes_obj, **cparams)
dest = bytearray(bytes_obj)
blosc2.decompress2(c, dst=dest, **dparams)
assert dest == bytes_obj
dest2 = blosc2.decompress2(c, **dparams)
> assert dest2 == bytes_obj
E AssertionError: assert b'x00\\x00\\x...x00\\x00\\x00' == b' ... '
E At index 0 diff: b'x' != b' '
E Use -v to get more diff
tests/test_compress2.py:91: AssertionError
============================================================= short test summary info ==============================================================
FAILED tests/test_compress2.py::test_compress2[641091-cparams1-dparams1-True] - AssertionError: assert b'\x00\x00\x0...0\x00\x00\x00' == b' ... '
FAILED tests/test_compress2.py::test_compress2[641091-cparams1-dparams1-False] - AssertionError: assert b'x00\\x00\\x...x00\\x00\\x00' == b' ... '
=========================================================== 2 failed, 22 passed in 0.24s ===========================================================
I don't think fixing these should be terribly complicated, but I don't have more time now. I am attaching my modifications here in case someone else would want to continue this effort.