Blosc
pip install broken in v1.8.3 on alpine linux
I've been using blosc for various projects for years now (thanks!). Recently we had CI start to fail because it could not build blosc on alpine linux during pip install blosc==1.8.1 with the error
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DTHREAD_STACK_SIZE=0x100000 -fPIC -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Ic-blosc/blosc -Ic-blosc/internal-complibs/zlib-1.2.8 -Ic-blosc/internal-complibs/zstd-1.3.8 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/lz4-1.8.3 -Ic-blosc/internal-complibs/lz4-1.8.3 -Ic-blosc/internal-complibs/zstd-1.3.8/common -Ic-blosc/internal-complibs/zstd-1.3.8 -I/usr/local/include/python3.7m -c c-blosc/blosc/shuffle.c -o build/temp.linux-x86_64-3.7/c-blosc/blosc/shuffle.o -DSHUFFLE_SSE2_ENABLED -msse2 -DSHUFFLE_AVX2_ENABLED -mavx2
c-blosc/blosc/shuffle.c:177:1: error: conflicting types for '_xgetbv'
177 | _xgetbv(uint32_t xcr) {
| ^~~~~~~
In file included from /usr/lib/gcc/x86_64-alpine-linux-musl/9.2.0/include/immintrin.h:43,
from c-blosc/blosc/blosc-common.h:71,
from c-blosc/blosc/shuffle.h:18,
from c-blosc/blosc/shuffle.c:10:
/usr/lib/gcc/x86_64-alpine-linux-musl/9.2.0/include/xsaveintrin.h:60:1: note: previous definition of '_xgetbv' was here
60 | _xgetbv (unsigned int __A)
| ^~~~~~~
Upgrading to the latest blosc version (1.8.3) appears to bring in a whole new build system that also doesn't work with a simple pip install blosc==1.8.3. Adding some additional dependencies, in a very specific order, appears to get the build to work, but it does add a lot of complexity. This works on alpine linux in python 3.7:
ENV CXXFLAGS="-D_BSD_SOURCE"
RUN apk add --no-cache --update linux-headers musl-dev libffi-dev openssl-dev python3-dev py3-pip gcc cmake ninja g++ && \
pip install scikit-build && \
pip install ninja && \
pip install blosc==1.8.3 && \
pip uninstall -y scikit-build ninja && \
apk del cmake ninja g++ && \
rm -rf /var/cache/apk/*
Are there any plans to simplify this, or is it just an artifact of upstream alpine changes that you can't deal with?
Also I'd like to note that the build time for blosc when doing pip install blosc==1.8.3 has gone up significantly. For example in CircleCI using their standard, default instance size it's taking ~18 minutes to pip install blosc! Just want to make sure that is expected in these latest updates.
commenting here because I'm having a similar issue building 1.8.3 in our CI setup.
Collecting blosc
Using cached blosc-1.8.3.tar.gz (809 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
cwd: None
Complete output (21 lines):
Collecting setuptools
Using cached setuptools-45.1.0-py3-none-any.whl (583 kB)
Collecting wheel
Using cached wheel-0.33.6-py2.py3-none-any.whl (21 kB)
Collecting scikit-build
Using cached scikit_build-0.10.0-py2.py3-none-any.whl (66 kB)
Collecting cmake
Using cached cmake-3.13.3-cp37-cp37m-manylinux1_x86_64.whl (15.9 MB)
Collecting ninja
Using cached ninja-1.9.0.post1.tar.gz (25 kB)
ERROR: Command errored out with exit status 1:
...
...
...
Complete output (5 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-lczudc8p/ninja/setup.py", line 7, in <module>
from skbuild import setup
ModuleNotFoundError: No module named 'skbuild'
Looks like the recent releases have some issues. As a temporary workaround I would suggest to pin to 1.8.1 which, as far as I can tell, is the last known version to work.
@rlizzo Since 1.8.3, python-blosc requires scikit-build (as specified in requirements-dev.txt). Can you try to install it and see if that works?
@dkleissa Can you send some hints on what part of the installation is taking more time? And which is the time that took 1.8.1?
