kivy
SciPy recipe fails with current development branch of P4A
WSL2 on Windows 11 Ubuntu 22.04.5 LTS Buildozer 1.5.1.dev0 $LEGACY_NDK: android-ndk-r21e with fortran addons
Relevant configuration lines for buildozer.spec:
requirements = python3,kivy,scipy
android.minapi = 24
android.archs = arm64-v8a
p4a.branch = develop
log_level = 2
Minimal Kivy test:
import kivy
import scipy
from kivy.app import App
from kivy.uix.button import Button
def callback(instance):
scipy.show_config()
class MyApp(App):
def build(self):
btn1 = Button(text='SciPy show config', size=(100, 50))
btn1.bind(on_press=callback)
return btn1
if __name__ == '__main__':
MyApp().run()
Issue:
buildozer -v android debug finished with the error: 'int_t' is not a type identifier
[INFO]: Building scipy for arm64-v8a
[INFO]: scipy apparently isn't already in site-packages
[DEBUG]: -> running pip install numpy --target /home/mrisco/kivy/test_scipy/.buildozer/android/platform/build-arm64-v8a/build/other_builds/hostpython3/desktop/hostpython3/native-build/Lib/site-packages --python-version 3.11.5 --only-binary=:all: --upgrade
[DEBUG]: Collecting numpy
[DEBUG]: Using cached numpy-2.2.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (62 kB)
[DEBUG]: Using cached numpy-2.2.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.4 MB)
[DEBUG]: Installing collected packages: numpy
[DEBUG]: Successfully installed numpy-2.2.3
[INFO]: Building compiled components in scipy
[INFO]: -> directory context /home/mrisco/kivy/test_scipy/.buildozer/android/platform/build-arm64-v8a/build/other_builds/scipy/arm64-v8a__ndk_target_24/scipy
[DEBUG]: -> running python3 setup.py build_ext -v -j 12
[DEBUG]: /home/mrisco/kivy/test_scipy/.buildozer/android/platform/build-arm64-v8a/build/other_builds/scipy/arm64-v8a__ndk_target_24/scipy/setup.py:136: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
[DEBUG]: from distutils.command.sdist import sdist
[DEBUG]: /home/mrisco/kivy/test_scipy/.buildozer/android/platform/build-arm64-v8a/build/other_builds/hostpython3/desktop/hostpython3/native-build/Lib/site-packages/_distutils_hack/__init__.py:11: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
[DEBUG]: warnings.warn(
[DEBUG]: /home/mrisco/kivy/test_scipy/.buildozer/android/platform/build-arm64-v8a/build/other_builds/hostpython3/desktop/hostpython3/native-build/Lib/site-packages/_distutils_hack/__init__.py:26: UserWarning: Setuptools is replacing distutils.
[DEBUG]: warnings.warn("Setuptools is replacing distutils.")
[DEBUG]: /home/mrisco/kivy/test_scipy/.buildozer/android/platform/build-arm64-v8a/build/other_builds/scipy/arm64-v8a__ndk_target_24/scipy/setup.py:519: DeprecationWarning:
[DEBUG]:
[DEBUG]: `numpy.distutils` is deprecated since NumPy 1.23.0, as a result
[DEBUG]: of the deprecation of `distutils` itself. It will be removed for
[DEBUG]: Python >= 3.12. For older Python versions it will remain present.
[DEBUG]: It is recommended to use `setuptools < 60.0` for those Python versions.
[DEBUG]: For more details, see:
[DEBUG]: https://numpy.org/devdocs/reference/distutils_status_migration.html
[DEBUG]:
[DEBUG]:
[DEBUG]: from numpy.distutils.core import setup
[DEBUG]: Cythonizing sources
[DEBUG]: Running scipy/stats/_generate_pyx.py
[DEBUG]: Running scipy/linalg/_generate_pyx.py
[DEBUG]: Running scipy/special/_generate_pyx.py
[DEBUG]: Processing scipy/ndimage/src/_cytest.pyx
[DEBUG]: Processing scipy/ndimage/src/_ni_label.pyx
[DEBUG]: Processing scipy/stats/_stats.pyx
[DEBUG]: Processing scipy/stats/_biasedurn.pyx
[DEBUG]: Processing scipy/stats/_qmc_cy.pyx
[DEBUG]: Processing scipy/stats/_sobol.pyx
[DEBUG]: Processing scipy/stats/_unuran/unuran_wrapper.pyx
[DEBUG]: Processing scipy/stats/_levy_stable/levyst.pyx
[DEBUG]: Processing scipy/stats/_boost/src/skewnorm_ufunc.pyx
[DEBUG]: Processing scipy/stats/_boost/src/beta_ufunc.pyx
[DEBUG]: Processing scipy/stats/_boost/src/binom_ufunc.pyx
[DEBUG]: Processing scipy/stats/_boost/src/invgauss_ufunc.pyx
[DEBUG]: Processing scipy/stats/_boost/src/ncx2_ufunc.pyx
[DEBUG]: Processing scipy/stats/_boost/src/hypergeom_ufunc.pyx
[DEBUG]: Processing scipy/stats/_boost/src/nbinom_ufunc.pyx
[DEBUG]: Processing scipy/stats/_boost/src/ncf_ufunc.pyx
[DEBUG]: Processing scipy/stats/_boost/src/nct_ufunc.pyx
[DEBUG]: Processing scipy/stats/_rcont/rcont.pyx
[DEBUG]: Processing scipy/sparse/_csparsetools.pyx.in
[DEBUG]:
[DEBUG]: Error compiling Cython file:
[DEBUG]: ------------------------------------------------------------
[DEBUG]: ...
[DEBUG]: needs_self_labeling = True
[DEBUG]:
[DEBUG]: # Take neighbor labels
[DEBUG]: PyArray_ITER_RESET(itstruct)
[DEBUG]: for ni in range(num_neighbors):
[DEBUG]: neighbor_use_prev = (<np.int_t *> PyArray_ITER_DATA(itstruct))[0]
[DEBUG]: ^
[DEBUG]: ------------------------------------------------------------
[DEBUG]:
[DEBUG]: _ni_label.pyx:329:42: 'int_t' is not a type identifier
Current recipe uses SciPy maintenance/1.11.3: https://github.com/kivy/python-for-android/blob/9b0eed1502ed2b8c8dceb404091e151ba87a2092/pythonforandroid/recipes/scipy/init.py#L18-L20 which uses a deprecated Numpy type np.int_t: https://github.com/scipy/scipy/blob/1d3a067c2ccd0a6efddeb3194163aa9a3879d26e/scipy/ndimage/src/_ni_label.pyx#L328-L331
for ni in range(num_neighbors):
neighbor_use_prev = (<np.int_t *> PyArray_ITER_DATA(itstruct))[0]
neighbor_use_adjacent = (<np.int_t *> (<char *> PyArray_ITER_DATA(itstruct) + ss))[0]
neighbor_use_next = (<np.int_t *> (<char *> PyArray_ITER_DATA(itstruct) + 2 * ss))[0]
That was fixed in the SciPy version 1.12.0: https://github.com/scipy/scipy/blob/4edfcaa3ce8a387450b6efce968572def71be089/scipy/ndimage/src/_ni_label.pyx#L328-L331
for ni in range(num_neighbors):
neighbor_use_prev = (<np.npy_bool *> PyArray_ITER_DATA(itstruct))[0]
neighbor_use_adjacent = (<np.npy_bool *> (<char *> PyArray_ITER_DATA(itstruct) + ss))[0]
neighbor_use_next = (<np.npy_bool *> (<char *> PyArray_ITER_DATA(itstruct) + 2 * ss))[0]
But the 1.12.0 release doesn't have the file /tools/cythonize.py
So I tried again with an old version of SciPy, I saw the PR https://github.com/scipy/scipy/pull/19320 about fixing the Cython version in 1.11.3 and releasing a final 1.11.4 version before the 1.12.0 release, using that version in the recipe also gave the original error.
Is there anything that I can do to fix this recipe? The same error is produced with current development branch.
I took a stab at it last summer and it was highly painful, hopefully things got better with recent releases, I don't know. But basically you can start writing a recipe, try to build, fix the build errors by updating the recipe, rebuild...iterate until you have them all fixed. Then do the same for the runtime errors. You could probably start with:
from pythonforandroid.recipe import MesonRecipe
class ScipyRecipe(MesonRecipe):
...
Then the workflow I'm using to rebuild is:
# clean the build between rebuilds (not sure it's currently the best approach
p4a clean_recipe_build scipy && p4a clean_dists
# build using the testapps
cd testapps/on_device_unit_tests/
python setup.py apk \
--sdk-dir $ANDROID_SDK_HOME \
--ndk-dir $ANDROID_NDK_HOME \
--arch=x86_64 \
--requirements python3,scipy
In fact I often build from the repository docker image, this is the way I like the best, but you can also build from your host. In case you want to give it a try from the container, this is how:
# build the image (or use the already published one)
docker build --tag=kivy/python-for-android .
# Docker run (note the volume mount so you don't have to rebuild the image after updating the recipe):
docker run -it \
--volume $(pwd)/pythonforandroid:/home/user/app/pythonforandroid \
--env ANDROID_NDK_HOME_LEGACY=/home/user/.android/android-ndk-legacy \
--env ANDROID_SDK_HOME=/home/user/.android/android-sdk \
--env ANDROID_NDK_HOME=/home/user/.android/android-ndk \
--rm kivy/python-for-android bash
# then from within the container, same thing as above
. venv/bin/activate
cd testapps/on_device_unit_tests/
python setup.py apk --debug \
--sdk-dir $ANDROID_SDK_HOME \
--ndk-dir $ANDROID_NDK_HOME \
--requirements python3,scipy \
--arch=armeabi-v7a # or whatever arch you like
Good luck! 🏃
Ok I went through all the pain, here you go: https://github.com/T-Dynamos/python-for-android/tree/scipy_update
You need to set ndk version to r27c (in spec) and ndk api to 24 (minapi).
Only works for 64 bit arches.
Here is my test:
Apk size was 53MB
Test code:
import numpy as np
from scipy.integrate import quad
def f(x):
return np.sin(x)
result, error = quad(f, 0, np.pi)
print(f"Result of the integral: {result}")
print(f"Estimated error: {error}")
Will open PR soon when will get more time (needs cleaning).
~~But unfortunately, updating ndk to 27c doesn't allows SDL to build.~~
~~I had to pull trick for that:~~
~~1. build with only python3 and kivy (in requirements) with ndk 25b~~
~~2. change ndk to 27c with scipy and numpy in requirements~~
~~then it should work.~~
Edit: NVM fixed in latest commit.
@T-Dynamos
Edit: NVM fixed in latest commit.
I'm getting this error with your fix:
[INFO]: Downloading sdl2
[INFO]: -> directory context /home/mrisco/kivy/test_scipy/.buildozer/android/platform/build-x86_64/packages/sdl2
[DEBUG]: -> running basename https://github.com/libsdl-org/SDL/releases/download/release-2.30.11/SDL2-2.30.11.tar.gz
[DEBUG]: SDL2-2.30.11.tar.gz
[DEBUG]: * Generated md5sum: bea190b480f6df249db29eb3bacfe41e
[DEBUG]: * Expected md5sum: a344eb827a03045c9b399e99af4af13d
...
ValueError: Generated md5sum does not match expected md5sum for sdl2 recipe
By changing the line 11 of pythonforandroid/recipes/sdl2/init.py to
md5sum = 'bea190b480f6df249db29eb3bacfe41e'
fix that.
But I got a new error:
[INFO]: Prebuilding scipy for x86_64
[INFO]: scipy has no prebuild_x86_64, skipping
[INFO]: Applying patches for scipy[x86_64]
[INFO]: Applying patch meson.patch
[DEBUG]: -> running patch -t -d /home/mrisco/kivy/test_scipy/.buildozer/android/platform/build-x86_64/build/other_builds/scipy/x86_64__ndk_target_24/scipy -p1 -i /home/mrisco/python/python-for-android/pythonforandroid/recipes/scipy/meson.patch
[DEBUG]: (Stripping trailing CRs from patch; use --binary to disable.)
[DEBUG]: can't find file to patch at input line 5
[DEBUG]: Perhaps you used the wrong -p or --strip option?
[DEBUG]: The text leading up to this was:
[DEBUG]: --------------------------
[DEBUG]: |Binary files scipy.git/.git/index and scipy.git.patch/.git/index differ
[DEBUG]: |diff '--color=auto' -uNr scipy.git/.git/logs/refs/remotes/origin/main scipy.git.patch/.git/logs/refs/remotes/origin/main
[DEBUG]: |--- scipy.git/.git/logs/refs/remotes/origin/main 2025-03-27 02:55:14.521123150 +0530
[DEBUG]: |+++ scipy.git.patch/.git/logs/refs/remotes/origin/main 2025-03-27 11:24:34.225186085 +0530
[DEBUG]: --------------------------
[DEBUG]: No file to patch. Skipping patch.
That can be fixed by removing the references to .git folder in the meson.patch file:
diff '--color=auto' -uNr scipy.git/meson.options scipy.git.patch/meson.options
--- scipy.git/meson.options 2025-03-27 02:55:14.586853766 +0530
+++ scipy.git.patch/meson.options 2025-03-27 02:07:29.736674085 +0530
@@ -2,6 +2,8 @@
description: 'option for BLAS library switching')
option('lapack', type: 'string', value: 'openblas',
description: 'option for LAPACK library switching')
+option('openblas_incldir', type: 'string', value: '', description: 'OpenBLAS include directory')
+option('openblas_libdir', type: 'string', value: '', description: 'OpenBLAS library directory')
option('use-g77-abi', type: 'boolean', value: false,
description: 'If set to true, forces using g77 compatibility wrappers ' +
'for LAPACK functions. The default is to use gfortran ' +
diff '--color=auto' -uNr scipy.git/scipy/meson.build scipy.git.patch/scipy/meson.build
--- scipy.git/scipy/meson.build 2025-03-27 02:55:14.632428649 +0530
+++ scipy.git.patch/scipy/meson.build 2025-03-27 11:25:33.756445056 +0530
@@ -268,10 +268,18 @@
endif
endif
+openblas_inc = get_option('openblas_incldir')
+openblas_lib = get_option('openblas_libdir')
+
+openblas_dep = declare_dependency(
+ include_directories: include_directories(openblas_inc),
+ link_args: ['-L' + openblas_lib, '-lopenblas']
+)
+
# pkg-config uses a lower-case name while CMake uses a capitalized name, so try
# that too to make the fallback detection with CMake work
if blas_name == 'openblas'
- blas = dependency(['openblas', 'OpenBLAS'])
+ blas = openblas_dep
elif blas_name != 'scipy-openblas' # if so, we found it already
blas = dependency(blas_name)
endif
@@ -295,7 +303,7 @@
# use that - no need to run the full detection twice.
lapack = blas
elif lapack_name == 'openblas'
- lapack = dependency(['openblas', 'OpenBLAS'])
+ lapack = openblas_dep
else
lapack = dependency(lapack_name)
endif