[JOSS-Review] Add instructions for running unit tests in /tests

[mikey247]

https://github.com/openjournals/joss-reviews/issues/9093

Hi @jonnor,

For the checklist’s "Automated tests" item, could you document how to run the unit tests in /tests ? if it already exists, please point me to it.

Please include:

• Prerequisites: runtime/version, dependencies, environment variables

• Setup: install commands

• Run commands: all tests and a single test

• Expected output: what success/failure looks like

Suggest adding this to README or a TESTING.md and linking it from the main README.

Thanks.

[jonnor]

Hi @mikey247 and thanks for the feedback! There are some instructions on how to run the tests on https://github.com/emlearn/emlearn-micropython/blob/master/docs/CONTRIBUTING.md - which is linked at the bottom of the README. Perhaps I should move it up before the "citation" section for better visibility.

It is missing how to run a single test and description of expected output, I will add that. Otherwise, I think it should mostly be there?

[mikey247]

I ran into this issue when I ran the make check command. @jonnor I've got micropython installed and followe the instructions in https://github.com/emlearn/emlearn-micropython/blob/master/docs/developing.md

[jonnor]

Do you have a checkout of MicroPython at Desktop/micropython ? Per this comment in the docs:

We assume that micropython git repository available. It is assumed to be at the same level as this repository in the file system. If using another location, adjust MPY_DIR accordingly. You should be using MicroPython 1.25 (or newer).

I will try to make that as well a bit more clear

[mikey247]

hi @jonnor

I followed the instructions in https://github.com/micropython/micropython/blob/master/ports/unix/README.md to build Micropython and ran into this error running he make command. Note the warnings on the make submodules command; I'm not sure what to make of it.

[mikey247]

I cloned the micropython repo in /Desktop and ran through the build steps in https://github.com/micropython/micropython/blob/master/ports/unix/README.md up until the make command, where I get this error.

[jonnor]

@mikey247 you will need the libffi headers and pkg-config.

To build the default "standard" variant and configuration, then you will also need:

pkg-config tool libffi library and headers

[mikey247]

I did this, I think, but with Brew (on Mac)

[jonnor]

Hmm. The first screenshot you shared had "pkg-config: file not found" errors, as output of the make submodules. Did you install it after that? Missing pkg-config and/or libffi would fit with the error that you get. But I do see an include path that has ffi in it in the failing build... But is from Mac OS / XCode though, not from homebrew - so maybe that is the problem. You could try setting PKG_CONFIG_PATH to where homebrew has put the libffi .pc files? I unfortunately do not have a Mac for testing.

But, you seem to have installed micropython executable from homebrew at the lastest version. It should be possible to use that in combination with having a MicroPython source checkout (for the dynruntime.mk file that your first error got).

[mikey247]

Yes, it was AFTER the first screenshot @jonnor

I think I've been able to fully setup Micropython, some test fail/skp but at last the libffi error is gone.

**Now back to emlearn, I now get this error. when I run make check command from here

• "/Users/michaelumeokoli/Desktop/micropython/py/mpconfig.h:195:18: error: redefinition of typedef 'mp_int_t' is a C11 feature [-Werror,-Wtypedef-redefinition]"*

[jonnor]

Hmm. Thanks for the details. This looks like something that might affect all kinds of dynamic module building for MicroPython. This feature is unfortunately not that widely used in MicroPython upstream yet, so it could be that it has problems on Mac OS.

Can you give the output of gcc -v? I seem to recall that this can be aliased to clang on Mac OS - so maybe we are seeing a compiler difference wrt this being an error.

In this case the typedefs are actually equivalent, so this warning being an error is the opposite of helpful... I think we may need a workaround for that, to disable these warnings/errors.

[jonnor]

We actually do have a CI build for Mac OS. However, the make check (using dynamic values) is commented out. Because there is actually no support to build these on Mac OS in upstream MicroPython: https://github.com/micropython/micropython/issues/5500

So, at the moment we can only expect "make check_unix" to work - which instead builds the modules as "external C modules" which are included into the firmware (micropython executable on PC), instead of "dynamic native modules" which are deployed as loadable .mpy files. Will update documentation to reflect this.

[jonnor]

Hi @mikey247 I have now overhauled the developing.md - and mentioned there the test running command which should work on Mac OS (we run it in CI). Hopefully that is sufficient for you and others on Mac OS to be able to reproduce it :)

[mikey247]

Hey @jonnor , I'm sorry, but I'm still having problems with the new updates when I run the make check_unix command.

[jonnor]

Hi @mikey247 What is the difference between the two runs? Are you running that in a Python environment where you have the dependencies installed (where pip install -r requirements.txt was ran previously)?

[mikey247]

pulled the updates and ran all the commands from scratch @jonnor

[jonnor]

Hmm, in that case something must have failed in one of the earlier steps, that prevented the emlearn package from being installed. Can you please provide the complete log of all the steps?

We have a build-macos CI job which tests the same process for each commit. Ref https://github.com/emlearn/emlearn-micropython/blob/master/.github/workflows/build.yaml#L179

[mikey247]

@jonnor I just followed the steps in https://github.com/emlearn/emlearn-micropython/blob/master/docs/developing.md, except for a new clone.

[jonnor]

@mikey247 that is good. But something is failing, and I cannot debug or fix the failure if I do not see the output of the commands.

[mikey247]

Hey @jonnor , I'm sorry, but I'm still having problems with the new updates when I run the make check_unix command.

Well, I put a screenshot here @jonnor , is there something else you want?

[jonnor]

@mikey247 Yes, the output of all the previous steps in the documentation. Including activating the Python environment (if you use any), installing the dependencies, etc.

[mikey247]

@jonnor, I deleted the old repo and started fresh, and ran the instructions in https://github.com/emlearn/emlearn-micropython/blob/master/docs/developing.md. I've attached an output file.

build-log.txt

[jonnor]

@mikey247 thanks for the full build log. I have no idea why libffi is not found. It should be a part of the XCode SDK... I have disabled it completely now. I have also updated our CI for Mac to use the Mac OS 26 SDK, which I saw is what you have. There I encountered some new build warnings that turned into errors, which I have documented a workaround for in the docs/developing.md - and with that I hope everything works... Can you update to master and try again?

[mikey247]

Hey @jonnor, Ive been able to get the tests to work.

Issue: MicroPython build failed with linker error Undefined symbols for architecture arm64: "_mp_module_ffi"

probable cause: maybe Stale/corrupted build artifacts in the build-standard directory. The build system had inconsistent state - possibly from an interrupted previous build, incomplete compilation, or corrupted intermediate files.

Solution: Clean the build directory to remove all artifacts, then rebuild:

../micropython/ports/unix
make clean
cd -
make check_unix

[jonnor]

@mikey247 fantastic! Thank you for your patience!

Yeah, that kind of error can happen when changing the build configuration (which I did when disabling libffi), so cleaning out the old build is the correct solution.

[mikey247]

Great, I will close this issue now! @jonnor