htm.core icon indicating copy to clipboard operation
htm.core copied to clipboard

CI: build in Docker with MUSL libc for static library

Open breznak opened this issue 6 years ago • 10 comments

  • [x] avoid glibc with -static build, as it does not work. https://stackoverflow.com/questions/26304531/compiling-with-static-libgcc-static-libstdc-still-results-in-dynamic-depende
  • [x] adds deterministic checks for MUSL libc based systems #707
  • [x] Also bump gtest to v 1.10
  • [x] for PyPI add build in Docker on Alpine linux that uses MUSL libc, which can be compiled into static transferable library

For #707 #19

breznak avatar Oct 30 '19 13:10 breznak

https://github.com/htm-community/htm.core/pull/736/checks?check_run_id=283078173#step:5:1176

I'm ~~un~~able to reproduce locally (but using the same Docker), @dkeeney when you have time, would you please test for me on another system? You'll need docker, and then follow the Dockerfile instructions at the top of the file (or see the CI section), please?

breznak avatar Oct 31 '19 14:10 breznak

I don't know anything about Docker so this is a good time to find out. I can put my other PR on hold for now.

Now, what you are trying to do is use the static C++ runtime library and you are doing it with this MUSL thing. I see three builds that fail...are they one for each platform?

dkeeney avatar Oct 31 '19 17:10 dkeeney

I ran into a MAJOR snag. Docker Desktop for windows requires Hyper-V to work. VirtualBox does not work at all with Hyper-V turned on. I use VirtualBox as my Ubuntu installation. I cannot install Docker inside another virtualization either. There does not appear to be any way around this short of buying another computer.

dkeeney avatar Oct 31 '19 20:10 dkeeney

There is an older Docker Toolbox that does run Docker under VirtualBox. I could use that but it might not be the environment that you want me to try.

dkeeney avatar Oct 31 '19 20:10 dkeeney

I don't know anything about Docker so this is a good time to find out.

thanks a lot for willing to try that for me, David!

are trying to do is use the static C++ runtime library and you are doing it with this MUSL thing

MUSL is an alternative stdlibc++ for linux, most commonly used is glibc (only which we've been using so far), MSVC has its version, etc. GLIBC is known for its static libraries (when you link stdlibc in your code) are not truly independent, transferable. Unlike musl, where this should work ok. The intention is to use such statically linked lib as a binary for all linux distributions on PyPI

I see three builds that fail...are they one for each platform?

it's the UX here messed up, it's the same platform - x86_64 on Alpine linux (with musl C) ran inside docker. I've restarted the PR and the old results just keep showing up.

Docker Desktop for windows requires Hyper-V to work. VirtualBox does not work at all with Hyper-V turned on. I use VirtualBox as my Ubuntu installation

can you enable HyperV in BIOS, or is it a HW limitation? If you could, you'd just test the docker from Windows, and then disable HyperV again to be able to run VBox. I was not aware of such complication on Win, sorry. On linux (running directly on HW) I run both VBox and Docker containers.

https://docs.docker.com/docker-for-windows/install/#what-to-know-before-you-install

breznak avatar Nov 01 '19 07:11 breznak

can you enable HyperV in BIOS

Ok, I think I understand what you are looking for. Yes it is possible to toggle HyperV in the BIOS. I am leaving this morning for a few days in San Jose with our grandson. Will be back Monday afternoon. I will give this a try when I return.

dkeeney avatar Nov 01 '19 12:11 dkeeney

Enjoy your weekend! :fallen_leaf: PS: I'm able to replicate the bug locally, crash in Classifier, leads to someting with NTA_CHECK macros and way how we handle exceptions. Preparing a PR that aims to streamline to process of our exceptions. (removing LogItem and the likes)

breznak avatar Nov 01 '19 12:11 breznak

(removing LogItem and the likes)

Oh, very good. That is still some old original code. Could use a cleanup.

dkeeney avatar Nov 01 '19 12:11 dkeeney

Do you still need me to try docker on my machine? I am reluctant to risk messing up my Ubuntu OS which is under virtualbox but if you still need me to do this I will give it a try.

dkeeney avatar Nov 09 '19 00:11 dkeeney

Do you still need me to try docker on my machine?

so far I'm good, or I can replicate the err seen in CI. Thank you very much for offering the help!

breznak avatar Nov 09 '19 22:11 breznak