Cross-compiling luv

[jonahbeckford]

I can do a PR myself (if it is small-ish like the proposal below) once this design question is answered.

Context

The way I do cross-compilation is using Dune workspace targets, which is just a wrapper around ocamlfind toolchains. I believe others who use opam-monorepo do the same thing.

Problem

I'm running into problems with luv because luv uses a custom build in src/c/dune. One example blocker is https://github.com/aantron/luv/blob/0ed668e87ca55d65f4f84422b0dd3381701c71b0/src/c/dune#L75 :

   (bash
    "sh configure --host `ocamlc -config | awk '/host/ {print $NF}'` \
      'CC=%{cc}' CFLAGS=-DNDEBUG --silent --enable-silent-rules")

For cross-compiling, you would want a build for ocamlc -config like above, but you would also want a ocamlfind -toolchain <toolchain> ocamlc -config build. Dune does this automatically by building a "native" context=default target (which uses no toolchain) and then building separately (ex. context="android") for any targets listed in dune-workspace.

Proposal

A couple points:

• I think plumbing the toolchain logic into the embedded libuv parts of src/c/dune is too complicated (ie. hard to maintain).
• I think using embedded libuv is undesirable in many common cross-compiling situations. For example, when cross-compiling Android ... it is better to have a prebuilt, well-tested, properly configured Android libuv library than to cross-compile and embed libuv internally with ocamlc -config flags.

So my idea would be to:

1. Extend the simple logic that is already present for LUV_USE_SYSTEM_LIBUV. Dune has a variable called context_name; it is also available as Jbuild_plugin.V1.context. If dune-workspace has two targets "native" and "android", then src/c/dune will be called with context_name="default" in the _build/default directory and then called again with context_name="default.android" in the _build/default.android directory.
2. If and only if LUV_USE_SYSTEM_LIBUV is set, have LUV_SYSTEM_LIBUV_${uppercase(sanitize(context_name))} be an environment variable that sets uv_library_flag. If it isn't defined just fallback to -luv. For example:

   LUV_USE_SYSTEM_LIBUV=yes
   LUV_SYSTEM_LIBUV_DEFAULT='-luv'
   LUV_SYSTEM_LIBUV_DEFAULT_ANDROID_AARCH64='-L/Android/prebuilt/llvm-aarch64/lib;-luv'
   LUV_SYSTEM_LIBUV_DEFAULT_ANDROID_ARM32='/Android/prebuilt/llvm-arm32v7a/lib/libuv.so'
   LUV_SYSTEM_LIBUV_DEFAULT_WINDOWS64='-LC:/Program Files/vcpkg/windows-x64/lib;-llibuv'
   

I used semicolons as a separator as one way to allow spaces in paths which is essential for Windows. Could also have escaped the space or passed in OCaml escaped strings.

What do you think? Suggestions?

Thanks.

[jonahbeckford]

One thought: The vcpkg port of libuv names the library libuv rather than uv; see https://github.com/microsoft/vcpkg/blob/05c445c1fb3afb90aa8a46ca2b7d2a9548a27121/ports/libuv/CMakeLists.txt#L54-L64 . So does the other popular C package manager Conan. The proposal I wrote above can handle these naming choices (LUV_SYSTEM_LIBUV_DEFAULT='-llibuv') even when cross-compilation is not needed.

[aantron]

That seems reasonable. If someone would prefer cross-compilation using the vendored libuv code, I/they can add it separately later.

[jonahbeckford]

Thinking aloud: We would end up with many many environment variables if every OCaml package does what my PR https://github.com/aantron/luv/pull/130 does. And I could see running into environment maximum size (32K on Windows) or command line max length limitations (ex. env X=Y A=B gcc; 8K on Windows) that won't scale.

The entire problem is that we don't have a good way in OCaml to ask for locations of C libraries. Today packages do one of the following:

• Expect that the C library is in the "system location" (ex. /usr/include, /usr/lib) as a result of depexts. But not only is "system location" ill-defined on Windows, it can't work when cross-compiling.
• pkg-config is another standard but it works poorly on Windows (ex. pkg-config wasn't designed for spaces; few packages bother to make .pc files on Windows) and it definitely won't work when cross-compiling
• environment variables obviously work but don't scale. And it seems too difficult for package maintainers to implement (especially the cross-compiling environment variable name logic)
• ocamlfind is mismatched (finds OCaml packages rather than C packages) but extending ocamlfind might work

I am fine with keeping environment variables for now, but we need a better long-term solution. I am willing to start a discuss.ocaml.org topic, unless you have strong opinions one way or the other.