Error Cross compiling for x86_64-pc-windows-gnu from Linux

[michaelvanstraten]

I have the following docker image for testing the compilation.

From rust:latest
RUN rustup target add x86_64-pc-windows-gnu
RUN apt-get update -y && apt-get install -y gcc-mingw-w64-x86-64 gcc-mingw-w64-i686 libnpth-mingw-w64-dev
RUN mkdir src
RUN git clone https://github.com/MagicalBitcoin/libtor-sys.git src

WORKDIR src
CMD cargo build -vv --features=vendored-openssl --target=x86_64-pc-windows-gnu

I am on the master branch of version 46.9.1+0.4.6.9 with commit ID 19bae6258c4d290bd4b0eb3f5650ac4487c93219.

There seams to be a problem with libevent, it errors out when checking the dependency:

Error

checking for libevent directory... 
  --- stderr
  configure: WARNING: using cross tools not prefixed with host triplet
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event.c: In function 'dump_inserted_event_fn':
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event.c:3751:18: warning: format '%d' expects argument of type 'int', but argument 5 has type 'long long int' [-Wformat=]
   3751 |  fprintf(output, "  %p [%s "EV_SOCK_FMT"]%s%s%s%s%s%s%s",
        |                  ^~~~~~~~~~~
  In file included from /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/minheap-internal.h:36,
                   from /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event-internal.h:40,
                   from /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event.c:62:
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/util-internal.h:461:25: note: format string is defined here
    461 | #define EV_I64_FMT "%I64d"
        |                     ~~~~^
        |                         |
        |                         int
        |                     %I64lld
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event.c: In function 'dump_active_event_fn':
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event.c:3785:18: warning: format '%d' expects argument of type 'int', but argument 5 has type 'long long int' [-Wformat=]
   3785 |  fprintf(output, "  %p [%s "EV_SOCK_FMT", priority=%d]%s%s%s%s%s active%s%s\n",
        |                  ^~~~~~~~~~~
  In file included from /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/minheap-internal.h:36,
                   from /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event-internal.h:40,
                   from /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event.c:62:
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/util-internal.h:461:25: note: format string is defined here
    461 | #define EV_I64_FMT "%I64d"
        |                     ~~~~^
        |                         |
        |                         int
        |                     %I64lld
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/evutil.c: In function 'evutil_check_ifaddrs':
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/evutil.c:714:13: warning: cast between incompatible function types from 'FARPROC' {aka 'long long int (*)()'} to 'ULONG (*)(ULONG,  ULONG,  void *, IP_ADAPTER_ADDRESSES_XP *, ULONG *)' {aka 'long unsigned int (*)(long unsigned int,  long unsigned int,  void *, IP_ADAPTER_ADDRESSES_XP *, long unsigned int *)'} [-Wcast-function-type]
    714 |  if (!(fn = (GetAdaptersAddresses_fn_t) GetProcAddress(lib, "GetAdaptersAddresses")))
        |             ^
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/evdns.c: In function 'load_nameservers_with_getnetworkparams':
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/evdns.c:3781:13: warning: cast between incompatible function types from 'FARPROC' {aka 'long long int (*)()'} to 'DWORD (*)(FIXED_INFO *, DWORD *)' {aka 'long unsigned int (*)(FIXED_INFO *, long unsigned int *)'} [-Wcast-function-type]
   3781 |  if (!(fn = (GetNetworkParams_fn_t) GetProcAddress(handle, "GetNetworkParams"))) {
        |             ^
  configure: WARNING: unrecognized options: --disable-shared, --enable-static, --disable-module-dircache
  configure: WARNING: using cross tools not prefixed with host triplet
  configure: WARNING: Could not find a linkable libevent.  If you have it installed somewhere unusual, you can specify an explicit path using --with-libevent-dir
  configure: WARNING: On Debian, you can install libevent using "apt-get install libevent-dev"
  configure: error: Missing libraries; unable to proceed.
  thread 'main' panicked at '
  command did not execute successfully, got: exit status: 1

Here is a full log of the build: log.txt

I would be very grade full if somebody could help me with this issue

[michaelvanstraten]

So sorry, took me two days and a GitHub workflow run to realize that the cross compilation of libevent from arm to x64 is responsible for this error, the crate has nothing to do with it. Works all fine in a Docker Container on a x64 AMD VPS. #12 is probably the same thing, caused by an apple arm processor.

[afilini]

It looks like libevent is building fine (albeit with a few warnings), but then tor doesn't like to link with it.

I don't have an arm laptop, I'll see if I can get my hands on one to try and reproduce the error, because I would like to see exactly why tor doesn't want to link with libevent. Maybe it's something we can fix

[michaelvanstraten]

I don't have an arm laptop, I'll see if I can get my hands on one to try and reproduce the error, because I would like to see exactly why tor doesn't want to link with libevent. Maybe it's something we can fix

If you want, I could run some tests if you tell me what to do.

[afilini]

I don't remember how to do this, but there should be a way to get the full log of all the checks run by configure. There should be at least one check that tries to link with libevent and fails, which then makes the whole build fail. If you could get the log of that specific test then we could try to figure out why it's failing and maybe fix it.

[michaelvanstraten]

Do you mean this?

--- stderr
  configure: WARNING: using cross tools not prefixed with host triplet
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event.c: In function 'dump_inserted_event_fn':
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event.c:3751:18: warning: format '%d' expects argument of type 'int', but argument 5 has type 'long long int' [-Wformat=]
   3751 |  fprintf(output, "  %p [%s "EV_SOCK_FMT"]%s%s%s%s%s%s%s",
        |                  ^~~~~~~~~~~
  In file included from /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/minheap-internal.h:36,
                   from /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event-internal.h:40,
                   from /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event.c:62:
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/util-internal.h:461:25: note: format string is defined here
    461 | #define EV_I64_FMT "%I64d"
        |                     ~~~~^
        |                         |
        |                         int
        |                     %I64lld
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event.c: In function 'dump_active_event_fn':
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event.c:3785:18: warning: format '%d' expects argument of type 'int', but argument 5 has type 'long long int' [-Wformat=]
   3785 |  fprintf(output, "  %p [%s "EV_SOCK_FMT", priority=%d]%s%s%s%s%s active%s%s\n",
        |                  ^~~~~~~~~~~
  In file included from /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/minheap-internal.h:36,
                   from /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event-internal.h:40,
                   from /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/event.c:62:
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/util-internal.h:461:25: note: format string is defined here
    461 | #define EV_I64_FMT "%I64d"
        |                     ~~~~^
        |                         |
        |                         int
        |                     %I64lld
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/evutil.c: In function 'evutil_check_ifaddrs':
  /src/target/x86_64-pc-windows-gnu/debug/build/libtor-sys-ae46822d76c19b23/out/libevent/libevent-src/evutil.c:714:13: warning: cast between incompatible function types from 'FARPROC' {aka 'long long int (*)()'} to 'ULONG (*)(ULONG,  ULONG,  void *, IP_ADAPTER_ADDRESSES_XP *, ULONG *)' {aka 'long unsigned int (*)(long unsigned int,  long unsigned int,  void *, IP_ADAPTER_ADDRESSES_XP *, long unsigned int *)'} [-Wcast-function-type]
    714 |  if (!(fn = (GetAdaptersAddresses_fn_t) GetProcAddress(lib, "GetAdaptersAddresses")))

[afilini]

No these are just warnings, they shouldn't make it all fail.

There's a log somewhere that lists all the "checks" run by configure to figure out the right compiling flags. There should be one or more for libevent where it tries to link with it and fails.

Those logs would help try figuring out why it doesn't want to use the libevent we provide

[i5hi]

I've been having an issue with libevent too when cross compiling for android on linux.

I get this warning and error:

 --- stderr
  configure: WARNING: unrecognized options: --disable-shared, --enable-static, --disable-module-dircache, --disable-rust
  configure: error: "You must specify an explicit --with-libevent-dir=x option when using --enable-static-libevent"
  thread 'main' panicked at '
  command did not execute successfully, got: exit status: 1

I'm using this command:

CC=/usr/bin/x86_64-linux-gnu-gcc \
PKG_CONFIG_ALLOW_CROSS=1 \
OPENSSL_STATIC=1 \
cargo build --target aarch64-linux-android --release

I get the same error if i use "vendored-openssl" in Config.toml and only use the CC env in cargo build.

[i5hi]

The same project builds fine when i remove libtor from Cargo.toml, so all other cross compiling configs are okay.

[i5hi]

Those logs would help try figuring out why it doesn't want to use the libevent we provide

configure: error: "You must specify an explicit --with-libevent-dir=x option when using --enable-static-libevent"

Does the --enable-static-libevent flag force it to use a local libevent?

[i5hi]

I made some progress by adding features = ["vendored-lzma","vendored-zstd"] } to the dependency in Cargo.toml now i get a linker error at a later stage.

[i5hi]

Got it working.

Cargo.toml

openssl = { version = "0.10", features = ["vendored"] }
libtor = { version = "47.7.0+0.4.7.x", features = ["vendored-lzma","vendored-zstd"] }

cargo build --target aarch64-linux-android --release

No flags required.

With windows I would assume the same should work.

[michaelvanstraten]

Got it working.

Cargo.toml

openssl = { version = "0.10", features = ["vendored"] }
libtor = { version = "47.7.0+0.4.7.x", features = ["vendored-lzma","vendored-zstd"] }

cargo build --target aarch64-linux-android --release

No flags required.

With windows I would assume the same should work.

This still errors out when compiling from linux-aarch64 to windwos-x64.

[i5hi]

Sorry i just noticed your CMD and youre using cargo build -vv --features=vendored-openssl --target=x86_64-pc-windows-gnu

Have you tried adding the other two vendored libs in --features in the above command?

[afilini]

@i5hi if you are targeting android I think your CC env variable was wrong, it was pointing to a non-android linux toolchain. Usually if you have the NDK in your PATH cargo can figure out the right compiler to use, otherwise point it manually to clang from the NDK.

Btw your final configuration should be equivalent to just including libtor with vendored-openssl, in theory there's no need to include openssl separately unless you need it for your project of course.

[i5hi]

@i5hi if you are targeting android I think your CC env variable was wrong, it was pointing to a non-android linux toolchain. Usually if you have the NDK in your PATH cargo can figure out the right compiler to use, otherwise point it manually to clang from the NDK.

Yes. After changing that I could run cargo run --release android without needing to use flags. I was using my local gcc instead of the target clang. Now i just let it chose the right linkers that i set in .cargo/config

Btw your final configuration should be equivalent to just including libtor with vendored-openssl, in theory there's no need to include openssl separately unless you need it for your project of course.

Yes, that makes sense but I get errors when i use vendored-openssl. Since the other way worked, i just left it for the time being. Will attempt to debug this later.

[afilini]

That's interesting, let me know if you figure out why that's happening.

[i5hi]

Will do.

On an aside, @afilini, what is the best rust http client that allows me to configure a socks5 port to make normal http requests over tor?

[afilini]

Not sure exactly what's best because I haven't compared many of them, but we've been using ureq with BDK since it's pretty lightweight and they seem to support socks5 proxies as well: https://docs.rs/ureq/latest/ureq/struct.Proxy.html

[michaelvanstraten]

For anybody who is interested, I have tested this again and on commit 7309e85f8 this still fails to cross compile.