rustc_codegen_cranelift icon indicating copy to clipboard operation
rustc_codegen_cranelift copied to clipboard

Using in Windows

Open thewh1teagle opened this issue 1 year ago • 14 comments

It's unclear how can I use this codegen in Windows. I would like to use it both locally and on Github actions for faster build with Rust on Windows. Does it improve speed for release builds?

rustup default nightly

What should I do after installing nightly rust and set it as deafult?

thewh1teagle avatar Jul 07 '24 12:07 thewh1teagle

Does it improve speed for release builds?

Cranelift doesn't support enough optimizations to make it useful in release mode.

It's unclear how can I use this codegen in Windows.

It isn't distibuted with rustup on Windows yet. You can download the latest version from https://github.com/rust-lang/rustc_codegen_cranelift/releases/tag/dev instead, extract it somewhere and then run the contained cargo-clif.exe in the place of cargo.exe.

bjorn3 avatar Jul 07 '24 12:07 bjorn3

I think it would be beneficial if there was a bigger notice in README.md that the usage instructions above the section mentioning precompiled binaries (for Windows) don't all easily transfer. I at least can't get rustc to accept the rustc_codegen_cranelift.dll in the "codegen-backends" in "sysroot" (mentioned in error when trying to use codegen-backend = "cranelift" without using rustup). I tried doing a few things to get it working per profile, including mirroring the layout that appears when using the rustup method on Linux. Only by passing -Zcodegen-backend=<absolute-path-to-dll> wholesale in a crate-wide rust flag can I get it working on Windows.

selvmaya avatar Jul 18 '24 19:07 selvmaya

and then run the contained cargo-clif.exe in the place of cargo.exe.

I added cargo-clif to PATH and changed the default toolchain to nightly but it give me error:

C:\Users\User\Documents\code\rustduck\src-tauri>rustup default
nightly-x86_64-pc-windows-msvc (default)

C:\Users\User\Documents\code\rustduck\src-tauri>cargo-clif build
warning: `C:\Users\User\.cargo\config` is deprecated in favor of `config.toml`
note: if you need to support cargo 1.38 or earlier, you can symlink `config` to `config.toml`
warning: `C:\Users\User\.cargo\config` is deprecated in favor of `config.toml`
note: if you need to support cargo 1.38 or earlier, you can symlink `config` to `config.toml`
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `rustc - --crate-name ___ --print=file-names -Cpanic=abort -Zpanic-abort-tests -Zcodegen-backend=C:\apps\bin\cargo-clif\bin\rustc_codegen_cranelift.dll --sysroot C:\apps\bin\cargo-clif --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=split-debuginfo --print=crate-name --print=cfg` (exit code: 1)
  --- stderr
  error: the option `Z` is only accepted on the nightly compiler

  help: consider switching to a nightly toolchain: `rustup default nightly`

  note: selecting a toolchain with `+toolchain` arguments require a rustup proxy; see <https://rust-lang.github.io/rustup/concepts/index.html>

  note: for more information about Rust's stability policy, see <https://doc.rust-lang.org/book/appendix-07-nightly-rust.html#unstable-features>

  error: 1 nightly option were parsed

thewh1teagle avatar Jul 20 '24 12:07 thewh1teagle

That was an issue with rustc. I reinstalled it and it got fixed.


I compared build time with cargo vs cargo-clif:

  1. With cargo-clif $start = Get-Date; cargo-clif build ; $elapsed = (Get-Date) - $start; Write-Output "Elapsed time: $elapsed" Elapsed time: 00:01:35.74

  2. With cargo $start = Get-Date; cargo build ; $elapsed = (Get-Date) - $start; Write-Output "Elapsed time: $elapsed" Elapsed time: 00:02:01.69

thewh1teagle avatar Jul 20 '24 12:07 thewh1teagle

Sorry for the late reply @mikkelens. With the precompiled binaries from the releases page you probably want to use cargo-clif.exe instead of using the cargo configs.

bjorn3 avatar Jul 20 '24 13:07 bjorn3

I also have problems compiling for windows. this is my cargo config

# Add the contents of this file to `config.toml` to enable "fast build" configuration. Please read the notes below.

# NOTE: For maximum performance, build using a nightly compiler
# If you are using rust stable, remove the "-Zshare-generics=y" below.
[unstable]
codegen-backend = true

[profile.dev]
codegen-backend = "cranelift"
[profile.dev.package."*"]
codegen-backend = "llvm"
[target.x86_64-unknown-linux-gnu]
linker = "clang"
#rustflags = ["-C", "link-arg=-fuse-ld=lld"]
rustflags = ["-C", "link-arg=-fuse-ld=/usr/bin/mold"]

[target.x86_64-apple-darwin]
rustflags = [
	"-C",
	"link-arg=-fuse-ld=/opt/homebrew/bin/zld",
	"-Zshare-generics=y",
]

[target.aarch64-apple-darwin]
rustflags = [
	"-C",
	"link-arg=-fuse-ld=/opt/homebrew/bin/zld",
	"-Zshare-generics=y",
]

[target.x86_64-pc-windows-msvc]
linker = "rust-lld.exe"
#rustflags = ["-Zshare-generics=n"]

# Optional: Uncommenting the following improves compile times, but reduces the amount of debug info to 'line number tables only'
# In most cases the gains are negligible, but if you are on macos and have slow compile times you should see significant gains.
#[profile.dev]
#debug = 1

cargo-clif build gives me this:

Compiling proc-macro2 v1.0.86 Compiling unicode-ident v1.0.12 Compiling cfg-if v1.0.0 Compiling serde v1.0.204 Compiling autocfg v1.3.0 Compiling windows_x86_64_gnu v0.52.6 Compiling getrandom v0.2.15 Compiling once_cell v1.19.0 error: linking with x86_64-w64-mingw32-gcc failed: exit code: 1 | = note: "x86_64-w64-mingw32-gcc" "-fno-use-linker-plugin" "-Wl,--dynamicbase" "-Wl,--disable-auto-image-base" "-m64" "-Wl,--high-entropy-va" "crt2.o" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\rsbegin.o" "C:\Users\mlp23\AppData\Local\Temp\rustctmeQdK\symbols.o" "C:\Users\mlp23\src\ob3d_rust\target\debug\build\windows_x86_64_gnu-e5e4b9dfcb0112d9\build_script_build-e5e4b9dfcb0112d9.build_script_build.5ee14548a281eb96-cgu.0.rcgu.o" "C:\Users\mlp23\src\ob3d_rust\target\debug\build\windows_x86_64_gnu-e5e4b9dfcb0112d9\build_script_build-e5e4b9dfcb0112d9.allocator_shim.rcgu.o" "-L" "C:\Users\mlp23\src\ob3d_rust\target\debug\deps" "-L" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib" "-Wl,-Bstatic" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libstd-8c05ab2476f28dde.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libpanic_abort-518a42450a8bc012.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libobject-7dc256a1c4e602c1.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libmemchr-b1097742e6866c22.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libaddr2line-eadc6a7baf6e5c75.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libgimli-cebbe372d286137b.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\librustc_demangle-4e688bb8c706118e.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libstd_detect-309883f26df9524d.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libhashbrown-c5a2307c4bb51095.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\librustc_std_workspace_alloc-3d68b96e12844dd5.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libminiz_oxide-6651de0a4fbf42e0.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libadler-ae6b461c2e2fbeef.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libunwind-15f0e9bf1570448e.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libcfg_if-a405d22dd6456aac.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\liblibc-242d7e49704c4071.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\liballoc-34ed69e46a0684e5.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\librustc_std_workspace_core-0926cf63825cc565.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libcore-50309013fb4dc6c2.rlib" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\libcompiler_builtins-b4d4e5dd180af752.rlib" "-Wl,-Bdynamic" "-lkernel32" "-ladvapi32" "-lntdll" "-luserenv" "-lws2_32" "-lkernel32" "-lsynchronization" "-lkernel32" "-lgcc_eh" "-l:libpthread.a" "-lmsvcrt" "-lmingwex" "-lmingw32" "-lgcc" "-lmsvcrt" "-lmingwex" "-luser32" "-lkernel32" "-Wl,--nxcompat" "-nostartfiles" "-L" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib" "-L" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\self-contained" "-o" "C:\Users\mlp23\src\ob3d_rust\target\debug\build\windows_x86_64_gnu-e5e4b9dfcb0112d9\build_script_build-e5e4b9dfcb0112d9.exe" "-Wl,--gc-sections" "-no-pie" "-Wl,-O1" "-nodefaultlibs" "C:\Users\mlp23\apps\cg_clif\dist\lib\rustlib\x86_64-pc-windows-gnu\lib\rsend.o" = note: ld: cannot find crt2.o: No such file or directory ld: cannot find -lkernel32: No such file or directory ld: cannot find -ladvapi32: No such file or directory ld: cannot find -lntdll: No such file or directory ld: cannot find -luserenv: No such file or directory ld: cannot find -lws2_32: No such file or directory ld: cannot find -lkernel32: No such file or directory ld: cannot find -lsynchronization: No such file or directory ld: cannot find -lkernel32: No such file or directory ld: cannot find -lgcc_eh: No such file or directory ld: cannot find -l:libpthread.a: No such file or directory ld: cannot find -lmsvcrt: No such file or directory ld: cannot find -lmingwex: No such file or directory ld: cannot find -lmingw32: No such file or directory ld: cannot find -lgcc: No such file or directory ld: cannot find -lmsvcrt: No such file or directory ld: cannot find -lmingwex: No such file or directory ld: cannot find -luser32: No such file or directory ld: cannot find -lkernel32: No such file or directory

mlp1802 avatar Jul 23 '24 16:07 mlp1802

Does it work with the x86_64-pc-windows-msvc toolchain? I think the x86_64-pc-windows-gnu toolchain is missing all MinGW libraries.

bjorn3 avatar Jul 23 '24 17:07 bjorn3

I'm not sure, what do you mean? I use "nightly" though I have a feeling that's not what you mean. I quick search brought me to this: https://www.msys2.org/

mlp1802 avatar Jul 23 '24 17:07 mlp1802

Are you using the artifacts from https://github.com/rust-lang/rustc_codegen_cranelift/releases/tag/dev or are you using rustc_codegen_cranelift from rustup (rustup component add rustc-codegen-cranelift-preview)?

bjorn3 avatar Jul 23 '24 17:07 bjorn3

I think I'm already using it:

 rustup toolchain list
stable-x86_64-pc-windows-gnu
stable-x86_64-pc-windows-msvc
nightly-2024-07-13-x86_64-pc-windows-gnu
nightly-x86_64-pc-windows-msvc (default) (override)

mlp1802 avatar Jul 23 '24 17:07 mlp1802

You downloaded the x86_64-pc-windows-gnu version of cg_clif rather than the x86_64-pc-windows-msvc version. Try using the x86_64-pc-windows-msvc version.

bjorn3 avatar Jul 23 '24 17:07 bjorn3

That works, thanks! but now my game runs on 15 fps for some reason (it was bad already on windows, 45 fps, but still playable). Anyway it works on linux and quite well I must say.

mlp1802 avatar Jul 23 '24 18:07 mlp1802

With the precompiled binaries from the releases page you probably want to use cargo-clif.exe instead of using the cargo configs.

This is true and what I expected after having trouble with the other method. The problem though is that this doesn't (seemingly) make it possible to use cranelift for only the top level crate (incrementally compiled), like is recommended in most contexts (using LLVM for the dependencies). I also am not sure if dynamic linking (e.g. bevy/dynamic_linking) is supported under these conditions. If there is a way to fix this I would love to know. It is at least nice to know that the straight-forward method is actually the generally recommended one on windows, and not just the easiest to get working. Thanks!

selvmaya avatar Jul 24 '24 16:07 selvmaya

The problem though is that this doesn't (seemingly) make it possible to use cranelift for only the top level crate (incrementally compiled), like is recommended in most contexts (using LLVM for the dependencies).

Yeah. I would like to see the rustup version work on windows too, but that is waiting on at least raw-dylib support (which someone is currently working on)

I also am not sure if dynamic linking (e.g. bevy/dynamic_linking) is supported under these conditions.

Dynamic linking should be supported. If not, that is a bug you can open an issue for.

bjorn3 avatar Aug 01 '24 19:08 bjorn3

cg_clif has been distributed as rustup component for a while now. See https://github.com/rust-lang/rustc_codegen_cranelift?tab=readme-ov-file#download-using-rustup for how to use it. And raw-dylib is fully working.

bjorn3 avatar Dec 04 '24 11:12 bjorn3