IoUring: use typed Flags and start syncing with liburing 2.12

[bernardassan]

IoUring Overhaul

• [x] Use Named Constants for IoUring
• [x] Move io_uring_sqe.zig into IoUring
• [x] Add newly exposed IoUring Types, Constants, and Operations
• [x] Improve api of init fn with Flags.Setup Incompatible Flags References https://github.com/axboe/liburing/issues/1075#issuecomment-1970744640 https://github.com/axboe/liburing/issues/811#issuecomment-1454967969 https://nick-black.com/dankwiki/index.php/Io_uring
• [x] Catch simple incompatible flag configurations during init_params fn
• [x] Improve naming and namespacing
• [x] Update IoUring to match liburing 2.12
• [x] follow Zig Style Guide where it makes sense
• [x] Ensure there are no regressions (It would be great if community members already using it can test it out and give feedback)
• [x] Remove IoUring bits and pieces from linux.zig
• [x] Update IoUring to use the new typed flags
• [x] Add int_flags to fix a real-world issue in ghostty https://github.com/ziglang/zig/issues/25566
• [x] Cleanup linux/test.zig and IoUring tests
• [x] Update the added flags to match options available in kernel 6.16

Future Enhancement

• Indicate what versions of the kernel support the new flags and operations
• Port all tests from liburing 2.12 to IoUring
• Implement functionalities that require int_flags and enter_ring_fd

[daurnimator]

Move io_uring_sqe.zig into IoUring

That seems like a regression? Why is that desirable?

[bernardassan]

Why is it a regression? It makes IoUring self-contained. When I'm done, io_uring_sqe is going to be IoUring.Sqe, and IoUring is exposed in linux.zig so you can access sqe with IoUring.Sqe.

[bernardassan]

Now have IoUring and Linux test tests passing

❯ zig test lib/std/std.zig --zig-lib-dir lib --test-filter "os.linux"
68/116 os.linux.test.test.futex2_wait...SKIP
69/116 os.linux.test.test.futex2_wake...SKIP
70/116 os.linux.test.test.futex2_requeue...SKIP
111/116 os.linux.IoUring.test.waitid...SKIP
116/116 os.linux.IoUring.test.bind/listen/connect...SKIP
111 passed; 5 skipped; 0 failed.

Full test suite also passes

❯ zig test lib/std/std.zig --zig-lib-dir lib
441/2932 debug.test.manage resources correctly...SKIP
1163/2932 crypto.25519.x25519.test.rfc7748 1,000 iterations...SKIP
1164/2932 crypto.25519.x25519.test.rfc7748 1,000,000 iterations...SKIP
1288/2932 crypto.scrypt.test.kdf...SKIP
1289/2932 crypto.scrypt.test.kdf rfc 1...SKIP
1290/2932 crypto.scrypt.test.kdf rfc 2...SKIP
1291/2932 crypto.scrypt.test.kdf rfc 3...SKIP
1292/2932 crypto.scrypt.test.kdf rfc 4...SKIP
1293/2932 crypto.scrypt.test.password hashing (crypt format)...SKIP
1294/2932 crypto.scrypt.test.strHash and strVerify...SKIP
1295/2932 crypto.scrypt.test.unix-scrypt...SKIP
1301/2932 crypto.pbkdf2.test.RFC 6070 16,777,216 iterations...SKIP
1304/2932 crypto.pbkdf2.test.Very large dk_len...SKIP
1408/2932 fs.test.test.delete a setAsCwd directory on Windows...SKIP
1409/2932 fs.test.test.invalid UTF-8/WTF-8 paths...SKIP
1832/2932 net.test.test.non-blocking tcp server...SKIP
1833/2932 posix.test.test.WTF-8 to WTF-16 conversion buffer overflows...SKIP
1837/2932 posix.test.test.readlink on Windows...SKIP
1861/2932 posix.test.test.POSIX file locking with fcntl...SKIP
1941/2932 os.linux.test.test.futex2_wait...SKIP
1942/2932 os.linux.test.test.futex2_wake...SKIP
1943/2932 os.linux.test.test.futex2_requeue...SKIP
1945/2932 os.windows.nls.test.upcaseW matches RtlUpcaseUnicodeChar...SKIP
2269/2932 json.static_test.test.test all types...SKIP
2305/2932 json.dynamic_test.test.polymorphic parsing...SKIP
2363/2932 os.linux.IoUring.test.waitid...SKIP
2368/2932 os.linux.IoUring.test.bind/listen/connect...SKIP
2905 passed; 27 skipped; 0 failed.
4 fuzz tests found.

[RaidoAun]

There currently does not seem to be a replacement for int_flags on the ring. liburing uses that to set NO_IOWAIT flag to then before doing enter in for example submit_and_wait instead of initializing flags as 0 it is instead set to a mask of int_flags which includes NO_IOWAIT. see: https://github.com/ziglang/zig/pull/25604

and: https://github.com/axboe/liburing/commit/8f0c124785f167a6fbe034e59c32b63a4caea702#diff-ddb623c7d8d649eff38f34b19b7a2f8b55e1d448ee94b813a9541d5304467629

otherwise I like these changes since they add type safety around the flags. While playing with this I found the bug in my own PR.

[bernardassan]

This is on my to-do list, but I wanted to first get IoUring up-to-date with liburing 2.12 before working on these changes. Because I realized some liburing operations (Check commit message https://github.com/ziglang/zig/pull/25394/commits/6bb7f304bf719dc3e748f4eca1e68e54802f664e) depend on int_flags and enter_ring_fd.

But because this is affecting a real-world projects ghostty which I use :smile: , let me add the set_io_await functionality that you PR here https://github.com/ziglang/zig/pull/25604 quickly so you can test it out.

[bernardassan]

@RaidoAun a port of your solution in https://github.com/ziglang/zig/pull/25604 is ready in this branch

And can you share your GitHub email so I add you as a Co-Author of that commit

[RaidoAun]

Thanks, I will take a look.

Im fine without being co authored, but if you insist then you can just use my public github email ([email protected]). It should also be visible on the commit on my PR.

[bernardassan]

That is true, I didn't expect it to get this big, but I guess because we are way behind liburing 2.12 that should have been expected. @RaidoAun, a helping hand is always welcome. It would be nice to have an extra hand on deck when working on the future enhancements stated in the PR description.

I agree it's a good time to start getting reviews from core team members and previous contributors, as there are only a few more operations I want to implement and leave the rest as future enhancements, which I would make another PR for.

[bernardassan]

cc @mlugg @ianic @andrewrk @nikneym @topecongiro @ozgrakkurt @psnszsn @joadnacer @Vexu @jacobly0 @mllken @vrischmann @hnakamur @lithdew @ayende @jorangreef @ifreund @daurnimator

[bernardassan]

On my Arch Linux machine

❯ zig test lib/std/std.zig --zig-lib-dir lib --test-filter "os.linux" All 117 tests passed.

[bernardassan]

Finally done with all planned work for this PR. I would appreciate reviews and feedback from core maintainers, previous maintainers, and anyone interested in this changeset.

All tests are passing, and I'm able to build a debug build of Zig on this branch.

[bernardassan]

@Vexu @alexrp I have fixed the build error.

zig build test-std -Dskip-release -Dskip-non-native -fwasmtime -fwine -fqemu -Denable-llvm ✅

I have been trying to port the self-hosted CI to GitHub runners so I can test PRs in my branch before upstreaming them. Mac and Windows have Zig hosted deps, which makes the process very easy, but I tried the Linux one, and there doesn't seem to be a Linux variant of the hosted deps. It seems to be preinstalled on the self-hosted CI. Is there a way I can get a copy of these deps?