zig icon indicating copy to clipboard operation
zig copied to clipboard

std.Build: add `lazyImport` (`@import` for lazy dependencies)

Open castholm opened this issue 2 years ago • 0 comments

Closes #18987 Follow-up to #18778 (which added lazy dependencies) and the discussion in #18808 that followed

This adds b.lazyImport, which (in the context of a build script) is to @import what b.lazyDependency is to b.dependency and can be used to import a lazy dependency's build.zig struct.

It enables the use case where a package wants to lazily depend on an @import-style dependency and not fetch it until it is first needed, for example when building for a specific target.

// build.zig of lazy dependency 'foo'
const std = @import("std");

pub fn build(_: *std.Build) void {}

pub const bytes = "Hello!";
// build.zig of consumer
const std = @import("std");

pub fn build(b: *std.Build) void {
    const wf = b.addWriteFiles();

    // the non-lazy way, which won't work for our lazy dependency
    //_ = wf.add("foo.txt", @import("foo").bytes);

    // if 'foo' has not yet been fetched, 'lazyImport' returns 'null' and
    // triggers a recompile of the build script
    if (b.lazyImport(@This(), "foo")) |foo| {
        // 'foo' here is the build.zig struct of the dependency
        // we can access its public decls
        _ = wf.add("foo.txt", foo.bytes);
    }

    b.installDirectory(.{
        .source_dir = wf.getDirectory(),
        .install_dir = .prefix,
        .install_subdir = "",
    });
}

Implementation details

Given the name of a dependency, we want to locate that package.

Just like with dependency and lazyDependency, in order to find the package we need to first obtain a list over which dependencies are available to the dependee, then find the package with the matching name in that list. For dependency/lazyDependency, this list is b.available_deps.

However, because lazyImport returns a comptime-only return type, we can't use runtime arguments like b.available_deps.

All the lists of dependencies of all participating packages can be found in @import("root").dependencies, but given only the name of the package have no way to know at comptime for which package we are calling lazyImport.

The only "clean" solution to this problem that I can come up with is to take a build.zig struct as the second argument:

pub inline fn lazyImport(
    b: *Build,
    /// The struct that corresponds to the build.zig of the package `b` was created for.
    /// When calling this function from a build.zig's `build` function, you usually pass `@This()`.
    comptime b_build_zig: type,
    comptime dep_name: []const u8,
) ?type {}

This enables us to compare b_build_zig against the build.zig struct of every package in @import("root").dependencies to figure out for which package we are calling lazyImport, then search through that package's dependencies as usual.

For good measure we can also throw in a runtime safety check that asserts that the b instance and the build.zig struct both represent the same package.

In practice this means that you almost always invoke the function with @This() as the second argument, such as in b.lazyImport(@This(), "foo").

Testing

This API can't currently easily be tested by the compiler's test suite.

If any maintainers would like to check out this branch and test it locally, I have prepared a little test project that uses lazyImport up to two levels of indirection, which I used myself to test the changes:

test.tar.gz

The project is set up as follows:

  • root lazily depends on alfa
  • root lazily depends on bravo
  • bravo lazily depends on charlie

I used python -m http.server to serve the packages over http://localhost:8000/; you might need to edit them if you want to serve them in a different way.

Running zig build will trigger one initial compile of the build script + two recompiles, then install two plaintext files.

castholm avatar Feb 18 '24 20:02 castholm