cc_binary dep on cc_library#linkstamp doesn't set up include paths correctly for non-root module deps

[ted-xie]

Suppose you have two bazel modules, A and B. B depends upon A.

A has a cc_library called "src/foo:foo". This library has linkstamp = build_embed.cc", where build_embed.cc has a local include path: #include "src/foo/build_embed.h"`.

A's "src/foo:foo" library is a dep of a binary called "src/foo:main", also within A.

B has two targets that just copy from A: "ersatz_main" and "ersatz_foo". Both of these targets just copy the output files of @A//src/foo:{foo,main} into B's repository.

Within B, it is possible to compile ersatz_foo correctly. However, it is not possible to compile ersatz_main. This error occurs:

external/foo+/src/foo/build_embed.cc:1:10: fatal error: src/foo/build_embed.h: No such file or directory
    1 | #include "src/foo/build_embed.h"

It seems that the compilation action for cc_binary targets with linkstamp set does not properly add all required include paths.

See https://github.com/ted-xie/linkstamp_issue_rules_cc for full repro.

Bazel version: 8.2.1 (also repros at 7.4.1) rules_cc version: 0.1.1 (also repros at 0.0.10)

[trybka]

I believe this is intentional. In theory this should include very few (if any) headers, and they be simple and co-located with the .cc file. I would say they shouldn't cross package boundaries, let alone repo boundaries.

The docs say,

A linkstamp compilation may not include any particular set of compiler flags and so should not depend on any particular header, compiler option, or other build variable.

I think linkstamp compile options intentionally do not provide more than the bare minimum include paths by design.

If I look at some exemplar internal builddata_globals.cc files, I see the comments:

// This file is somewhat magic: it gets compiled into an object file
// during the bazel *link* step, immediately before the linker is
// run. It is never built/cached separately, so that the data will
// always be accurate as of when the binary was linked.
//
// Therefore, it should be kept extremely simple, and should not
// depend on any system or other header files, so that the link step
// does not need to have a full compilation environment available.

and

// This is a private header declaring the globals provided by the
// magic builddata_globals.cc file, and used by the functions in
// builddata.cc (exposing the interface in builddata.h).
//
// It should not include *any* other headers, because they will not
// necessarily be present when this file is compiled during bazel's
// link step.

[ted-xie]

Thanks, Tom. This all makes sense, but I think these assumptions break for some simple Bazel use-cases. For example:

• rules_A is a repo with rules and tools for the language A
• proj_B depends on rules_A to compile its A deps
• A has a compiler with a linkstamp attr set with one #include

In this case, wouldn't proj_B be unable to build A's compiler? I think the A compiler's usage of linkstamp is fair here, but the behavior of linkstamp precludes this. Should rules_A refactor its compiler to not require any local includes in the linkstamped cc file?

[trybka]

I'm admittedly not well-versed in how cross-repo includes work in this scenario.

The way your rules are setup, I would have assumed foo:main as built via ersatz_main would still be built in the context of repo A. (i.e. what makes it different being built as a dep of that genrule vs. being built directly?)

[ted-xie]

Ah, ersatz_main was just a convenient way of demonstrating the problem. Building foo:main directly inside the other directory exhibits the same problem:

linkstamp_issue_rules_cc/other$ bazelisk build @foo//src/foo:main
[...]
external/foo+/src/foo/build_embed.cc:1:10: fatal error: src/foo/build_embed.h: No such file or directory
    1 | #include "src/foo/build_embed.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

[trybka]

Based on the docs and the implementation, I would say that any use of headers that happens to work is probably restricted to mono-repo only.

This is kind of the trade-off with the "do the compile as late as possible for proper timestamps" wherein you need to restrict it to "just stuff you can compile at link time".

I've seen a few other examples where you can do something clever with genrules instead: https://etherealwake.com/2021/09/bazel-linkstamp/#genrule-example

That might be the best path forward for bootstrapping these tools.

Curious if others have thoughts though, perhaps I've missed something.