repository_ctx: Allow renaming archive entries during extraction.

[jmillikin]

Adds a rename_files= parameter to the extract() and download_and_extract() methods of repository_ctx. This new parameter takes a dict of archive entries to rename, and is applied prior to any prefix adjustment.

[jmillikin]

Use cases include:

• Archives where multiple files have the same name in different case. On platforms with case-insensitive filesystems (macOS, Windows) these archives can't be extracted as-is, and the files can't be fixed up after extraction.
  • One such project is the Linux kernel, which has files named like net/netfilter/xt_dscp.c and net/netfilter/xt_DSCP.c (among others).
• Archives containing non-UTF8 filenames fail to extract on platforms or filesystems that enforce Unicode paths.
  • See https://github.com/bazelbuild/rules_go/issues/2771 (Go toolchains from golang.org fail to extract)

A longer example of extracting the Linux kernel on macOS (which is case-insensitive by default):

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

def _linux_repository(ctx):
    ctx.file("WORKSPACE", "workspace(name = {name})\n".format(
        name = repr(ctx.name),
    ))
    ctx.file("BUILD.bazel", """
genrule(
    name = "nf_srcs",
    srcs = glob(["net/netfilter/*"]),
    outs = ["nf_srcs.txt"],
    cmd = "ls -1 external/linux/net/netfilter/ > $@",
)
""")

    ctx.download_and_extract(
        url = ["https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.19.tar.xz"],
        sha256 = "ff240c579b9ee1affc318917de07394fc1c3bb49dac25ec1287370c2e15005a8",
        stripPrefix = "linux-5.19",
        rename_files = _rename_files("5.19"),
    )

def _rename_files(version):
    netfilter_srcs = {
        "xt_DSCP.c": "xt_DSCP_target.c",
        "xt_HL.c": "xt_HL_target.c",
        "xt_RATEEST.c": "xt_RATEEST_target.c",
        "xt_TCPMSS.c": "xt_TCPMSS_target.c",
    }
    renames = {}
    for orig_name, new_name in netfilter_srcs.items():
        prefix = "linux-{}/net/netfilter/".format(version)
        renames[prefix + orig_name] = prefix + new_name
    # and similar logic for netfilter headers under include/uapi/linux/
    return renames

linux_repository = repository_rule(
    implementation = _linux_repository,
)

$ tar -tf ~/downloads/linux-5.19.tar.xz | grep -i xt_dscp.c
linux-5.19/net/netfilter/xt_DSCP.c
linux-5.19/net/netfilter/xt_dscp.c
$ bazel build @linux//:nf_srcs
[...]
Target @linux//:nf_srcs up-to-date:
  bazel-bin/external/linux/nf_srcs.txt
$ cat bazel-bin/external/linux/nf_srcs.txt | grep -i xt_dscp
xt_DSCP_target.c
xt_dscp.c

[jmillikin]

Gentle ping