buchgr
Blake3 support
Bazel now supports blake3 as a hashing algorithm. It'd be great if we could start using it with bazel-remote as it promises impressive perfomance boosts
The main blocker to adding support for BLAKE3 is that we need to consider if an update to the cache directory format is required, since the current format does not explicitly say what digest function the blobs are stored in.
If it is safe to assume that we will never encounter digest collisions between SHA256 and BLAKE3, then maybe we can keep the current format unchanged. If that's the case then maybe it's OK for bazel-remote to not care what the format of on-disk blobs are.
(Aside: I think someone mentioned that BLAKE3 is only fast on amd64. Something to bear in mind.)
I think it's a safe assumption that BLAKE3 and SHA256 would not have a digest collision.
However, I think there are some assumptions that sha256 is the only hashing algorithm for bazel. Example: https://github.com/buchgr/bazel-remote/blob/0fd6a4470bdb879730a519be6fefd79193894239/server/grpc.go#L138
However, there are new fields in the protos that explain which digest function is used, so it would be trivial to add the appropriate paths for BLAKE3.
https://github.com/bazelbuild/remote-apis/blob/6c32c3b917cc5d3cfee680c03179d7552832bb3f/build/bazel/remote/execution/v2/remote_execution.proto#L1555
I tried using blake3 with bazel-remote and it works fine when I disable remote cache compression. To be precise, i needed to set storage mode to 'uncompressed' in bazel-remote. However, if I set it to zstd than it stops working.
