rules_js icon indicating copy to clipboard operation
rules_js copied to clipboard

[Bug]: Perf: Slow postprocessing.run action after `js_image_layer`'s entries write

Open stoiky opened this issue 5 months ago • 1 comments

What happened?

When using js_image_layer on js_binary targets with many inputs (100k+) we're observing very slow postprocessing.run events in the Bazel profiles. These occur every-time, also when fully cached.

The snips are from a CI linux runner profile. Image Image

Version

Development (host) and target OS/architectures: macOS & k8 + Engflow remote cache

Output of bazel --version: release 7.4.1

Version of the Aspect rules, or other relevant rules from your WORKSPACE or MODULE.bazel file: rules_js 2.8.1

Language(s) and/or frameworks involved: JS

How to reproduce

The entries.json generated is usually over 150MB with more than 500k lines in some of our services. 

So the js_binary should depend on node modules with many files.

Any other information?

No response

stoiky avatar Nov 26 '25 07:11 stoiky

Notes:

  • the slowest part is most likely the directory expansion introduced in https://github.com/aspect-build/rules_js/pull/2217 (unconfirmed)
  • something needs to expand the directories, https://github.com/aspect-build/rules_js/pull/2217 is probably a lot faster then something like node doing it

To investigate:

  • does the slow perf only occur when the analysis cache is busted?
  • can we find an alternate way of detecting changes to directories? maybe only for NpmPackageInfo providers?

jbedard avatar Nov 26 '25 22:11 jbedard