rules_js
rules_js copied to clipboard
[Bug]: Perf: Slow postprocessing.run action after `js_image_layer`'s entries write
What happened?
When using js_image_layer on js_binary targets with many inputs (100k+) we're observing very slow postprocessing.run events in the Bazel profiles. These occur every-time, also when fully cached.
The snips are from a CI linux runner profile.
Version
Development (host) and target OS/architectures: macOS & k8 + Engflow remote cache
Output of bazel --version: release 7.4.1
Version of the Aspect rules, or other relevant rules from your
WORKSPACE or MODULE.bazel file: rules_js 2.8.1
Language(s) and/or frameworks involved: JS
How to reproduce
The entries.json generated is usually over 150MB with more than 500k lines in some of our services.
So the js_binary should depend on node modules with many files.
Any other information?
No response
Notes:
- the slowest part is most likely the directory expansion introduced in https://github.com/aspect-build/rules_js/pull/2217 (unconfirmed)
- something needs to expand the directories, https://github.com/aspect-build/rules_js/pull/2217 is probably a lot faster then something like node doing it
To investigate:
- does the slow perf only occur when the analysis cache is busted?
- can we find an alternate way of detecting changes to directories? maybe only for
NpmPackageInfoproviders?