Support using a `fs.FS` for input and output files

[xremming]

I am building a deep integration with ESBuild to my Go application. Due to how it works, some of the files are embedded, and some are on the filesystem, and all access to files happen through the fs.FS interface.

In essence, I would like to be able to pass an fs.FS interface to all of the different methods and functions and for ESBuild to use it for all file accesses. Another separate fs.FS should be available to be passed for the output (as the embed.FS is read-only).

It should be a relatively straightforward change, but as I don't know the codebase, I cannot be sure.

[m13t]

I agree this would be a nice feature. However you can achieve this using plugins, at least for the inputs. See the example http plugin here: https://esbuild.github.io/plugins/#http-plugin

The outputs are stored in a byte slice regardless of whether they're written to disk or not, so you could fairly easily take the output and wrap them in a struct with receivers to implement the fs.FS interface.

[xremming]

Interesting, I didn't know the plugin system is this powerful. Will have to take a look at this. So far I have solved this by just copying files to a temporary directory and pointing esbuild there (which also works well enough).

[pablochacin]

@xremming I'm facing a similar problem. And started working on a file plugin, as @m13t'ssuggests.

However, a test with an early prototype brought an issue: esbuild is resolving the relative path specified in the Stdin options as an absolute path with respect to the current working directory and passes this path to the plugin.

Did you face this same issue? How did you solve it? Thanks in advance.

[firelizzard18]

I agree this would be a nice feature. However you can achieve this using plugins, at least for the inputs.

@m13t I don't think this is feasible for the general case. esbuild's resolution logic is rather complex and AFAIK if you tried to simulate fs.FS with a plugin like this you would have to reimplement all of that.

For example, say I have an fs.FS that contains node_modules. Resolving a node module import can get rather complex. With a plugin I can use OnResolve and OnResolve to load from the fs.FS, but that disables esbuild's resolution entirely so I have to do that all manually even down to index => index.js.

[firelizzard18]

@xremming fs.FS does not support writing. To be able to pass in a writable FS abstraction, esbuild would have to define its own interface. See https://github.com/golang/go/issues/45757 for more discussion.

[xremming]

Sorry I missed your comment @pablochacin. I was able to make it work with some tricks of copying files to a temporary directory and doing the build there, instead of using the plugin system for it. I seem to have the source somewhere on a different machine perhaps. Once I someday get back to it and finish it, it will be open-source in my xremming/esox project. I just haven't had the time to continue working on it in a while.

In essence:

1. Create a directory like ./.esox/build_${BUILD_ID}.
2. Run a "collect assets" step that outputs all of the files that need to be built to that directory (in my case this first build the go app, and then runs it with some special sauce that causes it to write all of the raw assets in the directory, and there is a way for packages to "register" resources).
3. Run esbuild in the build directory.
4. Copy the built files and the manifest file to a predefined directory of the app.
5. Rebuild the go app (which itself uses fs.FS with the //go:embed feature).

I believe I also had a thing there which made it so that when the app was started in dev mode, it would do this same process itself automatically (on every request?), except obviously the files would be read from the build directory directly.

A naive implementation will end up having some of the files multiple times in the final binary, a smarter implementation would use build tags to exclude the duplicate embeds when building the final binary.

Making a plugin that causes imports to also try to read from an fs.FS isn't really that hard but the dealbreaker for me was that I want my binary to be deployable as-is (without any additional files) but I also want to use packages from node_modules and these two are in conflict with each other.

---

Getting support for fs.FS for input files would be still great! I hope the #4200 PR does get merged in the future.

[pablochacin]

Hi @xremming

I actually implemented the plugin and as you mentioned, if is fairly straightforward, except for a little complexity introduce by esbuild expecting an absolute path as work dir, while fs uses always relative paths (IMHO a questionable design decision), so I had to create some glue code to move paths from/to fs.

[xremming]

I believe I also had to build a wrapper around the embedded fs.FS because of the relative/absolute path difference from the non-embed version. I also anyway ended up needing a way of building a single fs.FS from multiple ones.