ts_project with webpack devserver?

[chortis]

I am trying to configure a ts_project target into a webpack dev server, however due to ts_project deleting the output before it runs tsc, webpack will throw a ton of errors as it watches the delete and, then when tsc is run, it re-compiles successfully.

There are no examples of ts_project being used with a dev server that I can find (all use ts_library or no Typescript), so attempting to resolve these issues has proven quite a challenge. It is quite slow and a pretty poor developer experience.

Bazel and webpack however without the devserver compile successfully.

Here is the Bazel config:

ts_config(
    name = "project-tsconfig",
    src = "tsconfig.json",
    deps = ["//:tsconfig.json"] // a global ts config we extend from
)

ts_project(
   name = "build",
   srcs = glob(["src/**/*.ts"]),
   tsconfig = ":project-tsconfig",
   declaration = True,
   source_map = True
)

filegroup(
    name = "public",
    srcs = glob(
        include = [
            "public/**/*",
        ]
    )
)

webpack(
    name = "serve",
    data = [
        "@npm//html-webpack-plugin",
        "@npm//webpack-dev-server",
        "@npm//webpack-cli",
        ":build",
        ":public",
        "webpack.config.js"
    ],
    args = [
        "serve",
        "--config",
        "$(execpath webpack.config.js)"
    ],
    tags = [
        "ibazel_notify_changes",
        "nobazel_node_patches"
    ]
)

Here is the webpack config

module.exports = {
    entry: path.resolve(__dirname, 'src/index'),
    mode: 'development',
    cache: false,
    target: 'web',
    output: {
        filename: 'index.js',
        path: path.resolve(__dirname, 'public/js')
    },
    resolve: {
        symlinks: false
    },
    plugins: [
        new HtmlWebpackPlugin({
            inject: true,
            template: path.resolve(__dirname, 'public/index.html'),
            filename: 'index.html'
        })
    ],
    devServer: {
        historyApiFallback: true,
        host: 'localhost',
        open: true,
        port: 3102,
        liveReload: true,
        hot: false,
        static: [
            path.resolve(__dirname, 'public'),
        ]
    },
    watchOptions: {
        // Follow symlinks out of the sandbox & execroot so that
        // it can detect changes to the underlying files. Watching the symlinks
        // doesn't work since no change notifications is generated for a symlink
        // when the file it points to changes.
        // NB: this option only works under bazel with node patches disabled with --nobazel_node_patches
        followSymlinks: true,
        ignored: ['**/node_modules/**', '**/node_modules'],
        // Necessary to debounce the io from ibazel constantly writing
        aggregateTimeout: 2000,
        // Necessary for live reload to be able to receive the changes
        poll: 1000
    }
};

If anyone knows if this is possible, or any other ideas that would be good. Converting our project to ts_library would present it's own challenges. We have several packages already set up via ts_project that is working well.

[alexeagle]

As far as I know, there is no way to configure webpack's built-in file watcher to avoid it noticing files getting unlinked (deleted) while Bazel is updating the bazel-out tree. Instead you always have to have an external trigger that tells webpack when the output tree has stabilized and only then should it see changes. This requires writing a small webpack extension that understands the ibazel protocol to know it's time to reload.

@mrmeku is the expert on this and maybe already has something to show? @mistic is working on it as well IIUC

[a-ignatov-parc]

As an alternative that can be useful to you, I would like to share our internal setup that we use to get the best DX from bazel and webpack:

1. We wrote a simple webpack rule that collects TS source files with a bazel aspect and passes them as input to the webpack dev server.
2. Webpack is configured to transpile files using ts-loader to use the same tsc version as ts_project.

With this setup, we do not need to wait for type checking (they are performed in the editor during work) + fast HMR and no errors on files deletion, since the source files are deleted only by the developer.

Additional information:

• For webpack prod builds we also collect output from ts_project to initiate type checks and verify that everything is correct.
• webpack rule can also collect modules mapping from rules_nodejs targets and provide information to configure webpack alias if necessary.

[chortis]

@a-ignatov-parc Thanks very much!

Yeah I just implemented something like this and it works. However can you clarify the Webpack rule you wrote with the bazel aspect? I simply passed in absolute paths instead of relative paths.

I loosely followed this MR usingts-loader to get live reload to pick up the changes.

https://github.com/bazelbuild/rules_nodejs/pull/2431

[a-ignatov-parc]

However can you clarify the Webpack rule you wrote with the bazel aspect? I simply passed in absolute paths instead of relative paths.

If I understand you correctly and you are talking about absolute imports with workspace names, then this should work fine for simple cases. In our case, we have a lot of ts_project and ts_library from external workspaces that use module_name/package_name, and their imports may not map directly to workspace names. In such cases, we have to use the module_mappings_runtime_aspect aspect of rules_nodejs to collect the module mappings, which is used to set up aliases in webpack. Also, due to the presence of many different packages with dependencies, we need to collect transitive source and data files, which, as far as I know, can only be done using the bazel rule with aspects.

If none of this moments is required for you, then you should not worry about rules and aspects ;)

[mistic]

@chortis what @alexeagle wrote above is true. We need to pause the webpack compilation before ibazel notifies us we can resume it again. A few weeks ago I worked on a prototype of this with @mrmeku. I think he might be already working to finish it, in case he is not, I can try to to pickup that work and finish it by the end of the month or early next month.

[chortis]

@chortis what @alexeagle wrote above is true. We need to pause the webpack compilation before ibazel notifies us we can resume it again. A few weeks ago I worked on a prototype of this with @mrmeku. I think he might be already working to finish it, in case he is not, I can try to to pickup that work and finish it by the end of the month or early next month.

Yeah ok thanks for your help! I wouldn't say it's something that needs to be rushed. We can write a quick webpack plugin on our side as well to handle that as an interim solution.

[thesayyn]

We've done some work integrating webpack into bazel. That even includes making webpack-devserver work properly under bazel. See #2919

As far as I know, there is no way to configure webpack's built-in file watcher to avoid it noticing files getting unlinked (deleted) while Bazel is updating the bazel-out tree.

with the @bazel/webpack package, this is not the case anymore. In the new rule, webpack is aware that it is running under bazel, and does not do any work until bazel gives it a green light. When bazel gives the green light, it knows which files have changed and builds only those files hence we get incremental builds. This will be the case with webpack-dev-server as well.

[xnerhu]

@thesayyn Would there be any problems with using babel-loader with typescript preset instead of ts_project?

[thesayyn]

Theoretically there should not be any problem. You should be able to use it as you would outside of bazel. We will add examples for widely known plugins eventually.

[a-ignatov-parc]

Would there be any problems with using babel-loader with typescript preset instead of ts_project?

Sorry for jumping into your conversation, but I would like to share an important issue that everyone should consider when deciding how to transpile TS source files. The problem is the build reproducibility. Transpiling files from TS to JS using tsc, babel-preset-typescript, esbuild and swc might have difference because all these tools have different release lifecycle. Sometime the difference is not important, but sometime it can break your production, and the team should be prepared for situations where typescript released a new version with features/fixes which are not yet supported by other tools. Yes, by using esbuild we can significantly decrease the build time, but it comes at a price. For example, good test coverage, otherwise it is almost impossible to be sure that if the type check is passed, there will be no problems with release artifacts.

[xnerhu]

Would there be any problems with using babel-loader with typescript preset instead of ts_project?

Sorry for jumping into your conversation, but I would like to share an important issue that everyone should consider when deciding how to transpile TS source files. The problem is the build reproducibility. Transpiling files from TS to JS using tsc, babel-preset-typescript, esbuild and swc might have difference because all these tools have different release lifecycle. Sometime the difference is not important, but sometime it can break your production, and the team should be prepared for situations where typescript released a new version with features/fixes which are not yet supported by other tools. Yes, by using esbuild we can significantly decrease the build time, but it comes at a price. For example, good test coverage, otherwise it is almost impossible to be sure that if the type check is passed, there will be no problems with release artifacts.

Babel doesn't type-check, but for the tests https://www.npmjs.com/package/fork-ts-checker-webpack-plugin could be used.

[a-ignatov-parc]

@xnerhu sorry if I expressed my idea in a complex way. I'll try to explain it with an example.

Please imagine we are writing our code in TS and using [email protected] for type checking. We use either our editor, ts_project or the suggested fork-ts-checker-webpack-plugin to perform checks. We are using esbuild to compile our code.

Now let's say we wrote some pretty standard TS code:

new Foo!.Bar();

Type checking (no matter which utility we use to do this) will tell us that this code is valid and there are no errors.

We then compile it and deploy it to production and get a runtime error on that particular line of code. After examining the problem, we understand that esbuild did not compile this code correctly, and in order to solve the problem, we must wait until it is fixed in esbuild. Btw, this problem was fixed 8 days ago.

The main idea that I was trying to explain is that if we use tools other than tsc to compile the code, the possibility that we will run into similar problems increases, and we must take this into account.

[chortis]

We've done some work integrating webpack into bazel. That even includes making webpack-devserver work properly under bazel. See #2919

As far as I know, there is no way to configure webpack's built-in file watcher to avoid it noticing files getting unlinked (deleted) while Bazel is updating the bazel-out tree.

with the @bazel/webpack package, this is not the case anymore. In the new rule, webpack is aware that it is running under bazel, and does not do any work until bazel gives it a green light. When bazel gives the green light, it knows which files have changed and builds only those files hence we get incremental builds. This will be the case with webpack-dev-server as well.

@thesayyn This looks great. And will definitely be helpful.

One of the issues I am also seeing with webpack is using a ts_project output and piping it into a webpack production build. It appears the output isn't including the ts_project output into the sandbox, thus it can't find the entry points. I notice the example for react_webpack uses tsc,but I would expect it to work with ts_project as well since the output is the same and is recommended.

Below outlines what I'm seeing.

webpack(
   name = 'production', 
   outs = ["build/index.js"],
   data = [
       ":public_files",
       ":ts_project_output", // the compiles the tsc into bin, but it doesn't appear in the sandbox where webpack actually runs to produce the output
       "webpack.config.prod.js"
   ]
)

[jakelauer]

It looks like #2919 was closed without being merged - is this still a thing? We're in the midst of trying to do the same, except with ts_library instead of ts_project to integrate with webpack-dev-server.

[thesayyn]

we are on a path where we reduce the scope of rules_nodejs hence we created webpack rules under See: https://github.com/aspect-build/rules_webpack (attention to the readme)

[jakelauer]

@thesayyn Rad, thanks

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity for 6 months. It will be closed if no further activity occurs in 30 days. Collaborators can add a "cleanup" or "need: discussion" label to keep it open indefinitely. Thanks for your contributions to rules_nodejs!

[github-actions[bot]]

This issue was automatically closed because it went 30 days without any activity since it was labeled "Can Close?"