Bug report

What is the current behavior? d.ts files are not emitted after removing the output folder and running webpack again. I use ts-loader and have set declaration: true and outDir: 'dist' in tsconfig.json.

If the current behavior is a bug, please provide the steps to reproduce.

Run webpack build for the first time - bundle.js and typings are emitted. Remove 'dist' folder and run webpack again.

webpack.config.js:

{
  mode: isDevelopment ? 'development' : 'production',
  entry: {
    main: ['./src/index.tsx'],
  },
  output: {
    filename: 'bundle.js',
    path: path.resolve(__dirname, 'dist'),
    publicPath: '/',
  },
  cache: {
    type: 'filesystem',
    buildDependencies: {
      sources: [path.resolve(process.cwd(), 'src') + '/'],
      tsConfig: [path.resolve(process.cwd(), 'tsconfig.json')],
      config: [__filename]
    },
  },
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        include: path.join(__dirname, 'src'),
        use: {
          loader: 'ts-loader'
        }
      },
      {
        test: /\.jsx?$/,
        include: path.join(__dirname, 'src'),
        use: 'babel-loader',
      },
    ],
  },
  resolve: {
    extensions: ['.ts', '.tsx', '.js', '.jsx'],
  },
};

What is the expected behavior? I expect idempotent behavior - set of files emitted by webpack build is the same regardless if the cache is utilized or not.

Other relevant information: webpack version: 5.37.1 Node.js version: 14.15.4 Operating System: Ubuntu 20.04 Additional tools: ts-loader: 9.2.2, typescript: 4.0.3

This issue was moved from webpack/webpack#13465 by @alexander-akait. Original issue was by @MBelniak.

May 28 '21 11:05 webpack-bot

@johnnyreilly I think it will be great thing, cache will help with big projects very good, do you need help? I want to look at this in the next week (example usage of cache here https://github.com/webpack-contrib/terser-webpack-plugin/blob/master/src/index.js#L214, do not look at logic for extract comments)

May 28 '21 11:05 alexander-akait

If you'd like to look at this I'd be happy to look at the PR!

May 28 '21 12:05 johnnyreilly

@alexander-akait @johnnyreilly Hi guys, any update on this?

Jun 29 '21 09:06 MBelniak

I'm not working on this, but I'm happy to review PRs

Jun 29 '21 10:06 johnnyreilly

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Apr 17 '22 07:04 stale[bot]

Closing as stale. Please reopen if you'd like to work on this further.

Apr 28 '22 13:04 stale[bot]

As a workaround, it is possible to use fork-ts-checker-webpack-plugin alongside ts-loader and it supports typings caching

May 31 '22 09:05 vsternbach

Is there any news in this direction?

Sep 05 '22 15:09 yslpn

@johnnyreilly Maybe will be great to reopen, because it can be still useful

Sep 05 '22 16:09 alexander-akait

Reopened - if anyone would like to work on it that's great. I'm not planning to myself, so we'll need someone to step up if they want this to progress

Sep 05 '22 18:09 johnnyreilly

Hey there, in my company we're using TS-Loader in multiple Webpack packages, including some in a monorepo, and we wanted to implement Webpack Cache, but faced this issue where we hit the cache but don't get the typings as they're not part of Webpack's tracked output files. I would be really interested in contributing but would need heavy guidance at first as I don't know Webpack's source code / plugin API.

Mar 29 '23 13:03 a-coruble

@alexander-akait would you be able to advise?

Mar 29 '23 14:03 johnnyreilly

@johnnyreilly Yeah, can you provide a link on code where we generate them?

Mar 29 '23 14:03 alexander-akait

I don't know where they are generated - I thought you might!

Mar 29 '23 15:03 johnnyreilly

Any hints by where I could start learning / deep-diving in your codebase even if it's not the link to the part responsible for type generation. I would really like to contribute here 😄

Apr 04 '23 07:04 a-coruble

@johnnyreilly I mean where ts-loader generate types and emit them :smile: We need to add cache logic here, so I want to look how it is implemented right now and I will gave example/good advice how we can improve it, ideally what I want to see

Input (how and where it is)
Output (how and where it is)

Apr 05 '23 14:04 alexander-akait

@johnnyreilly @alexander-akait any news on that side? I tried to find where types get generated in the loader, but didn't find it by myself 😅

Apr 19 '23 08:04 a-coruble

I think that this is where types are supplied to webpack (output): https://github.com/TypeStrong/ts-loader/blob/5fbfb5c97d6d18ad285d300fbfaecbafe1189ed8/src/after-compile.ts#L75

And likewise any other references to the same method I think.

I think input will be the same as any other webpack loader - webpack supplies the file contents for processing

Apr 19 '23 09:04 johnnyreilly

Thank you, I will investigate it soon (today/tomorrow) and will send a PR (or ask more infromation)

Apr 19 '23 15:04 alexander-akait

Hey there, allowing myself to ask for an update on this, or as usual, some guidance to implement it myself?

May 11 '23 13:05 a-coruble

@johnnyreilly Okay, I looked at code and found that typescript emits these files itself, not ts-loader, what we can:

add these files to dependecies, but if you remove them ts-loader will run again, so we won't get any advantage here
say typescript doesn't emit them and emit them itself (in ts-loader), but it is tricky, because logic is deeply inside typescript (maybe you know how to achive it, I am not good in typescript API)
say, it is a limitation, but it mean if you enable output.clean you will lose these files.

Another big question - why developers have "declaration": true, "outDir": "dist", for ts-loader, may I am missing something :confused: , but that is a point to do it? When you build library you have typescript declarations and compiled files to JS (by typescript, no webpack here), when you build an application you don't need d.ts files inside the dist directory (why have them?).

May 16 '23 14:05 alexander-akait

We have this https://github.com/TypeStrong/ts-loader#declaration-files-dts

To output declaration files (.d.ts), you can set "declaration": true in your tsconfig and set "transpileOnly" to false.

If you use ts-loader with "transpileOnly": true along with fork-ts-checker-webpack-plugin, you will need to configure fork-ts-checker-webpack-plugin to output definition files, you can learn more on the plugin's documentation page: https://github.com/TypeStrong/fork-ts-checker-webpack-plugin#typescript-options

So I think the second point is more valid than the first, I will continue to looking for a way how to solve it, but not sure it is really possible

May 16 '23 14:05 alexander-akait

Anyway I found a place the problem - https://github.com/TypeStrong/ts-loader/blob/main/src/after-compile.ts#L29, shorty - when cache enabled, loader won't start (because why? it is only wasting time), so hooks will not registered and so nothing will be emit.

@johnnyreilly Why we have a such design? It is not valid accroding webpack logic, everything inside loader should be only inside loader, we have this.emitAsset and this.emitError and other methods to achive it. We need to remove it, i.e. rewrite to build-in loader API this and cache will work out of box without any problems

May 16 '23 15:05 alexander-akait

I tried to rewrite it, but some of tests are failed, so I need a developer who written it and we need to rewrite it, also it allows to improve perf very well when cache is enabled

May 16 '23 15:05 alexander-akait

The same asnwer https://github.com/vercel/ncc/issues/208#issuecomment-452275518 and this issue can be union with https://github.com/TypeStrong/ts-loader/issues/894, I am afraid without @johnnyreilly your help I can't rewrite it without problems and regressions

May 16 '23 15:05 alexander-akait

Also possible improvements:

use webpack built-in logger, so everything can be in stats
use a new API for resolver, so developers don't need to add this:

resolve: {
    // Add `.ts` and `.tsx` as a resolvable extension.
    extensions: [".ts", ".tsx", ".js"],
    // Add support for TypeScripts fully qualified ESM imports.
    extensionAlias: {
     ".js": [".js", ".ts"],
     ".cjs": [".cjs", ".cts"],
     ".mjs": [".mjs", ".mts"]
    }
  },

also it allows to custmozire resolver logic on module level

May 16 '23 15:05 alexander-akait

@johnnyreilly Why we have a such design? It is not valid accroding webpack logic, everything inside loader should be only inside loader, we have this.emitAsset and this.emitError and other methods to achive it.

To be honest, I can't remember anymore - ts-loader has been in my hands for 7 years and this is information I've now forgotten

The nature of our test pack is that it's quite brittle - certainly the comparison test part

I'm happy to review a rewrite, but I don't plan to do one myself.

May 16 '23 16:05 johnnyreilly

@johnnyreilly at the very least we need to research and provide a roadmap to rewrite it (step by step) otherwise no one will be able to solve it and we will accumulate problemsm without you I can't do it, because I don't know why this code exists. We can start with easy - just add comments to each functions https://github.com/TypeStrong/ts-loader/blob/main/src/after-compile.ts#L36 here

May 16 '23 17:05 alexander-akait

We can start with easy - just add comments to each functions https://github.com/TypeStrong/ts-loader/blob/main/src/after-compile.ts#L36 here

I'm not sure I know what you mean. Which functions? What comments would you like?

May 16 '23 17:05 johnnyreilly

@johnnyreilly I mean like https://github.com/TypeStrong/ts-loader/blob/main/src/after-compile.ts#L50, https://github.com/TypeStrong/ts-loader/blob/main/src/after-compile.ts#L67 and https://github.com/TypeStrong/ts-loader/blob/main/src/after-compile.ts#L74 and etc, shortly - we need to rewrite all of them and put logic inside loader, not in the hook, so I need to undestand why it here and what it is doing

May 16 '23 18:05 alexander-akait

Webpack 5 cache - typings not cached

Bug report