ds300
Support creating patches with Yarn 2+
Attempting to create new patch files with patch-package within a yarn 2+ project currently doesn't work. This has a few small changes to fixes all of them and support modern versions of yarn. Fixes: #496 #272
But first, why bother? Yarn 2 and up has its own built-in option for patching dependencies, so projects using it don't need patch-package anymore, right? On the contrary, patch-package not only has a much better overall user experience, but with the addition of support multiple patch files per package in v8.0.0, patch-package is a far more attractive solution for my needs. In particular, in one project there's a single critical dependency that I have made extensive customizations to over several years, and it's always a big effort to maintain those changes when they are combined into a single patch file. Yes, it's still technically possible to use yarn's native patch system to progressively layer patch files onto a single dependency, but it's a bit nightmarish. I would much prefer to continue using patch-package than jump through those types of hoops.
Now, for the issues I had to resolve to get patch-package with yarn 3 working:
- Problem: Initial tmp directory install fails because
--ignore-enginesoption is not supported- Cause: yarn v3 bails if you pass
--ignore-enginesbecause support for that option is removed - Solution: only pass
--ignore-enginesif yarn version starts with1.
- Cause: yarn v3 bails if you pass
- Problem: Our private repo config isn't being respected during the initial tmp directory install
- Cause:
makePatch.tscopies over any.yarnrcfile it finds, but we use a.yarnrc.ymlfile - Solution: also copy
.yarnrc.ymlfiles into the tmp directory
- Cause:
- Problem: Installs in the tmp directory take a long time, and eventually fail, with lots of error spew about cache mismatch, install state issues, etc
- Cause:
makePatch.tscopies over the entire.yarndirectory which includes a very large project-specific package cache, a state file for the current project's node modules, etc - Solution: only copy two specific subfolders of the
.yarndirectory:.yarn/releaseswhich contains the currently installed version of yarn, and.yarn/pluginswhich contains any additional plugins (in our case the workspace tools plugin was required)
- Cause:
Finally, one additional "fix" that's not quite so objective is included. The error messaging produced by patch-package in the above failure cases was extremely hard to interpret, and doesn't include any sort of stack trace to help identify where the error is even coming from:
/Users/nmannesc/dev/twitch/twilight/node_modules/patch-package/dist/makePatch.js:395
throw e;
^
{
status: 1,
signal: null,
output: [
null,
Buffer(350) [Uint8Array] [
121, 97, 114, 110, 32, 105, 110, 115, 116, 97, 108, 108,
32, 118, 49, 46, 50, 50, 46, 49, 48, 10, 105, 110,
102, 111, 32, 78, 111, 32, 108, 111, 99, 107, 102, 105,
108, 101, 32, 102, 111, 117, 110, 100, 46, 10, 91, 49,
47, 52, 93, 32, 82, 101, 115, 111, 108, 118, 105, 110,
103, 32, 112, 97, 99, 107, 97, 103, 101, 115, 46, 46,
46, 10, 105, 110, 102, 111, 32, 73, 102, 32, 121, 111,
117, 32, 116, 104, 105, 110, 107, 32, 116, 104, 105, 115,
32, 105, 115, 32,
... 250 more items
],
Buffer(2018) [Uint8Array] [
119, 97, 114, 110, 105, 110, 103, 32, 112, 97, 99, 107,
97, 103, 101, 46, 106, 115, 111, 110, 58, 32, 78, 111,
32, 108, 105, 99, 101, 110, 115, 101, 32, 102, 105, 101,
108, 100, 10, 119, 97, 114, 110, 105, 110, 103, 32, 78,
111, 32, 108, 105, 99, 101, 110, 115, 101, 32, 102, 105,
101, 108, 100, 10, 119, 97, 114, 110, 105, 110, 103, 32,
82, 101, 115, 111, 108, 117, 116, 105, 111, 110, 32, 102,
105, 101, 108, 100, 32, 34, 112, 97, 116, 99, 104, 58,
114, 101, 97, 99,
... 1918 more items
]
],
pid: 3504,
stdout: Buffer(350) [Uint8Array] [
121, 97, 114, 110, 32, 105, 110, 115, 116, 97, 108, 108,
32, 118, 49, 46, 50, 50, 46, 49, 48, 10, 105, 110,
102, 111, 32, 78, 111, 32, 108, 111, 99, 107, 102, 105,
108, 101, 32, 102, 111, 117, 110, 100, 46, 10, 91, 49,
47, 52, 93, 32, 82, 101, 115, 111, 108, 118, 105, 110,
103, 32, 112, 97, 99, 107, 97, 103, 101, 115, 46, 46,
46, 10, 105, 110, 102, 111, 32, 73, 102, 32, 121, 111,
117, 32, 116, 104, 105, 110, 107, 32, 116, 104, 105, 115,
32, 105, 115, 32,
... 250 more items
],
stderr: Buffer(2018) [Uint8Array] [
119, 97, 114, 110, 105, 110, 103, 32, 112, 97, 99, 107,
97, 103, 101, 46, 106, 115, 111, 110, 58, 32, 78, 111,
32, 108, 105, 99, 101, 110, 115, 101, 32, 102, 105, 101,
108, 100, 10, 119, 97, 114, 110, 105, 110, 103, 32, 78,
111, 32, 108, 105, 99, 101, 110, 115, 101, 32, 102, 105,
101, 108, 100, 10, 119, 97, 114, 110, 105, 110, 103, 32,
82, 101, 115, 111, 108, 117, 116, 105, 111, 110, 32, 102,
105, 101, 108, 100, 32, 34, 112, 97, 116, 99, 104, 58,
114, 101, 97, 99,
... 1918 more items
],
error: null
}
Node.js v20.5.1
In an attempt to at least get a useful stack trace on screen in error cases of this class, I've edited spanSafe.ts to create a new error object and attach it to the thrown object which provides a clear stack trace to the likely source of issues and it appears as one of the last things printed before the process exits:
/Users/nmannesc/dev/twitch/twilight/node_modules/patch-package/dist/makePatch.js:395
throw e;
^
{
status: 1,
signal: null,
output: [
null,
Buffer(351) [Uint8Array] [
121, 97, 114, 110, 32, 105, 110, 115, 116, 97, 108, 108,
32, 118, 49, 46, 50, 50, 46, 49, 48, 10, 105, 110,
102, 111, 32, 78, 111, 32, 108, 111, 99, 107, 102, 105,
108, 101, 32, 102, 111, 117, 110, 100, 46, 10, 91, 49,
47, 52, 93, 32, 82, 101, 115, 111, 108, 118, 105, 110,
103, 32, 112, 97, 99, 107, 97, 103, 101, 115, 46, 46,
46, 10, 105, 110, 102, 111, 32, 73, 102, 32, 121, 111,
117, 32, 116, 104, 105, 110, 107, 32, 116, 104, 105, 115,
32, 105, 115, 32,
... 251 more items
],
Buffer(2018) [Uint8Array] [
119, 97, 114, 110, 105, 110, 103, 32, 112, 97, 99, 107,
97, 103, 101, 46, 106, 115, 111, 110, 58, 32, 78, 111,
32, 108, 105, 99, 101, 110, 115, 101, 32, 102, 105, 101,
108, 100, 10, 119, 97, 114, 110, 105, 110, 103, 32, 78,
111, 32, 108, 105, 99, 101, 110, 115, 101, 32, 102, 105,
101, 108, 100, 10, 119, 97, 114, 110, 105, 110, 103, 32,
82, 101, 115, 111, 108, 117, 116, 105, 111, 110, 32, 102,
105, 101, 108, 100, 32, 34, 112, 97, 116, 99, 104, 58,
114, 101, 97, 99,
... 1918 more items
]
],
pid: 16211,
stdout: Buffer(351) [Uint8Array] [
121, 97, 114, 110, 32, 105, 110, 115, 116, 97, 108, 108,
32, 118, 49, 46, 50, 50, 46, 49, 48, 10, 105, 110,
102, 111, 32, 78, 111, 32, 108, 111, 99, 107, 102, 105,
108, 101, 32, 102, 111, 117, 110, 100, 46, 10, 91, 49,
47, 52, 93, 32, 82, 101, 115, 111, 108, 118, 105, 110,
103, 32, 112, 97, 99, 107, 97, 103, 101, 115, 46, 46,
46, 10, 105, 110, 102, 111, 32, 73, 102, 32, 121, 111,
117, 32, 116, 104, 105, 110, 107, 32, 116, 104, 105, 115,
32, 105, 115, 32,
... 251 more items
],
stderr: Buffer(2018) [Uint8Array] [
119, 97, 114, 110, 105, 110, 103, 32, 112, 97, 99, 107,
97, 103, 101, 46, 106, 115, 111, 110, 58, 32, 78, 111,
32, 108, 105, 99, 101, 110, 115, 101, 32, 102, 105, 101,
108, 100, 10, 119, 97, 114, 110, 105, 110, 103, 32, 78,
111, 32, 108, 105, 99, 101, 110, 115, 101, 32, 102, 105,
101, 108, 100, 10, 119, 97, 114, 110, 105, 110, 103, 32,
82, 101, 115, 111, 108, 117, 116, 105, 111, 110, 32, 102,
105, 101, 108, 100, 32, 34, 112, 97, 116, 99, 104, 58,
114, 101, 97, 99,
... 1918 more items
],
error: Error: command exited with non-zero status
at Object.spawnSafeSync (/Users/nmannesc/dev/twitch/twilight/node_modules/patch-package/dist/spawnSafe.js:23:32)
at Object.makePatch (/Users/nmannesc/dev/twitch/twilight/node_modules/patch-package/dist/makePatch.js:131:29)
at /Users/nmannesc/dev/twitch/twilight/node_modules/patch-package/dist/index.js:72:25
at Array.forEach (<anonymous>)
at Object.<anonymous> (/Users/nmannesc/dev/twitch/twilight/node_modules/patch-package/dist/index.js:71:22)
at Module._compile (node:internal/modules/cjs/loader:1233:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1287:10)
at Module.load (node:internal/modules/cjs/loader:1091:32)
at Module._load (node:internal/modules/cjs/loader:938:12)
at Module.require (node:internal/modules/cjs/loader:1115:19)
}
Node.js v20.5.1
I haven't familiarized myself enough with the integration test setup in this project to build this PR out into something that I would consider ready to merge, but I think it should be a good starting point. And, hopefully my intro above is a convincing case for why modern yarn versions are still worth supporting in this package.
After using this patch in our internal CI, I believe this PR is quite unfinished, I'll create a new PR based yours. For example, packageManager, --mode=skip-build support, and resolve resolution for the following correctly:
"@backstage/integration@npm:^1.5.0, @backstage/integration@npm:^1.7.0, @backstage/integration@npm:^1.7.2":
version: 1.7.2
resolution: "@backstage/integration@npm:1.7.2"
@noahm See #507
