shipitjs
Release in wrong directory after tar extract
Hi!
I noticed that --strip-components has been removed as an argument to the tar command upon extraction. It seems to have been removed in this commit:
https://github.com/shipitjs/shipit/pull/260/commits/3cf7454dd46ec9fdddf98d85ce40229e2d2d52c2
This broke shipit-deploy deployments (in my case) as the leading directory is no longer being stripped. What was previously extracted to src/... is now being extracted to tmp-12../src/..
Log after 5.3.0:
Running "cd /home/.../releases/20200323213903 && tar -xzf tmp-42Xp73RZdZvYq3.tar.gz" on host "ssh.example.com".
@ssh.example.com-err tar: tmp-42ulD5Bg9qK4yf/.gitignore: time stamp 2020-03-23 22:39:00 is 40.005396223 s in the future
...
@ssh.example.com-err npm WARN saveError ENOENT: no such file or directory, open '/home/..../releases/20200323213903/package.json'
It's looking for package.json in the correct directory, but package.json is now located in tmp-242../package.json.
Log pre 5.3.0:
Running "cd /home/.../releases/20200317222927 && tar --strip-components=1 -xzf tmp-41PrwQtteALZ5p.tar.gz" on host "ssh.example.com".
@ssh.example.com-err tar: .gitignore: time stamp 2020-03-17 23:29:24 is 36.715125466 s in the future
@ssh.example.com-err tar: src/index.ts: time stamp 2020-03-17 23:29:24 is 36.713136291 s in the future
shipitfile.js
module.exports = (shipit) => {
require('shipit-deploy')(shipit);
shipit.initConfig({
default: {
deployTo: '/home/...',
repositoryUrl: '[email protected]:...',
ignores: ['.git', 'node_modules'],
keepReleases: 3,
keepWorkspace: false,
deleteOnRollback: false,
shallowClone: true,
},
production: {
servers: '[email protected]:2222',
deployTo: '/home/...',
branch: 'master',
},
});
shipit.blTask('install', () =>
shipit.remote(`cd ${shipit.releasePath} && npm install`));
shipit.on('updated', () => shipit.start('install'));
};
My apologies if this is an issue of misconfiguration from my side! My configuration was working as of the previous release.
I do have the exact same problem - my shipit.releasePath now looks like this:
-rw-rw-r-- 1 usr usr 27 Apr 1 16:15 package-lock.json
-rw-rw-r-- 1 usr usr 41 Apr 1 16:15 REVISION
drwxr-xr-x 4 usr usr 4096 Apr 1 16:14 tmp-31663xXIxG9Ls1APQ
Sure, i could do that. Though, there's probably a reason, that @oleg-andreyev removed the --strip-components=1 from each tar command!?
@schwarmco I had some issues with unit tests, they simply were not working with --strip-components=1
I'll try to review my PR and remember the exact reason.
I've returned --strip-components=1 and now single test is failing
Summary of all failing tests
FAIL packages/ssh-pool/tests/integration.test.js (15.512s)
● ssh-pool › should copy to remote
Command failed: ssh -i /Users/oandreyev/Development/shipit/ssh/id_rsa [email protected] "cd ./ && cat test.1587159277564.txt"
cat: test.1587159277564.txt: No such file or directory
it seems when is running this test it does not extract files (see below)
deploy@shipit-test:~$ tar --strip-components=1 -xzf foo.tar.gz
deploy@shipit-test:~$ ls -la
total 48
drwxr-xr-x 5 deploy deploy 4096 Apr 17 22:08 .
drwxr-xr-x 3 root root 4096 Sep 3 2017 ..
-rw------- 1 deploy deploy 1540 Mar 10 21:44 .bash_history
-rw-r--r-- 1 deploy deploy 220 Sep 3 2017 .bash_logout
-rw-r--r-- 1 deploy deploy 3771 Sep 3 2017 .bashrc
drwx------ 2 deploy deploy 4096 Sep 3 2017 .cache
-rw-r--r-- 1 deploy deploy 0 Sep 3 2017 .cloud-locale-test.skip
-rw------- 1 deploy deploy 41 Mar 10 21:18 .lesshst
-rw-r--r-- 1 deploy deploy 655 Sep 3 2017 .profile
drwxrwxr-x 2 deploy deploy 4096 Sep 3 2017 .ssh
-rw------- 1 deploy deploy 642 Sep 3 2017 .viminfo
-rw-r--r-- 1 deploy deploy 141 Apr 17 22:05 foo.tar.gz
drwxrwxr-x 2 deploy deploy 4096 Apr 17 21:49 tmp
deploy@shipit-test:~$
but --strip-components=1 is remove
deploy@shipit-test:~$ tar -xzf foo.tar.gz
deploy@shipit-test:~$ ls
foo.tar.gz test.1587161041444.txt tmp
deploy@shipit-test:~$
It seems that scpCopyToRemote works okay performing a copy of a folder/, but it does not work okay with a single file.
I had this same problem. I was on 5.3.0. I downgraded and deploy one version at a time until it went away. The first version that correctly un tared the contents was 4.1.4. I wasn't able to install 4.2.0 because it wasn't available in npm.
https://github.com/shipitjs/shipit/compare/v4.1.4..v5.3.0
I do have the exact same problem - my
shipit.releasePathnow looks like this:-rw-rw-r-- 1 usr usr 27 Apr 1 16:15 package-lock.json -rw-rw-r-- 1 usr usr 41 Apr 1 16:15 REVISION drwxr-xr-x 4 usr usr 4096 Apr 1 16:14 tmp-31663xXIxG9Ls1APQ
me too, my releasepath look like that also. how do you fix that ? i have downgrade to 4.1.4 but doesnt solve my problem
I have the same issue with 5.3.0
In the meantime I just edited my local node_modules/ssh-pool/lib/commands/tar.js file and added back args = [...args, '--strip-components=1'] on line 39 so deployments continue to work.
I've created #279, which is showing that --strip-components=1 was initially wrong.
When you create a tar from a folder, this folder should be located in tar, and later when you extract it this folder must be present.
I and @gregberge made a mistake (B/C) by removing --strip-components=1 in the minor version.
As a possible solution to return --strip-components=1 in 5.3.1, and remove it in 6.0.0
I've monkey patched like this:
shipit.blTask('fix-tar', () => {
shipit.pool.connections.forEach((connection) => {
const { runLocally } = connection;
connection.runLocally = function(cmd, options) {
if (cmd.includes('tar -xzf')) {
cmd = cmd.replace('.tar.gz', '.tar.gz --strip-components=1');
}
return runLocally.call(this, cmd, options);
}
});
});
shipit.on('deploy', () => {
return shipit.start(['fix-tar']);
});
But I have a bad feeling about Shipit when I see such a critical bug unfixed for so long :(
