RemoteSSH hangs at checking "/.vscode-server/XXXX.log and /.vscode-server/XXXX.pid" for a running server

[haochenz96]

VS-Code RemoteSSH was working perfectly on my remote server until recently, when this issue started happening occasionally and getting back to normal after a couple of days (probably the previous installation/setting up processes timed out). But this time, it has not got back to normal after a a long while.

I looked through all the issues related to this, and seems like hanging at the "checking XXX.log and XXX.pid" issue has never been solved. I tried deleting everything and changing all combinations of settings (Use Floc, Use Local Server etc.) and nothing worked. Command line SSH login was fine too.

This issue seems to be related to RemoteSSH not exiting properly from previous usage on this specific server, since it never happens when I SSH into a less frequently used server.

Here are the details:

• VSCode Version: Version: 1.66.2 (Universal)
• Local OS Version: Darwin x64 20.3.0
• Remote OS Version: CentOS
• Remote Extension/Connection Type: SSH
• Logs:

[14:54:21.320] Log Level: 1
[14:54:21.322] [email protected]
[14:54:21.322] darwin x64
[14:54:21.348] SSH Resolver called for "ssh-remote+juno.mskcc.org", attempt 1
[14:54:21.349] "remote.SSH.useLocalServer": false
[14:54:21.349] "remote.SSH.showLoginTerminal": true
[14:54:21.349] "remote.SSH.remotePlatform": {}
[14:54:21.349] "remote.SSH.path": undefined
[14:54:21.349] "remote.SSH.configFile": undefined
[14:54:21.349] "remote.SSH.useFlock": true
[14:54:21.350] "remote.SSH.lockfilesInTmp": true
[14:54:21.350] "remote.SSH.localServerDownload": auto
[14:54:21.350] "remote.SSH.remoteServerListenOnSocket": true
[14:54:21.350] "remote.SSH.showLoginTerminal": true
[14:54:21.350] "remote.SSH.defaultExtensions": []
[14:54:21.350] "remote.SSH.loglevel": 1
[14:54:21.350] "remote.SSH.maxReconnectionAttempts": 1
[14:54:21.350] "remote.SSH.enableDynamicForwarding": true
[14:54:21.351] "remote.SSH.enableRemoteCommand": false
[14:54:21.351] "remote.SSH.serverPickPortsFromRange": {}
[14:54:21.352] "remote.SSH.serverInstallPath": {"juno.mskcc.org":"/juno/home/zhangh5/work"}
[14:54:21.356] SSH Resolver called for host: juno.mskcc.org
[14:54:21.356] Setting up SSH remote "juno.mskcc.org"
[14:54:21.360] Using commit id "dfd34e8260c270da74b5c2d86d61aee4b6d56977" and quality "stable" for server
[14:54:21.364] Install and start server if needed
[14:54:22.274] PATH: /usr/bin:/bin:/usr/sbin:/sbin
[14:54:22.275] Checking ssh with "ssh -V"
[14:54:22.281] > OpenSSH_8.1p1, LibreSSL 2.7.3

[14:54:22.283] Running script with connection command: ssh -T -D 55188 -o ConnectTimeout=15 'juno.mskcc.org' bash
[14:54:27.063] "install" terminal received data: "07f206fe916f: running"
[14:54:27.064] Got some output, clearing connection timeout
[14:54:27.072] "install" terminal received data: "Acquiring lock on /run/user/1926/vscode-remote-lock.zhangh5.dfd34e8260c270da74b5c2d86d61aee4b6d56977"
[14:54:27.092] "install" terminal received data: "Found existing installation at /juno/home/zhangh5/work/.vscode-server/bin/dfd34e8260c270da74b5c2d86d61aee4b6d56977...
Checking /juno/home/zhangh5/work/.vscode-server/.dfd34e8260c270da74b5c2d86d61aee4b6d56977.log and /juno/home/zhangh5/work/.vscode-server/.dfd34e8260c270da74b5c2d86d61aee4b6d56977.pid for a running server"

Steps to Reproduce:

1. delete the tmp folder (lock files), .vscode-server folder from previous installation attempts
2. open a ssh connection with the following settings: Lock Files in tmp --- YES Use Flock --- YES Use Local Server --- NO

Does this issue occur when you try this locally?: N/A Does this issue occur when you try this locally and all extensions are disabled?: N/A

[tanhakabir]

Hm and do you see /juno/home/zhangh5/work/.vscode-server/.dfd34e8260c270da74b5c2d86d61aee4b6d56977.pid exist in your remote machine?

It looks like we get stuck trying to read the file. Do you generally have home directory read/write access in this account on your remote?

[haochenz96]

No, neither of the .pid and .log exists.

I generally have home directory read/write access on the remote (that's why all othter .vscode-server/ files, such as the bin/ subdirectory, can be created).

At the moment the issue disappeared (i didn't do anything), but it's very likely goign to happen again based on my experience.

I think this issue could relate to #5799.

[tanhakabir]

Do you have any alias that changes the behavior of ps?

You can check ps aux --forest to see if the script is stuck in the same place as the issue you linked next time you see it get stuck.

Sorry it looks like we weren't able to track down the cause of this issue in the past.

[github-actions[bot]]

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.

Happy Coding!

[haochenz96]

Hey sorry for the late reply -was waiting for the issue to appear again and report to you.

The issue just appeared again today. The ps aux --forest command does not work since I am working on a cluster server.... i tried to grab all the processes with ps -ux, killed all the processes, deleted all the lock and vscode-server files, and tried to install the server again, but it still hang at the same step (checking XXX.pid for a running server).

[OmarSZamzam]

I have the exact same problem. Any working solution found?

[lingchen1991]

same issue here. Any solution?

[eleanorjboyd]

Hello! Could you describe more in depth the issue you are seeing @lingchen1991? Since this issue is a bit stale I wanted to gather information on current behavior as that might be more relevant. If you could send your file structure, what you are trying to do, replication steps, and SSH logs that would be great. Thanks!

[lingchen1991]

Hello! Could you describe more in depth the issue you are seeing @lingchen1991? Since this issue is a bit stale I wanted to gather information on current behavior as that might be more relevant. If you could send your file structure, what you are trying to do, replication steps, and SSH logs that would be great. Thanks!

Thanks for your kindly reply. In my case, the issue was fixed by deleting the ~/.vscode-server folder and then rebooting linux machine. Sorry for that cannot provide more details about this issue.

[eleanorjboyd]

@lingchen1991, thank you for your reply! That is still helpful as it seems you just needed a reset of your remote vscode setup. I will keep this issue open in case the original poster is still having problems.

[hmassalha]

Hi, I am facing the same problem. I tried to kill vscode processed on the ssh machine and delete ~/.vscode-server folder. The next run vscode manages to establish connections. But the moment my machine goes to sleep or I deliberately stops remote connection, vscode ran into the same problem. This issue was not before. I always used vscode as my terminal, navigator and editor. Any suggestions will be much appreciated.

[hmassalha]

Hi @eleanorjboyd, Thanks a lot for your intention to help solving this issue. Yesterday, I deleted ~/.vscode-server, removed Remote - SSH extension and kill the processes from terminal. Reinstalled the extension again and connect to the server worked well.

Few notes:

1. doing the above steps without removing ~/.vscode-server, the connection process will stuck log pid step at

[10:01:26.935] > Found existing installation at /nfs/users/nfs_h/hm11/.vscode-server-insiders/bin/1a908707b491de77cc8408ca6d5bf626cf435736...
> Checking /nfs/users/nfs_h/hm11/.vscode-server-insiders/.1a908707b491de77cc8408ca6d5bf626cf435736.log and /nfs/users/nfs_h/hm11/.vscode-server-insiders/.1a908707b491de77cc8408ca6d5bf626cf435736.pid for a running server

2. removing ~/.vscode-server without deleting/reinstalling Remote - SSH extension will stuck at

> Written by John Gilmore and Jay Fenlason.
[10:04:47.920] > Checking /nfs/users/nfs_h/hm11/.vscode-server-insiders/.1a908707b491de77cc8408ca6d5bf626cf435736.log and /nfs/users/nfs_h/hm11/.vscode-server-insiders/.1a908707b491de77cc8408ca6d5bf626cf435736.pid for a running server
Today, I ran into the same problem and I have to do the same steps again to reconnect.  

[eleanorjboyd]

Hello @hmassalha! Did you try the command kill vs code server on remote from the command palette of a local VS Code instance? I am wondering if this would solve those problems you outlined above since it should handle all dis-installation steps.

[hmassalha]

Hi @eleanorjboyd, thanks for the suggestion. I just tried that and ran into the following error

Yesterday I was working working from the office. Today I am connecting from home and I find the established connection from yesterday is not working anymore. It gave the same error in the previous note.

[eleanorjboyd]

Hello! Do you have an alias for ps set up? Can you try running this command and seeing what happens:

xyz=`ps ax` 
echo $xyz

[github-actions[bot]]

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.

Happy Coding!

[hmassalha]

I installed the last update from vscode-insiders and it is working now as before. Many thanks,