elementary
Files freeze after ssh server drops connection
What Happened?
At my university we have a few ssh servers and clusters for running computations. I often connect to these servers from Files, because it's a convenient way to copy files between the servers and my local machine.
These particular servers have an annoying habit of dropping the ssh connection after few minutes of inactivity (unless I'm connected to university wifi - eduroam - which is quite weird in itself). When this happens, and I go to Files and try to interact with the remote directories (navigate folders or refresh current folder view), the Files app freezes. The only point of action afterwards is to kill Files.
One way to prevent this freezing is to disconnect the server from sidebar whenever I suspect the connection was dropped (e.g. coming back 5 minutes or more after I used it last time), then I can reconnect and everything works without issue. But obviously I forget to do that way too often.
I know that the ssh/sftp feature is handled by gvfs backend, however I also use another app - Taxi, the (s)ftp client from AppCenter - which is using the same backend, but doesn't freeze after the connection drops. I know it uses the same feature as it doesn't even ask for password if I already used the password in Files.
Steps to Reproduce
I admit this will not be easy to reproduce, but it goes something like this:
- Connect to ssh server which tends to drop connection after some time of inactivity.
- Wait some time without doing anything so the connection gets dropped.
- Then try to browse folders or just refresh the current folder.
- If it didn't freeze yet, click anywhere in the window with a mouse. Now you're there - Files is frozen.
Expected Behavior
Files should not freeze just because a connection to a server was dropped. Taxi uses the same backend to connect to the servers and does not freeze when connection is dropped.
OS Version
5.x (Hera)
Software Version
Latest release (I have run all updates)
Log Output
I tried launching Files from terminal, but the freeze doesn't produce any output there. Also `coredumpctl` says there are no coredumps to show.
Hardware Info
All the laptops that I ever had with elementary OS. Acer, Dell, Slimbook, all with Intel chips. I think the issue exists maybe since Loki, but I never paid enough attention to it, until my friend pointed out the dropping connections to our servers (I use a few other servers at different institutes and those are all fine). I then noticed that Files tends to freeze after the connection drops.
This still applies to 6.1.
Same issue here, commented just to raise awareness. No way to debug this
carlosflores@Lenovo-ideapad-FLEX-5-1570-7d5362b2:~$ neofetch
eeeeeeeeeeeeeeeee carlosflores@Lenovo-ideapad-FLEX-5-1570-7d5362b2
eeeeeeeeeeeeeeeeeeeeeee ------------------------------------------------
eeeee eeeeeeeeeeee eeeee OS: elementary OS 6.1 Jólnir x86_64
eeee eeeee eee eeee Host: 80XB Lenovo ideapad FLEX 5-1570
eeee eeee eee eeee Kernel: 5.11.0-46-generic
eee eee eee eee Uptime: 20 hours, 50 mins
eee eee eee eee Packages: 2618 (dpkg), 95 (flatpak)
ee eee eeee eeee Shell: bash 5.0.17
ee eee eeeee eeeeee Resolution: 1920x1080
ee eee eeeee eeeee ee DE: Pantheon
eee eeee eeeeee eeeee eee WM: Mutter(Gala)
eee eeeeeeeeee eeeeee eee Terminal: io.elementary.t
eeeeeeeeeeeeeeeeeeeeeeee eeeee CPU: Intel i5-7200U (4) @ 2.500GHz
eeeeeeee eeeeeeeeeeee eeee GPU: Intel HD Graphics 620
eeeee eeeee Memory: 6577MiB / 7820MiB
eeeeeee eeeeeee
eeeeeeeeeeeeeeeee
I just realized one way to test this bug with any ssh server, as there is at least one other way to make the server drop the connection.
Have a tab with active connection to a ssh server and suspend the laptop / desktop for some time (e.g. over night, but even something like 15 minutes should be enough, or maybe a lunch break). After waking up, try to interact with the ssh server tab as described above (navigate folders or refresh the current folder view).
Edit: Forgot to mention I'm on OS 6.1 now.
This behavior is reproducible, when we let the system go to sleep while connected to a remote server. If the file was opened with a tab displaying remote location, on waking up, it crashes.
I have tried to reproduce this with version 6.5.0 (On Elementary 7) on an SFTP connection, simulating the "going to sleep" by going into aeroplane mode. I haven't experienced any crashing or freezing of the entire interface (the tab itself may block but other tabs and other elements of the UI continue to work). Reconnecting either completes the operation or asks for the password again. I'll try it with suspending the laptop.
Closing and opening Files such that a disconnected location is restored results in the tab continuing to be in a "working" state but the rest of the UI works as usual. Reconnecting results in the tab completing the load process.
A fair few changes have been made since this issue was first reported so a fresh report for Horus is required if it still happens under some circumstances.
I can still get it to freeze. I connected to a server (my model finished overnight, cool!), then closed the laptop and went for breakfast. Now I opened it again, focused Files and clicked on a folder (in a column view) that I didn't load before - and Files are frozen now, clicking other tabs or anything else isn't working.
PS: upon switching back to Files after writing this comment I was presented with the "wait or kill" dialog.
PPS: relaunching Files again doesn't work and presents the same "wait or kill" dialog. Kiling the gvfsd process for the connection finally let me open Files again.
PPPS: BTW after all of this the path bar in Files looks like this:
Switching between tabs or opening new tabs doesn't help it, it just stays there like that with new paths projected over it. The background path seems to be the one to the server (closing connection from sidebar and reconnecting also doesn't help). Closing Files with Ctrl+Q finally shows normal breadcrumbs.
@janxkoci Thanks for the update. Looks like there is a blocking io process going on that needs to be made asynchronous. Either that or an infinite loop (in that case the cpu usage will be high). Either way something is stopping any other Files process from getting any cpu time. To help identify what it is, could you run Files from the terminal under gdb (you may need to install gdb) with the command gdb io.elementary.files followed by run. Make sure all existing Files processes are closed first. When Files freezes, go back to the terminal and enter Ctrl-C followed by bt. This should give information about what happened up to the problem. If you install the libpantheon-files-core-dev package first the information will be more useful.
The pathbar appearance is just a side-effect of the animation being interrupted in the middle of a transition.
By the way, could you confirm which version of Files you are running? The latest is 6.5.0.
Ok my is older:
$ io.elementary.files --version
io.elementary.files 6.4.1
(Btw issuing io.elementary.files -h closes files and I cannot open them again - I expected to see help instead :open_mouth: )
@janxkoci Hmm that's a regression - not sure when that happened. I'll raise an issue and investigate.
Ah, the update to Files is in the queue, but Appcenter forces me to double-reboot to even install the update (it only downloads it now) and I didn't have time to reboot this week. :shrug:
Hopefully it helps. I havent been able to freeze Files by simulating the server disconnecting by turning wifi off while suspended using a public SFTP server. The tab freezes but not the whole app. However, this is not exactly the same as the server disconnecting. When I have time Ill try to set up an SSH server I can control.
I thought AppCenter does install native apps like Files straight-away (although you need to close and restart Files for it to have an effect) however there do seem to be a lot of mandatory restarts recently - I assumed due to kernel updates - so maybe something changed.
Ok this is crazy. I did the two reboots to install the update, then after the second booting I opened Files and the SSH session in one of the tabs, closed the laptop and went out witm my second laptop. Now I returned, opened, focused Files and tried to navigate another folder on the SSH server - instant freeze.
I realized I forgot to check the version after the update, so I popped up a terminal and - well this is weird:
$ io.elementary.files --version
Failed to register: Časový limit vypršel
It timed out because Files were frozen :open_mouth: I had to check in Appcenter, which shows the version as 6.5 now.
About the reboots - all "OS updates" (I think it means deb packages) are grouped under one item in Appcenter and when there is a kernel update, it only shows a "Download" button, not an "Update" button next to it (other updates show normal Update buttons, including flatpak apps and runtimes). So basically unless I reboot, all these updates get only downloaded but not installed, only the flatpaks do. :shrug:
The --version option is working for me and showing "6.5.0". I am running the daily development version compiled from source but there havent been any recent commits that I would expect to affect this option. Weirdly, the --help option is now working as well :shrug: Maybe something to do with experiments with trying to get Files to freeze.
Might be worth running sudo killall io.elementary.files to ensure all Files processes are killed. They will restart with the installed version when needed.
@janxkoci I have tried setting up an OpenSSH server on my local machine but have not succeeded in getting a freeze so far. One thing occurred to me is that when Files is frozen, there might be a modal dialog somewhere (e.g. beneath a window) that has stolen focus. Unlikely, but worth asking. https://www.cyberciti.biz/tips/open-ssh-server-connection-drops-out-after-few-or-n-minutes-of-inactivity.html might be of help but only if you have access to the server config file.
There is definitely no dialog behind the window or anything like that. If there is, it appears normally on top, asking to wait or kill the app. In fact, killing the app and trying to re-launch will often show no window and only the wait-or-kill dialog appears.
Afaik, the only way to avoid the freeze is to eject the connection from the sidebar before it times-out (e.g. server dropping the connection, or me suspending the laptop), or to close the remote tab/not interact with it.