Error: undefined caused by opening multiple notebooks simultaneously

[davidwagner]

Here's a new error that I saw for the first time when trying to open a notebook:

What I did that triggered this error: I went to http://data8.org/sp17/ and double-clicked on three links to three notebooks in rapid succession, causing each one to open in a separate tab. (I opened the lec16, lec17, and lec18 demos notebooks, i.e., Fri 2/24, .) This created three tabs with links to git-pull URLs. The first tab loaded the notebook. The next two tabs triggered the above error message.

This is very minor and low-impact. If I just re-click the link, everything works. But this makes me suspect there is something failing if a user tries to simultaneously do multiple pull/sync operations, and failing in a way that displays an unhelpful error message. So a small improvement could be to display a more helpful error message, or to write the system to wait a random amount of time and try again.

[yuvipanda]

Yep, git can not really do concurrent operations on a repository cleanly, and we try to make that the case by grabbing a lock before doing any operations. Clearly that isn't working properly somewhere. We need more robust locking + error handling here.

[r4mmer]

Maybe python fcntl module may help

This example makes a non-blocking attempt to lock a file, but if only LOCK_EX was used, it would wait until he gets the lock

from fcntl import flock, LOCK_EX, LOCK_NB

f = open('file.lock')
try:
    # EXclusive, Non-Blocking
    flock(f, LOCK_EX | LOCK_NB)
except IOError:
    # file was locked, try again later
else:
    # got lock!
    # do stuff
# release the lock!
f.close()

[sanjaydatasciencedojo]

You can make this change in your static file: https://github.com/jupyterhub/nbgitpuller/pull/115/commits/f7e0efaa23861ac0423e302afd8dd0306e9579ac

[shaneknapp]

just pinging this again... i was able to recreate this issue by trying to open two nbgitpuller links simultaneously on a hub, both links pointing to the same github repo. the first link i clicked loaded successfully, but the second failed out immediately.

there's nothing in the logs to indicate what went wrong.

we're running jupyterhub 4.1.5 and nbgitpuller 1.2.1.

@yuvipanda

[shaneknapp]

just pinging this again... i was able to recreate this issue by trying to open two nbgitpuller links simultaneously on a hub, both links pointing to the same github repo. the first link i clicked loaded successfully, but the second failed out immediately.

after changing the nbgitpuller url from https://<hub url>/hub/user-redirect/git-pull? to https://<hub url>/user/<username>/git-pull? i was able to open multiple links concurrently. :)

[yuvipanda]

This race condition continues to exist, I suspect the multiple links are perhaps because the race was not hit, or if the issue is caused by multiple different kinds of git operations (create vs pull?) idk.

I don't think there's been any progress made here though.

[balajialg]

@yuvipanda When I make multiple clicks to either the same link or a different link in quick succession (even before the notebook from the first link gets launched), I get the above error message as shared by Shane. However, when the same action is performed using the previous version of nbgitpuller (v1.1.1) this issue didn't occur for me. Is this intended as part of the release? If not, There seems to be a regression in 1.2.1 which might be causing this issue to a small subset of users. Around 15+ students were impacted by this issue after we made the upgrade.