DistributedLock icon indicating copy to clipboard operation
DistributedLock copied to clipboard

Granted lock never released

Open gtaylor44 opened this issue 4 years ago • 5 comments

Second time I've run into a particular issue.

A granted lock is never released after over a day.

Using the following code:

var myLock = new SqlDistributedLock($"ProductsJob_<Id>, "ConnectionString");
using (var handle = await myLock.TryAcquireAsync(TimeSpan.FromHours(1)))
...

app lock

After killing the lock

KILL 65

The lock is released an application continues to work.

Is there any work around for this?

gtaylor44 avatar Dec 22 '21 13:12 gtaylor44

Hi @gtaylor44 thanks for reporting. Without more information it is going to be difficult to investigate the issue.

In this situation, I can think of several possible options:

  • The handle is never getting disposed
  • TryAcquire is throwing after acquiring the lock so the handle never gets created to release it
  • Disposing of the handle is failing/not working for some reason
  • TryAcquire is returning null for some reason despite having acquired the lock

To start, it would be great to add some logging to your code so that we can start to narrow this down (perhaps you've already done this).

var guid = Guid.NewGuid();
Log($"Acquiring {myLock.Name} {guid}");
using (var handle = await myLock.TryAcquireAsync(TimeSpan.FromHours(1)))
{
    if (handle != null)
    {
        Log($"Acquired {myLock.Name} {guid}");

        ...

        Log($"Releasing {myLock.Name} {guid}");
    }
    else { Log($"Failed to acquire {myLock.Name} {guid}");
}
if (guid.HasValue) { Log($"Released {myLock.Name} {guid}"); }

Also, can you confirm that you're seeing this behavior with the newest version of the library?

madelson avatar Dec 22 '21 18:12 madelson

Thanks @madelson

I will add some additional logging. It's only happened twice in the last year and this process is run every 15 minutes.

gtaylor44 avatar Dec 22 '21 23:12 gtaylor44

@madelson

I'm currently on version 2.1.0

The last time I updated was when I first encountered this issue. I will update to 2.3.0 in the meantime.

gtaylor44 avatar Dec 22 '21 23:12 gtaylor44

It's only happened twice in the last year and this process is run every 15 minutes.

Wow I guess we might we waiting a long time for a repro, then. I wonder if it could be reproduced more quickly by a unit test acquiring and releasing in a tight loop.

Can you tell me more about your setup?

  • How many threads/processes are typically contending for the lock?
  • Is it common for Acquire to block or is there usually no contention?
  • How long does each thread hold the lock for before releasing it?
  • Have you tried releasing locks asynchronously using await using?

madelson avatar Dec 23 '21 15:12 madelson

@gtaylor44 any chance you've seen this come up again?

madelson avatar Nov 06 '22 22:11 madelson