RedLock.net icon indicating copy to clipboard operation
RedLock.net copied to clipboard

Lock stuck in Conflicted state

Open userAXC001 opened this issue 2 years ago • 0 comments

Package versions:

<PackageReference Include="RedLock.net" Version="2.3.1" />
<PackageReference Include="StackExchange.Redis" Version="2.5.43" />

I have an instance where keys get "stuck" in a conflicted state. Below is the lock information:

{
  "isAcquired": false,
  "lockId": "57ef994d-b449-4351-b387-05efdd9ec346",
  "status": 3,
  "instanceSummary": {
    "acquired": 0,
    "conflicted": 1,
    "error": 0
  },
  "extendCount": 0
}

The architecture is such that there are an n number of service instances running and each is configured as noted below. A possible important note here is that RedLock is being set up against a Redis Cluster.

services.AddSingleton(_ => ConnectionMultiplexer.Connect(
    ctx.Configuration.GetSection("connectionString").Value + ",allowAdmin=true",
    options => options.AsyncTimeout = 15000));
services.AddSingleton(sp =>
{
    var multiplexer = sp.GetService<ConnectionMultiplexer>();
    return RedLockFactory.Create(new List<RedLockMultiplexer> { multiplexer });
})

At any given time, one of the running service instances can acquire a lock and any other service that requests the same lock will fail to acquire. This seems to be working. But as noted above, it looks like a lock will inadvertently get "stuck".

If a processes is indeed taking a long time to processes, I would expect the extendCount to be greater than zero. I suspect a long running process is not the issue bc the code to acquire the lock is running in a consumer for a Azure Service Bus message. Azure Service Bus has a default limit of 5 minute processing time before it throwing a message lost exception, which is not occurring in this case.

A possibility is that one of the processes gets the lock, but isn't disposing it properly? If this is the case, wouldn't extendCount be greater than zero as well? Could it be something with the usage of Redis Cluster?

userAXC001 avatar Dec 09 '23 00:12 userAXC001