AccessViolation in SQLiteDistributedLock

[ax0l0tl]

Version 0.4.2, .net 8:

We encounter AccessViolationExceptions in SQLiteDistributedLock.UpdateExpiration. Mainly in debug mode, but we also encouter problems with the release version, were hangfire storage gets corrupted which might be caused by this exception.

It looks like two threads are accessing the same SQLiteConnection instance at the same time. First one is the heartbeat timer from SQLiteDistributedLock the second one in our case is RecurringJobScheduler tryiing to Release another DistributedLock.

Thread 29892 and thread 34292 use the same connection instance.

I think this can always happen, because the connection is shared between the consumer of the lock an the lock itself.

Can be reproduced with the following test:

[Fact]
//this will result in an access violation exception caused by multithreaded usage of the same connection
//(distributed lock heartbeat is trying to update expiry on owners connection).
public void Use_Connection_When_Heartbeat_Fires()
{
    UseConnection(database =>
    {
        using var slock = SQLiteDistributedLock.Acquire("resource1", TimeSpan.FromSeconds(10), database, new SQLiteStorageOptions
        {
            DistributedLockLifetime = TimeSpan.FromSeconds(5) //heartbeat every second
        });        

        var start = DateTimeOffset.Now;
        //generate some pressure on connection to increase probability of race condition with heartbeat timer thread
        while (DateTimeOffset.Now - start < TimeSpan.FromMinutes(1)) 
        {
            database.Database.Insert(new JobParameter()
            {
                ExpireAt = start.AddSeconds(15).UtcDateTime,
                JobId = 13,
                Name = "MyParameter",
                Value = "MyValue"
            });
        }
    });
}

Using SQLiteOpenFlags.FullMutex is a workaround for the problem, but I think that should not be the final solution.

UseSQLiteStorage(
	new SQLiteDbConnectionFactory(() => new SQLiteConnection(
		databasePath: databasePath,
		openFlags: SQLiteOpenFlags.ReadWrite | SQLiteOpenFlags.Create | SQLiteOpenFlags.FullMutex,
		storeDateTimeAsTicks: true
		)
		{
			BusyTimeout = TimeSpan.FromSeconds(value: 10)
		}
	)
);