Enhancements to SlidingCache<> for Improved Thread Safety and Performance

[TWhidden]

In the latest update to the SlidingCache<> implementation, we addressed a critical oversight related to concurrent modifications and access patterns. Specifically, we identified that a cache key could be removed during enumeration, triggering a KeyNotFoundException. To address these concerns and enhance the cache's robustness and efficiency, the following adjustments were made:

1. Atomic Locking with Interlocked.CompareExchange: We introduced an atomic lock mechanism using Interlocked.CompareExchange to prevent simultaneous entries into the Cleanup method. This enhancement aims to significantly reduce contention and ensure thread-safe operations within the cache cleanup process.

2. Deferred Cleanup Execution: The CleanupIfNeeded method has been relocated to a finally block following the TryGetValue execution. This will ensure that if PermitExpiredReturns is enabled, there is no race to remove the item while accessing it.

3. Support for Deterministic Element Caching: A new parameter has been added to SlidingCache<>, enabling the caching of expired but deterministic elements. This feature allows expired objects to remain accessible before the Cleanup process has executed, preventing unnecessary removals and recreations. This capability is particularly beneficial for ConstantExpressions and is enabled by default to optimize performance for predictable data patterns.

4. Configurable Expired Returns via CacheConfig: To provide greater control over cache behavior, the PermitExpiredReturns setting has been incorporated into the CacheConfig class. This addition allows developers to manage the allowance of expired item returns based on their specific application requirements and caching strategies.

These enhancements are designed to bolster the SlidingCache<>'s performance, reliability, and flexibility, particularly in high-concurrency environments.

[StefH]

@TWhidden Did you have time to verify this new PR?

[TWhidden]

It's currently running in a beta instance without any known issues. I have not rolled it to prod yet. I'll try to get that out this week or early next week. Sorry for the delay.

[StefH]

Did you have time to test it?

[TWhidden]

I have not pushed to production yet, however I did pull a memory snapshot of our demo/beta build which is an active system. I am working on a release to prod right now. FYI - this is using 9.3.9-preview-02 for reference. So far, I have not had any error popup.

[TWhidden]

Ok, pushed to production - will follow up in a couple days.

[StefH]

Hello @TWhidden, what is the status?

[TWhidden]

Looks solid. No leaks, no exceptions reported. 9 days running in prod.

All good!

[StefH]

Thanks for verifying. I'll release a new NuGet soon.