microsoft
Implement semantic cache
That's a feature that already exists in langchain and will be beneficial to save costs. The idea will be to ported from phyton to c#
https://github.com/zilliztech/GPTCache
https://python.langchain.com/docs/integrations/llms/llm_caching/
https://www.mongodb.com/developer/products/atlas/advanced-rag-langchain-mongodb/
Be able to save all the queries together with llm responses in some database and try to fetch it first from there, if not then call the LLM, and have some parameters to invalidate or update cache from time to time.
@markwallace-microsoft, going to assign this to @dmytrostruk since this is one of the samples he's working on using filters.
Also assigning @eavanvalkenburg since he was working on filters for Python
Example for Semantic Caching in .NET is here: https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Caching/SemanticCachingWithFilters.cs
Removing the .Net tag, since the sample exists. Leaving python as we haven't completed this yet.
Python version was actually created a bit ago: https://github.com/microsoft/semantic-kernel/blob/main/python/samples/concepts/caching/semantic_caching.py
