Support LLM/Model Caching

[warlockedward]

例行检查

• [ ] 我已确认目前没有类似 features
• [ ] 我已确认我已升级到最新版本
• [ ] 我已完整查看过项目 README，已确定现有版本无法满足需求
• [ ] 我理解并愿意跟进此 features，协助测试和提供反馈
• [x] 我理解并认可上述内容，并理解项目维护者精力有限，不遵循规则的 features 可能会被无视或直接关闭

功能描述 A semantic cache for Large Language Models (LLMs) that reduces response time for similar requests and improves user experience by caching pre-generated model results.

应用场景 Introducing a caching mechanism to optimize services helps enterprises and research institutions to reduce inference deployment costs, improve model performance and efficiency, and provide scalable services for large models.

相关示例 similar to gptcache, modelcache such open source projects