feat(docs): add documentation for usage of minio or other s3 compatible service as a cache backend or celery results backend
User description
SUMMARY
Adds documentation for usage of minio or other s3 compatible service as a cache backend or celery results backend. Update pages concerning cache and celery async queries. Examples of usage are given.
CHANGES
Modified:
CodeAnt-AI Description
Document MinIO/S3 as cache and Celery results backend
What Changed
- Added documentation and configuration examples showing how to use MinIO or other S3-compatible services as a Flask-Caching backend and as Celery RESULTS_BACKEND.
- Included example configuration snippets for RESULTS_BACKEND and FILTER_STATE_CACHE_CONFIG using MinIO (endpoint, credentials, bucket, secure flag, timeout, key prefix).
- Listed the Python packages to install when using S3/MinIO backends (e.g., minio-flask-cache, s3werkzeugcache) to avoid missing-dependency issues.
Impact
✅ Use MinIO for caches and Celery async results
✅ Clear configuration examples for S3-compatible caching
✅ Fewer missing-dependency surprises when enabling S3/MinIO backends
💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
@codeant-ai ask: Your question here
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
@codeant-ai ask: Can you suggest a safer alternative to storing this secret?
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
@codeant-ai: Your feedback here
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
@codeant-ai: Do not flag unused imports.
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
@codeant-ai: review
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.
Interaction Diagram by Bito
sequenceDiagram
participant Admin as Admin/Developer
participant Config as superset_config.py<br/>🟩 Added | ●●● High
participant Manager as ResultsBackendManager
participant MinIO as MinioCacheBackend<br/>🟩 Added | ●●● High
participant SQLLab as SQL Lab Module
participant Commands as sql_lab Commands
Admin->>Config: Configure MinIO backend
Config->>Manager: init_app(app)
Manager->>MinIO: Initialize cache backend
SQLLab->>Manager: Request results_backend
Manager-->>SQLLab: Return MinioCacheBackend instance
Commands->>MinIO: Store/retrieve query results
Critical path: Admin/Developer -> superset_config.py -> ResultsBackendManager -> MinioCacheBackend -> SQL Lab Module
Note: Documentation update adds MinIO/S3-compatible backend configuration examples. No code changes to core components; existing ResultsBackendManager and sql_lab modules continue using the configured backend transparently.
CodeAnt AI is reviewing your PR.
CodeAnt AI finished reviewing your PR.
Can you say more here about what using Minio and your package interface to it adds? I don't use async queries so might need a basic explanation. Redis is the default for the community and so it would help for readers to know that if their needs are X they should check this out.
I could see instead of including a full config like this which seems to put it on par with Redis - which I'm wary about given that it's a repo owned by a single person and with little activity - it being helpful to say, "in case X where you want to use S3 because Y, you can try this Minio interface" with a link to the repo.
Incidentally in reviewing this I see we link to https://pypi.org/project/s3werkzeugcache/ which seems stale, it's owned by a former Superset contributor and hasn't been touched in five years.
CodeAnt AI is running Incremental review
@sfirke thanks for the review.
Since reasons for why one could choose one cache backend vs. another are not mentioned for the other backends in the documentation, I didn't make this explicit.
In our case it is by pure simplicity. We already use S3/Minio in our infrastructure for other purposes and decided to remove the instance of Redis which was used only for Superset.
Redis was also used as celery broker but as the documentation already covers we switched to RabbitMq which was also already used in our infrastructure.
I also removed the example as requested and left just the link to the library (which should be enough to get anyone started).
The PR also copies the other links from cache.mdx to async-queries-celery.mdx to make it easier to set up any backend.