haystack
haystack copied to clipboard
Using Docker to speed up installation of dependencies and cache HF models
Related to https://github.com/deepset-ai/haystack/issues/2811
Problem:
- The current setup time of GH runners is not negligible, and subject to failure in case of network issues on the runner.
- The runner needs to fetch system deps, Python deps, external containers for docstores, models from HF, models for NLTK, and others.
Solution:
- We could speed up and make the process more robust by providing a Docker images that the runners should use to execute the tests.