haystack icon indicating copy to clipboard operation
haystack copied to clipboard

Using Docker to speed up installation of dependencies and cache HF models

Open ZanSara opened this issue 3 years ago • 0 comments

Related to https://github.com/deepset-ai/haystack/issues/2811

Problem:

  • The current setup time of GH runners is not negligible, and subject to failure in case of network issues on the runner.
  • The runner needs to fetch system deps, Python deps, external containers for docstores, models from HF, models for NLTK, and others.

Solution:

  • We could speed up and make the process more robust by providing a Docker images that the runners should use to execute the tests.

ZanSara avatar Jul 27 '22 10:07 ZanSara