Split Coverage Tests for Github Actions

[seanlaw]

As our test suite gets longer, the coverage tests, which are executed in pure Python, will continue to need more time to complete. GIven that Github Actions has a job time limit, it may be beneficial to further split up the coverage testing time into two separate jobs (instead of all in one). However, this will require storing the .coverage file between jobs. This can be accomplished by using "Uploading/Downloading Artifacts" (see this article):

To upload a file:

steps:
- uses: actions/checkout@v2

- run: mkdir -p path/to/artifact

- run: echo hello > path/to/artifact/world.txt

- uses: actions/upload-artifact@v2
  with:
    name: my-artifact
    path: path/to/artifact/world.txt

In our case, it might look something like:

  coverage-testing:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        python-version: ['3.7', '3.8', '3.9', '3.10']
    steps:
      - uses: actions/checkout@v2
      - name: Set Up Python
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Display Python Version
        run: python -c "import sys; print(sys.version)"
        shell: bash
      - name: Install STUMPY And Other Dependencies
        run: pip install --editable .[ci]
        shell: bash
      - name: Run Black
        run: black --check --diff ./
        shell: bash
      - name: Run Flake8
        run: flake8 ./
        shell: bash
      - name: Run Coverage Tests
        run: ./test.sh coverage
        shell: bash
      - uses: actions/upload-artifact@v2
        with:
          name: coverage_results
          path: ./.coverage        

Then, for the second half of the job you can retrieve the file:

steps:
- uses: actions/checkout@v2

- uses: actions/download-artifact@v2
  with:
    name: my-artifact

or in our case:

steps:
- uses: actions/checkout@v2
- uses: actions/download-artifact@v2
  with:
    name: coverage_results