:four_leaf_clover: `Proposal`: Plugin `githubactions-python` Enhancement (sub-task of #513)

[daniel-hutao]

What would you like to add? Why is this needed?

See #513 for the details

[ly1ex]

wanna try

[daniel-hutao]

@lyleshaw Go ahead and happy coding! @IronCore864 is very good at py, on the contrary I am not very good at it. So he will communicate with you more about this task later.

/assign @lyleshaw

[IronCore864]

githubactions-python Plugin Improvements

Config Improvement

The current config for this plugin is like the following:

tools:
- name: githubactions-python
  instanceID: default
  options:
    owner: YOUR_GITHUB_USERNAME
    org: YOUR_ORGANIZATION_NAME
    repo: YOUR_REPO_NAME
    language:
      name: python
      version: "3.8"
    branch: main

It seems the language.name is redundant since this plugin will only handle GitHub actions workflows for Python. This can be removed.

New Features

Refer to https://github.com/actions/setup-python and https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python for possible new features.

Examples:

• exclude a specific Python version
• support both pip and pipenv
• cache pip dependencies
• cache pipenv dependencies
• matrix testing
• install dependencies
• requirements files (pip)
• test the code (pytest)
• test coverage (pytest-cov)
• lint (flake8)
• run tests with tox (https://tox.wiki/en/latest/)

There is no need to implement every single one of them; use your experience and best judgement to decide which features to include first.

These features probably require config options change; please make sure you post a design first for review before coding.

[ly1ex]

Hi @IronCore864, I'm so sorry that only now starting deal with this issue due to the final exams.

• [x] exclude a specific Python version
• [ ] support both pip and pipenv
• [x] cache pip dependencies
• [x] cache pipenv dependencies
• [x] matrix testing
• [x] install dependencies
• [x] requirements files (pip)
• [x] test the code (pytest)
• [x] test coverage (pytest-cov)
• [x] lint (flake8)
• [x] run tests with tox (https://tox.wiki/en/latest/)

To support those features, I checked the setup-python's docs, and changed some parameters in githubactions-python's config.

Including delete options-language-name, add options-matrix to support matrix testing and exclude a specific Python version, add options-cache to control the way of cache

here comes the old config structure

tools:
- name: githubactions-python
  instanceID: default
  options:
    owner: YOUR_GITHUB_USERNAME
    org: YOUR_ORGANIZATION_NAME
    repo: YOUR_REPO_NAME
    language:
      name: python
      version: "3.8"
    branch: main

and the new config

tools:
- name: githubactions-python
  instanceID: default
  options:
    owner: YOUR_GITHUB_USERNAME
    org: YOUR_ORGANIZATION_NAME
    repo: YOUR_REPO_NAME
    matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        python-version: ["3.7", "3.8", "3.9", "3.10", pypy-2.7, pypy-3.8]
        exclude:
          - os: macos-latest
            python-version: "3.7"
    cacahe: 'pip'
    branch: main

BTW, to support the other features, I also made some changes on templates

old one:

name: Tests

on: [push, pull_request]

jobs:
  build:

    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [2.7, 3.5, 3.6, 3.7]

    steps:
    - uses: actions/checkout@v2
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v1
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install .[test]
    - name: Lint with flake8
      run: |
        pip install -Iv enum34==1.1.6 # https://bitbucket.org/stoneleaf/enum34/issues/27/enum34-118-broken
        pip install flake8
        flake8 . --count --show-source --statistics
    - name: Run unit tests
      run: |
        python -m unittest discover -v tests/unit

new one

name: Tests

on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest]
        python-version: ["3.7", "3.8"]
        exclude:
          - os: macos-latest
            python-version: "3.7"
    steps:
    - uses: actions/checkout@v2
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v1
      with:
        python-version: ${{ matrix.python-version }}
        cache: 'pip'
    - name: Install dependencies
      run: |
        pip install -r requirements.txt
    - name: Test with pytest
      run: |
        pip install pytest
        pip install pytest-cov
        pytest tests.py --doctest-modules --junitxml=junit/test-results.xml --cov=com --cov-report=xml --cov-report=html
    - name: Lint with flake8
      run: |
        pip install -Iv enum34==1.1.6 # https://bitbucket.org/stoneleaf/enum34/issues/27/enum34-118-broken
        pip install flake8
        flake8 . --count --show-source --statistics
    - name: Run unit tests
      run: |
        python -m unittest discover -v tests/unit
    - name: Install tox and any other packages
      run: pip install tox
    - name: Run tox
      run: tox -e py

I have a question, let's say pox tests or pytest these phases rely on established config files (like tox.ini or tests.py) so do we have to make it a parameter for the user to choose whether to turn on tests or not?

Please feel free to point out any improvements that can be made. And I have tested this workflow on my own repo: https://github.com/lyleshaw/python-devstream-demo

After the config structure is approved, I will start making changes to the code and submitting PRs.

[ly1ex]

Hi @IronCore864, I'm so sorry that only now starting deal with this issue due to the final exams.

• [x] exclude a specific Python version
• [ ] support both pip and pipenv
• [x] cache pip dependencies
• [x] cache pipenv dependencies
• [x] matrix testing
• [x] install dependencies
• [x] requirements files (pip)
• [x] test the code (pytest)
• [x] test coverage (pytest-cov)
• [x] lint (flake8)
• [x] run tests with tox (https://tox.wiki/en/latest/)

To support those features, I checked the setup-python's docs, and changed some parameters in githubactions-python's config.

Including delete options-language-name, add options-matrix to support matrix testing and exclude a specific Python version, add options-cache to control the way of cache

here comes the old config structure

tools:
- name: githubactions-python
  instanceID: default
  options:
    owner: YOUR_GITHUB_USERNAME
    org: YOUR_ORGANIZATION_NAME
    repo: YOUR_REPO_NAME
    language:
      name: python
      version: "3.8"
    branch: main

and the new config

tools:
- name: githubactions-python
  instanceID: default
  options:
    owner: YOUR_GITHUB_USERNAME
    org: YOUR_ORGANIZATION_NAME
    repo: YOUR_REPO_NAME
    matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        python-version: ["3.7", "3.8", "3.9", "3.10", pypy-2.7, pypy-3.8]
        exclude:
          - os: macos-latest
            python-version: "3.7"
    cacahe: 'pip'
    branch: main

BTW, to support the other features, I also made some changes on templates

old one:

name: Tests

on: [push, pull_request]

jobs:
  build:

    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [2.7, 3.5, 3.6, 3.7]

    steps:
    - uses: actions/checkout@v2
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v1
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install .[test]
    - name: Lint with flake8
      run: |
        pip install -Iv enum34==1.1.6 # https://bitbucket.org/stoneleaf/enum34/issues/27/enum34-118-broken
        pip install flake8
        flake8 . --count --show-source --statistics
    - name: Run unit tests
      run: |
        python -m unittest discover -v tests/unit

new one

name: Tests

on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest]
        python-version: ["3.7", "3.8"]
        exclude:
          - os: macos-latest
            python-version: "3.7"
    steps:
    - uses: actions/checkout@v2
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v1
      with:
        python-version: ${{ matrix.python-version }}
        cache: 'pip'
    - name: Install dependencies
      run: |
        pip install -r requirements.txt
    - name: Test with pytest
      run: |
        pip install pytest
        pip install pytest-cov
        pytest tests.py --doctest-modules --junitxml=junit/test-results.xml --cov=com --cov-report=xml --cov-report=html
    - name: Lint with flake8
      run: |
        pip install -Iv enum34==1.1.6 # https://bitbucket.org/stoneleaf/enum34/issues/27/enum34-118-broken
        pip install flake8
        flake8 . --count --show-source --statistics
    - name: Run unit tests
      run: |
        python -m unittest discover -v tests/unit
    - name: Install tox and any other packages
      run: pip install tox
    - name: Run tox
      run: tox -e py

I have a question, let's say pox tests or pytest these phases rely on established config files (like tox.ini or tests.py) so do we have to make it a parameter for the user to choose whether to turn on tests or not?

Please feel free to point out any improvements that can be made. And I have tested this workflow on my own repo: https://github.com/lyleshaw/python-devstream-demo

After the config structure is approved, I will start making changes to the code and submitting PRs.

PTAL @IronCore864 @daniel-hutao