python-pytest-cases
python-pytest-cases copied to clipboard
smarie
CHORE: upgrate build suite
what about
- use ruff check in nox lint session
- ruff format in nox format session
- use mypy in nox typing session
- move all config to pyproject.toml and use poetry or uv
- ...
Hi @smarie , I can tackle this one 😄
I would go in the following order:
- Update package management to use UV for dependency management and deployment
- Introduce Ruff and Mypy as linters and formatters used in the project
- Use Ruff to perform Pyupgrade actions on the repo to modernize it automatically
- (Optional) Consider replacing Nox with more modern tools like Tox
Please let me know if that makes sense to you, or if you have any caveats.
Hi @saroad2 , it seems that we are in line with 1 and 2 so yes, let's go for it (one at a time for PR review comfort). In my company when we use poetry, we create one group for each nox session. I imagine that this will be the same for uv (we are currently completing our migration, I did not look at the new templates yet). I will paste a pyproject and a nox file below to get started
For 3 and 4 let's rediscuss more. In particular:
- This should remain optional as part of a (possibly automated) PR, or simply a nox session
- nox is actually a sequel to tox, it was created to overcome the limitations of tox :) . Of course this is opinionated but I would rather stick to it - it has become a standard for industrial CI/CD. Note that nox can use uv as backend.
Here a pyproject file where I have setup most of what we need, and filled most relevant pytest-cases contents.
[project]
name = "pytest-cases"
dynamic = [ "version" ]
description = "Separate test code from test cases in pytest."
# Not sure there is a notion of long description anymore, to check ?
#long_description = file: docs/long_description.md
# long_description_content_type=text/markdown
# Shall this be also removed or is there a way to declare it ?
# license = BSD 3-Clause
requires-python = ">=3.9"
readme = "README.md"
license-files = ["LICENSE"]
authors = [
{ name = "Sylvain Marié", email = "[email protected]" },
]
# Is there a way to add maintainers names ? It could be appropriate to recognize commited maintainers if relevant.
# Our current way to recognize ponctual contributions is in the changelog with explicit gh account author links and pr links
keywords = [
"pytest",
"test",
"case",
"testcase",
"test-case",
"decorator",
"parametrize",
"parameter",
"data",
"dataset",
"file",
"separate",
"concerns",
"lazy",
"fixture",
"union"
]
classifiers = [
# Complete classifiers list is available here: https://pypi.org/classifiers/
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"License :: OSI Approved :: BSD License",
"Topic :: Software Development :: Libraries :: Python Modules",
"Topic :: Software Development :: Testing",
"Programming Language :: Python",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Framework :: Pytest"
]
# Project dependencies (excluding dev and tools ones)
dependencies = [
"decopatch", # todo add version lower bound
"makefun>=1.15.1",
# "packaging", to check : is this still needed ?
"pytest"
]
# Define here (if any) the "extra" dependencies here, installable with "pip install your_package[your_extra]"
# [project.optional-dependencies]
# your-extra = ["your-deps>=1,<2"]
[project.urls]
repository = "https://github.com/smarie/python-pytest-cases"
homepage = "https://smarie.github.io/python-pytest-cases/"
documentation = "https://smarie.github.io/python-pytest-cases/"
# Define here (if any) the console cmd to launch your application
# [project.scripts]
# my-cli = "pytest_cases:my_function"
# -----------------------------------------------
# Dependencies management
# -----------------------------------------------
# Below there is one group for each nox session.
# The extra 'dev' nox session (not used in ci cd) is to create the dev environment containing all tools.
[dependency-groups]
dev = [
"nox[uv]>=2025",
"pre-commit==4.0.1",
]
format = ["ruff>=0.9.0,<0.10"]
lint = [
"ruff>=0.9.0,<0.10",
# "junit2html>=30.1.3,<31",
]
typing = [
"mypy>=1.8,<2",
"typing-extensions>=4.9.0,<5",
]
# note: we probably have to extend some versions numbers or remove them to match the nox.parametrize versions matrix to work
test = [
"pytest>=6,<9",
"pytest-steps",
"pytest-harvest",
"pytest-asyncio"
]
doc = [........]
[tool.uv]
default-groups = []
# -----------------------------------------------
# Build-backend choice and requirements
# -----------------------------------------------
[build-system]
requires = ["hatchling>=1.14.1,<2", "uv-dynamic-versioning>=0.8.1,<0.9.0"]
build-backend = "hatchling.build"
[tool.hatch.version]
source = "uv-dynamic-versioning"
[tool.hatch.build.hooks.version]
path = "pytest_cases/_version.py"
[tool.hatch.build.targets.sdist]
include = [
"pytest_cases",
"pytest_cases/_version.py",
]
[tool.hatch.build.targets.wheel]
include = [
"pytest_cases",
"pytest_cases/_version.py",
]
# -----------------------------------------------
# Versioning management
# -----------------------------------------------
[tool.uv-dynamic-versioning]
vcs = "git"
style = "semver"
pattern = "^(?P<base>\\d+\\.\\d+\\.\\d+)(-?((?P<stage>[a-zA-Z]+)\\.?(?P<revision>\\d+)?))?$"
# -----------------------------------------------
# Development tools
# -----------------------------------------------
[tool.mypy]
plugins = [
"pydantic.mypy",
"numpy.typing.mypy_plugin"
]
ignore_missing_imports = true
follow_imports = "silent"
warn_redundant_casts = true
warn_no_return = false
warn_unused_ignores = true
disallow_any_generics = false
check_untyped_defs = true
no_implicit_reexport = false
# For strict mypy put 'true'
disallow_untyped_defs = false
[tool.pytest.ini_options]
addopts = "-s" # Make prints and logs to appear 'live' (not after test end)
[tool.coverage.run]
branch = true
omit = [
"*tests*",
".nox/*",
]
[tool.coverage.report]
fail_under = 80
ignore_errors = true
show_missing = true
exclude_lines = [
"def __repr__",
"if self.debug:",
"if settings.DEBUG",
"raise AssertionError",
"except ImportError:",
"raise NotImplementedError",
"if 0:",
"if __name__ == .__main__.:",
"if TYPE_CHECKING:",
"class .*\\bProtocol\\):",
"@(abc\\.)?abstractmethod",
]
[tool.ruff]
line-length = 120
exclude = [".git", ".mypy_cache", ".pytest_cache", ".nox/*", ".venv", ".*_version.py"]
extend-include = ["*.ipynb"] # Include explicitly Jupyter notebooks
[tool.ruff.lint]
preview = true
explicit-preview-rules = true
# By default all rules are enabled, if you won't respect one rule, you have to explicitly ignore it
select = ["ALL"]
extend-select = ["CPY001"] # To be removed when rule is not in preview anymore #75
ignore = [
# Whole rules families
"D", # pydocstyle: https://docs.astral.sh/ruff/rules/#pydocstyle-d
"ANN", # flake8-annotations: https://docs.astral.sh/ruff/rules/#flake8-annotations-ann
"FIX", # flake8-fixme: https://docs.astral.sh/ruff/rules/#flake8-fixme-fix
"COM", # flake8-commas: https://docs.astral.sh/ruff/rules/#flake8-commas-com
"EM", # flake8-errmsg: https://docs.astral.sh/ruff/rules/#flake8-errmsg-em
# Specific ignores see https://github.schneider-electric.com/AIHub-Common/pilot-python/issues/103
# TODOs
"TD002",
"TD003",
"TD004",
# elif and the like
"RET504",
"RET505",
"RET506",
"SIM102",
"PLR5501",
# dicts
"C408",
# pandas
"PD003",
# typing
"TCH003",
"TCH002",
# Specific rule code
# flake8-unused-arguments (ARG)
"ARG002", # unused-method-argument: https://docs.astral.sh/ruff/rules/unused-method-argument/
"ARG003", # unused-class-method-argument: https://docs.astral.sh/ruff/rules/unused-class-method-argument/
"ARG004", # unused-static-method-argument: https://docs.astral.sh/ruff/rules/unused-static-method-argument/
# tryceratops (TRY)
"TRY002", # raise-vanilla-class: https://docs.astral.sh/ruff/rules/raise-vanilla-class/
"TRY003", # raise-vanilla-args: https://docs.astral.sh/ruff/rules/raise-vanilla-args/
]
[tool.ruff.lint.per-file-ignores]
# Noxfile legacy
"noxfile.py" = ["A", "SIM"]
# Accept 'assert' inside tests
"tests/*" = ["S101"]
# Auto-generated file with singles quotes
"_version.py" = ["Q000", "UP009", "CPY001"]
[tool.ruff.lint.mccabe]
max-complexity = 10
[tool.ruff.lint.pylint]
max-args = 10 # Raise an error if a function has more than `n` arguments
[tool.ruff.lint.flake8-tidy-imports]
ban-relative-imports = "all"
[tool.ruff.lint.flake8-copyright]
min-file-size = 1 # Do not enforce on empty files, like `__init__.py`
notice-rgx = "blabla"
Here a nox file associated with the above pyproject.toml - there are a few things missing/remaining. The best would be to actually work by comparing current nox file with this one, and transfer the appropriate lines/sessions - this will be more efficient than replacing.
import argparse
import os
import shutil
from pathlib import Path
import nox
from nox import Session
MODULE_NAME = "pytest_cases"
nox.needs_version = ">=2025.2.9" # No upper bound needed
nox.options.reuse_existing_virtualenvs = True
nox.options.error_on_missing_interpreters = True
# Default backend : uv
nox.options.default_venv_backend = "uv"
# Specify which sessions will be run (in this order) at command invocation `nox`.
# If you want to run another specific session, explicitly invoke it ; for example `nox -s dev`.
nox.options.sessions = ["test", "format", "lint", "typing", "doc"]
# Python version(s) supported by this project, read from the `pyproject.toml` file.
PYTHON_VERSIONS = nox.project.python_versions(nox.project.load_toml("pyproject.toml"), max_version="4")
# Note: we need to check how reported html pages were integrated in mkdocs final page earlier. Indeed the dirs
# below (and nox doc session) correspond to a sphinx doc system, which I would not like to use.
PROJECT_DIR = Path(__file__).parent
MODULE_DIR = PROJECT_DIR / MODULE_NAME
REQUIREMENTS_DIR = PROJECT_DIR / "requirements"
DOC_DIR = PROJECT_DIR / "doc"
DOC_BUILT_DIR = PROJECT_DIR / "site"
DOC_EXTRA_DIR = DOC_DIR / "extra" # Folder for extra static pages
REPORT_DIR = DOC_EXTRA_DIR / "reports"
PYTEST_DIR = REPORT_DIR / "pytest"
COVERAGE_DIR = REPORT_DIR / "coverage"
LINTER_DIR = REPORT_DIR / "ruff"
# Learn more about Nox sessions usage here:
# https://pages.github.schneider-electric.com/AIHub-Common/usage-documentation/site/verifying/nox_guide.html
@nox.session(python=PYTHON_VERSIONS[-1])
def dev(session: Session) -> None:
"""Set up an environment for a developer, which can be used by an IDE as all in one.
Usage
-----
> nox -s dev
"""
session.run("uv", "sync", "--active", "--all-groups", external=True)
# possible use pre-commit hooks later
# session.run("pre-commit", "install")
@nox.session(python=PYTHON_VERSIONS[-1])
def format(session: Session) -> None:
"""Check that all Python code is compliant with Ruff format.
(`ruff format .` can be used to format manually to ensure compliance. This is not automatic on purpose)
Usage
-----
> nox -s format
"""
session.run("uv", "sync", "--active", "--locked", "--only-group=format", external=True)
session.run("ruff", "check", ".", "--select", "I") # Sort imports
session.run("ruff", "format", ".", "--check")
@nox.session(python=PYTHON_VERSIONS[-1])
def lint(session: Session) -> None:
"""Runs Ruff linter for Python files.
Usage
-----
> nox -s lint
"""
session.run("uv", "sync", "--active", "--locked", "--only-group=lint", external=True)
shutil.rmtree(LINTER_DIR, ignore_errors=True)
LINTER_DIR.mkdir(parents=True)
with (LINTER_DIR / "ruff.xml").open(mode="w") as f:
session.run(
"ruff",
"check",
MODULE_NAME,
"--output-format=junit",
"--exit-zero",
stdout=f,
stderr=None,
)
# TODO check if this is compliant with the doc session
session.run(
"junit2html",
str(LINTER_DIR / "ruff.xml"),
"--report-matrix",
str(LINTER_DIR / "ruff.html"),
)
# To have also the complete output in CI, which fails if any error spotted
session.run("ruff", "check", MODULE_NAME)
@nox.session(python=PYTHON_VERSIONS[-1])
def typing(session: Session) -> None:
"""Runs PEP484 type hinting checks.
Usage
-----
> nox -s typing
"""
session.run("uv", "sync", "--active", "--locked", "--group=typing", external=True)
session.run("mypy", MODULE_NAME)
ENVS = {
# python 3.14
(PY314, "pytest-latest"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": ""}},
(PY314, "pytest7.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<8"}},
(PY314, "pytest6.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<7"}},
# python 3.13
(PY313, "pytest-latest"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": ""}},
(PY313, "pytest7.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<8"}},
(PY313, "pytest6.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<7"}},
# python 3.12
(PY312, "pytest-latest"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": ""}},
(PY312, "pytest7.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<8"}},
(PY312, "pytest6.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<7"}},
# python 3.11
# We'll run 'pytest-latest' this last for coverage
(PY311, "pytest7.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<8"}},
(PY311, "pytest6.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<7"}},
# python 3.10
(PY310, "pytest-latest"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": ""}},
(PY310, "pytest7.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<8"}},
(PY310, "pytest6.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<7"}},
# python 3.9
(PY39, "pytest-latest"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": ""}},
(PY39, "pytest7.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<8"}},
(PY39, "pytest6.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<7"}},
# IMPORTANT: this should be last so that the folder docs/reports is not deleted afterwards
(PY311, "pytest-latest"): {"coverage": True, "pkg_specs": {"pip": ">19", "pytest": ""}},
}
ENV_PARAMS = tuple((k[0], v["coverage"], v["pkg_specs"]) for k, v in ENVS.items())
ENV_IDS = tuple(f"{k[0].replace('.', '-')}-env-{k[1]}" for k in ENVS)
@nox.session
@nox.parametrize("python,coverage,pkg_specs", ENV_PARAMS, ids=ENV_IDS)
def test(session: Session) -> None:
"""Launch tests and coverage.
Usage
-----
> nox -s test # Launch sessions for every supported Python version
> nox -s test-{python-version} # Launch session for the given python version, e.g. {python-version}=3.12
"""
session.run("uv", "sync", "--active", "--locked", "--group=test", external=True)
# TODO REPLACE ALL BELOW WITH CURRENT
pytest_session_dir = PYTEST_DIR / f"python-{session.python}"
coverage_session_dir = COVERAGE_DIR / f"python-{session.python}"
shutil.rmtree(pytest_session_dir, ignore_errors=True)
shutil.rmtree(coverage_session_dir, ignore_errors=True)
pytest_session_dir.mkdir(parents=True, exist_ok=True)
coverage_session_dir.mkdir(parents=True, exist_ok=True)
session.run(
"coverage",
"run",
"--source",
str(MODULE_DIR),
"-m",
"pytest",
"--junitxml",
f"{pytest_session_dir}/pytest.xml",
"--html",
f"{pytest_session_dir}/pytest.html",
)
# Generate html/xml reports even if coverage is not sufficient, to be able to investigate
session.run("coverage", "html", "-d", str(coverage_session_dir), success_codes=[0, 2])
session.run(
"coverage",
"xml",
"-o",
str(coverage_session_dir / "coverage.xml"),
success_codes=[0, 2],
)
session.run("coverage", "report")
@nox.session(venv_backend="uv", python=PYTHON_VERSIONS[-1])
def doc(session: Session) -> None:
"""Build the documentation.
Usage
-----
> nox -s doc -- -h # Display documentation
> nox -s doc # To generate the site
> nox -s doc -- -s # To serve the documentation on a local server for faster editing.
"""
session.run("uv", "sync", "--active", "--locked", "--group=doc", external=True)
# TODO FILL WITH DOC SESSION CONTENTS
@nox.session(python=PYTHON_VERSIONS[-1])
def build(session: Session, *, output_dir: str = "dist"):
"""Build wheel for this repository, and put it in the ``output_dir`` (default="dist").
Note: the ``output_dir`` is not cleaned.
Usage
-----
> nox -s build
"""
session.run("uv", "build", "--wheel", "--out-dir", output_dir, external=True)
Hi @smarie! Thank you for the response and the initial examples for the pyproject.toml file and the nox file. I must have been confused between nox and nose. I'll admit that I wasn't familiar with nox before, and now after taking a look at it, it seems pretty cool! So we can skip 4.
As for 3, we can add pyupgrade as a pre-commit hook that updates the code whenever we change a specific document. I'll admit that I prefer to update everything in one go, but I respect your preferences.
I'll start working on 1 + 2 in the following weeks.
Thanks @saroad2 ! Do not hesitate to open a PR as "draft" so that we can discuss any open question / design choice / roadblock on the way if needed