nedbat
Partial branches not detected in comprehensions
First of all, thank you for creating coverage.py—it is a fantastic tool, and I use it in all of my Python projects (at least those that have tests 😅).
Describe the bug
I discovered a situation in my code recently where I had a missing test that should have been flagged as a partial branch in my function. However, coverage.py did not flag the branch as partial, and I posited it was because I had written my function as a comprehension.
To confirm this theory, if I rewrite the above as multiline for loop, coverage.py correctly flags the partial branch:
To Reproduce
- What version of Python are you using? 3.11.0
- What version of coverage.py shows the problem? The output of
coverage debug sysis helpful. A. See command output below - What versions of what packages do you have installed? The output of
pip freezeis helpful. A. See requirements.txt in my demo repository (link below) - What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix. A. See my demo repo below
- What commands did you run?
- Write some code using any type of comprehension that contains an
ifclause - Write a test where the result of the comprehension has the same length as the input iterable (meaning the
ifcondition always evaluated toTrueduring the test run) - Run
coverage run -m unittest && coverage report && coverage html - Open
htmlcov/index.htmlin your browser; if you view your source file, you'll see that coverage.py covered all lines in your function - Now, if you rewrite that list comprehension as a multiline
forloop (i.e. appending to a list on each iteration), and rerun the command from step (4), you'll see that coverage.py does flag the partial branch (as expected)
- Write some code using any type of comprehension that contains an
Expected behavior
I expect that a partial branch in a comprehension (of any type) to be recognized/flagged by the coverage report.
Additional context
I've created a repository to demonstrate the issue, including code, tests, and instructions. https://github.com/caleb531/coverage-partial-branch-bug
Debug output
Output of running coverage debug sys:
-- sys -------------------------------------------------------
coverage_version: 6.5.0
coverage_module: /Users/caleb/Repositories/personal/coverage-partial-branch-bug/.virtualenv/lib/python3.11/site-packages/coverage/__init__.py
tracer: -none-
CTracer: available
plugins.file_tracers: -none-
plugins.configurers: -none-
plugins.context_switchers: -none-
configs_attempted: .coveragerc
configs_read: /Users/caleb/Repositories/personal/coverage-partial-branch-bug/.coveragerc
config_file: /Users/caleb/Repositories/personal/coverage-partial-branch-bug/.coveragerc
config_contents: b'# Configuration for coverage.py (https://pypi.python.org/pypi/coverage)\n\n[run]\n# Enable branch coverage\nbranch = True\n\n[report]\n\n# Regexes for lines to exclude from consideration\nexclude_lines =\n\n pragma: no cover\n\n # Ignore non-runnable code\n if __name__ == .__main__.:\n\n pass\n\n# Only check coverage for source files\ninclude =\n animal_utils.py\n'
data_file: -none-
python: 3.11.0 (main, Oct 26 2022, 19:06:18) [Clang 14.0.0 (clang-1400.0.29.202)]
platform: macOS-13.0.1-arm64-arm-64bit
implementation: CPython
executable: /Users/caleb/Repositories/personal/coverage-partial-branch-bug/.virtualenv/bin/python
def_encoding: utf-8
fs_encoding: utf-8
pid: 16250
cwd: /Users/caleb/Repositories/personal/coverage-partial-branch-bug
path: /Users/caleb/Repositories/personal/coverage-partial-branch-bug/.virtualenv/bin
/opt/homebrew/Cellar/[email protected]/3.11.0/Frameworks/Python.framework/Versions/3.11/lib/python311.zip
/opt/homebrew/Cellar/[email protected]/3.11.0/Frameworks/Python.framework/Versions/3.11/lib/python3.11
/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/lib/python3.11/lib-dynload
/Users/caleb/Repositories/personal/coverage-partial-branch-bug/.virtualenv/lib/python3.11/site-packages
environment: HOME = /Users/caleb
PYTHONDONTWRITEBYTECODE = 1
command_line: /Users/caleb/Repositories/personal/coverage-partial-branch-bug/.virtualenv/bin/coverage debug sys
sqlite3_version: 2.6.0
sqlite3_sqlite_version: 3.39.4
sqlite3_temp_store: 0
sqlite3_compile_options: ATOMIC_INTRINSICS=1, COMPILER=clang-14.0.0, DEFAULT_AUTOVACUUM,
DEFAULT_CACHE_SIZE=-2000, DEFAULT_FILE_FORMAT=4,
DEFAULT_JOURNAL_SIZE_LIMIT=-1, DEFAULT_MMAP_SIZE=0, DEFAULT_PAGE_SIZE=4096,
DEFAULT_PCACHE_INITSZ=20, DEFAULT_RECURSIVE_TRIGGERS,
DEFAULT_SECTOR_SIZE=4096, DEFAULT_SYNCHRONOUS=2,
DEFAULT_WAL_AUTOCHECKPOINT=1000, DEFAULT_WAL_SYNCHRONOUS=2,
DEFAULT_WORKER_THREADS=0, ENABLE_COLUMN_METADATA, ENABLE_FTS3,
ENABLE_FTS3_PARENTHESIS, ENABLE_FTS4, ENABLE_FTS5, ENABLE_GEOPOLY,
ENABLE_MATH_FUNCTIONS, ENABLE_PREUPDATE_HOOK, ENABLE_RTREE, ENABLE_SESSION,
MALLOC_SOFT_LIMIT=1024, MAX_ATTACHED=10, MAX_COLUMN=2000,
MAX_COMPOUND_SELECT=500, MAX_DEFAULT_PAGE_SIZE=8192, MAX_EXPR_DEPTH=1000,
MAX_FUNCTION_ARG=127, MAX_LENGTH=1000000000, MAX_LIKE_PATTERN_LENGTH=50000,
MAX_MMAP_SIZE=0x7fff0000, MAX_PAGE_COUNT=1073741823, MAX_PAGE_SIZE=65536,
MAX_SQL_LENGTH=1000000000, MAX_TRIGGER_DEPTH=1000,
MAX_VARIABLE_NUMBER=250000, MAX_VDBE_OP=250000000, MAX_WORKER_THREADS=8,
MUTEX_PTHREADS, SYSTEM_MALLOC, TEMP_STORE=1, THREADSAFE=1
Hey @nedbat I would appreciate your thoughts on this.
I'm also seeing this behaviour on a project using Python 3.10.13 with Coverage.py 7.3.1 on debian bullseye
For anyone else coming to this, it's noted in the version 7.2.6 release notes that:
Python 3.12 beta 1 now inlines comprehensions. Previously they were compiled as invisible functions and coverage.py would warn you if they weren't completely executed. This no longer happens under Python 3.12.
So I am assuming it is indeed expected behaviour that comprehensions should be flagged as uncovered correctly in earlier versions of Python?
