snakefmt icon indicating copy to clipboard operation
snakefmt copied to clipboard

IndexError when `snakefmt` parses `pyproject.toml` (upstream `toml` bug)

Open thisIsMikeKane opened this issue 8 months ago • 0 comments

Summary

Running snakefmt on a project whose pyproject.toml contains certain mixed-type arrays crashes with IndexError: list index out of range.

The same symptom was reported in the now-closed issue #112 but never resolved; the original reporter said "the error is gone". After debugging I’ve traced the root cause to the third-party toml library that snakefmt pulls in to parse pyproject.toml.

Steps to reproduce

1. Create a minimal pyproject.toml that triggers the bug

[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "sample-package"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = [
  "requests>=2.31",
]
[dependency-groups]
test = ["pytest>=8.2"]
dev = [
    "ruff>=0.4",
	# Commenting out this next line hides the issue
    { include-group = "test" },
]

2. Run snakefmt (version 0.10.2 here)

snakefmt

Actual behaviour

Traceback (most recent call last):
  ...
  File ".../site-packages/toml/decoder.py", line 1002, in load_array
    ... = ...  # → IndexError here
IndexError: list index out of range

Expected behaviour

snakefmt should either format the file or, at worst, emit a helpful error message—not crash.

Environment

  • snakefmt 0.11.0
  • Python 3.12.10
  • toml 0.10.2
  • OS – Ubuntu 22.04 (WSL2 on Windows 10)

snakefmt reads pyproject.toml for configuration, which is why the crash occurs before any files are formatted. (PyPI)

Root cause analysis

  • In the failing code path, snakefmt calls toml.load().
  • The error propagates from toml/decoder.py where the parser cannot handle arrays that mix types.
  • Very similar stack traces are discussed in these toml issues:
    • uiri/toml #449 - "Error parsing PEP 735 dependent dependency groups"
    • uiri/toml #444 - "Unable to parse array of mixed types (string + array) - IndexError: list index out of range"
    • uiri/toml #270 - "mixed types in array don't work"

Proposed fix

Because the toml library is effectively unmaintained and Python 3.11+ ships with the standard-library tomllib:

  1. Replace the dependency on toml with tomllib (and tomli as a back-port for < 3.11). Pros: no external dependency for modern Pythons, avoids the upstream bug entirely. Cons: minimal shim code to keep the interface identical.

Work-around

For now I can prevent the crash by keeping my pyproject.toml free of mixed-type arrays or supplying a minimal "dummy" snakefmt.toml.

Why this matters

  • Prevents unexpected crashes for anyone with PEP 735 dependent dependency groups.
  • Removes an unmaintained dependency.
  • Aligns snakefmt with Python’s standard-library solution going forward.

thisIsMikeKane avatar May 18 '25 01:05 thisIsMikeKane