`ElementTree.parse` type hinted to return unusable generic types

[hterik]

Bug Report

Since upgrading to mypy 1.15+, using ElementTree has become increasingly difficult. Likely due to https://github.com/python/typeshed/pull/13349. I'm not sure if this extra strictness is expected or if I'm just holding it wrong.

One example is the ElementTree.parse function that now returns a generic ET.ElementTree[ET.Element[str]], which is no longer compatible with basic type signatures like ET.ElementTree, which is interpreted as ElementTree[Element[str] | None].

This adds many challenges that make it difficult to work with.

• The generic argument here is very difficult to understand what it means and is lacking documentation. From what i've been able to deduce it's possible that the root node is of a different Element type or that the root node is None. While perhaps a valid scenario, it adds a big uphill to learning and usability.
• If one were to use these generic types, typing out ET.ElementTree[ET.Element[str]] is a lot more verbose than ET.ElementTree, and i'm questioning if the strictness here is worth it.
• If one tries to type hint the code with these generics. Python throws errors that 'ElementTree.Element' is not subscriptable. One workaround i found for this is to surround the type hints with double-quotes.

To Reproduce

import xml.etree.ElementTree as ET

def process_document(doc: ET.ElementTree) -> None:
    pass

def process_document2(doc: ET.ElementTree[ET.Element[str]]) -> None:    # TypeError: type 'xml.etree.ElementTree.Element' is not subscriptable
    pass

process_document(ET.parse("foo.xml"))     # mypy error: Argument 1 to "process_document" has incompatible type "ElementTree[Element[str]]"; expected "ElementTree[Element[str] | None]"  [arg-type]
process_document2(ET.parse("foo.xml"))

Run with mypy --strict

Expected Behavior

ET.parse returns types assignable to ET.ElementTree

Actual Behavior

Mypy returns errror

# mypy error: Argument 1 to "process_document" has incompatible type "ElementTree[Element[str]]"; expected "ElementTree[Element[str] | None]"  [arg-type]

Workarounds

• Use ET.ElementTree().parse() function instead of ET.parse(), it returns the root-node as a type that can be assigned to ET.Element without any generic arguments.
  This is actually a pretty decent option, if this is considered to be the primary way of using ElementTree from now on, the documentation should reflect that, right now it's suggesting ET.parse()
• Type out the complete "ElementTree[Element[str] | None]" wrapped with double-quotes.

Your Environment

• Mypy version used:
• Mypy command-line flags:
• Mypy configuration options from mypy.ini (and other config files):
• Python version used:

[JelleZijlstra]

See also #14036.

[tungol]

I think I made a mistake in setting a default of str for the generic element. Fixing that should alleviate some pain, and I think some of what you're running into is inconsistencies in how I wrote the signatures.

ElementTree being ElementTree[Element | None] is because the root value of the tree is None when no element is passed to ElementTree.__init__ and ElementTree.parse() hasn't been called yet. I'm inclined to agree that that's not worth the pain. Getting rid of the "None" case also frees us to make ElementTree generic over the type of Elements it's expected to have within it (Elements with str-only tags or Elements with str-and-Comment-or-PI tags), with the default being ElementTree[Any]

You can also avoid the "not subscriptable" error by using from __future__ import annotations.

[tungol]

With the stubs I've put together so far while working on your other ticket, this works without error at runtime or with mypy:

from __future__ import annotations
import xml.etree.ElementTree as ET


def process_document(doc: ET.ElementTree) -> None:
    pass


def process_document2(doc: ET.ElementTree[str]) -> None:
    pass


process_document(ET.parse("foo.xml"))
process_document2(ET.parse("foo.xml"))

I should be able to get an MR up in the next day or two.