cpython icon indicating copy to clipboard operation
cpython copied to clipboard

gh-111353: GenericAlias support and TypeVarLike resolution for `typing.get_type_hints`

Open NCPlayz opened this issue 2 years ago • 6 comments

get_type_hints can now resolve TypeVarLikes correctly for all generic classes, including TypedDicts. There were a lot of edge cases that I tested and subsequently wrote unit tests for. I've also split some pre-existing code in the typing library into their own utility functions (_copy_with to work for copying any generic type with new params and _make_substitution from _GenericAlias._make_substitution).

I've tried my hardest to document each step but please let me know if there are any steps that don't make sense or could be written another way.

  • Issue: gh-111353

NCPlayz avatar Oct 30 '23 18:10 NCPlayz

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

bedevere-app[bot] avatar Oct 30 '23 18:10 bedevere-app[bot]

Wow thank you for this large improvement! I'm sorry I'm not able to review this as it's been some time since I touched that specific code (I wrote some of it but I no longer remember its intricacies). I'll defer to the rest.

Thanks again. Your efforts are much appreciated!

Fidget-Spinner avatar Oct 30 '23 20:10 Fidget-Spinner

I think there is a lack of tests and implementation for handling TypeVarTuple. I have added a comment to the original issue, which includes an implementation for this feature.

zhPavel avatar Oct 30 '23 22:10 zhPavel

Thanks @zhPavel! I've implemented some preliminary support for TypeVarTuple, which seemed to work with the test cases I added. Though, I haven't really used TypeVarTuple too much so there may be some bits that are buggy, please let me know if you find anything!

NCPlayz avatar Oct 31 '23 12:10 NCPlayz

Thank you so much for your work on this. It's truly an essential addition that's been missing from the standard library.

While testing this PR, I stumbled upon a scenario where get_type_hints throws an error when used with TypedDict + Generic + Annotated:

def test_get_type_hints_annotated_generic_typeddict(self):
    class Foo(TypedDict, Generic[T]):
        a: T
        b: Annotated[str, 'annotation']

    self.assertEqual(get_type_hints(Foo[bool]), {'a': bool, 'b': str})

The error I encountered is as follows:

Traceback (most recent call last):
  File "/Users/.../cpython/Lib/test/test_typing.py", line 6482, in test_get_type_hints_annotated_generic_typeddict
    self.assertEqual(get_type_hints(Foo[bool]), {'a': bool, 'b': str})
                     ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../cpython/Lib/typing.py", line 2220, in get_type_hints
    to_sub = _substitute_type_hints(param_tracking[obj], hints)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../cpython/Lib/typing.py", line 2414, in _substitute_type_hints
    sub = _copy_with(value, new_args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../cpython/Lib/typing.py", line 387, in _copy_with
    return t.copy_with(new_args)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../cpython/Lib/typing.py", line 1969, in copy_with
    assert len(params) == 1
           ^^^^^^^^^^^^^^^^
AssertionError

ermakov-oleg avatar Mar 25 '24 14:03 ermakov-oleg

And one more example with an error:

def test_get_type_hints_unbound_typevar(self):
    class Foo(Generic[T]):
        x: list[T]
        y: KT

    self.assertEqual(gth(Foo), {'x': list[T], 'y': KT})  # ok
    self.assertEqual(gth(Foo[int]), {'x': list[int], 'y': KT})    # error

Error:

Traceback (most recent call last):
  File "/Users/.../cpython/Lib/test/test_typing.py", line 6571, in test_get_type_hints_extended_generic_rules_subclassing_with_generic
    self.assertEqual(gth(Foo[int]), {'x': list[int], 'y': KT})    # error
                     ^^^^^^^^^^^^^
  File "/Users/.../cpython/Lib/typing.py", line 2220, in get_type_hints
    to_sub = _substitute_type_hints(param_tracking[obj], hints)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../cpython/Lib/typing.py", line 2416, in _substitute_type_hints
    sub = mapping[value]
          ~~~~~~~^^^^^^^
KeyError: ~KT

ermakov-oleg avatar Mar 25 '24 15:03 ermakov-oleg