Attribute Error with Union and isinstance

[geoffreywiseman]

From the looks of a few other issues (#491, #391) it seems like pytype is supposed to handle isinstance checks, but ... I'm getting an error and I'm not sure if it's something that pytype is supposed to handle.

I have a variable that's a union:

confirmation_value: Union[str, Pattern]

And check if it's a pattern, and if so, a match:

if isinstance(confirmation_value, Pattern):
    return any(confirmation_value.match(item) for item in page_content)

This gives me an error:

File "/path/to/file.py", line 475, in <genexpr>: No attribute 'match' on str [attribute-error]
  In Union[Pattern[Union[bytes, str]], str]

Should pytype be handling this? Am I doing something wrong that I haven't noticed? Is there a way to let pytype know that it's ok? Or is this just beyond the limits of pytype? Just trying to understand.

[rchen152]

Huh, pytype appears to be having problems with PEP 526-style annotations. This passes checking:

from typing import Union
v = None  # type: Union[int, str]
if isinstance(v, str):
  print(v.upper())

But this doesn't:

from typing import Union
v: Union[int, str]
if isinstance(v, str):
  print(v.upper())

[JulesGM]

I also have errors with this type of situation.

def fn() -> Union[tf.Tensor, Tuple[tf.Tensor, ...]]:
   ...

tensor_or_tuple = fn()
if not isinstance(tensor_or_tuple, tf.Tensor):
    raise RuntimeError(type(tensor_or_tuple))
print(tensor_or_tuple.shape[1])  # pytype complains here, attribute-error, tuples don't have .shape

and

def fn() -> Union[tf.Tensor, Tuple[tf.Tensor, ...]]:
   ...

tensor_or_tuple = fn()
assert isinstance(tensor_or_tuple, tf.Tensor), type(tensor_or_tuple)
print(tensor_or_tuple.shape[1])  # pytype complains here, attribute-error, tuples don't have .shape

That's too bad

[cebtenzzre]

Here a similar false positive, though it seems to be specific to Sequence (not Tuple, str, or Pattern). Not sure if it's worth opening a separate issue.

from typing import Sequence, Union

class Foo:
    y: Sequence[int]
    def __init__(self, x: Union[int, Sequence[int]]):
        if isinstance(x, Sequence):
            reveal_type(x)
            self.y = x

$ pytype-single --check-attribute-types repr.py
File "repr.py", line 7, in __init__: Union[Sequence[int], int] [reveal-type]
File "repr.py", line 8, in __init__: Type annotation for y does not match type of assignment [annotation-type-mismatch]
  Annotation: Sequence[int]
  Assignment: int
  In assignment of type: Union[Sequence[int], int]

@rchen152 With current pytype, the important difference between your two examples seems to be the = None, not type comments. If v is assigned from e.g. an untyped parameter, current pytype gives no errors in either example, so maybe it is oversimplified?

[rchen152]

@JulesGM Your issue looks like it's caused by the fact that pytype infers Any for types from third_party libraries that aren't present in typeshed. It's a tricky problem to solve in general, but we'll be doing some work next year to improve support for tensorflow in particular.

@Cebtenzzre You're right that the important difference between the two code snippets in my example seems to be the presence of = None, not the style of annotation. Thanks! The Sequence issue appears to be a different bug: https://github.com/google/pytype/blob/22545931decb6e14a63e1ed66c81899c9277203b/pytype/special_builtins.py#L355 The _is_instance implementation checks values against a "class spec", which is expected to be a single class object. Since Sequence is an annotation, it's instead a special object called an AnnotationClass, so we need to convert it to a normal class. Should be fixed at head sometime today or tomorrow.

[rchen152]

Pytype no longer reports an error for either code snippet in https://github.com/google/pytype/issues/533#issuecomment-602271962, so this appears to be fixed.