python
Add static_assert to verify type constraints in mypy
Mypy's type checking works well in many cases, but there are some complex cases that are difficult to code for, and would be impossible or costly to check at runtime. I give an example of a case where this would be useful in #5666. While I give a slightly cumbersome possible solution there, another possibility is to use explicit casting:
from abc import ABC, abstractmethod
from typing import Generic, TypeVar, cast
_InputType = TypeVar('_InputType', contravariant=True)
_IntermediateType = TypeVar('_IntermediateType')
_OutputType = TypeVar('_OutputType', covariant=True)
class GenericBase(Generic[_InputType, _IntermediateType, _OutputType], ABC):
@abstractmethod
def first_step(self, pipeline_input: _InputType) -> _IntermediateType: ...
def second_step(self, state: _IntermediateType) -> _OutputType:
# By default, pass through state unmodified
return cast(_OutputType, state)
def execute(self, pipeline_input: _InputType) -> _OutputType:
state = self.first_step(pipeline_input)
return self.second_step(state)
This works, but it eliminates type safety, as I have to trust that the implementing class chooses _IntermediateType and _OutputType in a way that that cast makes sense. If these were not TypeVars, I could do a runtime check like assert issubclass(_IntermediateType, _OutputType) to verify that this cast is safe, but obviously Python does not have enough information at runtime to do an issubclass check with TypeVars or parameterized Generics.
I propose adding a static_assert statement, which would be ignored at runtime, but would be capable of evaluating statements about Types in static analysis. So for example, in the code above, I could add:
def second_step(self, state: _IntermediateType) -> _OutputType:
# By default, pass through state unmodified
static_assert issubclass(_IntermediateType, _OutputType)
return cast(_OutputType, state)
In the definition of the base class, this would be assumed to be true if there were any possible unification of _IntermediateType and _OutputType that would allow it to be true (defaulting to true or producing a warning if this is non-trivial to evaluate), but this static_assert would be re-evaluated, whenever this class was subclassed/instantiated or this method was called, and if it could ever be determined to be false, would cause Mypy to raise an error.
Any thoughts on this?
I realize the particular syntax I have suggested here might be difficult to implement, as it would require extending Python so it knows not to evaluate the issubclass call here. An alternative would be to define a function static_assert_issubclass(class, classinfo) that would behave like static_assert issubclass(class, classinfo) but would not require extending Python.
Mypy already has such feature, it is called cast(). Of course it is not exactly the same as you propose, but I believe cast() is sufficient for your purpose.
FWIW there's a type of cast (probably downcast() would be a good name) that I am missing more and more. It differs from cast() in two ways:
- It inserts a runtime check that the object being cast is an instance of the target class
- The static check ensures that its type is an instance of a supertype of the target type other than object (perhaps also disallowing ABC and a few others)
Thus, if we had these classes:
class A: ... class B(A): ... class C(B): ... class D(C): ...
def foo(d: D): ...
def bar() -> B: return D()
def baz() -> B: return C()
foo(downcast(D, bar())) # Passes both static and dynamic checks
foo(downcast(D, baz())) # Passes static check, fails dynamic check
class X: ...
foo(downcast(D, X())) # Fails static check (X and D have only object in common in their MRO)
Yes, downcast() may be useful indeed. We can try experimenting with it, but the main problem is scheduling/prioritization. There are many other important tasks that needs to be done soon.
Mypy already has such feature, it is called
cast(). Of course it is not exactly the same as you propose, but I believecast()is sufficient for your purpose.
@ilevkivskyi My whole point here is that I would like a type-checked cast. So for example, if cast worked the way I wanted, then this should be an error:
from typing import cast, Optional, Generic, TypeVar
class A:
pass
class B:
pass
a: Optional[A] = None
a = cast(A, B()) # Should be type error
and the case I would really like to be able to check is this one:
T = TypeVar('T')
U = TypeVar('U')
class C(Generic[T, U]):
def method(self, t: T) -> U:
return cast(U, t)
This case might or might not be an error, depending on how T and U are defined, but there is not to my knowledge a way to check that they are defined in compatible ways. If I remove the cast, it is always an error, which is not what I want either.
I agree that something like downcast() would be quite useful. I remember discussing similar ideas before, but the discussions never went anywhere. One issue with the proposed semantics would be that only instance types would likely be usable as the target type (or a tuple of classes). Maybe that's fine, though.
@gvanrossum Can you create a separate issue focused on downcast()?
+1 a static_assert would be useful. For my particular use case, I'm not looking to do any casting, but just to add static asserts to tests and such to make sure "compile"-time types are what I expect them to be. My current workflow involves sprinkling in reveal_type() and then verifying manually. Runtime type checks in tests are not sufficient because "compile"-time types may be overly broad (or incorrect).
A general/arbitrary static_assert may be tricky to implement, but as proposed above, maybe static_assert_is_subclass would be a good start, and if we go that route, it'd be good to also include a static_assert_is_same for exact type matches.
+1, I'm running into this when converting third party API responses into properly typed dataclasses. I want to be able to write:
raw_response: JsonDict
@dataclass
class ResponseDataclass:
field: str
ResponseDataclass(
field=downcast(str, raw_response['field'])
)
I can give a stab at implementing this if a PR is welcome?
As an aside, if we could declare lower bounds based on TypeVars we could implement downcast as follows:
TargetType = TypeVar('TargetType')
ValueType = TypeVar('ValueType', lower_bound=TargetType)
def downcast(target: Type[TargetType], value: ValueType) -> TargetType:
assert isinstance(value, target)
return value
Or even better if we also have Intersection:
TargetType = TypeVar('TargetType')
ValueType = TypeVar('ValueType', lower_bound=TargetType)
def downcast(target: Type[TargetType], value: ValueType) -> Intersection[TargetType, ValueType]:
assert isinstance(value, target)
return value
The latter would also take care of assigning the correct type parameters if e.g. target = dict and value: Union[str, Dict[str, str]].
@SemMulder Since we still don't have a consensus of how the feature would behave exactly, the easiest way to move forward might be to add it to mypy_extensions (https://github.com/python/mypy_extensions) and implement support in mypy. Once folks are happy with how it works, we can propose to add it to typing/typing_extensions. I'm open to accepting an implementation to mypy, but it would be good to discuss the precise semantics a bit first -- in particular, what would the behavior be with all possible types, including things like TypedDict. This can happen here or in the mypy issue tracker.
Is there currently a way to make a version of cast that has only property 1 of https://github.com/python/mypy/issues/5687#issuecomment-425654603?
So it behaves exactly like cast for static checking, but does an assert isinstance call at runtime (and of course only works with types compatible with isinstance)?
The function in untyped python is:
# checked cast
def chcast(class_or_class_tuple, x):
assert isinstance(x, class_or_class_tuple)
return x
@DustinWehr: With this annotation your function looks like a cast to mypy:
from typing import Any, Type, TypeVar
T = TypeVar('T')
def checked_cast(t: Type[T], x: Any) -> T:
assert isinstance(x, t), type(x)
return x
Note that I dropped the tuple support in the type argument to keep things simple.
If you're going to use this in actual code, it might be better to raise TypeError instead of AssertionError.
I have a real-world example where this would be useful. In my case I have the data role which determines if the passed value should go into the _data dict. Since the base class has typing.Any as the value, I don't want to restrict it in the function definition.
import typing
from PyQt5 import QtCore
T = typing.TypeVar('T')
class CustomModel(QtCore.QAbstractTableModel, typing.Generic[T]):
def __init__(self, parent: typing.Optional[QtCore.QObject] = None) -> None:
super().__init__(parent)
self._data: typing.Dict[typing.Tuple[int, int], T] = {}
def setData(self,
index: QtCore.QModelIndex,
value: typing.Any,
role: int = QtCore.Qt.DisplayRole,
) -> typing.Any:
if role == QtCore.Qt.DisplayRole or role == QtCore.Qt.EditRole:
# static_assert(value, T)
self._data[index.row(), index.column()] = value
return super().setData(index, value, role)
def main() -> None:
app = QtCore.QCoreApplication([])
model = CustomModel[int]()
index = model.index(0, 0)
model.setData(index, 0) # This is okay
index = model.index(0, 1)
model.setData(index, '0') # I want mypy to report an error here, but currently it does not because of typing.Any.
if __name__ == '__main__':
main()
There are a couple different feature requests here, but the relatively new assert_type can be used to do some simple static-only assertions.
Any additional requests should be discussed at https://github.com/python/typing
