Make `MemoryView` Generic, make `cast` accurate

[max-muoto]

As detailed in https://github.com/python/typeshed/issues/8182, the stubs for memoryview currently represent a sequence of ints, while at runtime this vary across ints, bytes, floats, and booleans at runtime after casting.

This PR fixes this by making MemoryView generic, while using 3.13 typevar defaults to avoid any type of large-scale regression, and adding appropiate overloads when casting.

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

freqtrade (https://github.com/freqtrade/freqtrade)
+ /tmp/mypy_primer/projects/_freqtrade_venv/lib/python3.12/site-packages/sqlalchemy/sql/compiler.py:677: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
+ https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
+ Please report a bug at https://github.com/python/mypy/issues
+ version: 1.10.1
+ note: use --pdb to drop into pdb
+ Traceback (most recent call last):
+   File "mypy/semanal.py", line 6714, in accept
+   File "mypy/nodes.py", line 1142, in accept
+   File "mypy/semanal.py", line 1621, in visit_class_def
+   File "mypy/semanal.py", line 1661, in analyze_class
+   File "mypy/semanal.py", line 2206, in analyze_base_classes
+   File "mypy/semanal.py", line 6733, in expr_to_analyzed_type
+   File "mypy/semanal_namedtuple.py", line 297, in check_namedtuple
+   File "mypy/semanal_namedtuple.py", line 486, in build_namedtuple_typeinfo
+   File "mypy/semanal.py", line 6063, in named_type
+ AssertionError: 

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/web.py:353:25: error: Argument 2 to "TCPSite" has incompatible type "str | memoryview[Any]"; expected "str | None"  [arg-type]

aioredis (https://github.com/aio-libs/aioredis)
- aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview | int")  [operator]
+ aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview[int] | int")  [operator]
- aioredis/connection.py:934: error: Item "memoryview" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "memoryview[int]" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/connection.py:934: error: Item "int" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "int" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Any | None]", variable has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
+ aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Any | None]", variable has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
- aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview, Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
+ aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview[int], Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
- aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]
+ aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

freqtrade (https://github.com/freqtrade/freqtrade)
+ /tmp/mypy_primer/projects/_freqtrade_venv/lib/python3.12/site-packages/sqlalchemy/sql/compiler.py:677: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
+ https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
+ Please report a bug at https://github.com/python/mypy/issues
+ version: 1.10.1
+ note: use --pdb to drop into pdb
+ Traceback (most recent call last):
+   File "mypy/semanal.py", line 6714, in accept
+   File "mypy/nodes.py", line 1142, in accept
+   File "mypy/semanal.py", line 1621, in visit_class_def
+   File "mypy/semanal.py", line 1661, in analyze_class
+   File "mypy/semanal.py", line 2206, in analyze_base_classes
+   File "mypy/semanal.py", line 6733, in expr_to_analyzed_type
+   File "mypy/semanal_namedtuple.py", line 297, in check_namedtuple
+   File "mypy/semanal_namedtuple.py", line 486, in build_namedtuple_typeinfo
+   File "mypy/semanal.py", line 6063, in named_type
+ AssertionError: 

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/web.py:353:25: error: Argument 2 to "TCPSite" has incompatible type "str | memoryview[Any]"; expected "str | None"  [arg-type]

aioredis (https://github.com/aio-libs/aioredis)
- aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview | int")  [operator]
+ aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview[int] | int")  [operator]
- aioredis/connection.py:934: error: Item "memoryview" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "memoryview[int]" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/connection.py:934: error: Item "int" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "int" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Any | None]", variable has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
+ aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Any | None]", variable has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
- aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview, Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
+ aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview[int], Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
- aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]
+ aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/web.py:353:25: error: Argument 2 to "TCPSite" has incompatible type "str | memoryview[Any]"; expected "str | None"  [arg-type]

aioredis (https://github.com/aio-libs/aioredis)
- aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview | int")  [operator]
+ aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview[int] | int")  [operator]
- aioredis/connection.py:934: error: Item "memoryview" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "memoryview[int]" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/connection.py:934: error: Item "int" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "int" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Any | None]", variable has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
+ aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Any | None]", variable has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
- aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview, Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
+ aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview[int], Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
- aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]
+ aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]

[JelleZijlstra]

Sounds like you found a mypy bug! Would be good to minimize it and report it over on the mypy repo.

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/web.py:353:25: error: Argument 2 to "TCPSite" has incompatible type "str | memoryview[Any]"; expected "str | None"  [arg-type]

aioredis (https://github.com/aio-libs/aioredis)
- aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview | int")  [operator]
+ aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview[int] | int")  [operator]
- aioredis/connection.py:934: error: Item "memoryview" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "memoryview[int]" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/connection.py:934: error: Item "int" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "int" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Any | None]", variable has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
+ aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Any | None]", variable has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
- aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview, Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
+ aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview[int], Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
- aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]
+ aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]

[max-muoto]

Most of the errors:

- aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview | int")  [operator]
+ aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview[int] | int")  [operator]
- aioredis/connection.py:934: error: Item "memoryview" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "memoryview[int]" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/connection.py:934: error: Item "int" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "int" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Any | None]", variable has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
+ aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Any | None]", variable has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
- aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview, Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
+ aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview[int], Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
- aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]
+ aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]

Are not net-new (just memoryview -> memoryview[int]).

For web.py in aiohttp,

class TCPSite(BaseSite):
    __slots__ = ("_host", "_port", "_reuse_address", "_reuse_port")

    def __init__(
        self,
        runner: "BaseRunner",
        host: Optional[str] = None,
        port: Optional[int] = None,
        *,

We see that the isinstance check should only narrow to str assuming TCPSite has accurate types.

[max-muoto]

Sounds like you found a mypy bug! Would be good to minimize it and report it over on the mypy repo.

Are you referring to one of the previous crashes, or something else? Or perhaps the aiohttp error not getting caught before.

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/web.py:353:25: error: Argument 2 to "TCPSite" has incompatible type "str | memoryview[Any]"; expected "str | None"  [arg-type]

aioredis (https://github.com/aio-libs/aioredis)
- aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview | int")  [operator]
+ aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview[int] | int")  [operator]
- aioredis/connection.py:934: error: Item "memoryview" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "memoryview[int]" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/connection.py:934: error: Item "int" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "int" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Any | None]", variable has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
+ aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Any | None]", variable has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
- aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview, Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
+ aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview[int], Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
- aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]
+ aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]

[JelleZijlstra]

Sounds like you found a mypy bug! Would be good to minimize it and report it over on the mypy repo.

Are you referring to one of the previous crashes, or something else? Or perhaps the aiohttp error not getting caught before.

I'm referring to the crash.

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/web.py:353:25: error: Argument 2 to "TCPSite" has incompatible type "str | memoryview[Any]"; expected "str | None"  [arg-type]

aioredis (https://github.com/aio-libs/aioredis)
- aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview | int")  [operator]
+ aioredis/connection.py:933: error: Unsupported right operand type for in ("bytes | memoryview[int] | int")  [operator]
- aioredis/connection.py:934: error: Item "memoryview" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "memoryview[int]" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/connection.py:934: error: Item "int" of "bytes | memoryview | int" has no attribute "split"  [union-attr]
+ aioredis/connection.py:934: error: Item "int" of "bytes | memoryview[int] | int" has no attribute "split"  [union-attr]
- aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Any | None]", variable has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
+ aioredis/client.py:4114: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Any | None]", variable has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]")  [assignment]
- aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview, Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
+ aioredis/client.py:4158: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[bytes | str | memoryview[int], Any | None]"; expected "SupportsKeysAndGetItem[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]"  [arg-type]
- aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview, Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]
+ aioredis/client.py:4172: error: Incompatible types in assignment (expression has type "dict[bytes | str | memoryview[int], Callable[[dict[str, str]], Awaitable[None]]]", variable has type "dict[Any, Any | None]")  [assignment]

[max-muoto]

Sounds like you found a mypy bug! Would be good to minimize it and report it over on the mypy repo.

Are you referring to one of the previous crashes, or something else? Or perhaps the aiohttp error not getting caught before.

I'm referring to the crash.

Sounds good, I'll put up an issue in that case.

[max-muoto]

Just wanted to bump this, and see if anyone was interested in reviewing?