python
"Awaitable" is insufficiently expressive, as one may await something other than Futures
In the wild, today, if you await something, there are several potentially valid things to bubble back out from the innermost await
- an
asyncio.Future - a curio "trap" (which I gather is defined as something like
Tuple[curio.traps.Traps, Any, ...]) - a Trio trap, (which I gather is defined as something like
Iterable[WaitTaskRescheduled]although I haven't quite found the bottom of that stack) - a
Deferred
Not all of these are mutually compatible. However, Awaitable is parameterized only on one type - the type of its result. This results in code like this:
def some_deferred() -> Awaitable[int]: ...
def some_future() -> Awaitable[int]: ...
async def x() -> Awaitable[List[int]]:
y = []
y.append(await some_deferred())
y.append(await some_future())
return y
type-checking even though it should fail.
I'd like to be able to express this as
def some_deferred() -> Awaitable[int, Deferred]: ...
def some_future() -> Awaitable[int]: ... # presumably Future is the default
async def x() -> Awaitable[List[int]]:
y = []
y.append(await some_deferred()) # <-- automatically specialize to `Awaitable[List[int], Deferred]` here, if possible?
y.append(await some_future()) # <-- type error!
return y
This is made slightly more complex by the fact that some awaitables are mutually intelligible - for example, anything that awaits a Deferred should eventually be able to await a Future directly, once we've made a few more changes to Twisted. We could also do this in the other direction if __await__ were changed to pass the loop along. Whereas, some of these types are incompatible by design; my understanding is that await implies a completely different, and somewhat higher-level, scheduling protocol in Trio, for example.
This looks like something quite tricky. After thinking for some time I didn't find a solution that wouldn't require a lot of special casing in type checkers.
(Also note that by agreement async defs are annotated without wrapping return type in Awaitable, see PEP 484.)
I'm actually not sure I understand the example. As Ivan says, the type annotation on x seems wrong—it should be -> List[int].
But I also don't see why you want a TypeError in your second code block. If you run await some_deferred() or await some_future(), either way you will get an int, so you are still simply making a list of ints. some_deferred() itself may return a Deferred[int], but that can just be implemented by making Deferred a subclass of Awaitable.
It seems Glyph would like mypy to tell him that he's (incorrectly,
presumably) mixing await
The problem is that there's nothing in the signature of the async function that tells us which it should be -- the class Awaitable doesn't actually occur there, according to PEP 484. So I'm with Ivan -- it's unclear how to proceed.
Some fundamental change in the type system would be needed. Or we would need to reconsider the choice in PEP 484 for annotating async functions -- but even so we would need to design its replacement, and generic classes currently don't support a variable number of arguments, so it would become a complex change proposal there.
On Fri, Mar 9, 2018 at 3:57 PM, Jelle Zijlstra [email protected] wrote:
I'm actually not sure I understand the example. As Ivan says, the type annotation on x seems wrong—it should be -> List[int].
But I also don't see why you want a TypeError in your second code block. If you run await some_deferred() or await some_future(), either way you will get an int, so you are still simply making a list of ints. some_deferred() itself may return a Deferred[int], but that can just be implemented by making Deferred a subclass of Awaitable.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/python/typing/issues/542#issuecomment-371979107, or mute the thread https://github.com/notifications/unsubscribe-auth/ACwrMl0I-85sbYKMYigcRmE3YcQaK_uzks5tcxb3gaJpZM4SjKi_ .
-- --Guido van Rossum (python.org/~guido)
@JelleZijlstra I think you do have a good point, that Deferred should be a subclass of Awaitable and appropriately generic.
One really nasty thing about the way Deferreds are constructed, unfortunately is that the stubs want to look something like this:
class Deferred(Generic[T]):
def addCallback(self, cb: Callable[[T], T2], ...) -> Deferred[T2]: ...
Except, uh, addCallback returns self and its type changes at runtime.
I think maybe just pretending that it doesn't is the best way forward for type annotations in Twisted but I do wish there were some way to express this.
It seems Glyph would like mypy to tell him that he's (incorrectly, presumably) mixing
await <future>andawait <deferred>in the same code.
Yep. Fixing this is pretty easy even now: await deferred.asFuture(loop) or await Deferred.fromFuture(future) depending on which way you're going. But you do still need to be explicit about it.