python
Add typing.Reader and typing.Writer
I recently stumbled at a TODO in typeshed that proposes to split IO[AnyStr] into smaller pieces. I think it is a good idea and propose to add two protocols:
class Reader(Protocol[AnyStr]):
def read(self, n: int = ...) -> AnyStr: ...
def readlines(self) -> List[AnyStr]: ...
def close(self) -> None: ...
class Writer(Protocol[AnyStr]):
def write(self, s: AnyStr) -> int: ...
def writelines(self, lines: List[AnyStr]) -> None: ...
def close(self) -> None: ...
Then IO[AnyStr] can subclass both, plus add some less often used methods (also IO should stay nominal). What do you think?
I like this idea. Using the current IO classes in type annotations is a bit problematic because many types are kind of like files, but don't fulfill the entire IO interface (typing.IO has 23 methods and properties). If we add Reader and Writer, many argument type annotations could be changed to just one of these two.
Of course many duck type providers for files don't even support close() or readlines(), so even the proposed Reader protocol may be too wide. (Those libraries can of course define their own Readable protocol.)
I think that close() should not be a part of these protocols -- typically closing a stream is up to the owner, i.e. the party that created (opened) it.
Also you've left out readline() -- that seems a mistake, it's probably the most commonly called I/O method.
Finally this doesn't help much for return types -- we often also want to describe that something returns a readable stream (or a writable one) but the interface promised by many APIs is much wider than the minimal protocols proposed here. This is one of the reasons I punted on this when originally designing the IO API -- the other reason was of course that we didn't have protocols then.
(There are other things one might want to express in the type system too, esp. whether seek() and tell() should work.)
Maybe a good way to go about this is to start replacing argument types in typeshed with custom protocols in the respective modules (marked private so they do not get reused outside the module) to get a feeling what is required in practice.
Maybe a good way to go about this is to start replacing argument types in typeshed with custom protocols in the respective modules (marked private so they do not get reused outside the module) to get a feeling what is required in practice.
I like this idea. If we don't do this, we risk implementing something that won't be that useful and will mostly confuse users.
As seen above I submitted a first pull request implementing my last suggestion. Maybe it would be a good idea to mark those somehow, for example with a link to this issue, to be able to grep them easily later?
You mean in the commit message like above?
I think adding the link in PR description may be better. GitHub only uses the commit message for PR description if there is a single commit in your PR, and as you can see the links to PRs are bigger than to commits.
I think that
close()should not be a part of these protocols -- typically, closing a stream is up to the owner, i.e., the party that created (opened) it.
This seems like an area where Python could look at Golang's interfaces. In the case of IO, they have (concerning this)
- io.Reader
- io.ReadCloser
- io.ReadWriter
- io.ReadWriteCloser
- ....
This allows the end-user to operate with the specificity needed for the problem space at the cost of a more complex interface system. Personally, I've stayed away from Python's IO as it's never been clear to me what the intended use is from the docs.
References
- https://golang.org/pkg/io/
- https://docs.python.org/3/library/typing.html#typing.IO
We have a good starting point for useful protocols already in the _typeshed module. These are based on the actual need of the standard library. I see the future in easy composing of protocols rather than adding various protocol with different permutations of methods. For example using intersection types (#213): SupportsRead[str] & SupportsClose.