MagicStack
Missing asyncio feature regarding support of SCTP sockets
-
uvloop version: v0.16.0
-
Python version: 3.10.4
-
Platform: Linux Please note that SCTP works only if you have libsctp(-dev) installed on your Linux system. MacOS doesn't support SCTP without further effort and Windows I cannot say, since I don't use WIndows.
-
Can you reproduce the bug with
PYTHONASYNCIODEBUGin env?: Please clarify if you expect further output from uvloop in this case. The error is an OS error as a result of uvloop OS socket configuration, so there is no additional output I can see other than the output below. I will edit my post accordingly if I am missing information or should try something else. Thanks! -
Does uvloop behave differently from vanilla asyncio? How?: Yes, in vanilla asyncio, SCTP sockets can be used out of the box, behaving in the same way TCP stream sockets do.
I am using SCTP as the transport stream protocol, which works fine with Python vanilla asyncio. Asyncio shows the same behaviour as for TCP sockets, which is expected since both transport protocols are stream protocols. However, uvloop seems to only allow TCP sockets.
Server side SCTP socket issue observation When using an SCTP listening server socket, starting the server works fine. As soon as a SCTP client tries to connect, no connection notification or client data ever reaches the Python application code, but also no errors or other messages are observed on application layer. Wireshark shows that the connection from the client is dropped by the server side right after the SCTP cookie ack handshake.
Client side SCTP socket issue observation Using an SCTP socket for a client raises an OS Error. From the stack trace it looks like uvloop is trying to use TCP under the hood instead of the SCTP socket.
Traceback (most recent call last):
File "/home/lukas/uvloop-bug-report/client.py", line 69, in <module>
asyncio.run(start_client(use_sctp=True))
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "uvloop/loop.pyx", line 1501, in uvloop.loop.Loop.run_until_complete
File "/home/lukas/uvloop-bug-report/client.py", line 55, in start_client
transport, _ = await asyncio.get_event_loop().create_connection(
File "uvloop/loop.pyx", line 2057, in create_connection
File "uvloop/loop.pyx", line 2045, in uvloop.loop.Loop.create_connection
File "uvloop/handles/tcp.pyx", line 177, in uvloop.loop.TCPTransport._open
File "uvloop/handles/tcp.pyx", line 34, in uvloop.loop.__tcp_open
OSError: [Errno 92] Protocol not available
To reproduce the issues, the following client/server code can be used on a Linux machine or wrapped in a Linux based Docker container if you don't have a Linux OS (I am happy to help with this if needed).
Server code
"""
The server side of the asyncio/uvloop SCTP socket demo.
Author: Lukas Stermann <[email protected]>
"""
import asyncio
import socket
from typing import Optional
import uvloop
class ServerProtocol(asyncio.Protocol):
"""
This is just a dummy server echo protocol for demo purposes.
"""
def __init__(self):
self.transport: Optional[asyncio.Transport] = None
def connection_made(self, transport: asyncio.Transport) -> None:
self.transport = transport
print('New connection from client')
def data_received(self, raw_data: bytes) -> None:
print(f'Data received from client: <{raw_data.decode()}>')
self.transport.write(b'pong')
def connection_lost(self, exc: Optional[Exception]) -> None:
print('Connection lost')
self.transport.close()
async def start_server(use_sctp: bool = True) -> None:
"""
This server starts a listening TCP/SCTP socket.
:param use_sctp: Whether the server shall use SCTP.
If set to False it uses TCP.
"""
socket_: socket.socket
if use_sctp:
socket_ = socket.socket(socket.AF_INET6,
socket.SOCK_STREAM,
socket.IPPROTO_SCTP)
else:
socket_ = socket.socket(socket.AF_INET6,
socket.SOCK_STREAM)
socket_.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
if socket.has_dualstack_ipv6():
socket_.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 0)
else:
print('This OS does not support IPv4/IPv6 dual stack. '
'Falling back to IPv6 only.')
socket_.setblocking(False)
socket_.bind(('', 10002))
socket_.listen()
connection_settings: dict = {
'protocol_factory': lambda: ServerProtocol(),
'sock': socket_}
server = await asyncio.get_event_loop().create_server(**connection_settings)
await server.serve_forever()
# Comment in the following line to see the issue with using an SCTP socket in uvloop.
# None of the print statements in the protocol are executed and the socket is closed
# right after the SCTP cookie handshake (use Wireshark to see this behaviour or see attached pcap).
uvloop.install()
# Switch between TCP and SCTP to see that for uvloop TCP works well, while SCTP doesn't
asyncio.run(start_server(use_sctp=True))
Client code
"""
The client side of the asyncio/uvloop SCTP socket demo.
Author: Lukas Stermann <[email protected]>
"""
import asyncio
import socket
from typing import Optional
import uvloop
class ClientProtocol(asyncio.Protocol):
"""
This is just a dummy client echo protocol for demo purposes.
"""
def __init__(self):
self.transport: Optional[asyncio.Transport] = None
def connection_made(self, transport: asyncio.Transport) -> None:
self.transport = transport
print('Connection established to server')
self.transport.write(b'ping')
def data_received(self, raw_data: bytes) -> None:
print(f'Answer received from server: <{raw_data.decode()}>')
def connection_lost(self, exc: Optional[Exception]) -> None:
print('Connection lost')
self.transport.close()
async def start_client(use_sctp: bool = True) -> None:
"""
This client sends a message over TCP/SCTP socket.
:param use_sctp: Whether the server shall use SCTP.
If set to False it uses TCP.
"""
connection_parameters: dict = {
'protocol_factory': lambda: ClientProtocol()
}
if use_sctp:
socket_ = socket.socket(socket.AF_INET, socket.SOCK_STREAM,
socket.IPPROTO_SCTP)
else:
socket_ = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket_.setblocking(False)
connection_parameters['sock'] = socket_
await asyncio.get_event_loop().sock_connect(
socket_, ('127.0.0.1', 10002))
transport, _ = await asyncio.get_event_loop().create_connection(
**connection_parameters)
await asyncio.sleep(2)
transport.close()
# Comment in the following line to see the issue with using an SCTP socket in uvloop.
# The client will crash with OSError saying the protocol is not supported
uvloop.install()
# Switch between TCP and SCTP to see that for uvloop TCP works well, while SCTP doesn't
asyncio.run(start_client(use_sctp=True))
Please also find attached a zipped PCAP for Wireshark. The first part of the trace there is the vanilla asyncio version with the uvloop.install() command commented out in both client and server code. The second trace shows the behavior when the server is using uvloop but the client is using vanilla asyncio.
Would be great if you would comment on this, cheers for your work, really appreciate that you are bringing performance to Python :)
