Python locks on asyncio socket operations

Question

I tried to make a program that connects two sockets, by writing the data read on one socket to the other and vice versa.

import socket as sock
import asyncio

BUFF_SIZE = 4096


async def create_relay(s, port, loop):
    while True:
        s1, _ = await loop.sock_accept(s)  # while awaiting sock_accept no other task is run
        s2 = sock.socket(sock.AF_INET, sock.SOCK_STREAM)
        await loop.sock_connect(s2, ('localhost', port))
        loop.create_task(relay(s1, s2, loop))
        loop.create_task(relay(s2, s1, loop))
        # await asyncio.sleep(1000)        # while awaiting sleep the relay tasks are run

async def relay(s1, s2, loop):
    data = await loop.sock_recv(s1, BUFF_SIZE)
    while data:
        await loop.sock_sendall(s2, data)
        data = await loop.sock_recv(s1, BUFF_SIZE)


def main():
    s = sock.socket(sock.AF_INET, sock.SOCK_STREAM)
    s.bind(('localhost', 9999))
    s.listen()
    loop = asyncio.get_event_loop()
    loop.run_until_complete(create_relay(s, 9990, loop))


if __name__ == '__main__':
    main()

While this program successfully sets up the sockets, I can't actually transfer any messages. Debugging the code has shown that the program gets stuck at s1, _ = await loop.sock_accept(s) and doesn't execute any of the other asynchronous code while awaiting another connection. If however I add another await, like await asyncio.sleep(1000) at the end of the create_relay loop the relay works at least partially and I can forward the messages in one direction.

Can somebody explain, why these lock-ups happen and how to avoid them?

user4815162342 · Accepted Answer · 2019-10-21 13:55:41Z

1

Can somebody explain, why these lock-ups happen and how to avoid them?

The code doesn't make the sockets non-blocking. You need to add:

s.setblocking(False)

in every place where you create or obtain a new socket.

Also, sock_connect, sock_accept, sock_recv, and sock_sendall are considered low-level APIs that are not meant to be directly used by application code. Instead, your code should use the stream APIs, which provide features such as buffering, and of course automatically create non-blocking sockets.

edited Oct 21, 2019 at 13:55

answered Oct 21, 2019 at 13:11

user4815162342

158k22 gold badges347 silver badges417 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python locks on asyncio socket operations

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related