Python simple socket client/server using asyncio

Question

I would like to re-implement my code using asyncio coroutines instead of multi-threading.

server.py

def handle_client(client):
    request = None
    while request != 'quit':
        request = client.recv(255).decode('utf8')
        response = cmd.run(request)
        client.send(response.encode('utf8'))
    client.close()

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('localhost', 15555))
server.listen(8)

try:
    while True:
        client, _ = server.accept()
        threading.Thread(target=handle_client, args=(client,)).start()
except KeyboardInterrupt:
    server.close()

client.py

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.connect(('localhost', 15555))
request = None

try:
    while request != 'quit':
        request = input('>> ')
        if request:
            server.send(request.encode('utf8'))
            response = server.recv(255).decode('utf8')
            print(response)
except KeyboardInterrupt:
    server.close()

I know there are some appropriate asynchronous network librairies to do that. But I just want to only use asyncio core library on this case in order to have a better understanding of it.

It would have been so nice to only add async keyword before handle client definition... Here a piece of code which seems to work, but I'm still confused about the implementation.

asyncio_server.py

def handle_client(client):
    request = None
    while request != 'quit':
        request = client.recv(255).decode('utf8')
        response = cmd.run(request)
        client.send(response.encode('utf8'))
    client.close()

def run_server(server):
    client, _ = server.accept()
    handle_client(client)

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('localhost', 15555))
server.listen(8)

loop = asyncio.get_event_loop()
asyncio.async(run_server(server))
try:
    loop.run_forever()
except KeyboardInterrupt:
    server.close()

How adapt this in the best way and using async await keywords.

Have you gone through some manner of tutorial entirely focused on asyncio yet? It may be more prudent to do that first, instead of translating something you already have working. I would recommend this to start you off. — Adam Barnes
– Adam Barnes, Commented Jan 29, 2018 at 17:38
I have updated the question. As I am still confused about the asyncio lib, I think this use case may be relevant for a better understanding. — srjjio
– srjjio, Commented Jan 29, 2018 at 17:48
If you are still confused about asyncio (it's indeed a mess of futures, tasks and coroutines) and want to avoid threads, you can try gevent which is a well established coroutine library or, imo the best option, rely on the operating system's select function. Here is the Python 3 documentation and a quick tutorial Wait for IO Efficiently — warownia1
– warownia1, Commented Jul 9, 2019 at 11:12

user4815162342 · Accepted Answer · 2024-03-04 18:03:20Z

The closest literal translation of the threading code would create the socket as before, make it non-blocking, and use asyncio low-level socket operations to implement the server. Here is an example, sticking to the more relevant server part (the client is single-threaded and likely fine as-is):

import asyncio, socket

async def handle_client(client):
    loop = asyncio.get_event_loop()
    request = None
    while request != 'quit':
        request = (await loop.sock_recv(client, 255)).decode('utf8')
        response = str(eval(request)) + '\n'
        await loop.sock_sendall(client, response.encode('utf8'))
    client.close()

async def run_server():
    server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server.bind(('localhost', 15555))
    server.listen(8)
    server.setblocking(False)

    loop = asyncio.get_event_loop()

    while True:
        client, _ = await loop.sock_accept(server)
        loop.create_task(handle_client(client))

asyncio.run(run_server())

The above works, but is not the intended way to use asyncio. It requires precisely matching the use of the socket API to the expectations of asyncio, such as the setblocking() call. There is no buffering out-of-the-box, so something as simple as reading a line from the client becomes a tiresome chore. This API level is really only intended for implementers of alternative event loops, which would provide their implementation of sock_recv, sock_sendall, etc.

Asyncio's public API provides two abstraction layers intended for consumption: the lower-level transport/protocol layer modeled after Twisted, and the higher-level streams layer. In new code, you almost certainly want to use the streams API, i.e. call asyncio.start_server and avoid raw sockets. That significantly reduces the line count:

import asyncio

async def handle_client(reader, writer):
    request = None
    while request != 'quit':
        request = (await reader.read(255)).decode('utf8')
        response = str(eval(request)) + '\n'
        writer.write(response.encode('utf8'))
        await writer.drain()
    writer.close()

async def run_server():
    server = await asyncio.start_server(handle_client, 'localhost', 15555)
    async with server:
        await server.serve_forever()

asyncio.run(run_server())

It works so fine, even if I'm still confused about why asynchronous and sockets seem to be such intricated to be wrapped in the same library...
@srjjio Network (socket) programming constitutes a huge chunk of the motivation for needing asynchronous IO in the first place. An asyncio library that didn't provide support for sockets wouldn't be of much use.
Does this keep a socket open for each client or is a new socket created for each request/response?
@DevPlayer No, run_server is a coroutine function, and run_until_complete expects an awaitable. To create an awaitable, you have to actually call the coroutine function. This is analogous to functions expecting an iterable - you can't just pass them a generator function, you have to actually call the generator to get an iterable.

Dalija Prasnikar · Accepted Answer · 2024-04-12 08:30:15Z

I have read the answers and comments above, trying to figure out how to use the asyncio lib for sockets. As it often happens with Python, the official documentation along with the examples is the best source of useful information. I got understanding of Transports and Protocols (low-level API), and Streams (high-level API) from the following examples that can be found in the end of the support article:

For example, TCP Echo Server:

import asyncio


class EchoServerProtocol(asyncio.Protocol):
    def connection_made(self, transport):
        peername = transport.get_extra_info('peername')
        print('Connection from {}'.format(peername))
        self.transport = transport

    def data_received(self, data):
        message = data.decode()
        print('Data received: {!r}'.format(message))

        print('Send: {!r}'.format(message))
        self.transport.write(data)

        print('Close the client socket')
        self.transport.close()


async def main():
    # Get a reference to the event loop as we plan to use
    # low-level APIs.
    loop = asyncio.get_running_loop()

    server = await loop.create_server(
        lambda: EchoServerProtocol(),
        '127.0.0.1', 8888)

    async with server:
        await server.serve_forever()


asyncio.run(main())

and TCP Echo Client:

import asyncio


class EchoClientProtocol(asyncio.Protocol):
    def __init__(self, message, on_con_lost):
        self.message = message
        self.on_con_lost = on_con_lost

    def connection_made(self, transport):
        transport.write(self.message.encode())
        print('Data sent: {!r}'.format(self.message))

    def data_received(self, data):
        print('Data received: {!r}'.format(data.decode()))

    def connection_lost(self, exc):
        print('The server closed the connection')
        self.on_con_lost.set_result(True)


async def main():
    # Get a reference to the event loop as we plan to use
    # low-level APIs.
    loop = asyncio.get_running_loop()

    on_con_lost = loop.create_future()
    message = 'Hello World!'

    transport, protocol = await loop.create_connection(
        lambda: EchoClientProtocol(message, on_con_lost),
        '127.0.0.1', 8888)

    # Wait until the protocol signals that the connection
    # is lost and close the transport.
    try:
        await on_con_lost
    finally:
        transport.close()


asyncio.run(main())

Hope it help someone searching for simple explanation of asyncio.

connection closed when using telnet client connect to this echo server?
Note that the pasted examples are certainly not a good example of using asyncio, as they show the older callback-based style, rather than async/await which is easier to use and much more represented nowadays (in python as well in other languages).

Collectives™ on Stack Overflow

Python simple socket client/server using asyncio

2 Answers 2

12 Comments

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

12 Comments

2 Comments

Linked

Related