Python Socket gives "[Errno 24] Too many open files"

Question

I have the following UDP class sending arrays of data at about 100Hz

from six import string_types
import socket
import struct

def convert_data(iterable):
    if isinstance(iterable, string_types):
        return str(iterable)
    data = tuple(iterable)
    format = "{0}H".format(len(data))
    print("Sending data:", format, data)
    if max(data) > 2**16 - 1:
        raise ValueError(max(data))
    if min(data) < 0:
        raise ValueError(min(data))
    return struct.pack(format, *data)

class UDP(object):
    def __init__(self, ip, port):
        self._ip = ip
        self._port = port
        self.socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        self.socket.connect((ip, port))

    def send_data(self, data):
        message = convert_data(data)
        return self.socket.sendall(message)

It gives the following error after successfully sending for about a minute:

Traceback (most recent call last):
  File "take_analogue_data.py", line 13, in <module>
  File "take_analogue_data.py", line 8, in main
  File "/home/pi/nio-integration/hardware/raspi/UDP.py", line 22, in __init__
  File "/usr/lib/python2.7/socket.py", line 187, in __init__
socket.error: [Errno 24] Too many open files

I have looked for a solution. This Stack Overflow answer suggests increasing the number of possible files. I really don't think this is the solution I am looking for though.

Is there something I can do? I was thinking that closing the connection each time might work, but I have already played around with a bunch of things. (I have tried send, sendall, and sendto -- none have worked)

Note: I am running Python2.6 on Raspbian Wheezy on a Raspberry Pi

Edit Another module is sending the data. It could look something like

import UDP
udp = UDP.UDP(IP, PORT)
while(True):
    udp.send_data(range(8))
    sleep(0.01)

Answer 1

Likely, you are creating a new socket for every single iteration of while(True):. Processes are limited to the number of file descriptors they can have open (sockets are fds.) You can check /etc/security/limits.conf to see what your limits are set to.

You should close your socket when you're done with it, or, ideally, only open one and reuse it if possible.

You said that your other module "could look something like this." Is that code snippet exactly what it looks like?

I doubt it, because if so that should only be making one socket. If you're instancing the UDP object within the while, then the above is definitely your issue.

Answer 2

class UDP(object):

    def __init__(self, ip, port):
        self._ip = ip
        self._port = port
        self.socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        self.socket.connect((ip, port))

What you have to know about UDP is that it is a session less and connectionless protocol which means that the: self.socket.connect((ip,int(port))) is not right and so you should remove it.

If you want to connect to a server use Tcp instead:

class TCP(object):

    def __init__(self, ip, port):
        self._ip = ip
        self._port = port
        self.socket = socket.socket() #Here is the change
        self.socket.connect((ip, port))

    def send_data(self, data):
        message = convert_data(data)
        return self.socket.sendall(message)

Hope it helped !