Fixing ulimit: open files: cannot modify limit: Operation not permitted

Question

I tested this on different GNU/Linux installations:

perl -e 'while(1){open($a{$b++}, "<" ,"/dev/null") or die $b;print " $b"}'

System A and D

The first limit I hit is 1024. It is easily raised by putting this into /etc/security/limits.conf:

*                hard    nofile          1048576

and then run:

ulimit -n 1048576
echo 99999999 | sudo tee /proc/sys/fs/file-max

Now the test goes to 1048576.

However, it seems I cannot raise it above 1048576. If I put 1048577 in limits.conf it is simply ignored.

What is causing that?

System B

On system B I cannot even get to 1048576:

echo 99999999 | sudo tee /proc/sys/fs/file-max

/etc/security/limits.conf:

*                hard    nofile          1048576

Here I get:

$ ulimit -n 65537
bash: ulimit: open files: cannot modify limit: Operation not permitted
$ ulimit -n 65536
#OK

Where did that limit come from?

System C

This system also has the 1048576 limit in limits.conf and 99999999 in /proc/sys/fs/file-max.

But here the limit is 4096:

$ ulimit -n 4097
-bash: ulimit: open files: cannot modify limit: Operation not permitted
$ ulimit -n 4096
# OK

How do I raise that to (at least) 1048576?

(Note to self: Don't do: echo 18446744073709551616 | sudo tee /proc/sys/fs/file-max)

Answer 1

Check that /etc/ssh/sshd_config contains:

UsePAM=yes

and that /etc/pam.d/sshd contains:

session    required   pam_limits.so

In the comment below @venimus states the 1M limit is hardcoded:

The kernel 2.6.x source states ./fs/file.c:30:int sysctl_nr_open __read_mostly = 1024*1024; which is 1048676

The 1048576 is per process. So by having multiple processes this limit can be overcome.