Setting large fs.pipe-max-size

Question

When I increase fs.pipe-max-size like so:

echo "fs.pipe-max-size = N" >> /etc/sysctl.conf
sysctl -p

(N is ~4-10Mbytes)

And use F_SETPIPE_SZ to change named pipe sizes to N, sometimes it fails with "operation not permitted" error.

The system has ~20 pipes and I set the same pipe buffer size on all of them.

The question is:

• is it because I hit some kind of a total kernel pipe buffer memory capacity (btw the system has 30G RAM)?
• Or is it because I use N that isn't divisible by a page size so F_SETPIPE_SZ might set the size above the fs.pipe-max-size limit and it will fail as "operation not permitted"? Makes sense, I think I saw in logs values larger than I asked.
• Or is it something totally else?

Answer 1

In the kernel implementation that handles the fcntl() system call on a pipe, there's:

    /*
     * If trying to increase the pipe capacity, check that an
     * unprivileged user is not trying to exceed various limits
     * (soft limit check here, hard limit check just below).
     * Decreasing the pipe capacity is always permitted, even
     * if the user is currently over a limit.
     */
    if (nr_slots > pipe->max_usage &&
            size > pipe_max_size && !capable(CAP_SYS_RESOURCE))
        return -EPERM;

    user_bufs = account_pipe_buffers(pipe->user, pipe->nr_accounted, nr_slots);

    if (nr_slots > pipe->max_usage &&
            (too_many_pipe_buffers_hard(user_bufs) ||
             too_many_pipe_buffers_soft(user_bufs)) &&
            pipe_is_unprivileged_user()) {
        ret = -EPERM;
        goto out_revert_acct;
    }

Those EPERMs correspond to the "operation not permitted" errors that you're seeing. I think it's likely you're hitting an unprivileged user resource limit. You can do a man 7 pipe and search for /proc file for the details, but here's a quick summary:

  /proc/sys/fs/pipe-max-size (since Linux 2.6.35)
         The maximum size (in bytes) of individual pipes that can
         be set by users without the CAP_SYS_RESOURCE capability.

  /proc/sys/fs/pipe-user-pages-hard (since Linux 4.5)
         The hard limit on the total size (in pages) of all pipes
         created or set by a single unprivileged user (i.e., one
         with neither the CAP_SYS_RESOURCE nor the CAP_SYS_ADMIN
         capability).

  /proc/sys/fs/pipe-user-pages-soft (since Linux 4.5)
         The soft limit on the total size (in pages) of all pipes
         created or set by a single unprivileged user (i.e., one
         with neither the CAP_SYS_RESOURCE nor the CAP_SYS_ADMIN
         capability).