How can I specify to xargs to run a command with all the arguments at once?

Some clarifications:

In

cmd1 $(cmd2)

Here is what happens with the output of cmd2:

• in bash, the NULs are discarded. YMMV with other shells (in zsh they're preserved, some shells ignore everything after the first NUL).
• the trailing newline characters are removed
• the result is split according to $IFS whose default value is SPC, TAB, NL in most shells, SPC, TAB, NL, NUL in zsh.
• each work is subject to globbing (except in zsh). For instance * becomes the list of non-hidden files in the current directory.
• the resulting list is passed as arguments to cmd1. The execve() system call to do that will fail if that list (combined with the environment) is too big (and in the case of Linux also if any single argument is larger than 128KiB).

In

cmd2 | xargs cmd1

Here's what happening with the output of cmd2:

• in some implementations, the bytes of that output are interpreted as characters. For instance in a locale using a UTF-8 charset the e2 80 86 byte sequence makes up a U+2006 character which is the SIX-PER-EM SPACE character, often considered as a blanks character. If some bytes don't form valid characters, depending on the implementation, they may be removed or cause a failure, the behaviour is not specified by POSIX if the input is not text
• If there are NUL bytes, again, that's not text, so the behaviour varies with the implementation
• If there are lines that are bigger than LINE_MAX (which on some systems can be as low as 1024), the behaviour is also unspecified.
• xargs parses its input as per its own expected format. In that format, whitespace and newline characters are understood as delimiter (so trailing newlines would be split all the same) the list of which varies with the implementation (including that U+2006 for instance in some), and '...', "...", and backslash are used as quoting operators to prevent those delimiters and other quoting operators to be treated specially.
• that results in a list or words.
• in some implementations, if any of those words is _, it's treated as an end-of-file logical indicator and everything after it is ignored.
• if any of those words is larger than some limit (in many systems, as low as 255 bytes), the behaviour is unspecified.
• the remaining words are passed as arguments to cmd1, but if that list is too big, instead of failing like in the command substitution case, xargs tries to fit as many as possible (with a bit of margin which varies with the implementation) for execve() to succeed, and runs the command as many times as needed to use all the arguments.

Now:

• if you wanted to pass the whole output of cmd2 as one argument to cmd1:
  • if that output contains NUL characters, you can't as that's a limitation of the execve() system call, not command substitution nor xargs.
  • if that output is bigger than the ARG_MAX limit, you can't (though there are ways to increase that limit on some systems), again a limitation of exexve(). You could if cmd1 was for instance a builtin or function of your shell (as there's no execve() involved then), but even then, in practice, only zsh supports arguments with NUL in that case.
  • on Linux, if it's bigger than 128KiB, you can't. That limit can't be raised.
  • Otherwise (if there's no NUL, and it's small enough), you can always do: out=$(cmd2; echo .); cmd1 "${out%.}" or with GNU xargs: cmd2 | xargs -0 cmd1

What you describe is already the default. xargs -n provides a maximum number of arguments to use, but if you don't specify it, xargs will use as many arguments as possible.

You mentioned the system limit in your question, and that you don't expect/need to circumvent that. xargs will take that system limit into account, but may use a lower value, as described in the usage for the --max-chars (-s) option:

 --max-chars=max-chars
 -s max-chars

Use at most max-chars characters per command line, including the command and initial-arguments and the terminating nulls at the ends of the argument strings. The largest allowed value is system-dependent, and is calculated as the argument length limit for exec, less the size of your environment, less 2048 bytes of headroom. If this value is more than 128KiB, 128Kib is used as the default value; otherwise, the default value is the maximum. 1KiB is 1024 bytes.

(emphasis mine)

You can check what it is with echo | xargs --show-limits. Example output:

Your environment variables take up 3712 bytes
POSIX upper limit on argument length (this system): 2091392
POSIX smallest allowable upper limit on argument length (all systems): 4096
Maximum length of command we could actually use: 2087680
Size of command buffer we are actually using: 131072
Maximum parallelism (--max-procs must be no greater): 2147483647

This shows that my hard limit is 2087680 (just under 2MiB). If your system also permits more than 128K, you can prevent splitting by using xargs -s 2087680 (adjust according to your own limit).

Older versions of xargs would even let you provide a higher limit than the OS permits. With 4.4.2, a warning was displayed, but the specified value accepted anyway, and you get the same error that you would see from running the command directly:

$ seq 1 2000000 | xargs -s 2100000 echo | wc -l
xargs: value for -s option should be < 2092927
xargs: echo: Argument list too long
0

But in versions 4.6, the hard limit is respected:

$ seq 1 2000000 | xargs -s 2100000 echo | wc -l
xargs: value for -s option should be <= 2091392
23