Shell builtin `printf` line limit?

Question

The /usr/bin/printf util argument list length is limited to the shell's maximum command line length, (i.e. getconf ARG_MAX, on my system that'd be 2097152); example:

# try using a list that's way too long
/usr/bin/printf '%s\n' $(seq $(( $(getconf ARG_MAX) * 2 ))) | tail -1

Output:

bash: /usr/bin/printf: Argument list too long

Today I'm informed that shell builtin printfs don't have that limit; test:

printf '%s\n' $(seq $(( $(getconf ARG_MAX) * 2 ))) | tail -1

Output:

4194304

Questions:

1. A skim of man bash dash doesn't seem to say much about this advantage of builtin printf. Where is it documented?

2. Do builtin printfs (e.g. bash) have an argument list maximum length in chars, and if so, what is that length?

Answer 1

It's not really the manual's job to advocate the use of any particular utility. It should primarily describe the available built-in utilities.

The advantages of using a built-in utility over an external one is primarily speed and the availability of extended features (printf in bash can, for example, write directly into a variable with -v varname, which no external printf could ever do).

Executing external utilities is slow in comparison to executing a built-in utility, especially if done often in e.g. a loop, and, as you have noticed, they also allow for longer argument lists (this is not something that only the built-in printf allows, but all built-in utilities).

The length of the argument list to the built-in printf utility in bash is limited by the resource restrictions on the bash process itself. On some systems, this may even mean that you could use most of the available RAM to construct its command line argument list.

The documentation from where you would find these various bits of information is

• The bash source code, where you will see that the argument list for printf is a dynamically allocated linked list, and also that it's not using execve() to run printf (which is what limits the length of an argument list when running an external utility).

An example of a shell where printf in not a built-in utility is the ksh shell of OpenBSD. The utility can also be disabled in bash using enable -n printf.

Answer 2

The /usr/bin/printf util argument list length is limited to the shell's maximum command line length, (i.e. getconf ARG_MAX, on my system that'd be 2097152); example:

That's not a shell's limit, but a limit of the OS (of the Linux kernel), specifically of its execve(2) syscall, and caused by the antiquated way in which the command line arguments and environment variables are passed to a started program.

(Notice that that limit also includes the environment variables!).

A skim of man bash dash doesn't seem to say much about this advantage of builtin printf. Where is it documented? Do builtin printfs have an argument list length, (e.g. bash), and if so, what is it?

Since shell built-ins don't go through execve(2) they don't have to have any limit like that. Modern shell usually don't use fixed size buffers and such, so the limit is usually imposed by the amount of available memory and the layout of the virtual address space -- ie it's unlimited for all intents and purposes.

Answer 3

The POSIX standard requires the shell to be implemented without a line limit.

So this is a naturalness that shells do not document.

BTW: because of this fact, a shell script may not be a text file, as a text file is a file that does not have lines longer than LINE_MAX ;-)