ARG_MAX, maximum length of arguments for a new process

command: arg list too long 

2009-06-03 (see recent changes)

Here you'll find

• More about the nature of this limit
• The effectively usable space ...and a way to determine it reliably
• ...alternatively: about the GNU autoconf check
• How to avoid the limit in a shell
• Number of arguments (and maximum length of one argument)
• Actual values for ARG_MAX (with footnotes)

More about the nature of this limit

You will see this error message, if you tried to call a program with too many arguments, that is,
most likely in connection with pattern matching:

    $ command * 

On some systems the limit is even hit with "grep pattern /usr/include/* /usr/include/*/*". (You should not be doing this but using find.)

It's only the exec() system call (and its direct variants) which return the corresponding error condition E2BIG (<sys/errno.h>).
The shell is not to blame, it just delivers this error to you.
Shell expansion is not a problem, because here exec() is not needed, yet.
It's only limited by the virtual memory system resources ^[1].
Thus the following commands work smoothly, because they only make use of a shell built-in (echo) or a control structure (for loop):

    /dir-with-many-files$ echo * | wc -c
    /dir-with-many-files$ for i in * ; do grep ARG_MAX "$i"; done

There are different ways to learn the upper limit

• command: getconf ARG_MAX ^[2]
• system call: sysconf(_SC_ARG_MAX)
• system header: ARG_MAX in e.g. <[sys/]limits.h> ^[3]

(However, on the few system that have no limit for ARG_MAX, these methods wrongly might print a limit.)

From Version 7 on the limit was defined by NCARGS (usually in <[sys/]params.h>),
Later, ARG_MAX was introduced with 4.4BSD and System V.

In contrast to the headers, sysconf and getconf tell the limit which is actually in effect.
This is relevant on systems which allow to change it at run time (AIX), by reconfiguration (UnixWare, IRIX),
by recompiling (e.g. Linux) or by applying patches (HP-UX 10) - see the end notes for more details.
(Usually these are solutions for special requirements only, because increasing the limit doesn't solve the problem.)

The effectively usable space

When looking at ARG_MAX/NCARGS, you have to consider the space comsumption by both argv[] and envp[] (arguments and environment).
Thus you have to decrease ARG_MAX at least by the results of "env|wc -c" and "env|wc -l * 4" ^[3] for a good estimation of the currently available space.

POSIX suggests to subtract 2048 additionally so that the process may savely modify its environment. A quick estimation with the getconf command:
(formula suggested by Gunnar Ritter in <3B70A6AD.3L8115910@bigfoot.de>, de.comp.os.unix.shell)

 expr `getconf ARG_MAX` - `env|wc -c` - `env|wc -l` \* 4 - 2048

...and a way to determine it reliably

alternatively: about the GNU autoconf check

In a loop with increasing n, the check tries an exec() with an argument length of 2ⁿ.
The maximum is ARG_MAX/2 if ARG_MAX is a power of 2.
Finally, the found value is divided by 2 for reasons of safety. The reasoning is:
"C++ compilers can tack on massive amounts of additional arguments".

How to avoid the limit in a shell

• iterate with the shell:
  • for i in *; do command "$i"; done     (simple, completely robust and portable, may be slow)

• iterate with find
  • find . -exec command {} \;     (simple, completely robust and portable, may be slow)
  • find . -exec command {} +       (optimizes speed)
  • find . -print0|xargs -0 command      (optimizes speed, if find doesn't implement "-exec +" but knows "-print0")
  • find . -print|xargs command     (if there's no white space in the arguments)
  find descends into directories. To avoid that portably, you can use "find . ! -name . -prune [...]"

Number of arguments (and maximum length of one argument)

  PAGE_SIZE*MAX_ARG_PAGES-sizeof(void *) / sizeof(void *)

Since Linux 2.6.23, this function tests if the number exceeds MAX_ARG_STRINGS in <linux/binfmts.h> (2^32-1 = 4294967296-1).
And as additional limit, one argument must not be longer than MAX_ARG_STRLEN (131072).

Actual values for ARG_MAX

The maximum length of arguments for a new process is varying so much among unix flavours, that i had a look at some systems:

    "Increase ARG_MAX so that `make clean' in src/lib/libc works again.
    (Adding YP pushed it over the limit.)"

  "((sizeof(struct arg_hunk *))*(0x10000/(sizeof)(struct arg_hunk)))
   space for other stuff on initial stack like aux vectors, saved
   registers, etc.."  

  "ARG_MAX is calculated as follows:
   NCARGS - space for other stuff on initial stack
   like aux vectors, saved registers, etc.."

<http://www.in-ulm.de/~mascheck/various/argmax/>