"$( )" command substitution vs. embedded ")"

This page doesn't serve any purpose but fun, it's just about a special point concerning the robustness of shell parsers.

2010-07-29 (see recent changes)

In some shells, $( ) collides with other, "unbalanced", yet valid, right parentheses.
And there are some more subtle issues.

The much more famous issue, $( ) vs. `...`, is mentioned in the additional end notes.

What does the standard say?

The SUSv3 rationale and SUSv4 rationale to shell commands, C.2.9, about "case x in (x)" embedded in "$( )":

"An optional left parenthesis before pattern was added to allow numerous historical KornShell scripts to conform. At one time, using the leading parenthesis was required if the case statement was to be embedded within a "$()" command substitution; this is no longer the case with the POSIX shell. Nevertheless, many historical scripts use the left parenthesis, if only because it makes matching-parenthesis searching easier in vi and other editors."

However, there are some cornercases for "$( )" even in shells which try to implement posix.

The following table lists the results for some testcases. These tests exploit the case command and here-documents. They were taken from the above URL or Usenet and the thread <mailman.20673.1135786095.20277.bug-bash@gnu.org> in gnu.bash.bug (Dan Jacobson and Eric Blake). Jilles Tjoelker pointed out the problem with '# \'.

Examples, each to be embedded in $( ) , see full testcases

A.1	"case x in x)"
A.2	"case x in x)" with a comment
A.3	"case x in (x)"
A.4	"case x in (x)" with a comment
A.5	"case x in (x)" with \n and without ";;" before "esac"
B	) in qoutes
C	) after a comment sign
D	) embedded in a here-document
E	) as delimiter for an embedded here-document (most academic)
F	unbalanced quote in an embedded here-document
G	\ at end of line
H	empty

Additional Notes:

• the original Almquist shell and its early variants fail with A.3 and A.4 (workaround) because their grammar doesn't accept an opening parentheses in a case command yet.
  Later ash variants usually accept the opening paren, e.g., BSD/OS 3.0, NetBSD 4.0, FreeBSD 4.1/5.1, (d)ash 0.3.5-4, Slackware 8.1, Minix 3.1.3, Busybox from birth.

  The ash parser doesn't need this workaround. However, this is also an issue of script portability.

• traditional Bourne shells are not listed because they don't know "$( )" yet and thus had not implemented the opening paren in a case pattern, yet.

• the original ash suffers from a SEGV for empty command substitutions

• A few words about the issue "$( )" vs. `...`:

  pro:	Without doubt, the "$( )" syntax is the cleaner design because there are no escaping rules to consider. And \ is special in `...` so you have to escape it twice.
  pro:	Another argument: backquotes are easily confused with apostrophes -- not so much when actually using them (in terminals with fixed width fonts), but especially in other sources of documentation using arbitrary fonts.
  weak:	An often mentioned advantage of the "$( )" syntax: it allows clean nesting for this reason. However, that's a weak argument: If you really have to nest command substitution, then you should split into separate commands to ease maintencance and to improve readability.
  contra:	Last but not least, the "$( )" form is not as traditionally portable as the `...` form, because traditional Bourne shells only implement the latter.

• ...and a footnote for the above:

  In ksh88 (at least from release a to i) you have to be aware of a subtle quoting issue inside $( ).
  Single quotes in embedded here-documents are converted to double quotes:

  echo $(cat << EOF
       "double quotes"
       'single quotes'
  EOF
  ) 

  results in

      "double quotes" "single quotes" 

  This happens for the $( ) form but not for the `` form.
  This construct is likely to be used, if you feed a large block of text to a command and want to save the command output.