Portable Shell Programming

    The point of a standard isn't even whether it's ideal or particularly sensible,
    it's that a compliant program produces consistent results on compliant platforms.
    Diverging in the name of whatever benefit just means that one has to work harder
    to produce a portable program. -- Richard L. Hamilton, in comp.unix.questions

2024-06-16 (see recent changes)

This is a link list. Well, there's one page of my own (bourne shell characteristics), but after all that one is not about portability...

1. Books

• Peter Seebach: "Beginning Portable Shell Scripting: From Novice to Professional" (Apress)

2. POSIX / SUS (Single UNIX Specification), and implementations

• Standards:

  Shell and Utilities are covered particularly by these volumes: SUS XCU / POSIX.2 / IEEE 1003.2

  Understanding of the "Base Definitions" is assumed for understanding "Shell and Utilities".
  (From the FAQ (broken link, see XBD instead): "readers should be familiar with it before using the other parts, [...] reduces duplication [...] and ensures consistent use of terminology.")
  The "XSI" option describes extensions which are required on UNIX implementations.
  See also "Utilities" for the built-ins which are not special.

  Deep links mostly seem to work. However, please register online at the OpenGroup, if you access their documents. Andrew Josey (OpenGroup) explains why:
  "We do prefer folks to register to access the specification if they can, so we can track interest in the specification -- which then helps us to justify making it available."

  • 2024
    POSIX 1003.1-2024 aka POSIX.1-2024 aka IEEE Std 1003.1-2024; Open Group Standard Base Specifications, Issue 8
    OpenGroup: t.b.a.

    some news:

    • readlink; realpath
    • find -print0 / -iname ; xargs -0
    • read -d
    • sed -E / -o pipefail

  • 2008 / 2018:
    SUSv4 Open Group Base Specifications Issue 7
    (old TOC & keyword search),
    brands: UNIX V7, Commands and Utilities V5 (list of certified systems)
    POSIX.2:2008, IEEE Std 1003.2-2008,
    ISO/IEC/IEEE 9945:2009 (or :2008),
    The Austin Group Common Specifications, 2008 TC1 (Technical Corrigendum) Draft 2
    • "Shell and Utilities" / "Shell Command Language" explains the language itself in detail (deep links toc and doc)
    • "Shell and Utilities" / "Utilities" / "sh" in contrast describes options, environment and command line editing (deep link)
    • The important difference between SUSv4 and POSIX: XCurses is not part of POSIX.

  • 2001/2004:
    SUSv3, Open Group Base Specifications Issue 6
    (toc & old keyword search)
    brands: SUSv3 / UNIX 03, Commands and Utilities V4 (list of certified systems)
    POSIX.1:2001 (includes POSIX.2:2001), equivalent to IEEE Std 1003.1-2001 / -2004 (including 1003.2-2001/-2004),
    ISO/IEC 9945-2:2003 (/Cor 1:2004),

    SUS and POSIX have been merged with this release.
    2004 includes two technical corrigenda

    • (broken link) Comparison of SUSv3 and POSIX, e.g. the table "Commands". The important difference: XCurses is not part of POSIX.

  • 1992:
    SUSv2, keyword search (entry to old structure),
    brands: SUSv2 / UNIX 98, Commands and Utilities issue 5 (formerly V3) (list of certified systems)
    related: POSIX.2:1990/92 , equivalent to IEEE Std 1003.1-1990/92 (and 1003.2(/.2a)-1992),
    related: ISO/IEC 9945-1:1996
    • POSIX shell and the SUS shell are not the same, but nearly identical.
      (broken link) About the differences between POSIX and SUSv2 (the table "Commands").

  • SUSv1,
    Commands and Utilities, Issue 4, Version 2,
    brands: SUSv1 / UNIX 95, Commands and Utilities V2 (list of certified systems)
    related: ISO/IEC 9945-2:1993

    Also try "Commands and Utilities, Issue 4, Version 2" via Base 95

  • How to test conformance?
    The Light Version of the "Open Group's full VSC Commands and Utilities Test Suite". See the entry "VSC lite Test Suite" or "VSC-PCTS2003", respectively.
    The complete suite is also available after you have registered as developer of standard implementations or Open Source Project.

  • The Wikipedia entry lists some background and certified operating systems.

• ksh88, ksh93:

  See packages notes and changes, and a current ksh93 package on www.research.att.com/sw/download/ for the following files.
  The naming is confusing at first, here's what they mean:

  • COMPATIBILITY, "ksh88 vs. ksh 93"
  • OBSOLETE, "ksh features that are obsolete in ksh93"
  • RELEASE88, "Changelog from ksh88 to ksh93"
  • RELEASE, "Changelog for ksh93"
  • RELEASE93: an older variant of RELEASE

  In Usenet, you can even find an archived version of the changelog for ksh88, RELEASEa (local copy).

• POSIX notes for bash (GNU home, maintainer's home)
  bash-2.05, bash-3.1.0 and bash-4.0.0 (local snapshots for my convenience), better see the bash distribution: bash/CWRU/POSIX.NOTES.

  bash knows a POSIX mode. But keep in mind, that it only switches behaviour where bash features would collide. Apart from that, bash specific features are not affected.

• dash is a modern Almquist shell variant. It's aiming at a POSIX-only feature set.

  The Linux distribution Ubuntu has switched to dash as system shell with release 6.10.
  https://wiki.ubuntu.com/DashAsBinSh lists issues about the migration from bash to dash.
  The release goals for the Debian distribution Lenny also picked up this plan, but the move was postponed.

• Free- and NetBSD have an Almquist shell descendant as /bin/sh, also aiming at POSIX.
  Their manual pages are online, hosted by freebsd.org (and there are even manual pages of numerous other Unix variants).

• pdksh - the Public Domain Korn Shell.

  "mostly finished to make it fully compatible with both POSIX and AT&T ksh (when the two don't conflict)"
  Interesting files are NOTES ("lists of known bugs in pdksh, at&t ksh, and posix"), PROJECTS ("Things to be done in pdksh") and the two logs ChangeLog and ChangeLog.0.
  At the time of this writing, pdksh isn't actively maintained anymore.

• posh - debian policy compliant ordinary variant of pdksh

  This pdksh based shell aims directly at Debian policy compliance, that is, SUS with an exception for echo -n. Numerous features have been removed (e.g. type, ulimit, time, job control builtins, some XSI kill and test extensions, rsh-functionality, aliases, some shell options). So this shell is probably the implementation which is nearest to "POSIX only, and without XSI".

• mksh - MirBSD variant of pdksh

  This project calls itself a heir of pdksh and could be considered a continuation of pdksh maintenance.

3. Traditional portability, the unix landscape

• The autoconf documentation contains a chapter about portable shell programming. (The "#! /" issue in earlier releases likely is a myth, though.)
  Keep in mind that autoconf strictly aims at maximizing portability for install scripts, thus earlier versions even suggested to avoid all unportable extensions after Version7.
  Traditional portability nowadays rather means SVR2 (or even SVR3).

• Paul Jarc's list of suspicious or nonportable constructs in shell programming.

• Gunnar Ritter's Heirloom Bourne shell, a port of the OpenSolaris(/SVR4-like) Bourne Shell

• Gunnar Ritter's Heirloom Toolchest, a modern reimplementation of the traditional toolchest providing several variants of each tool (SVR4, SVR4.2MP, SUSV2, SUSV3 and 4BSD).

• "Migrating from the System V Shell to the POSIX Shell" (unix.org)
  covers some main points, also concerning forward compatibility.

• Stéphane Chazelas mentions issues of numerous utilities (google usenet archive).

• Mark Hobley wrote checkbashism which tries to detect non-portable constructs.

• Vidar Holen created a haskell script which analyzes script snippets heuristically and tries to point out wrong usage and portability issues.
  He even has a website www.shellcheck.net where you can paste snippets into a form and get the results immediately.

• D.J.Bernsteins's UNIX portability notes (commands, system/library calls and headers intermixed).

• The autobook documentation contains notes about portability, but there are several problematic statements.

• The traditional Bourne shell family (Gunnar Ritters's implementation above being a modern member of this family),
  and some more and some less known characteristic features are also listed here (but the latter is really rather for fun).