Files
oldlinux-files/Ref-docs/POSIX/susv3/xrat/xcu_chap01.html
2024-02-19 00:21:47 -05:00

902 lines
43 KiB
HTML

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta name="generator" content="HTML Tidy, see www.w3.org">
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<link type="text/css" rel="stylesheet" href="style.css"><!-- Generated by The Open Group's rhtm tool v1.2.1 -->
<!-- Copyright (c) 2001 The Open Group, All Rights Reserved -->
<title>Rationale for Shell and Utilities</title>
</head>
<body bgcolor="white">
<basefont size="3"> <!--header start-->
<center><font size="2">The Open Group Base Specifications Issue 6<br>
IEEE Std 1003.1-2001<br>
Copyright &copy; 2001 The IEEE and The Open Group, All Rights reserved.</font></center>
<!--header end-->
<hr size="2" noshade>
<h2><a name="tag_02"></a>Rationale for Shell and Utilities</h2>
<h3><a name="tag_02_01"></a>Introduction</h3>
<h4><a name="tag_02_01_01"></a>Scope</h4>
<p>Refer to <a href="xbd_chap01.html#tag_01_01_01"><i>Scope</i></a> .</p>
<h4><a name="tag_02_01_02"></a>Conformance</h4>
<p>Refer to <a href="xbd_chap02.html#tag_01_02"><i>Conformance</i></a> .</p>
<h4><a name="tag_02_01_03"></a>Normative References</h4>
<p>There is no additional rationale provided for this section.</p>
<h4><a name="tag_02_01_04"></a>Change History</h4>
<p>The change history is provided as an informative section, to track changes from previous issues of
IEEE&nbsp;Std&nbsp;1003.1-2001.</p>
<p>The following sections describe changes made to the Shell and Utilities volume of IEEE&nbsp;Std&nbsp;1003.1-2001 since Issue 5
of the base document. The CHANGE HISTORY section for each utility describes technical changes made to that utility from Issue 5.
Changes between earlier issues of the base document and Issue 5 are not included.</p>
<p>The change history between Issue 5 and Issue 6 also lists the changes since the ISO&nbsp;POSIX-2:1993 standard.</p>
<h5><a name="tag_02_01_04_01"></a>Changes from Issue 5 to Issue 6 (IEEE&nbsp;Std&nbsp;1003.1-2001)</h5>
<p>The following list summarizes the major changes that were made in the Shell and Utilities volume of
IEEE&nbsp;Std&nbsp;1003.1-2001 from Issue 5 to Issue 6:</p>
<ul>
<li>
<p>This volume of IEEE&nbsp;Std&nbsp;1003.1-2001 is extensively revised so that it can be both an IEEE POSIX Standard and an Open
Group Technical Standard.</p>
</li>
<li>
<p>The terminology has been reworked to meet the style requirements.</p>
</li>
<li>
<p>Shading notation and margin codes are introduced for identification of options within the volume.</p>
</li>
<li>
<p>This volume of IEEE&nbsp;Std&nbsp;1003.1-2001 is updated to mandate support of FIPS 151-2. The following changes were made:</p>
<ul>
<li>
<p>Support is mandated for the capabilities associated with the following symbolic constants:</p>
<blockquote>
<pre>
_POSIX_CHOWN_RESTRICTED
_POSIX_JOB_CONTROL
_POSIX_SAVED_IDS
</pre>
</blockquote>
</li>
<li>
<p>In the environment for the login shell, the environment variables <i>LOGNAME</i> and <i>HOME</i> shall be defined and have the
properties described in the Base Definitions volume of IEEE&nbsp;Std&nbsp;1003.1-2001, <a href=
"../basedefs/xbd_chap07.html">Chapter 7, Locale</a>.</p>
</li>
</ul>
</li>
<li>
<p>This volume of IEEE&nbsp;Std&nbsp;1003.1-2001 is updated to align with some features of the Single UNIX Specification.</p>
</li>
<li>
<p>A new section on Utility Limits is added.</p>
</li>
<li>
<p>A section on the Relationships to Other Documents is added.</p>
</li>
<li>
<p>Concepts and definitions have been moved to a separate volume.</p>
</li>
<li>
<p>A RATIONALE section is added to each reference page.</p>
</li>
<li>
<p>The <a href="../utilities/c99.html"><i>c99</i></a> utility is added as a replacement for <i>c89</i>, which is withdrawn in this
issue.</p>
</li>
<li>
<p>IEEE&nbsp;Std&nbsp;1003.2d-1994 is incorporated, adding the <a href="../utilities/qalter.html"><i>qalter</i></a>, <a href=
"../utilities/qdel.html"><i>qdel</i></a>, <a href="../utilities/qhold.html"><i>qhold</i></a>, <a href=
"../utilities/qmove.html"><i>qmove</i></a>, <a href="../utilities/qmsg.html"><i>qmsg</i></a>, <a href=
"../utilities/qrerun.html"><i>qrerun</i></a>, <a href="../utilities/qrls.html"><i>qrls</i></a>, <a href=
"../utilities/qselect.html"><i>qselect</i></a>, <a href="../utilities/qsig.html"><i>qsig</i></a>, <a href=
"../utilities/qstat.html"><i>qstat</i></a>, and <a href="../utilities/qsub.html"><i>qsub</i></a> utilities.</p>
</li>
<li>
<p>IEEE&nbsp;P1003.2b draft standard is incorporated, making extensive updates and adding the <a href=
"../utilities/iconv.html"><i>iconv</i></a> utility.</p>
</li>
<li>
<p>IEEE PASC Interpretations are applied.</p>
</li>
<li>
<p>The Open Group's corrigenda and resolutions are applied.</p>
</li>
</ul>
<h5><a name="tag_02_01_04_02"></a>New Features in Issue 6</h5>
<p>The following table lists the new utilities introduced since the ISO&nbsp;POSIX-2:1993 standard (as modified by
IEEE&nbsp;Std&nbsp;1003.2d-1994). Apart from the <a href="../utilities/c99.html"><i>c99</i></a> and <a href=
"../utilities/iconv.html"><i>iconv</i></a> utilities, these are all part of the XSI extension.</p>
<center>
<table border="1" cellpadding="3" align="center">
<tr valign="top">
<th colspan="8" align="center">
<p class="tent"><b>New Utilities in Issue 6</b></p>
</th>
</tr>
<tr valign="top">
<td align="left">
<p class="tent"><br>
<a href="../utilities/admin.html"><i>admin</i></a><br>
<a href="../utilities/c99.html"><i>c99</i></a><br>
<a href="../utilities/cal.html"><i>cal</i></a><br>
<a href="../utilities/cflow.html"><i>cflow</i></a><br>
&nbsp;</p>
</td>
<td align="left">
<p class="tent"><br>
<a href="../utilities/compress.html"><i>compress</i></a><br>
<a href="../utilities/cxref.html"><i>cxref</i></a><br>
<a href="../utilities/delta.html"><i>delta</i></a><br>
<a href="../utilities/fuser.html"><i>fuser</i></a><br>
&nbsp;</p>
</td>
<td align="left">
<p class="tent"><br>
<a href="../utilities/gencat.html"><i>gencat</i></a><br>
<a href="../utilities/get.html"><i>get</i></a><br>
<a href="../utilities/hash.html"><i>hash</i></a><br>
<a href="../utilities/iconv.html"><i>iconv</i></a><br>
&nbsp;</p>
</td>
<td align="left">
<p class="tent"><br>
<a href="../utilities/ipcrm.html"><i>ipcrm</i></a><br>
<a href="../utilities/ipcs.html"><i>ipcs</i></a><br>
<a href="../utilities/link.html"><i>link</i></a><br>
<a href="../utilities/m4.html"><i>m4</i></a><br>
&nbsp;</p>
</td>
<td align="left">
<p class="tent"><br>
<a href="../utilities/nl.html"><i>nl</i></a><br>
<a href="../utilities/prs.html"><i>prs</i></a><br>
<a href="../utilities/sact.html"><i>sact</i></a><br>
<a href="../utilities/sccs.html"><i>sccs</i></a><br>
&nbsp;</p>
</td>
<td align="left">
<p class="tent"><br>
<a href="../utilities/tsort.html"><i>tsort</i></a><br>
<a href="../utilities/ulimit.html"><i>ulimit</i></a><br>
<a href="../utilities/uncompress.html"><i>uncompress</i></a><br>
<a href="../utilities/unget.html"><i>unget</i></a><br>
&nbsp;</p>
</td>
<td align="left">
<p class="tent"><br>
<a href="../utilities/unlink.html"><i>unlink</i></a><br>
<a href="../utilities/uucp.html"><i>uucp</i></a><br>
<a href="../utilities/uustat.html"><i>uustat</i></a><br>
<a href="../utilities/uux.html"><i>uux</i></a><br>
&nbsp;</p>
</td>
<td align="left">
<p class="tent"><br>
<a href="../utilities/val.html"><i>val</i></a><br>
<a href="../utilities/what.html"><i>what</i></a><br>
<a href="../utilities/zcat.html"><i>zcat</i></a><br>
&nbsp;</p>
</td>
</tr>
</table>
</center>
<h4><a name="tag_02_01_05"></a>Terminology</h4>
<p>Refer to <a href="xbd_chap01.html#tag_01_01_04"><i>Terminology</i></a> .</p>
<h4><a name="tag_02_01_06"></a>Definitions</h4>
<p>Refer to <a href="xbd_chap03.html#tag_01_03"><i>Definitions</i></a> .</p>
<h4><a name="tag_02_01_07"></a>Relationship to Other Documents</h4>
<h5><a name="tag_02_01_07_01"></a>System Interfaces</h5>
<p>It has been pointed out that the Shell and Utilities volume of IEEE&nbsp;Std&nbsp;1003.1-2001 assumes that a great deal of
functionality from the System Interfaces volume of IEEE&nbsp;Std&nbsp;1003.1-2001 is present, but never states exactly how much
(and strictly does not need to since both are mandated on a conforming system). This section is an attempt to clarify the
assumptions.</p>
<h5><a name="tag_02_01_07_02"></a>File Removal</h5>
<p>This is intended to be a summary of the <a href="../functions/unlink.html"><i>unlink</i>()</a> and <a href=
"../functions/rmdir.html"><i>rmdir</i>()</a> requirements. Note that it is possible using the <a href=
"../functions/unlink.html"><i>unlink</i>()</a> function for item 4. to occur.</p>
<h5><a name="tag_02_01_07_03"></a>Concepts Derived from the ISO C Standard</h5>
<p>This section was introduced to address the issue that there was insufficient detail presented by such utilities as <a href=
"../utilities/awk.html"><i>awk</i></a> or <a href="../utilities/sh.html"><i>sh</i></a> about their procedural control statements
and their methods of performing arithmetic functions.</p>
<p>The ISO&nbsp;C standard was selected as a model because most historical implementations of the standard utilities were written
in C. Thus, it was more likely that they would act in the desired manner without modification.</p>
<p>Using the ISO&nbsp;C standard is primarily a notational convenience so that the many procedural languages in the Shell and
Utilities volume of IEEE&nbsp;Std&nbsp;1003.1-2001 would not have to be rigorously described in every aspect. Its selection does
not require that the standard utilities be written in Standard C; they could be written in Common Usage C, Ada, Pascal, assembler
language, or anything else.</p>
<p>The sizes of the various numeric values refer to C-language data types that are allowed to be different sizes by the ISO&nbsp;C
standard. Thus, like a C-language application, a shell application cannot rely on their exact size. However, it can rely on their
minimum sizes expressed in the ISO&nbsp;C standard, such as {LONG_MAX} for a <b>long</b> type.</p>
<p>The behavior on overflow is undefined for ISO&nbsp;C standard arithmetic. Therefore, the standard utilities can use &quot;bignum''
representation for integers so that there is no fixed maximum unless otherwise stated in the utility description. Similarly,
standard utilities can use infinite-precision representations for floating-point arithmetic, as long as these representations
exceed the ISO&nbsp;C standard requirements.</p>
<p>This section addresses only the issue of semantics; it is not intended to specify syntax. For example, the ISO&nbsp;C standard
requires that 0L be recognized as an integer constant equal to zero, but utilities such as <a href=
"../utilities/awk.html"><i>awk</i></a> and <a href="../utilities/sh.html"><i>sh</i></a> are not required to recognize 0L (though
they are allowed to, as an extension).</p>
<p>The ISO&nbsp;C standard requires that a C compiler must issue a diagnostic for constants that are too large to represent. Most
standard utilities are not required to issue these diagnostics; for example, the command:</p>
<blockquote>
<pre>
<tt>diff -C 2147483648 file1 file2
</tt>
</pre>
</blockquote>
<p>has undefined behavior, and the <a href="../utilities/diff.html"><i>diff</i></a> utility is not required to issue a diagnostic
even if the number 2147483648 cannot be represented.</p>
<h4><a name="tag_02_01_08"></a>Portability</h4>
<p>Refer to <a href="xbd_chap01.html#tag_01_01_18"><i>Portability</i></a> .</p>
<h5><a name="tag_02_01_08_01"></a>Codes</h5>
<p>Refer to <a href="xbd_chap01.html#tag_01_01_18_01"><i>Codes</i></a> .</p>
<h4><a name="tag_02_01_09"></a>Utility Limits</h4>
<p>This section grew out of an idea that originated with the original POSIX.1, in the tables of system limits for the <a href=
"../functions/sysconf.html"><i>sysconf</i>()</a> and <a href="../functions/pathconf.html"><i>pathconf</i>()</a> functions. The idea
being that a conforming application can be written to use the most restrictive values that a minimal system can provide, but it
should not have to. The values provided represent compromises so that some vendors can use historically limited versions of UNIX
system utilities. They are the highest values that a strictly conforming application can assume, given no other information.</p>
<p>However, by using the <a href="../utilities/getconf.html"><i>getconf</i></a> utility or the <a href=
"../functions/sysconf.html"><i>sysconf</i>()</a> function, the elegant application can be tailored to more liberal values on some
of the specific instances of specific implementations.</p>
<p>There is no explicitly stated requirement that an implementation provide finite limits for any of these numeric values; the
implementation is free to provide essentially unbounded capabilities (where it makes sense), stopping only at reasonable points
such as {ULONG_MAX} (from the ISO&nbsp;C standard). Therefore, applications desiring to tailor themselves to the values on a
particular implementation need to be ready for possibly huge values; it may not be a good idea to allocate blindly a buffer for an
input line based on the value of {LINE_MAX}, for instance. However, unlike the System Interfaces volume of
IEEE&nbsp;Std&nbsp;1003.1-2001, there is no set of limits that return a special indication meaning &quot;unbounded&quot;. The
implementation should always return an actual number, even if the number is very large.</p>
<p>The statement:</p>
<blockquote>
<pre>
&quot;It is not guaranteed that the application ...''
</pre>
</blockquote>
<p>is an indication that many of these limits are designed to ensure that implementors design their utilities without arbitrary
constraints related to unimaginative programming. There are certainly conditions under which combinations of options can cause
failures that would not render an implementation non-conforming. For example, {EXPR_NEST_MAX} and {ARG_MAX} could collide when
expressions are large; combinations of {BC_SCALE_MAX} and {BC_DIM_MAX} could exceed virtual memory.</p>
<p>In the Shell and Utilities volume of IEEE&nbsp;Std&nbsp;1003.1-2001, the notion of a limit being guaranteed for the process
lifetime, as it is in the System Interfaces volume of IEEE&nbsp;Std&nbsp;1003.1-2001, is not as useful to a shell script. The <a
href="../utilities/getconf.html"><i>getconf</i></a> utility is probably a process itself, so the guarantee would be without value.
Therefore, the Shell and Utilities volume of IEEE&nbsp;Std&nbsp;1003.1-2001 requires the guarantee to be for the session lifetime.
This will mean that many vendors will either return very conservative values or possibly implement <a href=
"../utilities/getconf.html"><i>getconf</i></a> as a built-in.</p>
<p>It may seem confusing to have limits that apply only to a single utility grouped into one global section. However, the
alternative, which would be to disperse them out into their utility description sections, would cause great difficulty when <a
href="../functions/sysconf.html"><i>sysconf</i>()</a> and <a href="../utilities/getconf.html"><i>getconf</i></a> were described.
Therefore, the standard developers chose the global approach.</p>
<p>Each language binding could provide symbol names that are slightly different from those shown here. For example, the C-Language
Binding option adds a leading underscore to the symbols as a prefix.</p>
<p>The following comments describe selection criteria for the symbols and their values:</p>
<dl compact>
<dt>{ARG_MAX}</dt>
<dd><br>
This is defined by the System Interfaces volume of IEEE&nbsp;Std&nbsp;1003.1-2001. Unfortunately, it is very difficult for a
conforming application to deal with this value, as it does not know how much of its argument space is being consumed by the
environment variables of the user.<br>
</dd>
<dt>{BC_BASE_MAX}</dt>
<dt>{BC_DIM_MAX}</dt>
<dt>{BC_SCALE_MAX}</dt>
<dd><br>
These were originally one value, {BC_SCALE_MAX}, but it was unreasonable to link all three concepts into one limit.</dd>
<dt>{CHILD_MAX}</dt>
<dd><br>
This is defined by the System Interfaces volume of IEEE&nbsp;Std&nbsp;1003.1-2001.</dd>
<dt>{COLL_WEIGHTS_MAX}</dt>
<dd><br>
The weights assigned to <b>order</b> can be considered as &quot;passes&quot; through the collation algorithm.</dd>
<dt>{EXPR_NEST_MAX}</dt>
<dd><br>
The value for expression nesting was borrowed from the ISO&nbsp;C standard.</dd>
<dt>{LINE_MAX}</dt>
<dd><br>
This is a global limit that affects all utilities, unless otherwise noted. The {MAX_CANON} value from the System Interfaces volume
of IEEE&nbsp;Std&nbsp;1003.1-2001 may further limit input lines from terminals. The {LINE_MAX} value was the subject of much debate
and is a compromise between those who wished to have unlimited lines and those who understood that many historical utilities were
written with fixed buffers. Frequently, utility writers selected the UNIX system constant BUFSIZ to allocate these buffers;
therefore, some utilities were limited to 512 bytes for I/O lines, while others achieved 4096 bytes or greater.
<p>It should be noted that {LINE_MAX} applies only to input line length; there is no requirement in IEEE&nbsp;Std&nbsp;1003.1-2001
that limits the length of output lines. Utilities such as <a href="../utilities/awk.html"><i>awk</i></a>, <a href=
"../utilities/sed.html"><i>sed</i></a>, and <a href="../utilities/paste.html"><i>paste</i></a> could theoretically construct lines
longer than any of the input lines they received, depending on the options used or the instructions from the application. They are
not required to truncate their output to {LINE_MAX}. It is the responsibility of the application to deal with this. If the output
of one of those utilities is to be piped into another of the standard utilities, line length restrictions will have to be
considered; the <a href="../utilities/fold.html"><i>fold</i></a> utility, among others, could be used to ensure that only
reasonable line lengths reach utilities or applications.</p>
</dd>
<dt>{LINK_MAX}</dt>
<dd><br>
This is defined by the System Interfaces volume of IEEE&nbsp;Std&nbsp;1003.1-2001.</dd>
<dt>{MAX_CANON}</dt>
<dt>{MAX_INPUT}</dt>
<dt>{NAME_MAX}</dt>
<dt>{NGROUPS_MAX}</dt>
<dt>{OPEN_MAX}</dt>
<dt>{PATH_MAX}</dt>
<dt>{PIPE_BUF}</dt>
<dd><br>
These limits are defined by the System Interfaces volume of IEEE&nbsp;Std&nbsp;1003.1-2001. Note that the byte lengths described by
some of these values continue to represent bytes, even if the applicable character set uses a multi-byte encoding.</dd>
<dt>{RE_DUP_MAX}</dt>
<dd><br>
The value selected is consistent with historical practice. Although the name implies that it applies to all REs, only BREs use the
interval notation <b>\{</b><i>m,n</i><b>\}</b> addressed by this limit.</dd>
<dt>{POSIX2_SYMLINKS}</dt>
<dd><br>
The {POSIX2_SYMLINKS} variable indicates that the underlying operating system supports the creation of symbolic links in specific
directories. Many of the utilities defined in IEEE&nbsp;Std&nbsp;1003.1-2001 that deal with symbolic links do not depend on this
value. For example, a utility that follows symbolic links (or does not, as the case may be) will only be affected by a symbolic
link if it encounters one. Presumably, a file system that does not support symbolic links will not contain any. This variable does
affect such utilities as <a href="../utilities/ln.html"><i>ln</i></a> <b>-s</b> and <a href="../utilities/pax.html"><i>pax</i></a>
that attempt to create symbolic links.
<p>{POSIX2_SYMLINKS} was developed even though there is no comparable configuration value for the system interfaces.</p>
</dd>
</dl>
<p>There are different limits associated with command lines and input to utilities, depending on the method of invocation. In the
case of a C program <i>exec</i>-ing a utility, {ARG_MAX} is the underlying limit. In the case of the shell reading a script and
<i>exec</i>-ing a utility, {LINE_MAX} limits the length of lines the shell is required to process, and {ARG_MAX} will still be a
limit. If a user is entering a command on a terminal to the shell, requesting that it invoke the utility, {MAX_INPUT} may restrict
the length of the line that can be given to the shell to a value below {LINE_MAX}.</p>
<p>When an option is supported, <a href="../utilities/getconf.html"><i>getconf</i></a> returns a value of 1. For example, when C
development is supported:</p>
<blockquote>
<pre>
<tt>if [ "$(getconf POSIX2_C_DEV)" -eq 1 ]; then
echo C supported
fi
</tt>
</pre>
</blockquote>
<p>The <a href="../functions/sysconf.html"><i>sysconf</i>()</a> function in the C-Language Binding option would return 1.</p>
<p>The following comments describe selection criteria for the symbols and their values:</p>
<dl compact>
<dt>POSIX2_C_BIND</dt>
<dt>POSIX2_C_DEV</dt>
<dt>POSIX2_FORT_DEV</dt>
<dt>POSIX2_FORT_RUN</dt>
<dt>POSIX2_SW_DEV</dt>
<dt>POSIX2_UPE</dt>
<dd><br>
It is possible for some (usually privileged) operations to remove utilities that support these options or otherwise to render these
options unsupported. The header files, the <a href="../functions/sysconf.html"><i>sysconf</i>()</a> function, or the <a href=
"../utilities/getconf.html"><i>getconf</i></a> utility will not necessarily detect such actions, in which case they should not be
considered as rendering the implementation non-conforming. A test suite should not attempt tests such as:
<pre>
<tt>rm /usr/bin/c99
getconf POSIX2_C_DEV
</tt>
</pre>
</dd>
<dt>POSIX2_LOCALEDEF</dt>
<dd><br>
This symbol was introduced to allow implementations to restrict supported locales to only those supplied by the
implementation.</dd>
</dl>
<h4><a name="tag_02_01_10"></a>Grammar Conventions</h4>
<p>There is no additional rationale provided for this section.</p>
<h4><a name="tag_02_01_11"></a>Utility Description Defaults</h4>
<p>This section is arranged with headings in the same order as all the utility descriptions. It is a collection of related and
unrelated information concerning:</p>
<ol>
<li>
<p>The default actions of utilities</p>
</li>
<li>
<p>The meanings of notations used in IEEE&nbsp;Std&nbsp;1003.1-2001 that are specific to individual utility sections</p>
</li>
</ol>
<p>Although this material may seem out of place here, it is important that this information appear before any of the utilities to
be described later.</p>
<h5><a name="tag_02_01_11_01"></a>NAME</h5>
<p>There is no additional rationale provided for this section.</p>
<h5><a name="tag_02_01_11_02"></a>SYNOPSIS</h5>
<p>There is no additional rationale provided for this section.</p>
<h5><a name="tag_02_01_11_03"></a>DESCRIPTION</h5>
<p>There is no additional rationale provided for this section.</p>
<h5><a name="tag_02_01_11_04"></a>OPTIONS</h5>
<p>Although it has not always been possible, the standard developers tried to avoid repeating information to reduce the risk that
duplicate explanations could each be modified differently.</p>
<p>The need to recognize <b>--</b> is required because conforming applications need to shield their operands from any arbitrary
options that the implementation may provide as an extension. For example, if the standard utility <i>foo</i> is listed as taking no
options, and the application needed to give it a pathname with a leading hyphen, it could safely do it as:</p>
<pre>
<tt>foo -- -myfile
</tt>
</pre>
<p>and avoid any problems with <b>-m</b> used as an extension.</p>
<h5><a name="tag_02_01_11_05"></a>OPERANDS</h5>
<p>The usage of <b>-</b> is never shown in the SYNOPSIS. Similarly, the usage of <b>--</b> is never shown.</p>
<p>The requirement for processing operands in command-line order is to avoid a &quot;WeirdNIX&quot; utility that might choose to sort the
input files alphabetically, by size, or by directory order. Although this might be acceptable for some utilities, in general the
programmer has a right to know exactly what order will be chosen.</p>
<p>Some of the standard utilities take multiple <i>file</i> operands and act as if they were processing the concatenation of those
files. For example:</p>
<blockquote>
<pre>
<tt>asa file1 file2
</tt>
</pre>
</blockquote>
<p>and:</p>
<blockquote>
<pre>
<tt>cat file1 file2 | asa
</tt>
</pre>
</blockquote>
<p>have similar results when questions of file access, errors, and performance are ignored. Other utilities such as <a href=
"../utilities/grep.html"><i>grep</i></a> or <a href="../utilities/wc.html"><i>wc</i></a> have completely different results in these
two cases. This latter type of utility is always identified in its DESCRIPTION or OPERANDS sections, whereas the former is not.
Although it might be possible to create a general assertion about the former case, the following points must be addressed:</p>
<ul>
<li>
<p>Access times for the files might be different in the operand case <i>versus</i> the <a href=
"../utilities/cat.html"><i>cat</i></a> case.</p>
</li>
<li>
<p>The utility may have error messages that are cognizant of the input filename, and this added value should not be suppressed. (As
an example, <a href="../utilities/awk.html"><i>awk</i></a> sets a variable with the filename at each file boundary.)</p>
</li>
</ul>
<h5><a name="tag_02_01_11_06"></a>STDIN</h5>
<p>There is no additional rationale provided for this section.</p>
<h5><a name="tag_02_01_11_07"></a>INPUT FILES</h5>
<p>A conforming application cannot assume the following three commands are equivalent:</p>
<pre>
<tt>tail -n +2 file
(sed -n 1q; cat) &lt; file
cat file | (sed -n 1q; cat)
</tt>
</pre>
<p>The second command is equivalent to the first only when the file is seekable. In the third command, if the file offset in the
open file description were not unspecified, <a href="../utilities/sed.html"><i>sed</i></a> would have to be implemented so that it
read from the pipe 1 byte at a time or it would have to employ some method to seek backwards on the pipe. Such functionality is not
defined currently in POSIX.1 and does not exist on all historical systems. Other utilities, such as <a href=
"../utilities/head.html"><i>head</i></a>, <a href="../utilities/read.html"><i>read</i></a>, and <a href=
"../utilities/sh.html"><i>sh</i></a>, have similar properties, so the restriction is described globally in this section.</p>
<p>The definition of &quot;text file&quot; is strictly enforced for input to the standard utilities; very few of them list exceptions to
the undefined results called for here. (Of course, &quot;undefined&quot; here does not mean that historical implementations necessarily
have to change to start indicating error conditions. Conforming applications cannot rely on implementations succeeding or failing
when non-text files are used.)</p>
<p>The utilities that allow line continuation are generally those that accept input languages, rather than pure data. It would be
unusual for an input line of this type to exceed {LINE_MAX} bytes and unreasonable to require that the implementation allow
unlimited accumulation of multiple lines, each of which could reach {LINE_MAX}. Thus, for a conforming application the total of all
the continued lines in a set cannot exceed {LINE_MAX}.</p>
<p>The format description is intended to be sufficiently rigorous to allow other applications to generate these input files.
However, since &lt;blank&gt;s can legitimately be included in some of the fields described by the standard utilities, particularly
in locales other than the POSIX locale, this intent is not always realized.</p>
<h5><a name="tag_02_01_11_08"></a>ENVIRONMENT VARIABLES</h5>
<p>There is no additional rationale provided for this section.</p>
<h5><a name="tag_02_01_11_09"></a>ASYNCHRONOUS EVENTS</h5>
<p>Because there is no language prohibiting it, a utility is permitted to catch a signal, perform some additional processing (such
as deleting temporary files), restore the default signal action (or action inherited from the parent process), and resignal
itself.</p>
<h5><a name="tag_02_01_11_10"></a>STDOUT</h5>
<p>The format description is intended to be sufficiently rigorous to allow post-processing of output by other programs,
particularly by an <a href="../utilities/awk.html"><i>awk</i></a> or <a href="../utilities/lex.html"><i>lex</i></a> parser.</p>
<h5><a name="tag_02_01_11_11"></a>STDERR</h5>
<p>This section does not describe error messages that refer to incorrect operation of the utility. Consider a utility that
processes program source code as its input. This section is used to describe messages produced by a correctly operating utility
that encounters an error in the program source code on which it is processing. However, a message indicating that the utility had
insufficient memory in which to operate would not be described.</p>
<p>Some utilities have traditionally produced warning messages without returning a non-zero exit status; these are specifically
noted in their sections. Other utilities shall not write to standard error if they complete successfully, unless the implementation
provides some sort of extension to increase the verbosity or debugging level.</p>
<p>The format descriptions are intended to be sufficiently rigorous to allow post-processing of output by other programs.</p>
<h5><a name="tag_02_01_11_12"></a>OUTPUT FILES</h5>
<p>The format description is intended to be sufficiently rigorous to allow post-processing of output by other programs,
particularly by an <a href="../utilities/awk.html"><i>awk</i></a> or <a href="../utilities/lex.html"><i>lex</i></a> parser.</p>
<p>Receipt of the SIGQUIT signal should generally cause termination (unless in some debugging mode) that would bypass any attempted
recovery actions.</p>
<h5><a name="tag_02_01_11_13"></a>EXTENDED DESCRIPTION</h5>
<p>There is no additional rationale provided for this section.</p>
<h5><a name="tag_02_01_11_14"></a>EXIT STATUS</h5>
<p>Note the additional discussion of exit values in <i>Exit Status for Commands</i> in the <a href=
"../utilities/sh.html"><i>sh</i></a> utility. It describes requirements for returning exit values greater than 125.</p>
<p>A utility may list zero as a successful return, 1 as a failure for a specific reason, and greater than 1 as &quot;an error
occurred&quot;. In this case, unspecified conditions may cause a 2 or 3, or other value, to be returned. A strictly conforming
application should be written so that it tests for successful exit status values (zero in this case), rather than relying upon the
single specific error value listed in IEEE&nbsp;Std&nbsp;1003.1-2001. In that way, it will have maximum portability, even on
implementations with extensions.</p>
<p>The standard developers are aware that the general non-enumeration of errors makes it difficult to write test suites that test
the <i>incorrect</i> operation of utilities. There are some historical implementations that have expended effort to provide
detailed status messages and a helpful environment to bypass or explain errors, such as prompting, retrying, or ignoring
unimportant syntax errors; other implementations have not. Since there is no realistic way to mandate system behavior in cases of
undefined application actions or system problems-in a manner acceptable to all cultures and environments-attention has been limited
to the correct operation of utilities by the conforming application. Furthermore, the conforming application does not need detailed
information concerning errors that it caused through incorrect usage or that it cannot correct.</p>
<p>There is no description of defaults for this section because all of the standard utilities specify something (or explicitly
state &quot;Unspecified&quot;) for exit status.</p>
<h5><a name="tag_02_01_11_15"></a>CONSEQUENCES OF ERRORS</h5>
<p>Several actions are possible when a utility encounters an error condition, depending on the severity of the error and the state
of the utility. Included in the possible actions of various utilities are: deletion of temporary or intermediate work files;
deletion of incomplete files; and validity checking of the file system or directory.</p>
<p>The text about recursive traversing is meant to ensure that utilities such as <a href="../utilities/find.html"><i>find</i></a>
process as many files in the hierarchy as they can. They should not abandon all of the hierarchy at the first error and resume with
the next command-line operand, but should attempt to keep going.</p>
<h5><a name="tag_02_01_11_16"></a>APPLICATION USAGE</h5>
<p>This section provides additional caveats, issues, and recommendations to the developer.</p>
<h5><a name="tag_02_01_11_17"></a>EXAMPLES</h5>
<p>This section provides sample usage.</p>
<h5><a name="tag_02_01_11_18"></a>RATIONALE</h5>
<p>There is no additional rationale provided for this section.</p>
<h5><a name="tag_02_01_11_19"></a>FUTURE DIRECTIONS</h5>
<p>FUTURE DIRECTIONS sections act as pointers to related work that may impact the interface in the future, and often cautions the
developer to architect the code to account for a change in this area. Note that a future directions statement should not be taken
as a commitment to adopt a feature or interface in the future.</p>
<h5><a name="tag_02_01_11_20"></a>SEE ALSO</h5>
<p>There is no additional rationale provided for this section.</p>
<h5><a name="tag_02_01_11_21"></a>CHANGE HISTORY</h5>
<p>There is no additional rationale provided for this section.</p>
<h4><a name="tag_02_01_12"></a>Considerations for Utilities in Support of Files of Arbitrary Size</h4>
<p>This section is intended to clarify the requirements for utilities in support of large files.</p>
<p>The utilities listed in this section are utilities which are used to perform administrative tasks such as to create, move, copy,
remove, change the permissions, or measure the resources of a file. They are useful both as end-user tools and as utilities invoked
by applications during software installation and operation.</p>
<p>The <a href="../utilities/chgrp.html"><i>chgrp</i></a>, <a href="../utilities/chmod.html"><i>chmod</i></a>, <a href=
"../utilities/chown.html"><i>chown</i></a>, <a href="../utilities/ln.html"><i>ln</i></a>, and <a href=
"../utilities/rm.html"><i>rm</i></a> utilities probably require use of large file-capable versions of <a href=
"../functions/stat.html"><i>stat</i>()</a>, <a href="../functions/lstat.html"><i>lstat</i>()</a>, <a href=
"../functions/ftw.html"><i>ftw</i>()</a>, and the <b>stat</b> structure.</p>
<p>The <a href="../utilities/cat.html"><i>cat</i></a>, <a href="../utilities/cksum.html"><i>cksum</i></a>, <a href=
"../utilities/cmp.html"><i>cmp</i></a>, <a href="../utilities/cp.html"><i>cp</i></a>, <a href="../utilities/dd.html"><i>dd</i></a>,
<a href="../utilities/mv.html"><i>mv</i></a>, <i>sum</i>, and <a href="../utilities/touch.html"><i>touch</i></a> utilities probably
require use of large file-capable versions of <a href="../functions/creat.html"><i>creat</i>()</a>, <a href=
"../functions/open.html"><i>open</i>()</a>, and <a href="../functions/fopen.html"><i>fopen</i>()</a>.</p>
<p>The <a href="../utilities/cat.html"><i>cat</i></a>, <a href="../utilities/cksum.html"><i>cksum</i></a>, <a href=
"../utilities/cmp.html"><i>cmp</i></a>, <a href="../utilities/dd.html"><i>dd</i></a>, <a href="../utilities/df.html"><i>df</i></a>,
<a href="../utilities/du.html"><i>du</i></a>, <a href="../utilities/ls.html"><i>ls</i></a>, and <i>sum</i> utilities may require
writing large integer values. For example:</p>
<ul>
<li>
<p>The <a href="../utilities/cat.html"><i>cat</i></a> utility might have a <b>-n</b> option which counts &lt;newline&gt;s.</p>
</li>
<li>
<p>The <a href="../utilities/cksum.html"><i>cksum</i></a> and <a href="../utilities/ls.html"><i>ls</i></a> utilities report file
sizes.</p>
</li>
<li>
<p>The <a href="../utilities/cmp.html"><i>cmp</i></a> utility reports the line number at which the first difference occurs, and
also has a <b>-l</b> option which reports file offsets.</p>
</li>
<li>
<p>The <a href="../utilities/dd.html"><i>dd</i></a>, <a href="../utilities/df.html"><i>df</i></a>, <a href=
"../utilities/du.html"><i>du</i></a>, <a href="../utilities/ls.html"><i>ls</i></a>, and <i>sum</i> utilities report block
counts.</p>
</li>
</ul>
<p>The <a href="../utilities/dd.html"><i>dd</i></a>, <a href="../utilities/find.html"><i>find</i></a>, and <a href=
"../utilities/test.html"><i>test</i></a> utilities may need to interpret command arguments that contain 64-bit values. For <a href=
"../utilities/dd.html"><i>dd</i></a>, the arguments include <i>skip</i>= <i>n</i>, <i>seek</i>= <i>n</i>, and <i>count</i>=
<i>n</i>. For <a href="../utilities/find.html"><i>find</i></a>, the arguments include <b>-size</b> <i>n</i>. For <a href=
"../utilities/test.html"><i>test</i></a>, the arguments are those associated with algebraic comparisons.</p>
<p>The <a href="../utilities/df.html"><i>df</i></a> utility might need to access large file systems with <a href=
"../functions/statvfs.html"><i>statvfs</i>()</a>.</p>
<p>The <a href="../utilities/ulimit.html"><i>ulimit</i></a> utility will need to use large file-capable versions of <a href=
"../functions/getrlimit.html"><i>getrlimit</i>()</a> and <a href="../functions/setrlimit.html"><i>setrlimit</i>()</a> and be able
to read and write large integer values.</p>
<h4><a name="tag_02_01_13"></a>Built-In Utilities</h4>
<p>All of these utilities can be <i>exec</i>-ed. There is no requirement that these utilities are actually built into the shell
itself, but many shells need the capability to do so because the Shell and Utilities volume of IEEE&nbsp;Std&nbsp;1003.1-2001, <a
href="../utilities/xcu_chap02.html#tag_02_09_01_01">Section 2.9.1.1, Command Search and Execution</a> requires that they be found
prior to the <i>PATH</i> search. The shell could satisfy its requirements by keeping a list of the names and directly accessing the
file-system versions regardless of <i>PATH .</i> Providing all of the required functionality for those such as <a href=
"../utilities/cd.html"><i>cd</i></a> or <a href="../utilities/read.html"><i>read</i></a> would be more difficult.</p>
<p>There were originally three justifications for allowing the omission of <i>exec</i>-able versions:</p>
<ol>
<li>
<p>It would require wasting space in the file system, at the expense of very small systems. However, it has been pointed out that
all 16 utilities in the table can be provided with 16 links to a single-line shell script:</p>
<pre>
<tt>$0 "$@"
</tt>
</pre>
</li>
<li>
<p>It is not logical to require invocation of utilities such as <a href="../utilities/cd.html"><i>cd</i></a> because they have no
value outside the shell environment or cannot be useful in a child process. However, counter-examples always seemed to be available
for even the most unusual cases:</p>
<pre>
<tt>find . -type d -exec cd {} \; -exec foo {} \;
</tt> (which invokes &quot;foo&quot; on accessible directories)<tt><br>
ps ... | sed ... | xargs kill
<br>
find . -exec true \; -a ...
</tt> (where &quot;true&quot; is used for temporary debugging)
</pre>
</li>
<li>
<p>It is confusing to have a utility such as <a href="../utilities/kill.html"><i>kill</i></a> that can easily be in the file system
in the base standard, but that requires built-in status for the User Portability Utilities option (for the <tt>%</tt> job control
job ID notation). It was decided that it was more appropriate to describe the required functionality (rather than the
implementation) to the system implementors and let them decide how to satisfy it.</p>
</li>
</ol>
<p>On the other hand, it was realized that any distinction like this between utilities was not useful to applications, and that the
cost to correct it was small. These arguments were ultimately the most effective.</p>
<p>There were varying reasons for including utilities in the table of built-ins:</p>
<dl compact>
<dt><i>alias</i>, <i>fc</i>, <i>unalias</i></dt>
<dd><br>
The functionality of these utilities is performed more simply within the shell itself and that is the model most historical
implementations have used.</dd>
<dt><i>bg</i>, <i>fg</i>, <i>jobs</i></dt>
<dd><br>
All of the job control-related utilities are eligible for built-in status because that is the model most historical implementations
have used.</dd>
<dt><i>cd</i>, <i>getopts</i>, <i>newgrp</i>, <i>read</i>, <i>umask</i>, <i>wait</i></dt>
<dd><br>
The functionality of these utilities is performed more simply within the context of the current process. An example can be taken
from the usage of the <a href="../utilities/cd.html"><i>cd</i></a> utility. The purpose of the <a href=
"../utilities/cd.html"><i>cd</i></a> utility is to change the working directory for subsequent operations. The actions of <a href=
"../utilities/cd.html"><i>cd</i></a> affect the process in which <a href="../utilities/cd.html"><i>cd</i></a> is executed and all
subsequent child processes of that process. Based on the POSIX standard process model, changes in the process environment of a
child process have no effect on the parent process. If the <a href="../utilities/cd.html"><i>cd</i></a> utility were executed from
a child process, the working directory change would be effective only in the child process. Child processes initiated subsequent to
the child process that executed the <a href="../utilities/cd.html"><i>cd</i></a> utility would not have a changed working directory
relative to the parent process.</dd>
<dt><i>command</i></dt>
<dd><br>
This utility was placed in the table primarily to protect scripts that are concerned about their <i>PATH</i> being manipulated. The
&quot;secure&quot; shell script example in the <a href="../utilities/command.html"><i>command</i></a> utility in the Shell and Utilities
volume of IEEE&nbsp;Std&nbsp;1003.1-2001 would not be possible if a <i>PATH</i> change retrieved an alien version of <a href=
"../utilities/command.html"><i>command</i></a>. (An alternative would have been to implement <a href=
"../utilities/getconf.html"><i>getconf</i></a> as a built-in, but the standard developers considered that it carried too many
changing configuration strings to require in the shell.)</dd>
<dt><i>kill</i></dt>
<dd>Since <a href="../utilities/kill.html"><i>kill</i></a> provides optional job control functionality using shell notation (
<tt>%1</tt> , <tt>%2</tt> , and so on), some implementations would find it extremely difficult to provide this outside the
shell.</dd>
<dt><i>true</i>, <i>false</i></dt>
<dd><br>
These are in the table as a courtesy to programmers who wish to use the <tt>"while&nbsp;true"</tt> shell construct without
protecting <a href="../utilities/true.html"><i>true</i></a> from <i>PATH</i> searches. (It is acknowledged that
<tt>"while&nbsp;:"</tt> also works, but the idiom with <a href="../utilities/true.html"><i>true</i></a> is historically
pervasive.)</dd>
</dl>
<p>All utilities, including those in the table, are accessible via the <a href="../functions/system.html"><i>system</i>()</a> and
<a href="../functions/popen.html"><i>popen</i>()</a> functions in the System Interfaces volume of IEEE&nbsp;Std&nbsp;1003.1-2001.
There are situations where the return functionality of <a href="../functions/system.html"><i>system</i>()</a> and <a href=
"../functions/popen.html"><i>popen</i>()</a> is not desirable. Applications that require the exit status of the invoked utility
will not be able to use <a href="../functions/system.html"><i>system</i>()</a> or <a href=
"../functions/popen.html"><i>popen</i>()</a>, since the exit status returned is that of the command language interpreter rather
than that of the invoked utility. The alternative for such applications is the use of the <i>exec</i> family.</p>
<hr size="2" noshade>
<center><font size="2"><!--footer start-->
UNIX &reg; is a registered Trademark of The Open Group.<br>
POSIX &reg; is a registered Trademark of The IEEE.<br>
[ <a href="../mindex.html">Main Index</a> | <a href="../basedefs/contents.html">XBD</a> | <a href=
"../utilities/contents.html">XCU</a> | <a href="../functions/contents.html">XSH</a> | <a href="../xrat/contents.html">XRAT</a>
]</font></center>
<!--footer end-->
<hr size="2" noshade>
</body>
</html>