From the original OpenBSD commit message:
restore the traditional behavior of -f implying -a; apparently Keith
Bostic forgot to restore it when the -f flag was put back on 2nd of
September 1989, after being removed on 16th of August as a
consequence of issues getting it working over NFS, so deviation from
traditional UNIX behavior in all BSDs looks like an historical
accident; as a side effect, this change accommodates behavior of
this option to IEEE Std 1003.1-2008 (``POSIX.1'').
joint work with jmc@ (who found the inaccuracy in our
implementation), schwarze@ (who provided a detailed tracking of
historical facts) and millert@
Submitted by: Igor Sobrado
Discussed with: mckusick
Obtained from: OpenBSD project
MFC after: 2 weeks
borrowed where syntax status=noxfer means no transfer statistics
and status=none means no status information at all.
This feature is useful because the statistics information can
sometimes be annoying, and redirecting stderr to /dev/null would
mean error messages also gets silenced.
Obtained from: OpenBSD
MFC after: 2 weeks
all the SUBDIR entries in parallel, instead of serially. Apply this
option to a selected number of Makefiles, which can greatly speed up the
build on multi-core machines, when using make -j.
This can be extended to more Makefiles later on, whenever they are
verified to work correctly with parallel building.
I tested this on a 24-core machine, with make -j48 buildworld (N = 6):
before stddev after stddev
======= ====== ======= ======
real time 1741.1 16.5 959.8 2.7
user time 12468.7 16.4 14393.0 16.8
sys time 1825.0 54.8 2110.6 22.8
(user+sys)/real 8.2 17.1
E.g. the build was approximately 45% faster in real time. On machines
with less cores, or with lower -j settings, the speedup will not be as
impressive. But at least you can now almost max out a machine with
buildworld!
Submitted by: jilles
MFC after: 2 weeks
Only store exit status for a process if that process has not terminated yet.
Test (slow):
exit 7 & p1=$!; until exit 8 & p2=$!; [ "$p1" = "$p2" ]; do wait "$p2";
done; sleep 0.1; wait %1; echo $?
should write "7".
Interestingly, the pkill tool lives in bin, not usr.bin. Haven't bothered
to check if this is because the tool moved or because the tests were
originally added in the wrong place.
When killing a %job started without job control, kill all processes in it.
As with process groups and zombies, if any process in the job can be killed
or has already terminated, the command is successful.
This also fixes occasional failures of the builtins/kill1.0 test.
Change {atf,plain,tap}.test.mk to be internal implementation details of
bsd.test.mk. Makefiles that build tests should now only include bsd.test.mk
and declaratively specify what they want to build, without worrying about
the internal implementation of the mk files.
The reason for this change is to permit building test programs of different
interfaces from a single directory, which is something I had a need for
while porting tests over from src/tools/regression/.
Additionally, this change makes it possible to perform some other requested
changes to bsd.test.mk in an easier manner. Coming soon.
If a job has terminated but is still known, silently do nothing when using
the kill builtin with the job specifier. Formerly, the shell called kill()
with the process group ID that might have been reused.
Redo expari() like evalvar(). This makes the logic more understandable and
avoids possible problems if arithmetic expansion occurs if CTLESC characters
are not generated (looking backwards for CTLARI is not generally possible in
that case but the old code tried anyway).
This adds an extra argstr() recursion.
If an alias's value ends with a space or tab, the next word is also
checked for aliases.
This is a POSIX feature. It is useful with utilities like command and
nohup (alias them to themselves followed by a space).
Add the space to avoid alias recursion when the alias is expanded, not when
it is added.
As a result, displaying an alias via command -v, command -V or type no
longer erroneously appends a space. Adjust the tests so they now require
this bug to be absent.
The SIGWINCH handler triggers breakage in libedit which is hard to fix; see
PR bin/169773.
Also, window size changes while a program is in foreground (and it rather
than sh will receive SIGWINCH) will now be picked up automatically.
Downside: it is now certain that a resize is only processed after pressing
<Enter>. If libedit is fixed, sh will most likely have to be changed also.
PR: bin/180146
Redo r260506 by using the new TEST_METADATA functionality of bsd.test.mk
to mark the sh(1) and test(1) tests as not supporting root. This is to
get rid of hand-crafted Kyuafiles for these very simple cases.
MFC after: 5 days
Redo r260586 by using the new functionality in tap.test.mk to transparently
support perl-based test programs.
As a side-effect, we get rid of an explicit path to /usr/bin/perl by
replacing it with /usr/local/bin/perl (or as defined in tap.test.mk).
This also fixes the name of the legacy_test source file because this should
have always been legacy_test.pl and not legacy_test.sh. My mistake when
originally moving the code around without realizing that this was a perl
script.
MFC after: 5 days
The effect of this is that the test program is marked as skipped when perl
is missing, instead of marking it as broken due to an execution failure.
MFC after: 3 days
One of the tests for test(1) fails and some of the tests for sh(1) are
silently bypassed when running as root.
To fix these tests and ensure they all run, mark the test programs for
sh(1) and test(1) as requiring an unprivileged user. (This should and
will be the default in Kyua but isn't yet.)
MFC after: 1 week
preadbuffer() maintained a flag whether there was any non-whitespace
character. This flag is only useful when history is enabled (in that case,
lines containing only whitespace are not added to history). Instead, check
using strspn() when history is enabled.
There is an approximate 2% speedup when running
sh -c '. /etc/rc.subr; . /etc/defaults/rc.conf; source_rc_confs'
with hot cache.
This change is a proof of concept on how to easily integrate existing
tests from the tools/regression/ hierarchy into the /usr/tests/ test
suite and on how to adapt them to the new layout for src.
To achieve these goals, this change:
- Moves tests from tools/regression/bin/<tool>/ to bin/<tool>/tests/.
- Renames the previous regress.sh files to legacy_test.sh.
- Adds Makefiles to build and install the tests and all their supporting
data files into /usr/tests/bin/.
- Plugs the legacy_test test programs into the test suite using the new
TAP backend for Kyua (appearing in 0.8) so that the code of the test
programs does not have to change.
- Registers the new directories in the BSD.test.dist mtree file.
Reviewed by: freebsd-testing
Approved by: rpaulo (mentor)
Although <&0 does nothing, it is a redirection affecting standard input and
should therefore disable the </dev/null redirection implicit in a background
command.
If job control is not enabled, background jobs started with ... & ignore
SIGINT and SIGQUIT so that they are not affected by such signals that are
intended for the foreground job. However, this should not prevent
reassigning a different action for these signals (as if the shell invocation
inherited these signal actions from its parent).
Austin group issue #751
Example:
{ trap - INT; exec sleep 10; } & wait
A Ctrl+C should terminate the sleep command.
user. Kqueue now saves the ucred of the allocating thread, to
correctly decrement the counter on close.
Under some specific and not real-world use scenario for kqueue, it is
possible for the kqueues to consume memory proportional to the square
of the number of the filedescriptors available to the process. Limit
allows administrator to prevent the abuse.
This is kernel-mode side of the change, with the user-mode enabling
commit following.
Reported and tested by: pho
Discussed with: jmg
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
The getpgrp() call is unnecessary: if there is no job control then the
result was not used at all and if there is job control then we are not a
subshell and our process group ID is equal to our process ID (rootpid).
Formerly, return always returned from a function if it was called from a
function, even if there was a closer dot script. This was for compatibility
with the Bourne shell which only allowed returning from functions.
Other modern shells and POSIX return from the function or the dot script,
whichever is closest.
Git 1.8.4's rebase --continue depends on the POSIX behaviour.
Reported by: Christoph Mallon, avg
The change in r238888 was incomplete. It was still possible for a trapped
signal to arrive before the shell went to sleep (sigsuspend()) because a
check was missing or because the signal arrived before in_waitcmd was set.
On SMP, this bug sometimes caused the builtins/wait4.0 test to take 1 second
to execute; it then might or might not fail. On UP, the test almost always
failed.
The erflag argument was only used by old-style (``) command substitutions.
We can remove it and handle the special case in the command substitution
code.
NEOF needs to be a non-null pointer distinct from valid union node pointers.
It is not dereferenced.
The new NEOF is much like SIG_ERR except that it is an object pointer
instead of a function pointer.
The variable tokpushback can now be static.
ps(1) utility, e.g. "ps -O fib".
bin/ps/keyword.c:
Add the "fib" keyword and default its column name to "FIB".
bin/ps/ps.1:
Add "fib" as a supported keyword.
sys/compat/freebsd32/freebsd32.h:
sys/kern/kern_proc.c:
sys/sys/user.h:
Add the default fib number for a process (p->p_fibnum)
to the user land accessible process data of struct kinfo_proc.
Submitted by: Oliver Fromme <olli@fromme.com>, gibbs
As per POSIX, a simple command must have at least one redirection,
assignment word or command word.
These occured in rare cases such as eval "f()" .
The extension of allowing no commands inside { }, if, while, for, etc.
remains.
POSIX does not require ++ and -- in arithmetic. It is probably more useful
to reject them than to treat ++x and --x as x silently.
Note that the behaviour of increment and decrement can be obtained via
(x+=1), ((x+=1)-1), (x-=1) and ((x-=1)+1).
PR: bin/176444
If a job is specified to 'wait', wait for it to complete. Formerly, in
interactive mode, the job was deleted if it stopped.
If no jobs are specified in interactive mode, 'wait' still waits for all jobs
to complete or stop.
In non-interactive mode, WUNTRACED is not passed to wait3() so stopped jobs
are not detected.
PR: bin/181435
and CIFS file attributes as BSD stat(2) flags.
This work is intended to be compatible with ZFS, the Solaris CIFS
server's interaction with ZFS, somewhat compatible with MacOS X,
and of course compatible with Windows.
The Windows attributes that are implemented were chosen based on
the attributes that ZFS already supports.
The summary of the flags is as follows:
UF_SYSTEM: Command line name: "system" or "usystem"
ZFS name: XAT_SYSTEM, ZFS_SYSTEM
Windows: FILE_ATTRIBUTE_SYSTEM
This flag means that the file is used by the
operating system. FreeBSD does not enforce any
special handling when this flag is set.
UF_SPARSE: Command line name: "sparse" or "usparse"
ZFS name: XAT_SPARSE, ZFS_SPARSE
Windows: FILE_ATTRIBUTE_SPARSE_FILE
This flag means that the file is sparse. Although
ZFS may modify this in some situations, there is
not generally any special handling for this flag.
UF_OFFLINE: Command line name: "offline" or "uoffline"
ZFS name: XAT_OFFLINE, ZFS_OFFLINE
Windows: FILE_ATTRIBUTE_OFFLINE
This flag means that the file has been moved to
offline storage. FreeBSD does not have any special
handling for this flag.
UF_REPARSE: Command line name: "reparse" or "ureparse"
ZFS name: XAT_REPARSE, ZFS_REPARSE
Windows: FILE_ATTRIBUTE_REPARSE_POINT
This flag means that the file is a Windows reparse
point. ZFS has special handling code for reparse
points, but we don't currently have the other
supporting infrastructure for them.
UF_HIDDEN: Command line name: "hidden" or "uhidden"
ZFS name: XAT_HIDDEN, ZFS_HIDDEN
Windows: FILE_ATTRIBUTE_HIDDEN
This flag means that the file may be excluded from
a directory listing if the application honors it.
FreeBSD has no special handling for this flag.
The name and bit definition for UF_HIDDEN are
identical to the definition in MacOS X.
UF_READONLY: Command line name: "urdonly", "rdonly", "readonly"
ZFS name: XAT_READONLY, ZFS_READONLY
Windows: FILE_ATTRIBUTE_READONLY
This flag means that the file may not written or
appended, but its attributes may be changed.
ZFS currently enforces this flag, but Illumos
developers have discussed disabling enforcement.
The behavior of this flag is different than MacOS X.
MacOS X uses UF_IMMUTABLE to represent the DOS
readonly permission, but that flag has a stronger
meaning than the semantics of DOS readonly permissions.
UF_ARCHIVE: Command line name: "uarch", "uarchive"
ZFS_NAME: XAT_ARCHIVE, ZFS_ARCHIVE
Windows name: FILE_ATTRIBUTE_ARCHIVE
The UF_ARCHIVED flag means that the file has changed and
needs to be archived. The meaning is same as
the Windows FILE_ATTRIBUTE_ARCHIVE attribute, and
the ZFS XAT_ARCHIVE and ZFS_ARCHIVE attribute.
msdosfs and ZFS have special handling for this flag.
i.e. they will set it when the file changes.
sys/param.h: Bump __FreeBSD_version to 1000047 for the
addition of new stat(2) flags.
chflags.1: Document the new command line flag names
(e.g. "system", "hidden") available to the
user.
ls.1: Reference chflags(1) for a list of file flags
and their meanings.
strtofflags.c: Implement the mapping between the new
command line flag names and new stat(2)
flags.
chflags.2: Document all of the new stat(2) flags, and
explain the intended behavior in a little
more detail. Explain how they map to
Windows file attributes.
Different filesystems behave differently
with respect to flags, so warn the
application developer to take care when
using them.
zfs_vnops.c: Add support for getting and setting the
UF_ARCHIVE, UF_READONLY, UF_SYSTEM, UF_HIDDEN,
UF_REPARSE, UF_OFFLINE, and UF_SPARSE flags.
All of these flags are implemented using
attributes that ZFS already supports, so
the on-disk format has not changed.
ZFS currently doesn't allow setting the
UF_REPARSE flag, and we don't really have
the other infrastructure to support reparse
points.
msdosfs_denode.c,
msdosfs_vnops.c: Add support for getting and setting
UF_HIDDEN, UF_SYSTEM and UF_READONLY
in MSDOSFS.
It supported SF_ARCHIVED, but this has been
changed to be UF_ARCHIVE, which has the same
semantics as the DOS archive attribute instead
of inverse semantics like SF_ARCHIVED.
After discussion with Bruce Evans, change
several things in the msdosfs behavior:
Use UF_READONLY to indicate whether a file
is writeable instead of file permissions, but
don't actually enforce it.
Refuse to change attributes on the root
directory, because it is special in FAT
filesystems, but allow most other attribute
changes on directories.
Don't set the archive attribute on a directory
when its modification time is updated.
Windows and DOS don't set the archive attribute
in that scenario, so we are now bug-for-bug
compatible.
smbfs_node.c,
smbfs_vnops.c: Add support for UF_HIDDEN, UF_SYSTEM,
UF_READONLY and UF_ARCHIVE in SMBFS.
This is similar to changes that Apple has
made in their version of SMBFS (as of
smb-583.8, posted on opensource.apple.com),
but not quite the same.
We map SMB_FA_READONLY to UF_READONLY,
because UF_READONLY is intended to match
the semantics of the DOS readonly flag.
The MacOS X code maps both UF_IMMUTABLE
and SF_IMMUTABLE to SMB_FA_READONLY, but
the immutable flags have stronger meaning
than the DOS readonly bit.
stat.h: Add definitions for UF_SYSTEM, UF_SPARSE,
UF_OFFLINE, UF_REPARSE, UF_ARCHIVE, UF_READONLY
and UF_HIDDEN.
The definition of UF_HIDDEN is the same as
the MacOS X definition.
Add commented-out definitions of
UF_COMPRESSED and UF_TRACKED. They are
defined in MacOS X (as of 10.8.2), but we
do not implement them (yet).
ufs_vnops.c: Add support for getting and setting
UF_ARCHIVE, UF_HIDDEN, UF_OFFLINE, UF_READONLY,
UF_REPARSE, UF_SPARSE, and UF_SYSTEM in UFS.
Alphabetize the flags that are supported.
These new flags are only stored, UFS does
not take any action if the flag is set.
Sponsored by: Spectra Logic
Reviewed by: bde (earlier version)
Replace the RESET blocks with regular functions and a reset() function that
calls them all.
This code generation tool is unusual and does not appear to provide much
benefit. I do not think isolating the knowledge about which modules need to
be reset is worth an almost 500-line build tool and wider scope for
variables used by the reset functions.
Also, relying on reset functions is often wrong: the cleanup should be done
in exception handlers so that no stale state remains after 'command eval'
and the like.
These cleanup operations are not needed because they are already performed
after an optimized command substitution (whether there was an error or not).
Although using -i with -c does not seem very useful, it seems inappropriate
to read commands from the terminal in this case.
Side effect: if the -s -c extension is used and the -s option is turned off
using 'set +s' during the interactive part, the shell now exits after an
error or interrupt. Note that POSIX only specifies -s as option to sh, not
to set.
See also Austin Group issue #718.
This is required by POSIX, at least for pids that are not known child
processes.
Other problems with job specifications still cause wait to abort with
exit status 2.
PR: 176916
This is only part of the PR; the behaviour for unknown/invalid pids/jobs
remains unchanged (aborts the builtin with status 2).
PR: 176916
Submitted by: Vadim Goncharov
This reverts commit r247274.
As maintainer of sh, I disapprove of this feature addition.
It is too specific and can be done without easily using find(1) or stat(1).
I will add some hints to the test(1) man page shortly.
In general, FreeBSD sh is not the place to invent new shell language
features. This is how it has been maintained and adding features randomly
does not work with that.
The new syntax (e.g. [ FILE1 -ntca FILE2 ]) looks cryptic to me.
* Work around kernel bugs that cause a spurious [EINTR] return if a
debugger (such as truss(1)) is attached.
* Write an error message if an error other than [EINTR] occurs.
PR: bin/178664
The linked list of stack marks may cause problems if the allocation stack is
used between an exception and a higher-level popstackmark(), as it may then
touch a stack mark that is local to a function which has returned.
Also, the adjustment compares to a pointer passed to realloc(), which is
undefined behaviour.
Instead of adjusting stack marks when reallocating stack blocks, ensure that
such an adjustment is never necessary by fixing a small piece of memory in
place at a stack mark. This also simplifies the code.
To avoid the problems reported in bin/175922, it remains necessary to call
setstackmark() after popstackmark() if the stack mark remains in use.
