This is a backport of upstream commit d1e9f03b89:
ck_pr/aarch64: Specify output operands for ck_pr_md_store_*
As in commit 2f9acab, we want to specify output operand widths so that
MSAN compiler instrumentation correctly updates the shadow map. In
particular, LLVM's implementation depends on having type information for
output operands, even when that's not otherwise necessary. Without it,
KMSAN in FreeBSD generates false positives on aarch64.
Reviewed by: cognet
MFC after: 2 weeks
Sponsored by: Klara, Inc.
Sponsored by: Juniper Networks, Inc.
This is a variant of CK_*_FOREACH from FreeBSD queue.h which starts
iteration at the specified item. If the item pointer is NULL, iteration
starts from the beginning of the list.
Upstream commit 74366be35a6f4635f248a3c62d2d23245a4eb0f4.
MFC after: 2 weeks
Sponsored by: Klara, Inc.
This does not appear to change generated code with the default
toolchain. However, KMSAN makes use of output operand specifications to
instrument inline asm, and with incorrect specifications we get false
positives in code that uses the CK_(S)LIST macros.
This was submitted upstream:
https://github.com/concurrencykit/ck/pull/175
The commit applies the same change locally to make KMSAN usable until
something equivalent is merged upstream.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Now that armv4/v5 is gone, remove the bits that implemented atomic operations
by disabling interrupts.
Those were specific to FreeBSD and never reached upstream.
sparcv9 atomics compatible with the FreeBSD kernel by using instructions
which access the appropriate address space.
Atomic operations within the kernel must access the nucleus address space
instead of the default primary one. Without this change but the increased
use of CK in the kernel, machines started to panic after some minutes of
uptime due to an unresolvable fault in ck_pr_cas_64_value().
This fixes two problems, one where epoch calls could occur before all
the readers had exited the epoch section, and one where the epoch calls
could be unnecessarily delayed.
Approved by: re (glebius)
It should fix ck_pr_[load|store]_ptr on mips and riscv, make sure no
*fence instructions are used on i386, as older cpus don't support it, and
make sure we don't rely on gcc builtins that can lead to calls to
libatomic when linked with -O0.
MFC after: 1 week
Defaulting to CK_MD_RMO has the unfortunate side effect of generating
memory barriers that are useless on those arches, and the even more
unfortunate side effect of generating lfence/sfence/mfence on i386, even
if older CPUs don't support it.
This should fix the panic reported when using IPFW on a Pentium 3.
Note that mfence and sfence might still be used in a few case, but that
shouldn't happen in FreeBSD right now, and should be fixed upstream first.
MFC after: 1 week
