The zfsd tests rely on this having the same layout as the real
zpool_handle, which changed in the last OpenZFS import.
Fixes: 62e7d3c89e ("ddt: add support for prefetching tables into the ARC")
Reported by: Jenkins
Notable upstream pull request merges:
#158175536c0dee Sync AUX label during pool import
#15889c7ada64bb ddt: dedup table quota enforcement
#1589062e7d3c89 ddt: add support for prefetching tables into the ARC
#15894e26b3771e spa_preferred_class: pass the entire zio
#15894d54d0fff3 dnode: allow storage class to be overridden by object type
#1619755427add3 Several improvements to ARC shrinking
#16217 -multiple JSON output for various zfs and zpool subcommands
#1624824e6585e7 libzfs.h: Set ZFS_MAXPROPLEN and ZPOOL_MAXPROPLEN
to ZAP_MAXVALUELEN
#162649dfc5c4a0 Fix long_free_dirty accounting for small files
#16268ed0db1cc8 Make txg_wait_synced conditional in zfsvfs_teardown,
for FreeBSD
#16288d60debbf5 Fix sa_add_projid to lookup and update SA_ZPL_DXATTR
#16308ec580bc52 zfs: add bounds checking to zil_parse
#16310c21dc56ea Fix zdb_dump_block for little endian
#163157ddc1f737 zil: add stats for commit failure/fallback
#16326b0bf14cdb abd: lift ABD zero scan from zio_compress_data()
to abd_cmp_zero()
#16337c8184d714 Block cloning conditionally destroy ARC buffer
#16338dbe07928b Add support for multiple lines to the sharenfs property
for FreeBSD
#163741a3e32e6a Cleanup DB_DNODE() macros usage
#16374ed87d456e Skip dnode handles use when not needed
#16346fb6d8cf22 Add some missing vdev properties
#16364670147be5 zvol: ensure device minors are properly cleaned up
#16382dea8fabf7 FreeBSD: Fix RLIMIT_FSIZE handling for block cloning
#16387aef452f10 Improve zfs_blkptr_verify()
#16395cbcb52243 Fix the names of some FreeBSD sysctls in
include/tunables.cfg
#164015b9f3b766 Soften pruning threshold on not evictable metadata
#16404cdd53fea1 FreeBSD: Add missing memory reclamation accounting
#164041fdcb653b Once more refactor arc_summary output
#164191f5bf91a8 Fix memory corruption during parallel zpool import
with -o cachefile
#16426cf6e8b218 zstream: remove duplicate highbit64 definition
Obtained from: OpenZFS
OpenZFS commit: 9c56b8ec78
Notable upstream pull request merges:
#16209 --multi-- icp: rip out everything we don't use
#1623020c8bdd85 FreeBSD: Update use of UMA-related symbols in
arc_available_memory
#16242121a2d335 FreeBSD: unregister mountroot eventhandler on unload
#162585de3ac223 vdev_open: clear async fault flag after reopen
#16270436731276 zvol: Fix suspend lock leaks
#16273c87cb22ba head_errlog: fix use-after-free
#16284f72e081fb FreeBSD: Use a statement expression to implement
SET_ERROR()
#16300a10faf5ce FreeBSD: Use the new freeuio() helper to free dynamically
allocated UIOs
#16302a7fc4c85e zstd: don't call zstd_mempool_reap if there are no buffers
#16334dc91e7452 zdb: dump ZAP_FLAG_UINT64_KEY ZAPs properly
Obtained from: OpenZFS
OpenZFS commit: 1147a27978
ZFS' libspl needs to be made aware that we have strlcat(3) and
strlcpy(3) to avoid some more complicated declaration duplication, so
go ahead and define these HAVE_ macros now.
libprocstat has to define `_KERNEL` and include kernel headers in order
to get what it wants, but this results in sys/cdefs.h being included too
late and we pick up the build breaking version of the __RENAME
definition. Just explicitly include sys/cdefs.h earlier rather than
disabling _FORTIFY_SOURCE. The zfs/ subdir only builds an object that
holds some structures and sizes, so just disable _FORTIFY_SOURCE there
entirely rather than trying to move #define _KERNEL into the file..
While we're here, make sure that we disable _FORTIFY_SOURCE in the
bootloader because we don't have the symbol renaming support today to do
it as cleanly as we'd like. ssp/ssp.h needs to be pulled into the libsa
environment so that other bits can understand that ssp is disabled in
the consistent __SSP_FORTIFY_LEVEL way that we try to do.
Reviewed by: allanjude (previous version), markj
Sponsored by: Klara, Inc.
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D45676
The idea here is to avoid a memory access and conditional branch per
probe site. Instead, the probe is represented by an "unreachable"
unconditional function call. asm goto is used to store the address of
the probe site (represented by a no-op sled) and the address of the
function call into a tracepoint record. Each SDT probe carries a list
of tracepoints.
When the probe is enabled, the no-op sled corresponding to each
tracepoint is overwritten with a jmp to the corresponding label. The
implementation uses smp_rendezvous() to park all other CPUs while the
instruction is being overwritten, as this can't be done atomically in
general. The compiler moves argument marshalling code and the
sdt_probe() function call out-of-line, i.e., to the end of the function.
Per gallatin@ in D43504, this approach has less overhead when probes are
disabled. To make the implementation a bit simpler, I removed support
for probes with 7 arguments; nothing makes use of this except a
regression test case. It could be re-added later if need be.
The approach taken in this patch enables some more improvements:
1. We can now automatically fill out the "function" field of SDT probe
names. The SDT macros let the programmer specify the function and
module names, but this is really a bug and shouldn't have been
allowed. The intent was to be able to have the same probe in
multiple functions and to let the user restrict which probes actually
get enabled by specifying a function name or glob.
2. We can avoid branching on SDT_PROBES_ENABLED() by adding the ability
to include blocks of code in the out-of-line path. For example:
if (SDT_PROBES_ENABLED()) {
int reason = CLD_EXITED;
if (WCOREDUMP(signo))
reason = CLD_DUMPED;
else if (WIFSIGNALED(signo))
reason = CLD_KILLED;
SDT_PROBE1(proc, , , exit, reason);
}
could be written
SDT_PROBE1_EXT(proc, , , exit, reason,
int reason;
reason = CLD_EXITED;
if (WCOREDUMP(signo))
reason = CLD_DUMPED;
else if (WIFSIGNALED(signo))
reason = CLD_KILLED;
);
In the future I would like to use this mechanism more generally, e.g.,
to remove branches and marshalling code used by hwpmc, and generally to
make it easier to add new tracepoint consumers without having to add
more conditional branches to hot code paths.
Reviewed by: Domagoj Stolfa, avg
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D44483
The old way is racy and can cause two instances, running in parallel, to
attempt to load dtrace_test, and only one will succeed. This caused
errors when running dtrace tests in parallel.
MFC after: 1 week
When compiling dt_lex.l, flex produces warnings of the form:
dt_lex.l:413: warning, trailing context made variable due to preceding '|' action
dt_lex.l:412: warning, dangerous trailing context
dt_lex.l:412: warning, dangerous trailing context
Here, trailing context refers to the use of "$", which expands to "/\n".
The meaning behind these warnings is described in the first two
paragraphs of the flex manual's DEFICIENCIES/BUGS section:
Some trailing context patterns cannot be properly matched and generate
warning messages ("dangerous trailing context"). These are patterns
where the ending of the first part of the rule matches the beginning of
the second part, such as "zx*/xy*", where the 'x*' matches the 'x' at
the beginning of the trailing context. (Note that the POSIX draft
states that the text matched by such patterns is undefined.)
For some trailing context rules, parts which are actually fixed-length
are not recognized as such, leading to the above mentioned performance
loss. In particular, parts using '|' or {n} (such as "foo{3}") are
always considered variable-length.
Here, the warnings appear to be bogus in this case. The lexer has no
problem matching either of the referenced patterns, e.g.,
printf("foobar
or
# 1 "asdfasdf
Introduce a small amount of code duplication to silence the warning.
MFC after: 2 weeks
Notable upstream pull request merges:
#1594041ae864b6 Replace P2ALIGN with P2ALIGN_TYPED and delete P2ALIGN
#161285137c132a zpool import output is not formated properly
#16138efbef9e6c FreeBSD: Add zfs_link_create() error handling
#1614604bae5ec9 Disable high priority ZIO threads on FreeBSD and Linux
#16151cc3869153 zfs_ioc_send: use a dedicated taskq thread for send
#16151adda768e3 spa: remove spa_taskq_dispatch_sync()
#16151515c4dd21 spa: flatten spa_taskq_dispatch_ent()
#161510a543db37 spa_taskq_dispatch_ent: simplify arguments
#16153975a13259 Add support for parallel pool exports
#1615389acef992 Simplified the scope of the namespace lock
#16159136c05321 ZAP: Fix leaf references on zap_expand_leaf() errors
#16162af5dbed31 Fix scn_queue races on very old pools
#161653400127a7 Fix ZIL clone records for legacy holes
#16167414acbd37 Unbreak FreeBSD cross-build on MacOS broken in 051460b8b#16172eced2e2f1 libzfs: Fix mounting datasets under thread limit pressure
#16178b64afa41d Better control the thread pool size when mounting datasets
#16181fa99d9cd9 zfs_dbgmsg_print: make FreeBSD and Linux consistent
#16191e675852bc dbuf: separate refcount calls for dbuf and dbuf_user
#16198a043b60f1 Correct level handling in zstream recompress
#1620434906f8bb zap: reuse zap_leaf_t on dbuf reuse after shrink
#16206d0aa9dbcc Use memset to zero stack allocations containing unions
#162078865dfbca Fix assertion in Persistent L2ARC
#1620808648cf0d Allow block cloning to be interrupted by a signal
#16210e2357561b FreeBSD: Add const qualifier to members of struct
opensolaris_utsname
#16214800d59d57 Some improvements to metaslabs eviction
#1621602c5aa9b0 Destroy ARC buffer in case of fill error
#1622501c8efdd5 Simplify issig()
Obtained from: OpenZFS
OpenZFS commit: e2357561b9
ZED uses vdev props for setting disk fault/degrade thresholds, this
patch enables zfsd to use the same vdev props for these same tasks.
OpenZFS on Linux is using vdev props for ZED disk fault/degrade
thresholds. Originally the thresholds supported were for io and checksum
events and recently this was updated to process slow io events as
well, see
cbe882298e
This patch enables us to use the same vdev props in zfsd as ZED uses.
After this patch is merged both OSs will use the same vdev props to set
retirement thresholds.
It's probably important to note that the threshold defaults are
different between OS. I've kept the existing defaults inside zfsd and
DID NOT match them to what ZED does.
Differential Revision: https://reviews.freebsd.org/D44043
MFC after: 2 weeks
Relnotes: yes
Reviewed by: asomers, allanjude
Sponsored by: Axcient
Submitted by: Alek Pinchuk <apinchuk@axcient.com>
This is similar to other translators and will be used in static
probes where the interface is not known.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D43728
If there are no CTF sections then ctfmerge just has nothing to do; it
should not be an error.
Note that ctfmerge has an option to require CTF:
-t Make sure that all object files have a CTF section.
Before this change, this option explicitly exited without error if none
of the object files have CTF sections, with the comment:
If we're verifying that C files have CTF, it's safe to
assume that in this case, we're building only from assembly
inputs.
PR: 276930
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43878
In particular, avoid loading the user's .profile file, since that can
have undesirable side effects. Most tests were already careful to do
this.
MFC after: 1 week
This option can be used to specify a format to use in DTrace output.
The following formats are supported:
- json
- xml
- html
- none (default DTrace output)
This is implemented using libxo and integrated into libdtrace. Client
code only works with the following API:
- dtrace_oformat_setup(dtrace_hdl_t *) -- to be called when output is starting.
- dtrace_oformat_teardown(dtrace_hdl_t *) -- to be called when output is finished
- dtrace_oformat(dtrace_hdl_t *) -- check if oformat is enabled.
- dtrace_set_outfp(FILE *) -- sets the output file for oformat.
- Ensure that oformat is correctly checked in the drop handler and record
processing callbacks.
This commit also adds tests which check if the generated output is
valid (JSON, XML) and extends the dtrace(1) describing the structured output.
Reviewed by: markj
Discussed with: phil
MFC after: 2 months
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D41745
The header gives an offset in 32-bit words, and the translator is
supposed to convert that to a byte count. But, the conversion was
incorrect.
Reviewed by: tuexen, rscheff
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43264
decoding capability of TH_AE to dtrace, including
the example provided with tcpdebug.
MFC after: 1 week
Reviewed By: markj, mav
Sponsored by: NetApp, Inc.
Differential Revision: https://reviews.freebsd.org/D43243
OpenZFS upstream refactored the asm versions of sha2 to be usable on all
32-bit arm flavors, so it is not necessary to limit this to armv6 and
armv7.
Suggested by: jhb
We already implemented execvpe internally with an _ prefix in libc so
go ahead and expose it for compatibility with Linux.
This reverts c605eea952.
Bump __FreeBSD_version for the addition and add definitions to supress
compat shims in libzfs (zfs changes were merged from upstream).
PR: 275370 (request and exp-run (thanks antoine!))
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D42846
The following upstream commit:
727497ccdf module/icp/asm-arm/sha2: enable non-SIMD asm kernels on armv5/6
does indeed enable sha2 asm for earlier arm CPUs, but since libicp's
Makefile was not updated, this leads to:
ld: error: undefined reference due to --no-allow-shlib-undefined: zfs_sha256_block_armv7
Fix it by compiling sha256-armv7.S and sha512-armv7.S for armv6 too.
Fixes: 3494f7c019
If ZFS reports that a disk had at least 8 I/O operations over 60s that
were each delayed by at least 30s (implying a queue depth > 4 or I/O
aggregation, obviously), fault that disk. Disks that respond this
slowly can degrade the entire system's performance.
MFC after: 2 weeks
Sponsored by: Axcient
Reviewed by: delphij
Differential Revision: https://reviews.freebsd.org/D42825
Previously zfsd would crash in the presence of a pool with a
top-level-vdev that had previously been removed. The crash happened
because the configuration nvlist of such a TLV contains an empty
ZPOOL_CONFIG_CHILDREN array, which led to a pop_front from an empty
list, which has undefined behavior.
The crash only happened in stable/14 and later, probably do to
differences in libcxx, but the change should be MFCed anyway.
PR: 273663
Reported by: Marek Zarychta <zarychtam@plan-b.pwste.edu.pl>
MFC after: 1 week
Sponsored by: Axcient
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D41818
Notable upstream pull request merges:
#15024 Add missed DMU_PROJECTUSED_OBJECT prefetch
#15029 Do not request data L1 buffers on scan prefetch
#15036 FreeBSD: catch up to __FreeBSD_version 1400093
#15039 Fix raw receive with different indirect block size
#15047 FreeBSD: Fix build on stable/13 after 1302506
#15049 Fix the ZFS checksum error histograms with larger record sizes
#15052 Reduce bloat in ereport.fs.zfs.checksum events
#15056 Avoid extra snprintf() in dsl_deadlist_merge()
#15061 Ignore pool ashift property during vdev attachment
#15063 Don't panic if setting vdev properties is unsupported for this vdev type
#15067 spa_min_alloc should be GCD, not min
#15071 Add explicit prefetches to bpobj_iterate()
#15072 Adjust prefetch parameters
#15076 Refactor dmu_prefetch()
#15079 set autotrim default to 'off' everywhere
#15080 ZIL: Fix config lock deadlock
#15088 metaslab: tuneable to better control force ganging
#15096 Avoid waiting in dmu_sync_late_arrival()
#15097 BRT should return EOPNOTSUPP
#15103 Remove zl_issuer_lock from zil_suspend()
#15107 Remove fastwrite mechanism
#15113 libzfs: sendrecv: send_progress_thread: handle SIGINFO/SIGUSR1
#15122 ZIL: Second attempt to reduce scope of zl_issuer_lock
#15129 zpool_vdev_remove() should handle EALREADY error return
#15132 ZIL: Replay blocks without next block pointer
#15148 zfs_clone_range should return descriptive error codes
#15153 ZIL: Avoid dbuf_read() before dmu_sync()
#15172 copy_file_range: fix fallback when source create on same txg
#15180 Update outdated assertion from zio_write_compress
Obtained from: OpenZFS
OpenZFS commit: 804414aad2
