Notable upstream pull request merges:
#16244acb6e71ed Added output to `zpool online` and `offline`
#1668494a03dd1e Pack dmu_buf_impl_t by 16 bytes
#166906187b1943 On the first vdev open ignore impossible ashift hints
#16692673efbbf5 zdb: add extra -T flag to show histograms of BRT refcounts
#166932bf152021 Fix gcc uninitialized warning in FreeBSD zio_crypt.c
#16694b16e09619 Reduce dirty records memory usage
#167015945676bc ZFS send should use spill block prefetched from
send_reader_thread
#167341c9a4c8cb Fix user properties output for zpool list
Obtained from: OpenZFS
OpenZFS commit: 1c9a4c8cb4
It was disabled by default in fe52b7f60e. We planned to (but did not)
remove the option before FreeBSD 14. Remove it now, for FreeBSD 15.
Relnotes: Yes
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31558
Notable upstream pull request merges:
#9416 -multiple zio_compress: introduce max size threshold
#10018a10e552b9 Adding Direct IO Support
#15147e419a63bf xattr dataset prop: change defaults to sa
#154547e957fde7 send/recv: open up additional stream feature flags
#158100d77e738e Defer resilver only when progress is above a threshold
#159213cf2bfa57 Allocate zap_attribute_t from kmem instead of stack
#16483 -multiple dmu_objset: replace dnode_hash impl with cityhash4
#164858be2f4c3d zio_resume: log when unsuspending the pool
#1649188433e640 sys/types32.h: Remove struct timeval32 from libspl header
#16496f245541e2 zfs_file: implement zfs_file_deallocate for FreeBSD 14
#16511308f7c2f1 Fix an uninitialized data access
#1652929c9e6c32 Fix handling of DNS names with '-' in them for sharenfs
#16531ddf5f34f0 Avoid fault diagnosis if multiple vdevs have errors
#165396f50f8e16 zfs_log: add flex array fields to log record structs
#16546d40d40913 Evicting too many bytes from MFU metadata
#165513014dcb76 Reduce and handle EAGAIN errors on AIO label reads
#1655480645d658 FreeBSD: restore zfs_znode_update_vfs()
#16565832f66b21 FreeBSD: Sync taskq_cancel_id() returns with Linux
#1656748d1be254 Properly release key in spa_keystore_dsl_key_hold_dd()
#16569141368a4b Restrict raidz faulted vdev count
#16583c84a37ae9 lua: add flex array field to TString type
#1658486737c592 Avoid computing strlen() inside loops
#16587d34d4f97a snapdir: add 'disabled' value to make .zfs inaccessible
#16593224393a32 feature: large_microzap
#16597412105977 Temporarily disable Direct IO by default
#166054ebe674d9 ARC: Cache arc_c value during arc_evict()
Backported pull request merges:
#16613ab777f436 Return boolean_t in inline functions of
lib/libspl/include/sys/uio.h
#16616efeb60b86 FreeBSD: ignore some includes when not building kernel
#16635 ---TBD--- zdb: fix printf format in dump_zap()
Obtained from: OpenZFS
OpenZFS commit: b109925820
OpenZFS tag: 2.3.0-rc1
These were reported by `mandoc -T lint ...` as errors; this commit only
handles unnecessary .El commands.
The rendered output (in ascii and html) is not affected by this commit.
Signed-off-by: Graham Percival <gperciva@tarsnap.com>
Reviewed by: mhorne
MFC after: 3 days
Sponsored by: Tarsnap Backup Inc.
Pull Request: https://github.com/freebsd/freebsd-src/pull/1447
Without this patch, vdevs faulted via AUX state would not be replaced
once the appropriate drive is replaced. ZFS does not internally use
that state, but a drive can be manually forced into such a state with a
command like zinject.
Submitted by: Goran Mekić <meka@tilda.center>
Sponsored by: ConnectWise
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D46866
Notable upstream pull request merges:
#15892 -multiple Fast Dedup: Introduce the FDT on-disk format and feature flag
#15893 -multiple Fast Dedup: “flat” DDT entry format
#15895 -multiple Fast Dedup: FDT-log feature
#162396be8bf555 zpool: Provide GUID to zpool-reguid(8) with -g
#16277 -multiple Fast Dedup: prune unique entries
#163165807de90a Fix null ptr deref when renaming a zvol with snaps and snapdev=visible
#1634377a797a38 Enable L2 cache of all (MRU+MFU) metadata but MFU data only
#1644683f359245 FreeBSD: fix build without kernel option MAC
#16449963e6c9f3 Fix incorrect error report on vdev attach/replace
#16505b10992582 spa_prop_get: require caller to supply output nvlist
Obtained from: OpenZFS
OpenZFS commit: b109925820
The zfsd tests rely on this having the same layout as the real
zpool_handle, which changed in the last OpenZFS import.
Fixes: 62e7d3c89e ("ddt: add support for prefetching tables into the ARC")
Reported by: Jenkins
Notable upstream pull request merges:
#158175536c0dee Sync AUX label during pool import
#15889c7ada64bb ddt: dedup table quota enforcement
#1589062e7d3c89 ddt: add support for prefetching tables into the ARC
#15894e26b3771e spa_preferred_class: pass the entire zio
#15894d54d0fff3 dnode: allow storage class to be overridden by object type
#1619755427add3 Several improvements to ARC shrinking
#16217 -multiple JSON output for various zfs and zpool subcommands
#1624824e6585e7 libzfs.h: Set ZFS_MAXPROPLEN and ZPOOL_MAXPROPLEN
to ZAP_MAXVALUELEN
#162649dfc5c4a0 Fix long_free_dirty accounting for small files
#16268ed0db1cc8 Make txg_wait_synced conditional in zfsvfs_teardown,
for FreeBSD
#16288d60debbf5 Fix sa_add_projid to lookup and update SA_ZPL_DXATTR
#16308ec580bc52 zfs: add bounds checking to zil_parse
#16310c21dc56ea Fix zdb_dump_block for little endian
#163157ddc1f737 zil: add stats for commit failure/fallback
#16326b0bf14cdb abd: lift ABD zero scan from zio_compress_data()
to abd_cmp_zero()
#16337c8184d714 Block cloning conditionally destroy ARC buffer
#16338dbe07928b Add support for multiple lines to the sharenfs property
for FreeBSD
#163741a3e32e6a Cleanup DB_DNODE() macros usage
#16374ed87d456e Skip dnode handles use when not needed
#16346fb6d8cf22 Add some missing vdev properties
#16364670147be5 zvol: ensure device minors are properly cleaned up
#16382dea8fabf7 FreeBSD: Fix RLIMIT_FSIZE handling for block cloning
#16387aef452f10 Improve zfs_blkptr_verify()
#16395cbcb52243 Fix the names of some FreeBSD sysctls in
include/tunables.cfg
#164015b9f3b766 Soften pruning threshold on not evictable metadata
#16404cdd53fea1 FreeBSD: Add missing memory reclamation accounting
#164041fdcb653b Once more refactor arc_summary output
#164191f5bf91a8 Fix memory corruption during parallel zpool import
with -o cachefile
#16426cf6e8b218 zstream: remove duplicate highbit64 definition
Obtained from: OpenZFS
OpenZFS commit: 9c56b8ec78
Notable upstream pull request merges:
#16209 --multi-- icp: rip out everything we don't use
#1623020c8bdd85 FreeBSD: Update use of UMA-related symbols in
arc_available_memory
#16242121a2d335 FreeBSD: unregister mountroot eventhandler on unload
#162585de3ac223 vdev_open: clear async fault flag after reopen
#16270436731276 zvol: Fix suspend lock leaks
#16273c87cb22ba head_errlog: fix use-after-free
#16284f72e081fb FreeBSD: Use a statement expression to implement
SET_ERROR()
#16300a10faf5ce FreeBSD: Use the new freeuio() helper to free dynamically
allocated UIOs
#16302a7fc4c85e zstd: don't call zstd_mempool_reap if there are no buffers
#16334dc91e7452 zdb: dump ZAP_FLAG_UINT64_KEY ZAPs properly
Obtained from: OpenZFS
OpenZFS commit: 1147a27978
ZFS' libspl needs to be made aware that we have strlcat(3) and
strlcpy(3) to avoid some more complicated declaration duplication, so
go ahead and define these HAVE_ macros now.
libprocstat has to define `_KERNEL` and include kernel headers in order
to get what it wants, but this results in sys/cdefs.h being included too
late and we pick up the build breaking version of the __RENAME
definition. Just explicitly include sys/cdefs.h earlier rather than
disabling _FORTIFY_SOURCE. The zfs/ subdir only builds an object that
holds some structures and sizes, so just disable _FORTIFY_SOURCE there
entirely rather than trying to move #define _KERNEL into the file..
While we're here, make sure that we disable _FORTIFY_SOURCE in the
bootloader because we don't have the symbol renaming support today to do
it as cleanly as we'd like. ssp/ssp.h needs to be pulled into the libsa
environment so that other bits can understand that ssp is disabled in
the consistent __SSP_FORTIFY_LEVEL way that we try to do.
Reviewed by: allanjude (previous version), markj
Sponsored by: Klara, Inc.
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D45676
The idea here is to avoid a memory access and conditional branch per
probe site. Instead, the probe is represented by an "unreachable"
unconditional function call. asm goto is used to store the address of
the probe site (represented by a no-op sled) and the address of the
function call into a tracepoint record. Each SDT probe carries a list
of tracepoints.
When the probe is enabled, the no-op sled corresponding to each
tracepoint is overwritten with a jmp to the corresponding label. The
implementation uses smp_rendezvous() to park all other CPUs while the
instruction is being overwritten, as this can't be done atomically in
general. The compiler moves argument marshalling code and the
sdt_probe() function call out-of-line, i.e., to the end of the function.
Per gallatin@ in D43504, this approach has less overhead when probes are
disabled. To make the implementation a bit simpler, I removed support
for probes with 7 arguments; nothing makes use of this except a
regression test case. It could be re-added later if need be.
The approach taken in this patch enables some more improvements:
1. We can now automatically fill out the "function" field of SDT probe
names. The SDT macros let the programmer specify the function and
module names, but this is really a bug and shouldn't have been
allowed. The intent was to be able to have the same probe in
multiple functions and to let the user restrict which probes actually
get enabled by specifying a function name or glob.
2. We can avoid branching on SDT_PROBES_ENABLED() by adding the ability
to include blocks of code in the out-of-line path. For example:
if (SDT_PROBES_ENABLED()) {
int reason = CLD_EXITED;
if (WCOREDUMP(signo))
reason = CLD_DUMPED;
else if (WIFSIGNALED(signo))
reason = CLD_KILLED;
SDT_PROBE1(proc, , , exit, reason);
}
could be written
SDT_PROBE1_EXT(proc, , , exit, reason,
int reason;
reason = CLD_EXITED;
if (WCOREDUMP(signo))
reason = CLD_DUMPED;
else if (WIFSIGNALED(signo))
reason = CLD_KILLED;
);
In the future I would like to use this mechanism more generally, e.g.,
to remove branches and marshalling code used by hwpmc, and generally to
make it easier to add new tracepoint consumers without having to add
more conditional branches to hot code paths.
Reviewed by: Domagoj Stolfa, avg
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D44483
The old way is racy and can cause two instances, running in parallel, to
attempt to load dtrace_test, and only one will succeed. This caused
errors when running dtrace tests in parallel.
MFC after: 1 week
When compiling dt_lex.l, flex produces warnings of the form:
dt_lex.l:413: warning, trailing context made variable due to preceding '|' action
dt_lex.l:412: warning, dangerous trailing context
dt_lex.l:412: warning, dangerous trailing context
Here, trailing context refers to the use of "$", which expands to "/\n".
The meaning behind these warnings is described in the first two
paragraphs of the flex manual's DEFICIENCIES/BUGS section:
Some trailing context patterns cannot be properly matched and generate
warning messages ("dangerous trailing context"). These are patterns
where the ending of the first part of the rule matches the beginning of
the second part, such as "zx*/xy*", where the 'x*' matches the 'x' at
the beginning of the trailing context. (Note that the POSIX draft
states that the text matched by such patterns is undefined.)
For some trailing context rules, parts which are actually fixed-length
are not recognized as such, leading to the above mentioned performance
loss. In particular, parts using '|' or {n} (such as "foo{3}") are
always considered variable-length.
Here, the warnings appear to be bogus in this case. The lexer has no
problem matching either of the referenced patterns, e.g.,
printf("foobar
or
# 1 "asdfasdf
Introduce a small amount of code duplication to silence the warning.
MFC after: 2 weeks
Notable upstream pull request merges:
#1594041ae864b6 Replace P2ALIGN with P2ALIGN_TYPED and delete P2ALIGN
#161285137c132a zpool import output is not formated properly
#16138efbef9e6c FreeBSD: Add zfs_link_create() error handling
#1614604bae5ec9 Disable high priority ZIO threads on FreeBSD and Linux
#16151cc3869153 zfs_ioc_send: use a dedicated taskq thread for send
#16151adda768e3 spa: remove spa_taskq_dispatch_sync()
#16151515c4dd21 spa: flatten spa_taskq_dispatch_ent()
#161510a543db37 spa_taskq_dispatch_ent: simplify arguments
#16153975a13259 Add support for parallel pool exports
#1615389acef992 Simplified the scope of the namespace lock
#16159136c05321 ZAP: Fix leaf references on zap_expand_leaf() errors
#16162af5dbed31 Fix scn_queue races on very old pools
#161653400127a7 Fix ZIL clone records for legacy holes
#16167414acbd37 Unbreak FreeBSD cross-build on MacOS broken in 051460b8b#16172eced2e2f1 libzfs: Fix mounting datasets under thread limit pressure
#16178b64afa41d Better control the thread pool size when mounting datasets
#16181fa99d9cd9 zfs_dbgmsg_print: make FreeBSD and Linux consistent
#16191e675852bc dbuf: separate refcount calls for dbuf and dbuf_user
#16198a043b60f1 Correct level handling in zstream recompress
#1620434906f8bb zap: reuse zap_leaf_t on dbuf reuse after shrink
#16206d0aa9dbcc Use memset to zero stack allocations containing unions
#162078865dfbca Fix assertion in Persistent L2ARC
#1620808648cf0d Allow block cloning to be interrupted by a signal
#16210e2357561b FreeBSD: Add const qualifier to members of struct
opensolaris_utsname
#16214800d59d57 Some improvements to metaslabs eviction
#1621602c5aa9b0 Destroy ARC buffer in case of fill error
#1622501c8efdd5 Simplify issig()
Obtained from: OpenZFS
OpenZFS commit: e2357561b9
ZED uses vdev props for setting disk fault/degrade thresholds, this
patch enables zfsd to use the same vdev props for these same tasks.
OpenZFS on Linux is using vdev props for ZED disk fault/degrade
thresholds. Originally the thresholds supported were for io and checksum
events and recently this was updated to process slow io events as
well, see
cbe882298e
This patch enables us to use the same vdev props in zfsd as ZED uses.
After this patch is merged both OSs will use the same vdev props to set
retirement thresholds.
It's probably important to note that the threshold defaults are
different between OS. I've kept the existing defaults inside zfsd and
DID NOT match them to what ZED does.
Differential Revision: https://reviews.freebsd.org/D44043
MFC after: 2 weeks
Relnotes: yes
Reviewed by: asomers, allanjude
Sponsored by: Axcient
Submitted by: Alek Pinchuk <apinchuk@axcient.com>
This is similar to other translators and will be used in static
probes where the interface is not known.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D43728
If there are no CTF sections then ctfmerge just has nothing to do; it
should not be an error.
Note that ctfmerge has an option to require CTF:
-t Make sure that all object files have a CTF section.
Before this change, this option explicitly exited without error if none
of the object files have CTF sections, with the comment:
If we're verifying that C files have CTF, it's safe to
assume that in this case, we're building only from assembly
inputs.
PR: 276930
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43878
In particular, avoid loading the user's .profile file, since that can
have undesirable side effects. Most tests were already careful to do
this.
MFC after: 1 week
This option can be used to specify a format to use in DTrace output.
The following formats are supported:
- json
- xml
- html
- none (default DTrace output)
This is implemented using libxo and integrated into libdtrace. Client
code only works with the following API:
- dtrace_oformat_setup(dtrace_hdl_t *) -- to be called when output is starting.
- dtrace_oformat_teardown(dtrace_hdl_t *) -- to be called when output is finished
- dtrace_oformat(dtrace_hdl_t *) -- check if oformat is enabled.
- dtrace_set_outfp(FILE *) -- sets the output file for oformat.
- Ensure that oformat is correctly checked in the drop handler and record
processing callbacks.
This commit also adds tests which check if the generated output is
valid (JSON, XML) and extends the dtrace(1) describing the structured output.
Reviewed by: markj
Discussed with: phil
MFC after: 2 months
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D41745
The header gives an offset in 32-bit words, and the translator is
supposed to convert that to a byte count. But, the conversion was
incorrect.
Reviewed by: tuexen, rscheff
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43264
decoding capability of TH_AE to dtrace, including
the example provided with tcpdebug.
MFC after: 1 week
Reviewed By: markj, mav
Sponsored by: NetApp, Inc.
Differential Revision: https://reviews.freebsd.org/D43243
OpenZFS upstream refactored the asm versions of sha2 to be usable on all
32-bit arm flavors, so it is not necessary to limit this to armv6 and
armv7.
Suggested by: jhb
We already implemented execvpe internally with an _ prefix in libc so
go ahead and expose it for compatibility with Linux.
This reverts c605eea952.
Bump __FreeBSD_version for the addition and add definitions to supress
compat shims in libzfs (zfs changes were merged from upstream).
PR: 275370 (request and exp-run (thanks antoine!))
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D42846
The following upstream commit:
727497ccdf module/icp/asm-arm/sha2: enable non-SIMD asm kernels on armv5/6
does indeed enable sha2 asm for earlier arm CPUs, but since libicp's
Makefile was not updated, this leads to:
ld: error: undefined reference due to --no-allow-shlib-undefined: zfs_sha256_block_armv7
Fix it by compiling sha256-armv7.S and sha512-armv7.S for armv6 too.
Fixes: 3494f7c019
If ZFS reports that a disk had at least 8 I/O operations over 60s that
were each delayed by at least 30s (implying a queue depth > 4 or I/O
aggregation, obviously), fault that disk. Disks that respond this
slowly can degrade the entire system's performance.
MFC after: 2 weeks
Sponsored by: Axcient
Reviewed by: delphij
Differential Revision: https://reviews.freebsd.org/D42825
