Notable upstream pull request merges:
#1594041ae864b6 Replace P2ALIGN with P2ALIGN_TYPED and delete P2ALIGN
#161285137c132a zpool import output is not formated properly
#16138efbef9e6c FreeBSD: Add zfs_link_create() error handling
#1614604bae5ec9 Disable high priority ZIO threads on FreeBSD and Linux
#16151cc3869153 zfs_ioc_send: use a dedicated taskq thread for send
#16151adda768e3 spa: remove spa_taskq_dispatch_sync()
#16151515c4dd21 spa: flatten spa_taskq_dispatch_ent()
#161510a543db37 spa_taskq_dispatch_ent: simplify arguments
#16153975a13259 Add support for parallel pool exports
#1615389acef992 Simplified the scope of the namespace lock
#16159136c05321 ZAP: Fix leaf references on zap_expand_leaf() errors
#16162af5dbed31 Fix scn_queue races on very old pools
#161653400127a7 Fix ZIL clone records for legacy holes
#16167414acbd37 Unbreak FreeBSD cross-build on MacOS broken in 051460b8b#16172eced2e2f1 libzfs: Fix mounting datasets under thread limit pressure
#16178b64afa41d Better control the thread pool size when mounting datasets
#16181fa99d9cd9 zfs_dbgmsg_print: make FreeBSD and Linux consistent
#16191e675852bc dbuf: separate refcount calls for dbuf and dbuf_user
#16198a043b60f1 Correct level handling in zstream recompress
#1620434906f8bb zap: reuse zap_leaf_t on dbuf reuse after shrink
#16206d0aa9dbcc Use memset to zero stack allocations containing unions
#162078865dfbca Fix assertion in Persistent L2ARC
#1620808648cf0d Allow block cloning to be interrupted by a signal
#16210e2357561b FreeBSD: Add const qualifier to members of struct
opensolaris_utsname
#16214800d59d57 Some improvements to metaslabs eviction
#1621602c5aa9b0 Destroy ARC buffer in case of fill error
#1622501c8efdd5 Simplify issig()
Obtained from: OpenZFS
OpenZFS commit: e2357561b9
ZED uses vdev props for setting disk fault/degrade thresholds, this
patch enables zfsd to use the same vdev props for these same tasks.
OpenZFS on Linux is using vdev props for ZED disk fault/degrade
thresholds. Originally the thresholds supported were for io and checksum
events and recently this was updated to process slow io events as
well, see
cbe882298e
This patch enables us to use the same vdev props in zfsd as ZED uses.
After this patch is merged both OSs will use the same vdev props to set
retirement thresholds.
It's probably important to note that the threshold defaults are
different between OS. I've kept the existing defaults inside zfsd and
DID NOT match them to what ZED does.
Differential Revision: https://reviews.freebsd.org/D44043
MFC after: 2 weeks
Relnotes: yes
Reviewed by: asomers, allanjude
Sponsored by: Axcient
Submitted by: Alek Pinchuk <apinchuk@axcient.com>
This is similar to other translators and will be used in static
probes where the interface is not known.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D43728
If there are no CTF sections then ctfmerge just has nothing to do; it
should not be an error.
Note that ctfmerge has an option to require CTF:
-t Make sure that all object files have a CTF section.
Before this change, this option explicitly exited without error if none
of the object files have CTF sections, with the comment:
If we're verifying that C files have CTF, it's safe to
assume that in this case, we're building only from assembly
inputs.
PR: 276930
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43878
In particular, avoid loading the user's .profile file, since that can
have undesirable side effects. Most tests were already careful to do
this.
MFC after: 1 week
This option can be used to specify a format to use in DTrace output.
The following formats are supported:
- json
- xml
- html
- none (default DTrace output)
This is implemented using libxo and integrated into libdtrace. Client
code only works with the following API:
- dtrace_oformat_setup(dtrace_hdl_t *) -- to be called when output is starting.
- dtrace_oformat_teardown(dtrace_hdl_t *) -- to be called when output is finished
- dtrace_oformat(dtrace_hdl_t *) -- check if oformat is enabled.
- dtrace_set_outfp(FILE *) -- sets the output file for oformat.
- Ensure that oformat is correctly checked in the drop handler and record
processing callbacks.
This commit also adds tests which check if the generated output is
valid (JSON, XML) and extends the dtrace(1) describing the structured output.
Reviewed by: markj
Discussed with: phil
MFC after: 2 months
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D41745
The header gives an offset in 32-bit words, and the translator is
supposed to convert that to a byte count. But, the conversion was
incorrect.
Reviewed by: tuexen, rscheff
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43264
decoding capability of TH_AE to dtrace, including
the example provided with tcpdebug.
MFC after: 1 week
Reviewed By: markj, mav
Sponsored by: NetApp, Inc.
Differential Revision: https://reviews.freebsd.org/D43243
OpenZFS upstream refactored the asm versions of sha2 to be usable on all
32-bit arm flavors, so it is not necessary to limit this to armv6 and
armv7.
Suggested by: jhb
We already implemented execvpe internally with an _ prefix in libc so
go ahead and expose it for compatibility with Linux.
This reverts c605eea952.
Bump __FreeBSD_version for the addition and add definitions to supress
compat shims in libzfs (zfs changes were merged from upstream).
PR: 275370 (request and exp-run (thanks antoine!))
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D42846
The following upstream commit:
727497ccdf module/icp/asm-arm/sha2: enable non-SIMD asm kernels on armv5/6
does indeed enable sha2 asm for earlier arm CPUs, but since libicp's
Makefile was not updated, this leads to:
ld: error: undefined reference due to --no-allow-shlib-undefined: zfs_sha256_block_armv7
Fix it by compiling sha256-armv7.S and sha512-armv7.S for armv6 too.
Fixes: 3494f7c019
If ZFS reports that a disk had at least 8 I/O operations over 60s that
were each delayed by at least 30s (implying a queue depth > 4 or I/O
aggregation, obviously), fault that disk. Disks that respond this
slowly can degrade the entire system's performance.
MFC after: 2 weeks
Sponsored by: Axcient
Reviewed by: delphij
Differential Revision: https://reviews.freebsd.org/D42825
Previously zfsd would crash in the presence of a pool with a
top-level-vdev that had previously been removed. The crash happened
because the configuration nvlist of such a TLV contains an empty
ZPOOL_CONFIG_CHILDREN array, which led to a pop_front from an empty
list, which has undefined behavior.
The crash only happened in stable/14 and later, probably do to
differences in libcxx, but the change should be MFCed anyway.
PR: 273663
Reported by: Marek Zarychta <zarychtam@plan-b.pwste.edu.pl>
MFC after: 1 week
Sponsored by: Axcient
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D41818
Notable upstream pull request merges:
#15024 Add missed DMU_PROJECTUSED_OBJECT prefetch
#15029 Do not request data L1 buffers on scan prefetch
#15036 FreeBSD: catch up to __FreeBSD_version 1400093
#15039 Fix raw receive with different indirect block size
#15047 FreeBSD: Fix build on stable/13 after 1302506
#15049 Fix the ZFS checksum error histograms with larger record sizes
#15052 Reduce bloat in ereport.fs.zfs.checksum events
#15056 Avoid extra snprintf() in dsl_deadlist_merge()
#15061 Ignore pool ashift property during vdev attachment
#15063 Don't panic if setting vdev properties is unsupported for this vdev type
#15067 spa_min_alloc should be GCD, not min
#15071 Add explicit prefetches to bpobj_iterate()
#15072 Adjust prefetch parameters
#15076 Refactor dmu_prefetch()
#15079 set autotrim default to 'off' everywhere
#15080 ZIL: Fix config lock deadlock
#15088 metaslab: tuneable to better control force ganging
#15096 Avoid waiting in dmu_sync_late_arrival()
#15097 BRT should return EOPNOTSUPP
#15103 Remove zl_issuer_lock from zil_suspend()
#15107 Remove fastwrite mechanism
#15113 libzfs: sendrecv: send_progress_thread: handle SIGINFO/SIGUSR1
#15122 ZIL: Second attempt to reduce scope of zl_issuer_lock
#15129 zpool_vdev_remove() should handle EALREADY error return
#15132 ZIL: Replay blocks without next block pointer
#15148 zfs_clone_range should return descriptive error codes
#15153 ZIL: Avoid dbuf_read() before dmu_sync()
#15172 copy_file_range: fix fallback when source create on same txg
#15180 Update outdated assertion from zio_write_compress
Obtained from: OpenZFS
OpenZFS commit: 804414aad2
libproc's PR_REQUESTED is not implemented on FreeBSD. Remove dead code
in dtrace that would handle it.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D41225
dtrace stored its PR_RLC and PR_KLC flags in the proc_handle's flags,
where they collided with PATTACH_FORCE and PATTACH_RDONLY, respectively.
Thus, Psetflags(PR_KLC) effectively also set the PATTACH_RDONLY flag.
Since the flags are private to dtrace (at least on FreeBSD), store them in
dtrace's own dt_proc structure instead.
On FreeBSD, either PR_RLC or PR_KLC was always set, so remove code that
would handle the case where neither was set.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D41121
The illumos #ifdef's in this file make it harder to read and maintain.
There is little change upstream, and increasing changes for FreeBSD.
Remove the illumos code with `unifdef`. The only manual changes here
are the #includes and #defines at the top, and removing blank lines.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D41173
They were previously being defined by cddl/lib/Makefile.inc, and as of
commit 4ae6991228 were being overridden by defaults in bsd.own.mk,
which changed SHLIBDIR to /usr/lib.
Reported by: Domagoj Stolfa <ds815@cam.ac.uk>
Fixes: 4ae6991228 ("dtrace: Add WITH_DTRACE_ASAN")
This option is a blanket for all the DTrace-related software. The option
when enabled passes in -fsanitize=address -fsanitize=undeifned, enabling
ASAN and UBSAN in the following components:
- libdtrace
- dtrace(1)
- lockstat(1)
- plockstat(1)
The option defaults to "no" and is intended as a developer aid.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D41157
At the time we ensured that the more important functional tests were
working, but neglected to update the unit tests.
MFC after: 2 weeks
MFC with: 92642bba4d
Sponsored by: Axcient
