1
0
mirror of https://git.FreeBSD.org/src.git synced 2025-01-20 15:43:16 +00:00
Commit Graph

1460 Commits

Author SHA1 Message Date
Eric van Gyzen
dad11f990e dtrace: prevent forked child from running after an error condition
The pid/killonerror test uses an invalid probe specifier to verify that
the child process is killed.  It occasionally fails because the "date"
command is allowed to run long enough to print the date.  This is harmless
in this case, but is clearly not ideal.

When the dt_proc_control thread is about to exit, and the dtrace command
forked the child, do not make the child runnable.

Reviewed by:	markj
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D40976
2023-07-12 10:33:28 -05:00
Alan Somers
92642bba4d zfsd: listen for sysevent.fs.zfs instead of misc.fs.zfs
At some point the names of these devd events changed.  Probably it
happened when importing OpenZFS.  Before that, FreeBSD's sysevent_alloc
method didn't create a "class" nvpair in the event, which led to
log_sysevent using the event's ev_subclass field as its type.

MFC after:	2 weeks
Sponsored by:	Axcient
Differential Revision: https://reviews.freebsd.org/D39437
2023-07-10 16:36:57 -06:00
Christos Margiolis
911f026039 dtrace: move kinst tests to common
Reviewed by:	markj
Approved by:	markj (mentor)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D40414
2023-07-04 18:46:59 +03:00
Michael Tuexen
2176c9ab71 dtrace: improve siftr probe
Improve consistency of the field names with tcpsinfo_t:
* Use mss instead of max_seg_size.
* Use lport and rport instead of tcp_localport and tcp_foreignport.

Use t_flags instead of flags to improve consistency with t_flags2.

Add laddr and raddr, since the addresses were missing when compared
to the output of siftr.

Reviewed by:		cc
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D40834
2023-07-02 03:08:51 +02:00
Michael Tuexen
f85b82daf1 dtrace: third update to the siftr probe
This adds the changes corresponding to
https://cgit.freebsd.org/src/commit/?id=60167184abd54ca12c6bf7ab60f2a08b41342f84
to the siftr probe.
This concludes the series up updates.

Reviewed by:		rscheff
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D40825
2023-07-01 09:30:34 +02:00
Michael Tuexen
643d067119 dtrace: another update of the siftr probe
This adds the changes corresponding to
https://cgit.freebsd.org/src/commit/?id=d090464ecd4af5cd400ef5cbbfe8409d019eac34
to the siftr probe.

Reviewed by:		rscheff
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D40824
2023-07-01 02:45:16 +02:00
Michael Tuexen
cade9a8e37 dtrace: update siftr probe
This adds the changes corresponding to
https://cgit.freebsd.org/src/commit/?id=1241e8e7aed5d709a6cf62024e9ae750981c03ae
to the siftr probe.

Reviewed by:		rscheff
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D40823
2023-07-01 02:26:52 +02:00
Michael Tuexen
f5541b85a5 dtrace: fix constants used by the siftr probe
The constants used in the C code were changed in
https://svnweb.freebsd.org/changeset/base/343646
but changing them also in the dtrace translator was missed.

Reviewed by:		cc, rscheff
MFC after:		3 days
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D40805
2023-06-30 22:15:47 +02:00
Mark Johnston
258a0d760a ctfconvert: Integer encoding types are unsigned
Before this change, encodings in the user-defined range were being
sign-extended.

MFC after:	1 week
Sponsored by:	Innovate UK
2023-06-28 16:29:49 -04:00
Martin Matuska
4e8d558c9d zfs: merge openzfs/zfs@feff9dfed
Notable upstream pull request merges:
   Update compatibility.d files
   ZIL: Reduce scope of per-dataset zl_issuer_lock
   zil: Add some more statistics
   btree: Implement faster binary search algorithm
   Fix inconsistent definition of zfs_scrub_error_blocks_per_txg
   Fix concurrent resilvers initiated at same time
   Fix NULL pointer dereference when doing concurrent 'send' operations
   ZIL: Allow to replay blocks of any size
   Fix the L2ARC write size calculating logic
   Introduce zfs_refcount_(add|remove)_few()
   Improve l2arc reporting in arc_summary
   Finally drop long disabled vdev cache
   Fix the L2ARC write size calculating logic (2)
   Use list_remove_head() where possible
   ZIL: Fix race introduced by f63811f072

Obtained from:	OpenZFS
OpenZFS commit:	feff9dfed3
2023-06-10 19:31:17 +02:00
Elyes Haouas
f6a119cf8d zfsd: Fix typo in comment
Signed-off-by: Elyes Haouas <ehaouas@noos.fr>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/653
2023-06-01 20:33:42 -06:00
Christos Margiolis
5c134fba22 kinst: fix memcpy() tracing crash
Tracing memcpy() would crash the kernel, because we'd also trace the
memcpy() calls from kinst_invop(). To fix this, introduce kinst_memcpy()
whose arguments are 'volatile', so that we avoid having the compiler
replace it with a regular memcpy().

Reviewed by:	markj
Approved by:	markj (mentor)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D40284
2023-05-26 18:43:37 +03:00
Christos Margiolis
ed35c7cf77 dtrace.1: fix mandoc -Tlint
Reviewed by:	markj
Approved by:	markj (mentor)
Differential Revision:	https://reviews.freebsd.org/D40230
2023-05-23 17:29:19 +03:00
Christos Margiolis
21a16d55cc dtrace: add register bindings for arm64
Reviewed by:	mhorne, markj
Approved by:	markj (mentor)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D39956
2023-05-23 17:19:25 +03:00
Christos Margiolis
db05f9fbfb dtrace: implement dtrace_instr_size() for arm64
Reviewed by:	markj
Approved by;	markj (mentor)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D39955
2023-05-23 17:19:15 +03:00
Christos Margiolis
1e136a9cbd dtrace(1): add -d flag to dump D script post-dt_sugar
By specifying the -d flag, libdtrace will dump the D script after it has
applied syntactical sugar transformations (e.g if/else). This is useful
for both understanding what dt_sugar does, as well as debugging it.

Reviewed by:	markj
Approved by:	markj (mentor)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D38732
2023-05-23 17:18:39 +03:00
Christos Margiolis
6265da14c9 libdtrace: get rid of illumos ifdefs in dt_module_update(), fix dm_file and dm_modid
Because dt_module_update() is highly OS-specific, the ifdefs make it
hard to read and follow what is going on. Also handle dm_modid, and
remove handling of the ".filename" section, since we can easily fetch
the filename from the module's pathname (k_stat->pathname).

Reviewed by:	markj
Approved by:	markj (mentor)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D39177
2023-05-23 15:54:58 +03:00
Christos Margiolis
1fef7abdc7 dtrace: add register bindings for RISC-V
Reviewed by:	mhorne, markj
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D39611
2023-04-20 13:35:57 -04:00
Christos Margiolis
080e56a6c9 dtrace: expose dtrace_instr_size() to userland and implement it for riscv
dtrace_instr_size() is needed by the forthcoming RISC-V port of kinst,
as well as by libdtrace in D38825 for both amd64 and RISC-V.

Reviewed by:	markj, mhorne
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D39489
2023-04-20 13:35:57 -04:00
Simon J. Gerraty
d9a4274795 Update/fix Makefile.depend for userland 2023-04-18 17:14:23 -07:00
Alan Somers
f698c1e99b zfsd: add support for hotplugging spares
If you remove an unused spare and then reinsert it, zfsd will now online
it in all pools.

Do not MFC without 2a58b312b6 (but it's ok to MFC that one without this
one).

Submitted by:	Ameer Hamza <ahamza@ixsystems.com> (zfsd), Me (tests)
MFC after:	2 weeks
MFC with:	2a58b312b6
Sponsored by:	iX Systems, Axcient
Pull Request:	https://github.com/freebsd/freebsd-src/pull/697
2023-04-06 11:58:55 -06:00
Mark Johnston
27f35b7dd4 libdtrace: Do not set SHF_ALLOC on SUNW_dof relocation sections
The section will contain static relocations which do not need to be
preserved after linking, and moreover these relocations may reference
symbols that end up getting removed.

Do not set SHF_ALLOC and instead let the linker decide what needs to be
done.

PR:		258872
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2023-04-04 14:08:16 -04:00
Martin Matuska
2a58b312b6 zfs: merge openzfs/zfs@431083f75
Notable upstream pull request merges:
   Fix short-lived txg caused by autotrim
   ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
   Implementation of block cloning for ZFS
   SHA2 reworking and API for iterating over multiple implementations
   Sync thread should avoid holding the spa config write lock
         when possible
   txg_sync should handle write errors in ZIL
   More adaptive ARC eviction
   Fix NULL pointer dereference in zio_ready()
   zfs redact fails when dnodesize=auto
   improve error message of zfs redact
   Skip memory allocation when compressing holes
   FreeBSD: don't verify recycled vnode for zfs control directory
   partially revert PR 14304 (eee9362a7)
   Fix per-jail zfs.mount_snapshot setting
   Fix data race between zil_commit() and zil_suspend()
   System-wide speculative prefetch limit
   Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()
   Do not hold spa_config in ZIL while blocked on IO
   Move dmu_buf_rele() after dsl_dataset_sync_done()
   Ignore too large stack in case of dsl_deadlist_merge
   Use .section .rodata instead of .rodata on FreeBSD
   ICP: AES-GCM: Refactor gcm_clear_ctx()
   ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx()
   Handle unexpected errors in zil_lwb_commit() without ASSERT()
   icp: Prevent compilers from optimizing away memset()
         in gcm_clear_ctx()
   Revert zfeature_active() to static
   Remove bad kmem_free() oversight from previous zfsdev_state_list
         patch
   Optimize the is_l2cacheable functions
   FreeBSD: zfs_znode_alloc: lock the vnode earlier
   FreeBSD: fix false assert in cache_vop_rmdir when replaying ZIL
   spl: Add cmn_err_once() to log a message only on the first call
   Fix incremental receive silently failing for recursive sends
   Restore ASMABI and other Unify work
   Fix detection of IBM Power8 machines (ISA 2.07)
   Better handling for future crypto parameters
   zcommon: Refactor FPU state handling in fletcher4
   Fix prefetching of indirect blocks while destroying
   Fixes in persistent error log
   FreeBSD: Remove extra arc_reduce_target_size() call
   Additional limits on hole reporting
   Drop lying to the compiler in the fletcher4 code
   panic loop when removing slog device
   Update vdev state for spare vdev
   Fix cloning into already dirty dbufs
   Revert "Do not hold spa_config in ZIL while blocked on IO"

Obtained from:	OpenZFS
OpenZFS commit:	431083f75b
2023-04-03 16:49:30 +02:00
Christos Margiolis
3afba490c1 libdtrace: fix indendation in dt_printd()
No functional change.

Reviewed by:	markj
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D39145
2023-03-20 09:34:20 -04:00
Zhenlei Huang
f8e1aa85fe ctf: Remove unused function prototype for getpname()
This function prototype should have been removed along with the
implementation.

Fixes:		3dd5524264 ctfdump: Use getprogname()
MFC after:	1 day
2023-02-26 12:18:25 +08:00
Yuri
996d07a059 zpool: install vdevprops(7) man page
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/666
2023-02-25 09:24:20 -07:00
Mark Johnston
49e3972afc lockstat: Use gelf.h instead of playing games with the preprocessor
This reverts a portion of 1477dd823ee ("Merge OpenZFS support in to
HEAD.").  No functional change intended.

MFC after:	1 week
2023-02-24 20:16:51 -05:00
Mark Johnston
f4f5e69c84 lockstat: Use the correct type for a symbol size
No functional change intended.

MFC after:	1 week
2023-02-24 20:11:08 -05:00
Zhenlei Huang
7f0ebf0876 Delete obsolete Solaris compat header file stdlib.h
This drops function `getexecname()` redirection.

Historically `getexecname()` is a compatibility definition. Since
openzfs has its own implementation of function `getexecname()` in libspl
and has been merged into base, the compat header file stdlib.h is
no longer needed and should not be used.

Also without this fix libspl will end up an incompatible version of
`getprogname()` with libc. In particular, if zfs is enabled, programs
such as pgrep in /rescue can be wrongly statically linked with libspl
and will not function properly.

PR:		269738
Reviewed by:	markj
Fixes:		9e5787d228 Merge OpenZFS support in to HEAD
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D38733
2023-02-24 02:00:09 +08:00
Zhenlei Huang
3dd5524264 ctfdump: Use getprogname()
Also remove no longer used function `getpname()`.

Reviewed by:	markj
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D38740
2023-02-24 00:28:35 +08:00
Emmanuel Vadot
f0408cd92f pkgbase: Create a FreeBSD-ctf-tools package
It contains the ctf tools that most users will never use.

Sponsored by:	Beckhoff Automation GmbH & Co. KG
Differential Revision:	https://reviews.freebsd.org/D38223
2023-02-08 08:48:58 +01:00
Mitchell Horne
f711d5c3d0 libdtrace: add riscv support
Largely untested, as we can't really do anything with user probes
without an implementation of fasttrap. However, this is enough to
generate an embedded dtrace program with `dtrace -G` and link the
generated ELF file.

Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D38301
2023-02-06 15:26:53 -04:00
Mitchell Horne
2166649fa1 libdtrace: drop remaining mips support
Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D38300
2023-02-06 15:26:53 -04:00
John Baldwin
9a93b6cf3d zfs: Disable -Wunused-variable for environ in main in zfs_main.c.
There is an extern declaration of the global variable environ (which
has no portable header that defines it) that is unused on FreeBSD due
to zfs_setproctitle_init() being a stub.

Reviewed by:	emaste
Reported by:	GCC
Differential Revision:	https://reviews.freebsd.org/D38231
2023-01-30 09:31:19 -08:00
Ed Maste
ac4c695ad6 Retire WITHOUT_CXX option
Several important base system components are written in C++, and the
WITHOUT_CXX option produced a system that was not fully functional.
Just accept this, and remove the option to build without C++ support.

This reverts commit adc3c128c6.

Reviewed by:	brooks, kevans, jhb (earlier)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D33108
2023-01-26 21:13:16 -05:00
Kristof Provost
ecdd0b48cb dtrace: remove stray {
Fixes:	da81cc6035
PR:	269128
2023-01-24 08:39:37 +01:00
Andrew Gallatin
da81cc6035 dtrace: conditionally load the systrace_linux klds when loading dtrace.
When dtrace starts, it tries to detect if the dtrace klds are loaded,
and if not, it loads them by loading the dtraceall kld. This module
depends on most dtrace modules, including systrace for the native
freebsd and freebsd32 ABIs. However, it does not depend on the
systrace_linux klds, as they in turn depend on the linux ABI klds, and
we don't want to load an ABI module that the user has not explicitly
requested. This can leave a naive user in a state where they think all
syscall providers have been loaded, yet linux ABI syscalls are
"invisible" to dtrace.

To fix this, check to see if the linux ABI modules are loaded. If they
are, then load their systrace klds.

Reviewed by: markj, (emaste & jhb, earlier versions)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D37986
2023-01-23 20:36:24 -05:00
Mark Johnston
be39466a10 libdtrace: Change the binding of USDT probe symbols to STB_WEAK
Otherwise, if multiple object files contain references to the same
probe, newish lld will refuse to link them by default, raising a
duplicate global symbol definition error.  Previously, duplicate global
symbols with identical absolute st_values were permitted by both lld and
GNU ld.

Since dtrace has no use for probe function symbols after the relocation
performed by dtrace -G, make the symbols weak as well, following a
suggestion from MaskRay.

Reported by:	dim
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2022-12-11 11:27:22 -05:00
Mark Johnston
add8a6c9ac dtrace tests: Extend the kinst regression test
Trace a function which disables interrupts.
2022-12-08 15:07:48 -05:00
Gleb Smirnoff
e68b379244 tcp: embed inpcb into tcpcb
For the TCP protocol inpcb storage specify allocation size that would
provide space to most of the data a TCP connection needs, embedding
into struct tcpcb several structures, that previously were allocated
separately.

The most import one is the inpcb itself.  With embedding we can provide
strong guarantee that with a valid TCP inpcb the tcpcb is always valid
and vice versa.  Also we reduce number of allocs/frees per connection.
The embedded inpcb is placed in the beginning of the struct tcpcb,
since in_pcballoc() requires that.  However, later we may want to move
it around for cache line efficiency, and this can be done with a little
effort.  The new intotcpcb() macro is ready for such move.

The congestion algorithm data, the TCP timers and osd(9) data are
also embedded into tcpcb, and temprorary struct tcpcb_mem goes away.
There was no extra allocation here, but we went through extra pointer
every time we accessed this data.

One interesting side effect is that now TCP data is allocated from
SMR-protected zone.  Potentially this allows the TCP stacks or other
TCP related modules to utilize that for their own synchronization.

Large part of the change was done with sed script:

s/tp->ccv->/tp->t_ccv./g
s/tp->ccv/\&tp->t_ccv/g
s/tp->cc_algo/tp->t_cc/g
s/tp->t_timers->tt_/tp->tt_/g
s/CCV\(ccv, osd\)/\&CCV(ccv, t_osd)/g

Dependency side effect is that code that needs to know struct tcpcb
should also know struct inpcb, that added several <netinet/in_pcb.h>.

Differential revision:	https://reviews.freebsd.org/D37127
2022-12-07 09:00:48 -08:00
Ed Maste
7543a9c028 dwatch: install hard links with same mode as target
Previoulsy when using NO_ROOT we recorded METALOG entries for dwatch
hard links with different file modes than their link targets, which is
not permitted.

Reviewed by:	bapt
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D37477
2022-11-23 15:10:33 -05:00
John Baldwin
7e3e70ce21 libicp: Link with -z noexecstack for ld.bfd.
The assembly source files do not contain GNU-stack annotations, so
ld.bfd defaults to using an executable stack.
2022-11-22 08:37:12 -08:00
Martin Matuska
dbd5678dca zfs: merge openzfs/zfs@2163cde45
Notable upstream pull request merges:
   Add options to zfs redundant_metadata property
   Allow mounting snapshots in .zfs/snapshot as a regular user
   quota: disable quota check for ZVOL
   quota: extend quota for dataset
   Fix memory leaks in dmu_send()/dmu_send_obj()
   Avoid unnecessary metaslab_check_free calling
   PAM: Fix unchecked return value from zfs_key_config_load()
   Handle possible null pointers from malloc/strdup/strndup()
   zstream: allow decompress to fix metadata for uncompressed
         records
   zvol_wait logic may terminate prematurely
   FreeBSD: Fix a pair of bugs in zfs_fhtovp()
   Stop ganging due to past vdev write errors
   Optimize microzaps
   Fix draid2+2s metadata error on simultaneous 2 drive failures
   zed: Avoid core dump if wholedisk property does not exist
   Propagate extent_bytes change to autotrim thread
   FreeBSD: vn_flush_cached_data: observe vnode locking contract
   Fix ARC target collapse when zfs_arc_meta_limit_percent=100
   Add ability to recompress send streams with new compression
         algorithm
   Deny receiving into encrypted datasets if the keys are not
         loaded
   Fix arc_p aggressive increase
   zed: Prevent special vdev to be replaced by hot spare
   Expose zfs_vdev_open_timeout_ms as a tunable
   FreeBSD: Fix out of bounds read in zfs_ioctl_ozfs_to_legacy()
   Adds the `-p` option to `zfs holds`
   Handle and detect 's unlock regression

Obtained from:	OpenZFS
OpenZFS commit:	2163cde450
2022-11-16 21:27:42 +01:00
Emmanuel Vadot
a71ea7bea0 pkgbase: Put zfs utilities and lib in their own package
It is useful to have zfs utilities and lib in a separate package as
it allow users to create image that can support ZFS (i.e. not with
WITHOUT_ZFS in src.conf set) without bloating the default image with
all zfs tools (for example for jails).

Differential Revision:	https://reviews.freebsd.org/D36225
2022-10-26 19:46:30 +02:00
Gordon Bergling
4bd5e6221e zfsd(8): Fix a typo in an error message
- s/faild/failed/

MFC after:	1 week
2022-10-25 12:55:42 +02:00
Mark Johnston
53a31c87df libdtrace: Fix some CPU register number definitions to match the kernel
MFC after:	1 week
2022-10-12 16:06:37 -04:00
Mark Johnston
19a847e5f2 kinst: Add a rudimentary regression test case
The test instruments a number of large, frequently called kernel
functions while generating load in the background.

MFC after:	3 months
2022-10-11 18:19:55 -04:00
Christos Margiolis
2179a159ea libdtrace: Add kinst support
kinst does not instantiate its probes automatically, it only does so on
demand via an ioctl interface implemented by /dev/kinst.  This change
modifies libdtrace to perform that work when the script references the
kinst provider, similar to the way pid provider probes are implemented.

Reviewed by:	markj
MFC after:	3 months
Sponsored by:	Google, Inc. (GSoC 2022)
Differential Revision:	https://reviews.freebsd.org/D36852
2022-10-11 18:19:08 -04:00
Mark Johnston
bdd101c4d4 dtrace: Add a "regs" variable
This allows invop-based providers (i.e., fbt and kinst) to expose the
register file of the CPU at the point where the probe fired.  It does
not work for SDT providers because their probes are implemented as plain
function calls and so don't save registers.  It's not clear what
semantics "regs" should have for them anyway.

This is akin to "uregs", which nominally provides access to the
userspace registers.  In fact, DIF already had a DIF_VAR_REGS variable
defined, it was simply unimplemented.

Usage example: print the contents of %rdi upon each call to
amd64_syscall():

    fbt::amd64_syscall:entry {printf("%x", regs[R_RDI]);}

Note that the R_* constants are defined in /usr/lib/dtrace/regs_x86.d.
Currently there are no similar definitions for non-x86 platforms.

Reviewed by:	christos
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D36799
2022-10-04 13:05:54 -04:00
John Baldwin
e67b246734 libzpool: Disable -Wuse-after-free for dbuf.c.
The debug traces for reference counting in ZFS use the pointer of the
owning object as a "tag" for references to check that when an object
drops a reference it had actually held one.  In a couple of places ZFS
drops references after freeing the owning object.  In userland GCC
realizes this is a use after free.  However, since only the value of
the pointer is used, and it isn't indirected, the use is harmless.

Reviewed by:	emaste
Differential Revision:	https://reviews.freebsd.org/D36818
2022-10-03 16:10:43 -07:00
Martin Matuska
c7046f76c2 zfs: merge openzfs/zfs@c629f0bf6
Notable upstream pull request merges:
   Fix BLAKE3 tuneable and module loading on Linux and FreeBSD
   FreeBSD: Organize sysctls
   FreeBSD: add kqfilter support for zvol cdev
   Importing from cachefile can trip assertion
   Apply arc_shrink_shift to ARC above arc_c_min
   Improve too large physical ashift handling
   Revert "Avoid panic with recordsize > 128k, raw sending and
         no large_blocks"
   Add zfs.sync.snapshot_rename
   zfs_enter rework
   zfs recv hangs if max recordsize is less than received
         recordsize

Obtained from:	OpenZFS
OpenZFS commit:	c629f0bf62
2022-09-21 14:17:13 +02:00
Jens Schweikhardt
7c20397b72 In man page cross refs to sudo, indicate it comes from ports/security/sudo. 2022-08-14 11:44:43 +02:00
Mark Johnston
6237c3e74f ctfconvert: Actually use the asprintf() helper
Fixes:	1165fc9a52 ("ctfconvert: Give bitfield types names distinct from the base type")
2022-08-02 20:49:50 -04:00
Mark Johnston
1165fc9a52 ctfconvert: Give bitfield types names distinct from the base type
CTF integers have an explicit width and so can be used to represent
bitfields.  Bitfield types emitted by ctfconvert(1) share the name of
the base integer type, so a struct field with type "unsigned int : 15"
will have a type named "unsigned int".

To avoid ambiguity when looking up types by name, add a suffix to names
of bitfield types to distinguish them from the base type.  Then, if
ctfmerge happens to order bitfield types before the corresponding base
type in a CTF file, a name lookup will return the base type, which is
always going to be the desired behaviour.

PR:		265403
Reported by:	cy
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2022-08-02 20:32:17 -04:00
Mark Johnston
6a05f14381 dtrace tests: Rename some test type names to avoid a conflict
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2022-08-02 20:32:08 -04:00
Mark Johnston
e1700a36a9 dtrace tests: Override RLIMIT_CORE for a test which triggers a core dump
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2022-08-02 20:32:04 -04:00
Kristof Provost
06016a40d6 libnvpair: install libnvpair.h
This is included by libbe/be.h.

Sponsored by:	Rubicon Communications, LLC ("Netgate")
2022-07-21 18:36:01 +02:00
Martin Matuska
a0b956f5ac zfs: merge openzfs/zfs@cb01da680
Notable upstream pull request merges:
   Avoid panic with recordsize > 128k, raw sending and no large_blocks
   Fix dnode byteswapping
   Add a "zstream decompress" subcommand
   Scrub mirror children without BPs
   Several sorted scrub optimizations
   Fix and disable blocks statistics during scrub
   Several B-tree optimizations
   Avoid two 64-bit divisions per scanned block
   Avoid memory copies during mirror scrub
   Avoid memory copy when verifying raidz/draid parity

Obtained from:	OpenZFS
OpenZFS commit:	cb01da6805
2022-07-08 00:02:36 +02:00
Brooks Davis
3bf6636512 cddl/*: add a WITH(OUT)_DTRACE option
Add an option to enable/disable DTrace without disabling ZFS.  New
architectures such as CHERI may support ZFS before they support DTrace
and the old model of WITHOUT_CDDL disabling both wasn't helpful.

For compatiblity, the CDDL option remains and WITHOUT_CDDL implies
WITHOUT_DTRACE.  WITHOUT_DTRACE also implies WITHOUT_CTF.

As part of this change, largely convert cddl/*/Makefile to using the
more compact SUBDIR.${MK_<FOO>}+= form rather than using intermediate
variables.

Reviewed by:	markj
Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D35718
2022-07-06 14:03:48 +01:00
Kornel Dulęba
9dbacce2d4 lockstat: Fix construction of comparision predicates
Passing "0x%p" to sprintf results in double "0x" being printed.
This causes a dtrace script compilation failure when "-d" flag
is specified.
Fix that by removing the extraneous "0x".

Reviewed by:	markj
Approved by:	mw(mentor)
Obtained from:	Semihalf
Sponsored by:	Alstom
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D35690
2022-07-04 16:22:22 +02:00
Brooks Davis
711d50bd9e dtrace: Remove local mips support
Remove the stub pid probe and all the build glue.

Reviewed by:	imp, jhb
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D35541
2022-07-01 08:33:17 +01:00
Jessica Clarke
b0a8333a88 libicp_rescue: Fix build for powerpc64le
This is a repeat of 13cee53cef35 ("libicp: Fix build for powerpc64le")
but for the rescue version of the library.

Fixes:	1f1e2261e3 ("zfs: merge openzfs/zfs@deb121309")
2022-07-01 04:28:15 +01:00
Jessica Clarke
9921563f43 libicp: Fix build for powerpc64le
The C sources use the assembly routines for little-endian powerpc64, and
the assembly files have ppc64le in their names, but the guard here was
for big-endian powerpc64.

Fixes:	1f1e2261e3 ("zfs: merge openzfs/zfs@deb121309")
2022-07-01 01:30:26 +01:00
Martin Matuska
1f1e2261e3 zfs: merge openzfs/zfs@deb121309
Notable upstream pull request merges:
   Introduce BLAKE3 checksums as an OpenZFS feature
   Reduce ZIO io_lock contention on sorted scrub
   Improve sorted scan memory accounting
   AVL: Remove obsolete branching optimizations
   FreeBSD: Improve crypto_dispatch() handling

Obtained from:	OpenZFS
OpenZFS commit:	deb1213098
2022-06-23 17:49:33 +02:00
Martin Matuska
e3aa18ad71 zfs: merge openzfs/zfs@b9d98453f
Notable upstream pull request merges:
   Fix inflated quiesce time caused by lwb_tx during zil_commit()
   zstd early abort
   Verify BPs as part of spa_load_verify_cb()
   More speculative prefetcher improvements
   Expose zpool guids through kstats
   Refactor Log Size Limit
   FreeBSD: libspl: Add locking around statfs globals
   Cancel in-progress rebuilds when we finish removal
   zed: Take no action on scrub/resilver checksum errors
   Remove wrong assertion in log spacemap

Obtained from:	OpenZFS
OpenZFS commit:	b9d98453f9
2022-06-03 18:17:53 +02:00
Martin Matuska
716fd348e0 zfs: merge openzfs/zfs@c0cf6ed67
Notable upstream pull request merges:
   zvol_wait: Ignore locked zvols
   Improve log spacemap load time
   Improved zpool status output, list all affected datasets
   FreeBSD: Use NDFREE_PNBUF if available
   Make zfs_max_recordsize default to 16M
   Fix error handling in FreeBSD's get/putpages VOPs
   FreeBSD: Fix translation from ABD to physical pages
   zfs: holds: dequadratify
   Corrected edge case in uncompressed ARC->L2ARC handling
   Improve mg_aliquot math
   Reduce dbuf_find() lock contention
   FreeBSD: use zero_region instead of allocating a dedicated page

Obtained from:	OpenZFS
OpenZFS commit:	c0cf6ed679
2022-05-19 00:55:59 +02:00
Mark Johnston
5727eceabc ctf: Link CTF toolchain man pages to ctf.5
Also expand the CTF acronym to provide a bit of context.

PR:		259790
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2022-04-21 11:20:28 -04:00
Mark Johnston
45dd2eaac3 ctfdump: Remove definitions of warn() and vwarn()
The presence of the latter causes a link error when building a
statically linked ctfdump(1) because libc defines the same symbol.
libc's warn() is defined as a weak symbol and so does not cause the same
problem, but let's just use libc's version.

Reported by:	stephane rochoy <stephane.rochoy@stormshield.eu>
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2022-04-21 11:18:53 -04:00
Mark Johnston
6e563a1b60 libctf: Fix recursive descent into anonymous SOU fields
PR:		262412
Tested by:	dhw, gallatin
Fixes:		a6fb869173 ("libctf: Handle CTFv3 containers")
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2022-04-12 17:29:53 -04:00
Li-Wen Hsu
0088bc444e
DTrace test: skip flakey common.misc.t_dtrace_contrib.tst_dynopt_d in CI
PR:		237641
Sponsored by:	The FreeBSD Foundation
2022-03-24 11:37:50 +08:00
Li-Wen Hsu
e62aee66b0
DTrace test: Add a new keyword SKIPCI to gentest.sh
This is for marking a test case is flakey and should not be executed in
the CI environment.

Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D34635
2022-03-24 11:30:27 +08:00
Li-Wen Hsu
16e02ae401
dtrace tests: Fix expected outout for tst.system.d
This is follow up of d500a85e64

PR:		262415
Sponsored by:	The FreeBSD Foundation
2022-03-09 11:39:12 +08:00
Martin Matuska
c03c5b1c80 zfs: merge openzfs/zfs@a86e08941 (master) into main
Notable upstream pull request merges:
  :  log xattr=sa create/remove/update to ZIL
  : Cross-platform xattr user namespace compatibility
  : Report dnodes with faulty bonuslen
  : FreeBSD: Fix zvol_cdev_open locking
  : spl: Don't check FreeBSD rwlocks for double initialization
  : Fix clearing set-uid and set-gid bits on a file when
          replying a write
  : Add enumerated vdev names to 'zpool iostat -v' and
          'zpool list -v'
  : Enable encrypted raw sending to pools with greater ashift
  : Receive checks should allow unencrypted child datasets
  : Avoid dirtying the final TXGs when exporting a pool
  : Fix ENOSPC when unlinking multiple files from full pool

Obtained from:	OpenZFS
OpenZFS commit:	a86e089415
2022-03-08 18:53:02 +01:00
Mark Johnston
3fe1f21fb3 ctf: Avoid passing a caddr_t to roundup2()
For some reason I can't reproduce this locally, but Jenkins complains.

Reported by:	Jenkins
Fixes:		bdf290cd3e ("ctf: Add v3 support to CTF tools, ctf{convert,dump,merge}")
2022-03-07 11:20:57 -05:00
Mark Johnston
cb6f722562 ctf: Fix a -Wunused-but-set-variable warning
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2022-03-07 10:43:19 -05:00
Mark Johnston
bdf290cd3e ctf: Add v3 support to CTF tools, ctf{convert,dump,merge}
ctfdump handles v2 and v3.  ctfconvert now emits only CTFv3, whereas
ctfmerge can merge v2 and v3 containers into v3 containers.

MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D34364
2022-03-07 10:43:19 -05:00
Mark Johnston
a6fb869173 libctf: Handle CTFv3 containers
In general, the patch adds indirection to minimize the amount of code
that needs to know about differences between v2 and v3.  Specifically,
some new ctf_get_ctt_* functions are added, and new LCTF_* macros are
added to use the underlying container's version to do the right thing.

CTF containers can have parent/child relationships, wherein a type ID in
one container refers to a type in the parent.  It is permitted for the
parent and child to have different versions.

MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D34363
2022-03-07 10:43:19 -05:00
Mark Johnston
994297b01b ctf: Add definitions for CTFv3
These are based on definitions added to binutils' libctf.  Specifically:
- Type IDs are now encoded in 32 bits rather than 16, changing the
  layout of ctf_type_t, ctf_array_t, ctf_member_t and ctf_lmember_t.
- Type info is encoded in 32 bits rather than 16.  The type "kind" is
  extended from 5 bits to 6, and the type "vlen" is extended from 10
  bits to 25.

The main upside is that we remove the current limit, imposed by CTFv2,
of 2^{15} distinct types in the main kernel executable.  Other limits,
such as that on the number of elements in an enum, imposed by the vlen
limit, are also raised.

This change adds v2 and v3 flavours of macros and type definitions which
differ between the two versions.  Compatibility is preserved for now by
having generic names refer to the v2 definitions, so, e.g., ctf_type_t
is still a v2 type.

No functional change intended.

Reviewed by:	Domagoj Stolfa
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D34360
2022-03-07 10:43:18 -05:00
Mark Johnston
2d5d2a986c ctf: Import ctf.h from OpenBSD
Use it instead of the existing ctf.h from OpenSolaris.  This makes it
easier to use CTF in the core kernel, and to extend the CTF format to
support wider type IDs.

The imported ctf.h is modified to depend only on _types.h, and also to
provide macros which use the "parent" bit of a type ID to refer to types
in a parent CTF container.

No functional change intended.

Reviewed by:	Domagoj Stolfa, emaste
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D34358
2022-03-07 10:43:18 -05:00
Mark Johnston
45c23c2608 libctf: Use const ctf_file_t references in string lookup routines
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2022-02-23 18:43:51 -05:00
Mark Johnston
1ef441f699 libdtrace: Add a missing newline to an error message
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2022-02-23 11:41:23 -05:00
Mark Johnston
565518046c libctf: Remove checks for CTFv1
Per commit 7db423d692 ("libctf: Rip out CTFv1 support") this support
is obsolete.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2022-02-23 11:41:22 -05:00
Mark Johnston
7be9a3b453 ctfconvert: Rip out STABS support
It is unused on FreeBSD and complicates some efforts to modify the CTF
format to permit wider type IDs, so remove it.  No functional change
intended.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2022-02-10 15:41:10 -05:00
Mark Johnston
7db423d692 libctf: Rip out CTFv1 support
CTFv1 was obsolete before libctf was imported into FreeBSD, and
ctfconvert/ctfmerge can emit only CTFv2.  Make ctf.h a bit easier to
maintain by ripping v1 support out.  No functional change intended.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2022-02-10 15:39:59 -05:00
Mark Johnston
2e4311906d libctf: Use ctf_type_t instead of struct ctf_type
For consistency with other CTF toolchain code.  No functional change
intended.

Fixes:	105fd928b0 ("libctf: Improve check for duplicate SOU definitions in ctf_add_type()")
MFC after:	1 week
2022-02-10 13:21:37 -05:00
Chuck Silvers
f339a3ef63 dtrace: remove unnecessary fflush()
This call was added back in the early days of dtrace porting and
no one knows why anymore.  The extra flushing causes lots of
unnecessary CPU overhead when a script produces lots of output,
as well as easily losing output because the command can't keep up.

Sponsored by:	Netflix
Reviewed by:	imp, markj
Differential Revision:	https://reviews.freebsd.org/D34216
2022-02-09 17:09:26 -08:00
Dimitry Andric
5f2aca8394 Disable clang 14 warning about bitwise operators in zstd
Parts of zstd, used in openzfs and other places, trigger a new clang 14
-Werror warning:

```
sys/contrib/zstd/lib/decompress/huf_decompress.c:889:25: error: use of bitwise '&' with boolean operands [-Werror,-Wbitwise-instead-of-logical]
                        (BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished)
                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

While the warning is benign, it should ideally be fixed upstream and
then vendor-imported, but for now silence it selectively.

MFC after:	3 days
2022-02-08 21:46:08 +01:00
Martin Matuska
e92ffd9b62 zfs: merge openzfs/zfs@17b2ae0b2 (master) into main
Notable upstream pull request merges:
   Fix error propagation from lzc_send_redacted
   Updated the lz4 decompressor
   FreeBSD: Provide correct file generation number
   Verify dRAID empty sectors
   FreeBSD: Update argument types for VOP_READDIR
   Reduce number of arc_prune threads
   FreeBSD: Fix zvol_*_open() locking
   lz4: Cherrypick fix for CVE-2021-3520
   FreeBSD: Fix leaked strings in libspl mnttab
   Fix handling of errors from dmu_write_uio_dbuf() on FreeBSD
   Introduce a flag to skip comparing the local mac when raw sending
   Avoid memory allocations in the ARC eviction thread

Obtained from:	OpenZFS
OpenZFS commit:	17b2ae0b24
2022-01-22 23:05:15 +01:00
Andriy Gapon
256c8c5df2 dt_unring_buf: set dtbd_oldest to the start of the first record
It was set to the start of the buffer and that can be different from the
start of teh first record because of a misalignment.

This change follows the example of dt_realloc_buf().

Reviewed by:	tsoome, markj
MFC after:	4 weeks
Differential Revision:	https://reviews.freebsd.org/D33649
2022-01-11 15:44:04 +02:00
Mark Johnston
963f5dc7a3 ctfconvert: Handle arrays of empty structs
Members with such a type will legitimately have a size of zero, so don't
emit a warning.

PR:		260818
Reviewed by:	bz
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D33700
2021-12-31 12:55:01 -05:00
Domagoj Stolfa
30ec3138ed dtrace: Disable getf() as it is broken on FreeBSD
getf() on FreeBSD calls _sx_slock(), _sx_sunlock() and fget_locked().
Furthermore, it does not set the per-core fault flag, meaning it
usually ends up in a double fault panic once getf() does get called,
especially from fbt.

Reviewing the DTrace Toolkit + a number of other scripts scattered
around FreeBSD, I have not been able to find one use of getf(). Given
how broken the implementation currently is, we disable it until it
can be implemented properly.

Also comment out a test in aggs/tst.subr.d for getf().

Reviewed by:	markj
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D33378
2021-12-17 13:10:22 -05:00
Greg V
9e9c651cac cddl: fix missing ZFS library dependencies
In 9fae47666 zfsd got a libspl dependency to avoid undefined references.
However that workaround did not help external consumers of libzfs_core.

Fix all missing dependencies lld 13 and the rtld complain about.

Reviewed by:	freqlabs, markj
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D32521
2021-11-15 09:42:07 -05:00
Martin Matuska
6ba2210ee0 zfs: merge openzfs/zfs@ec64fdb93 (master) into main
Notable upstream pull request merges:
   Avoid panic in case of pool errors and missing L2ARC
   skip snapshot in zfs_iter_mounted()
   Fix NFS and large reads on older kernels
   Fail invalid incremental recursive send gracefully
   FreeBSD: Really zero the zero page
   Reject zfs send -RI with nonexistent fromsnap
   Correct refcount_add in dmu_zfetch
   zpool should call zfs_nicestrtonum() with non-NULL handle

Obtained from:	OpenZFS
OpenZFS commit:	ec64fdb93d
2021-10-21 15:06:06 +02:00
Mark Johnston
105fd928b0 libctf: Improve check for duplicate SOU definitions in ctf_add_type()
When copying a struct or union from one CTF container to another,
ctf_add_type() checks whether it matches an existing type in the
destination container.  It does so by looking for a type with the same
name and kind as the new type, and if one exists, it iterates over all
members of the source type and checks whether a member with matching
name and offset exists in the matched destination type.  This can
produce false positives, for example because member types are not
compared, but this is not expected to arise in practice.  If the match
fails, ctf_add_type() returns an error.

The procedure used for member comparison breaks down in the face of
anonymous struct and union members.  ctf_member_iter() visits each
member in the source definition and looks up the corresponding member in
the desination definition by name using ctf_member_info(), but this
function will descend into anonymous members and thus fail to match.
Fix the problem by introducing a custom comparison routine which does
not assume member names are unique.  This should also be faster for
types with many members; in the previous scheme, membcmp() would perform
a linear scan of the desination type's members to perform a lookup by
name.  The new routine steps through the members of both types in a
single loop.

PR:		258763
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-10-04 12:28:22 -04:00
Mark Johnston
a40c4ae866 dtrace.1: Document a couple of preprocessor-related options
Suggested by:	swills
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2021-09-07 14:04:50 -04:00
Mark Johnston
13d9437a60 dtrace.1: Document -x ldpath
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2021-09-07 11:19:29 -04:00
Emmanuel Vadot
a3266ba269 pkgbase: Create a FreeBSD-dwatch package
While dwatch is useful some users might not want it by default.
Create a package for it.

Differential Revision:	https://reviews.freebsd.org/D31796
2021-09-07 10:20:40 +02:00
Dimitry Andric
9fae476669 Explicitly link zfsd with libspl to avoid undefined references
Because lld 13.0.0 is more strict about undefined references when
linking to shared libraries, it produces the following errors for zfsd:

ld: error: /home/dim/obj/home/dim/src/llvm-13-update/amd64.amd64/tmp/usr/lib/libzfs_core.so: undefined reference to libspl_assertf [--no-allow-shlib-undefined]
ld: error: /home/dim/obj/home/dim/src/llvm-13-update/amd64.amd64/tmp/usr/lib/libnvpair.so: undefined reference to libspl_assertf [--no-allow-shlib-undefined]
ld: error: /home/dim/obj/home/dim/src/llvm-13-update/amd64.amd64/tmp/usr/lib/libavl.so: undefined reference to libspl_assertf [--no-allow-shlib-undefined]
*** [zfsd.full] Error code 1

Fix this by adding libspl (where libspl_assertf lives) to zfsd's LIBADD.

MFC after:	3 days
2021-08-28 20:23:28 +02:00
Martin Matuska
48b4fe0503 zfs: attach zpool_influxdb to build
From the zpool_influxdb.8 manual page:
  zpool_influxdb produces InfluxDB-line-protocol-compatible metrics from
  zpools.  Like the zpool command, zpool_influxdb reads the current pool
  status and statistics.  Unlike the zpool command which is intended for
  humans, zpool_influxdb formats the output in the InfluxDB line protocol.
  The expected use is as a plugin to a metrics collector or aggregator,
  such as Telegraf.

zpool_influxdb is installed into /usr/libexec/zfs/

Differential revision:	https://reviews.freebsd.org/D31094
MFC after:	3 days
2021-07-07 20:15:12 +02:00
Ryan Moeller
53b438b242 zfsd: Check for error from zpool_vdev_online
Onlining a vdev can fail. Log the error if it does.

Reviewed by:	mav, asomers
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D30882
2021-07-06 16:00:18 +00:00