1
0
mirror of https://git.FreeBSD.org/src.git synced 2024-12-02 08:42:48 +00:00
Commit Graph

9077 Commits

Author SHA1 Message Date
Konstantin Belousov
e704f88f3d amd64: initialize APs kpmap_store in init_secondary()
The APs pcpu area is zeroed in init_secondary() by pcpu_init(), so the
early initialization in pmap_bootstrap() is nop.

Fixes:	42f722e721cd010ae5759a4b0d3b7b93c2b9cad2ESC
Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D39945
2023-05-04 18:39:22 +03:00
Konstantin Belousov
42f722e721 amd64: store pcids pmap data in pcpu zone
This change eliminates the struct pmap_pcid array embedded into struct
pmap and sized by MAXCPU, which would bloat with MAXCPU increase.  Also
it removes false sharing of cache lines, since the array elements are
mostly locally accessed by corresponding CPUs.

Suggested by:	mjg
Reviewed by:	markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D39890
2023-05-02 14:32:47 +03:00
Konstantin Belousov
9c8cbf3819 amd64 pmap_pcid_alloc(): pass a pointer to struct pmap_pcid instead of cpuid
Cpuid is used to index the pmap->pm_pcids array only.

Reviewed by:	markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D39890
2023-05-02 14:32:40 +03:00
Konstantin Belousov
9e0143694a amd64: add pmap_get_pcid() helper
Reviewed by:	markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D39890
2023-05-02 14:32:35 +03:00
Konstantin Belousov
86b61ccb34 amd64 pmap: add pmap_pinit_pcids() helper
to initialize pm_pcids array for a new user pmap

Reviewed by:	markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D39890
2023-05-02 14:32:29 +03:00
Konstantin Belousov
32bb28d8ad amd64: move definition of the struct pmap_pcids into _pmap.h
and rename the structure to pmap_pcid.

Reviewed by:	markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D39890
2023-05-02 14:32:20 +03:00
Dmitry Chagin
80d8a4a003 linux(4): Make struct stat64 to match Linux actual one 2023-04-28 11:55:04 +03:00
Dmitry Chagin
cd0fca82bb linux(4): Regen for mknod syscall changes 2023-04-28 11:55:04 +03:00
Dmitry Chagin
ca3333dd4a linux(4): Use Linux dev_t type for mknod syscalls dev argument
As of version 2.6.0 of the Linux kernel, dev_t is a 32-bit unsigned integer
on all platforms. Prior the 2.6 kernel dev_t type was an unsigned short.
However, since the firs commit of the Linuxulator, mknod syscall get int dev
argument.
Also, there is some confusion here, while the kernel declares a dev_t type
as a 32-bit sized, the user-space dev_t type can be size of 64 bits, e.g.,
in the Glibc library.
To avoid confusion and to help porting of the Linuxulator to other platforms
use explicit l_dev_t for dev argument of mknod syscalls.
2023-04-28 11:55:02 +03:00
Dmitry Chagin
19973638be linux(4): Move dev_t type declaration under /compat/linux
As of version 2.6.0 of the Linux kernel, dev_t is a 32-bit unsigned integer
on all platforms. Move it into the MI linux.h under /compat/linux.
2023-04-28 11:55:02 +03:00
Dmitry Chagin
e0bfe0d62c linux(4): Make struct newstat to match actual Linux one
In the struct stat the st_dev, st_rdev are unsigned long.
2023-04-28 11:55:01 +03:00
Dmitry Chagin
023e688496 linux(4): Regen for struct l_old_stat changes 2023-04-28 11:55:01 +03:00
Dmitry Chagin
2370c7321f linux(4): Update syscalls.master to reflect struct l_old_stat 2023-04-28 11:54:59 +03:00
Dmitry Chagin
391fd1e1a1 linux(4): Mark old fstat syscal as unimplemented
It looks like the old fstat system call never been implemented.
2023-04-28 11:54:59 +03:00
Dmitry Chagin
a408fc097f linux(4): Rename obsolete old struct l_stat to struct l_old_stat 2023-04-28 11:54:59 +03:00
Mark Johnston
74ac712f72 vmm: Dynamically allocate a couple of per-CPU state save areas
This avoids bloating the BSS when MAXCPU is large.

No functional change intended.

PR:		269572
Reviewed by:	corvink, rew
Tested by:	rew
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D39805
2023-04-26 10:08:42 -04:00
Vitaliy Gusev
0912408a28
vmm: fix HLT loop while vcpu has requested virtual interrupts
This fixes the detection of pending interrupts when pirval is 0 and the
pending bit is set

More information how this situation occurs, can be found here:
c5b5f2d808/sys/amd64/vmm/intel/vmx.c (L4016-L4031)

Reviewed by:		corvink, markj
Fixes:			02cc877968bbcd57695035c67114a67427f54549 ("Recognize a pending virtual interrupt while emulating the halt instruction.")
MFC after:		1 week
Sponsored by:		vStack
Differential Revision:	https://reviews.freebsd.org/D39620
2023-04-26 10:38:46 +02:00
Mateusz Guzik
95e4f5ef7c x86: whack pmspcv from GENERIC
The driver is enormous and rarely used.

      text      data       bss        dec         hex   filename
  23076646   1870505   4415872   29363023   0x1c00b4f   kernel.before
  20017433   1870305   4416000   26303738   0x1915cfa   kernel.after

People using the driver will need to add pmspcv_load="YES" to
their loader.conf.

Reviewed by:	jhb
Relnotes:	yes
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D39816
2023-04-25 18:09:44 +00:00
Mark Johnston
47cf1b37f4 vmm: Expose some more AVX512 CPUID bits to guests
This is required to announce support for some accelerated AES
operations.  AVX512BW indicates support for the AVX512-FP16 extension
and AVX512VL indicates support for the use of AVX512 instructions with
vector lengths smaller than 512 bits.

VAES and VPCLMULQDQ extensions indicate that VEX-prefixed AES-NI and
pclmulqdq instructions are supported.

All of these bits are needed for OpenSSL to use VAES to accelerate
AES-GCM transforms.

Reviewed by:	corvink, kib, jhb
MFC after:	2 weeks
Sponsored by:	Stormshield
Sponsored by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D39781
2023-04-25 13:35:14 -04:00
Dmitry Chagin
56c5230afd linux(4): Fix LINUX_AT_COUNT comments
Differential Revision:	https://reviews.freebsd.org/D39645
MFC after:		1 month
2023-04-22 22:16:43 +03:00
Dmitry Chagin
7d8c983983 linux(4): Deduplicate linux_copyout_auxargs()
Export default MINSIGSTKSZ value for the x86 until we do not preserve AVX
registers in the signal context.

Differential Revision:	https://reviews.freebsd.org/D39644
MFC after:		1 month
2023-04-22 22:16:02 +03:00
Warner Losh
559b94a122 syscall.master: Fix comments
Have more accruate comments. While #if, #else, etc are copied to the
header files, lines that don't start with # are not.  And #include files
are only output to sysinc (which winds up at the front of init_sysent.c
which seems a bit odd). This is all radically undocumented, and likely
has drifted somewhat from 4.4BSD and what other systems do (they've
drifted too, fwiw).

Sponsored by:		Netflix
2023-04-20 16:18:02 -06:00
Dmitry Chagin
de4da6cd04 x86: Move i386 timerreg.h to x86
Reviewed by:		emaste, jhb
Differential Revision:	https://reviews.freebsd.org/D39656
MFC after:		1 month
2023-04-20 19:42:59 +03:00
Dmitry Chagin
d1f4c44aa8 x86: Move i386 ppireg.h to x86
Differential Revision:	https://reviews.freebsd.org/D39655
MFC after:		1 month
2023-04-20 19:42:59 +03:00
Konstantin Belousov
617a11eab6 x86: initialize use_xsave once
The explanation from https://reviews.freebsd.org/D39637 by stevek:
The "use_xsave" variable is a global and that is only supposed to be
initialized early before scheduling gets started. However, with the way
the ifuncs for "fpusave" and "fpurestore" are implemented, the value
could be changed at runtime when scheduling is active if "use_xsave"
was set to 0 by the tunable. This leaves a window of opportunity where
"use_xsave" gets re-initialized to 1 and a context switch could occur
with a thread that was not set up to be able to use xsave functionality.
This can lead to an "privileged instruction fault".

The fix is to protect "use_xsave" from being initialized more than once.

Reported and reviewed by:	stevek
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D39660
2023-04-19 02:22:28 +03:00
Konstantin Belousov
1e0e335b0f amd64: fix PKRU and swapout interaction
When vm_map_remove() is called from vm_swapout_map_deactivate_pages()
due to swapout, PKRU attributes for the removed range must be kept
intact.  Provide a variant of pmap_remove(), pmap_map_delete(), to
allow pmap to distinguish between real removes of the UVA mappings
and any other internal removes, e.g. swapout.

For non-amd64, pmap_map_delete() is stubbed by define to pmap_remove().

Reported by:	andrew
Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D39556
2023-04-15 02:53:59 +03:00
Julien Grall
ab7ce14b1d xen/intr: introduce dev/xen/bus/intr-internal.h
Move the xenisrc structure which needs to be shared between the core Xen
interrupt code and architecture-dependent code into a separate header.  A
similar situation exists for the NR_EVENT_CHANNELS constant.

Turn xi_intsrc into a type definition named xi_arch to reflect the new
purpose of being an architectural variable for the interrupt source.

This was originally implemented by Julien Grall, but has been heavily
modified.  The core side was renamed "intr-internal.h" and is #include'd
by "arch-intr.h" instead of the other way around.  This allows the
architecture to add function definitions which use struct xenisrc.

The original version only moved xi_intsrc into xen_arch_isrc_t.  Moving
xi_vector was done by the submitter.

The submitter had also moved xi_activehi and xi_edgetrigger into
xen_arch_isrc_t.  Those disappeared with the removal of PVHv1 support.

Copyright note.  The current xenisrc structure was introduced at
76acc41fb7 by Justin T. Gibbs.  Traces remain, but the strength of
Copyright claims from before 2013 seem pretty weak.

Reviewed by: royger
Submitted by: Elliott Mitchell <ehem+freebsd@m5p.com>, 2021-03-17 19:09:01
Original implementation: Julien Grall <julien@xen.org>, 2015-10-20 09:14:56
Differential Revision: https://reviews.freebsd.org/D30648
[royger]
 - Adjust some line lengths
 - Fix comment about NR_EVENT_CHANNELS after movement.
 - Use #include instead of symlinks.
2023-04-14 15:58:53 +02:00
Elliott Mitchell
af610cabf1 xen/intr: adjust xen_intr_handle_upcall() to match driver filter
xen_intr_handle_upcall() has two interfaces.  It needs to be called by
the x86 assembly code invoked by the APIC.  Second, it needs to be called
as a driver_filter_t for the XenPCI code and for architectures besides
x86.

Unfortunately the driver_filter_t interface was implemented as a wrapper
around the x86-APIC interface.  Now create a simple wrapper for the
x86-APIC code, which calls an architecture-independent
xen_intr_handle_upcall().

When called via intr_event_handle(), driver_filter_t functions expect
preemption to be disabled.  This removes the need for
critical_enter()/critical_exit() when called this way.

The lapic_eoi() call is only needed on x86 in some cases when invoked
directly as an APIC vector handler.

Additionally driver_filter_t functions have no need to handle interrupt
counters.  The intrcnt_add() calling function was reworked to match the
current situation.  intrcnt_add() is now only called via one path.

The increment/decrement of curthread->td_intr_nesting_level had
previously been left out.  Appears this was mostly harmless, but this
was noticed during implementation and has been added.

CONFIG_X86 is a leftover from use with Linux.  While the barrier isn't
needed for FreeBSD on x86, it will be needed for FreeBSD on other
architectures.

Copyright note.  xen_intr_intrcnt_add() was introduced at 76acc41fb7
by Justin T. Gibbs.  xen_intrcnt_init() was introduced at fd036deac1
by John Baldwin.

sys/x86/xen/xen_arch_intr.c was originally created by Julien Grall in
2015 for the purpose of holding the x86 interrupt interface.  Later it
was found xen_intr_handle_upcall() was better earlier, and the x86
interrupt interface better later.  As such the filename and header list
belong to Julien Grall, but what those were created for is later.

Reviewed by: royger
Differential Revision: https://reviews.freebsd.org/D30006
2023-04-14 15:58:52 +02:00
Elliott Mitchell
ecdcad6516 xen: remove CONFIG_XEN_COMPAT, purge Xen 3.0 compatibility
This overlaps the purpose of __XEN_INTERFACE_VERSION__.  Remove Xen 3.0.2
compatibility.  __XEN_INTERFACE_VERSION__ has compatibility to Xen 3.2.8
enabled.  As Xen 3.3 was released almost 15 years ago, it seems unlikely
anyone hasn't updated.

Reviewed by: royger
2023-04-14 15:58:48 +02:00
Elliott Mitchell
b2c50bb934 xen/efi: make Xen PV EFI clock optional
The present implementation is only for x86.  Other architectures need
adjustments for querying presence of EFI.

Xen's EFI support is also quite troublesome on non-x86.  This is being
slowly remedied, but until in better shape the EFI clock functionality
should be disabled.

Reviewed by: royger
Differential Revision: https://reviews.freebsd.org/D31065
2023-04-14 15:58:47 +02:00
Henri Hennebert
71883128e5 rtsx: Add plug-and-play info
Add MODULE_PNP_INFO() to the driver to make it autoload if not linked
statically into the kernel. Remove the device from amd64/i386 GENERIC.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D35074
2023-04-13 11:12:50 -03:00
Dmitry Chagin
50111714f5 linux(4): Regen for close_range syscall
MFC after:		2 weeks
2023-04-04 23:23:37 +03:00
Dmitry Chagin
1c27dce1f8 linux(4): Modify close_range syscall to match Linux
MFC after:		2 weeks
2023-04-04 23:23:24 +03:00
Alexander V. Chernikov
3091d980f5 netlink: add NETLINK to the DEFAULTS for each architecture
NETLINK is going to replace rtsock and a number of other ioctl/sysctl interfaces.
In-base utilies such as route(8), netstat(8) and soon ifconfig(8)
 are being converted to use netlink sockets as a transport between
 kernel and userland.
In the current configuration, it still possible have the kernel
 without NETLINK (`nooptions NETLINK`) and use the aforementioned
 utilies by buidling the world with `WITHOUT_NETLINK` src.conf knob.
However, this approach does not cover the cases when person unintentionally
 builds a custom kernel without netlink and tries to use the standard userland.

This change adds `option NETLINK` to the default options for each
 architecture, fixing the custom kernel issue.
For arm, this change uses `std.armv6` and `std.armv7` (netlink already in)
 instead of DEFAULTS.

Reviewed By: imp
Differential Revision: https://reviews.freebsd.org/D39339
2023-04-02 15:27:21 +00:00
Konstantin Belousov
cd137909c3 amd64 wakeup: recalculate mitigations after APICs are woken
APICs are needed to broadcast IPIs for MSR writes.

PR:	270489
Reviewed by:	dchagin, emaste, jhb
Tested by:	dchagin, manu
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D39302
2023-03-29 21:45:20 +03:00
Elliott Mitchell
9f3be3a6ec xen: switch to using core atomics for synchronization
Now that the atomic macros are always genuinely atomic on x86, they can
be used for synchronization with Xen.  A single core VM isn't too
unusual, but actual single core hardware is uncommon.

Replace an open-coding of evtchn_clear_port() with the inline.

Substantially inspired by work done by Julien Grall <julien@xen.org>,
2014-01-13 17:40:58.

Reviewed by: royger
MFC after: 1 week
2023-03-29 09:51:42 +02:00
John Baldwin
0f735657aa bhyve: Remove vmctx member from struct vm_snapshot_meta.
This is a userland-only pointer that isn't relevant to the kernel and
doesn't belong in the ioctl structure shared between userland and the
kernel.  For the kernel, the old structure for the ioctl is still
supported under COMPAT_FREEBSD13.

This changes vm_snapshot_req() in libvmmapi to accept an explicit
vmctx argument.

It also changes vm_snapshot_guest2host_addr to take an explicit vmctx
argument.  As part of this change, move the declaration for this
function and its wrapper macro from vmm_snapshot.h to snapshot.h as it
is a userland-only API.

Reviewed by:	corvink, markj
Differential Revision:	https://reviews.freebsd.org/D38125
2023-03-24 11:49:06 -07:00
John Baldwin
7d9ef309bd libvmmapi: Add a struct vcpu and use it in most APIs.
This replaces the 'struct vm, int vcpuid' tuple passed to most API
calls and is similar to the changes recently made in vmm(4) in the
kernel.

struct vcpu is an opaque type managed by libvmmapi.  For now it stores
a pointer to the VM context and an integer id.

As an immediate effect this removes the divergence between the kernel
and userland for the instruction emulation code introduced by the
recent vmm(4) changes.

Since this is a major change to the vmmapi API, bump VMMAPI_VERSION to
0x200 (2.0) and the shared library major version.

While here (and since the major version is bumped), remove unused
vcpu argument from vm_setup_pptdev_msi*().

Add new functions vm_suspend_all_cpus() and vm_resume_all_cpus() for
use by the debug server.  The underyling ioctl (which uses a vcpuid of
-1) remains unchanged, but the userlevel API now uses separate
functions for global CPU suspend/resume.

Reviewed by:	corvink, markj
Differential Revision:	https://reviews.freebsd.org/D38124
2023-03-24 11:49:06 -07:00
Konstantin Belousov
2b4b3789f8 acpi_wakeup.c: apply the reviewer' editorial corrections to the comment text.
Fixes:	02904a06c7
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D39146
2023-03-18 17:47:19 +02:00
Konstantin Belousov
02904a06c7 amd64: properly recalculate mitigations knobs after resume
Revision r333125 AKA 986c4ca387 forced clear cpu_stdext_feature3
on suspend, since at that time microcode update was not reloaded
early on resume. Then, revision 050f5a8405 started re-reading
cpu_stdext_feature3 again. Since modern CPUs do not require mitigations
from the Skylake era, this went unnoticed for some time.

Keep zeroing cpu_stdext_feature3 on suspend, but re-read it in more
controlled way on resume after microcode is reloaded, and recalculate
active workarounds based on actual microcode capabilities.

Reported and tested by:	romain
Reviewed by:	emaste, markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D39146
2023-03-18 17:40:05 +02:00
Konstantin Belousov
ff6d60946a amd64 acpi_wakeup.c: fix typo
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2023-03-17 15:10:34 +02:00
Vitaliy Gusev
94a3876d7e
vmm: fix missing ipi statistic
ipi counters are missing in bhyvectl's output because vm_maxcpu is 0
when initializing them. That's because vmm_stat_register is executed
before vmm_init.

Instead of directly fixing it, there's a better solution in illumos
which is cherry picked:
65a3bc8373

It replaces the matrix statistic by two counters per vcpu. One for
counting the ipis to the vcpu and one counting the ipis received by the
vcpu. This has several advantages:

- A matrix statistic becomes huge when using many vcpus.
- A matrix statistic easily reaches the MAX_VMM_STAT_ELEMS limit.
- Two counters are enough in most cases. DTrace can be used for more
  advanced debugging purposes.
- A matrix statistic wastes memory. The matrix size is determined by
  vm_maxcpu regardless of the number of vcpus assigned to the vm.

Reviewed by:		corvink, markj
Fixes:			ee98f99d7a ("vmm: Convert VM_MAXCPU into a loader tunable hw.vmm.maxcpu.")
MFC after:		1 week
Sponsored by:		vStack
Differential Revision:	https://reviews.freebsd.org/D39038
2023-03-17 13:50:08 +01:00
Dmitry Chagin
9e7f03e9c6 linux(4): Drop unncessary struct l_ifconf declaration from amd64/linux
Its needed only for amd64/linux32 Linuxulator.

Differential Revision:	https://reviews.freebsd.org/D38793
2023-03-04 12:11:38 +03:00
Dmitry Chagin
cabbfb60d0 linux(4): Reduce code duplication between MD files
Move struct ifnet definitions under compat/linux.

Reviewed by:		emaste
Differential Revision:	https://reviews.freebsd.org/D38791
2023-03-04 12:11:38 +03:00
John-Mark Gurney
2fee875629
abstract out the vm detection via smbios..
This makes the detection of VMs common between platforms that
have SMBios.

Reviewed by:		imp, kib
Differential Revision:	https://reviews.freebsd.org/D38800
2023-03-02 16:54:21 -08:00
Vitaliy Gusev
8104fc31a2
bhyve: fix restore of kernel structs
vmx_snapshot() and svm_snapshot() do not save any data and error occurs at
resume:

Restoring kernel structs...
vm_restore_kern_struct: Kernel struct size was 0 for: vmx
Failed to restore kernel structs.

Reviewed by:		corvink, markj
Fixes:			39ec056e6d ("vmm: Rework snapshotting of CPU-specific per-vCPU data.")
MFC after:		2 weeks
Sponsored by:		vStack
Differential Revision:	https://reviews.freebsd.org/D38476
2023-02-28 13:37:53 +01:00
Vitaliy Gusev
281b496f22
vmm: fix restore of TSC offset
After suspend/resume Ubuntu 20.04 and 22.04 installer can hang if
tsc-early clocksource has a big skew.

Reviewed by:		corvink, jhb
Fixes:			a7db532e3a ("vmm: Simplify saving of absolute TSC values in snapshots.")
MFC after:		2 weeks
Sponsored by:		vStack
Differential Revision:	https://reviews.freebsd.org/D38474
2023-02-28 13:37:44 +01:00
Mike Karels
dd6f6030cc amd64 kernel config: clean up whitespace
Most options in kernel config files use "options<space><tab>OPTION".
This allows the option to be commented out without shifting columns.
A few options had two tabs, and some had spaces.  Make them consistent.
2023-02-24 08:36:28 -06:00
Mateusz Guzik
6b9acd1bfb Exclude MMCCAM kernels from make universe
They don't provide any value and are quite arbitrary.

Note arm64 GENERIC-MMCCAM was already excluded, just not the NODEBUG
variant.

The option is already build-tested with arm64 LINT kernel.

Reviewed by:	manu
Differential Revision:	https://reviews.freebsd.org/D38458
2023-02-16 07:29:53 +00:00
Dmitry Chagin
c8a79231a5 linux(4): Rename linux_timer.h to linux_time.h
To avoid confusing people, rename linux_timer.h to linux_time.h,
as linux_timer.c is the implementation of timer syscalls only,
while linux_time.c contains implementation of all stuff declared
in linux_time.h.

MFC after:		2 weeks
2023-02-14 17:46:33 +03:00
Dmitry Chagin
2456a45929 linux(4): Cleanup includes under amd64/linux
Cleanup unneeded includes, sort the rest according to style(9).
No functional changes.

MFC after:		2 weeks
2023-02-14 17:46:32 +03:00
Dmitry Chagin
31e938c531 linux(4): Cleanup vm includes from linux_util.h
Include vm headers directly where they needed. The linux_util.h included
in a most source files of the Linuxulator, avoid collecting a rarely used
includes here.

MFC after:		2 weeks
2023-02-14 17:46:30 +03:00
Dmitry Chagin
06c07e1203 Complete removal of opt_compat.h
Since Linux emulation layer build options was removed there is no reason
to keep opt_compat.h.

Reviewed by:		emaste
Differential Revision:	https://reviews.freebsd.org/D38548
MFC after:		2 weeks
2023-02-13 19:07:38 +03:00
Dmitry Chagin
10d16789a3 linux(4): Get rid of the opt_compat.h include.
Since e013e369 COMPAT_LINUX, COMPAT_LINUX32 build options are removed,
so include of opt_compat.h is no more needed.

MFC after:		2 weeks
2023-02-12 20:24:32 +03:00
Mark Johnston
b265a2e0d7 vmm: Fix AP startup compatibility for old bhyve executables
These changes unbreak AP startup when using a 13.1-RELEASE bhyve
executable with a newer kernel:
- Correct the destination mask for the VM_EXITCODE_IPI message generated
  by an INIT or STARTUP IPI in vlapic_icrlo_write_handler().
- Only initialize vlapics on active vCPUs.  13.1-RELEASE bhyve activates
  AP vCPUs only after the BSP starts them with an IPI, and vmm now
  allocates vcpu structures lazily, so the STARTUP handling in
  vm_handle_ipi() could trigger a page fault.
- Fix an off-by-one setting the vcpuid in a VM_EXITCODE_SPINUP_AP
  message.

Fixes:	7c326ab5bb ("vmm: don't lock a mtx in the icr_low write handler")
Reviewed by:	jhb, corvink
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D38446
2023-02-09 16:14:33 -05:00
Mark Johnston
ba34de1b3b vmm: Remove an unneeded initialization of "retu"
vm_handle_ipi() unconditionally initializes "retu".  No functional
change intended.

Reviewed by:	jhb, corvink
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D38446
2023-02-09 16:14:33 -05:00
Mark Johnston
f3bbd0e818 vmm: Collapse identical case statements in vlapic_icrlo_write_handler()
No functional change intended.

Reviewed by:	jhb, corvink
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D38446
2023-02-09 16:14:33 -05:00
Dag-Erling Smørgrav
43d4680b39 MINIMAL: Update and clean up.
* Add GEOM_LABEL, required to boot a default UEFI install.

* Add enough of virtio to boot in bhyve.

* Reduce diff between amd64 and i386.

* Reduce diff to GENERIC.

MFC after:	1 week
Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D38468
2023-02-09 18:24:45 +01:00
Konstantin Belousov
ee84487120 amd64 ia32 vdso: always define some __vdso_ symbols
... regardless of the kernel config options.
It is reported that llvm16 ld.lld warns about undefined symbols
referenced by the VERSION script.

Reviewed by:	emaste, val_packett.cool
Discussed with:	jrtc27
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D38392
2023-02-09 04:36:40 +02:00
Mateusz Guzik
b2c68dc6d9 amd64: ansify
Reported by:    clang 15
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2023-02-07 22:52:06 +00:00
Mateusz Guzik
819ed47204 amd64 pmap: patch up a comment in pmap_init_pv_table
Requested by:	jhb
2023-02-06 22:33:28 +00:00
Yuri
e4d3f1e40a hv_hid: Hyper-V HID driver
Hyper-V HID driver using hidbus/hms.

Reviewed by:	wulf
MFC after:	1 week
PR:		221074
Differential revision:	https://reviews.freebsd.org/D38140
2023-02-05 18:32:08 +03:00
Elliott Mitchell
d27d543c78 vmm: purge EOL release compatibility
Remove FreeBSD 11 support

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/603
Differential Revision: https://reviews.freebsd.org/D35560
2023-02-04 09:13:10 -07:00
Dmitry Chagin
ce20c00e85 linux(4): Remove stale comment that no longer applies.
MFC after:		1 week
2023-02-02 20:21:37 +03:00
Dmitry Chagin
6ad07a4b2b linux(4): Microoptimize rt_sendsig() on amd64.
Drop proc lock earlier, before copying user stuff.

Pointed out by:		kib
Reviewed by:		kib
Differential Revision:	https://reviews.freebsd.org/D38326
MFC after:		1 week
2023-02-02 20:21:37 +03:00
Dmitry Chagin
a95cb95e12 linux(4): Preserve fpu fxsave state across signal delivery on amd64.
PR:			240768
Reviewed by:		kib
Differential Revision:	https://reviews.freebsd.org/D38302
MFC after:		1 week
2023-02-02 20:21:37 +03:00
Dmitry Chagin
95b8603427 linux(4): Deduplicate linux_trans_osrel().
MFC after:		1 week
2023-02-02 17:58:07 +03:00
Dmitry Chagin
6039e966ff linux(4): Deduplicate linux_copyout_strings().
It is still present in the 32-bit Linuxulator on amd64.

MFC after:		1 week
2023-02-02 17:58:07 +03:00
Dmitry Chagin
9e550625f8 linux(4): Deduplicate linux_fixup_elf().
Use native routines to fixup initial process stack. On Arm64 linux_elf_fixup() is
noop, as it do the stack fixup (room for argc) in the linux_copyout_strings().

MFC after:		1 week
2023-02-02 17:58:07 +03:00
Dmitry Chagin
7446514533 linux(4): Microoptimize linux_elf.h for future use.
In order to reduce code duplication move coredump support definitions
into the appropriate header and hide private definitions.

MFC after:		1 week
2023-02-02 17:58:06 +03:00
Konstantin Belousov
2555f175b3 Move kstack_contains() and GET_STACK_USAGE() to MD machine/stack.h
Reviewed by:	jhb
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D38320
2023-02-02 00:59:26 +02:00
Dmitry Chagin
575e48f1c4 linux(4): Deduplicate MI futex structures.
MFC after:	1 week
2023-02-01 21:57:04 +03:00
Dmitry Chagin
5c32146723 amd64: Eliminate write only cpu_fxsr.
Reviewed by:		kib
Differential Revision:	https://reviews.freebsd.org/D38289
MFC after:		1 week
2023-02-01 18:17:06 +03:00
Corvin Köhne
892feec221
vmm: avoid spurious rendezvous
A vcpu only checks if a rendezvous is in progress or not to decide if it
should handle a rendezvous. This could lead to spurios rendezvous where
a vcpu tries a handle a rendezvous it isn't part of. This situation is
properly handled by vm_handle_rendezvous but it could potentially
degrade the performance. Avoid that by an early check if the vcpu is
part of the rendezvous or not.

At the moment, rendezvous are only used to spin up application
processors and to send ioapic interrupts. Spinning up application
processors is done in the guest boot phase by sending INIT SIPI
sequences to single vcpus. This is known to cause spurious rendezvous
and only occurs in the boot phase. Sending ioapic interrupts is rare
because modern guest will use msi and the rendezvous is always send to
all vcpus.

Reviewed by:		jhb
MFC after:		1 week
Sponsored by:		Beckhoff Automation GmbH & Co. KG
Differential Revision:	https://reviews.freebsd.org/D37390
2023-02-01 12:36:36 +01:00
Eric Joyner
5354596764
vtd: Increase DRHD_MAX_UNITS
Observed on a couple Ice Lake-SP platforms (Intel Coyote Pass, Dell
R750), there are more than 8 DRHD sections enumerated in the DMAR ACPI
section.  Since the previous limit was 8, this resulted in some of these
not being parsed by vtd when the iommu is initialized; in this case when
PCI devices are being passthru'd to a bhyve VM.

This omission later causes a kernel panic later in initialization when
devices could not be found in a valid DRHD scope because the DHRD
containing the device's scope was not added to vtd.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

PR:		268486
Sponsored by:	Intel Corporation
Reviewed by:	rew@, corvink@
MFC after:	1 day
Differential Revision:	https://reviews.freebsd.org/D38285
2023-01-31 13:57:42 -08:00
Konstantin Belousov
153643a5bc amd64: do not enable PKRU if user disabled saving PKRU register in xsave mask
This is done by reverting CR4_PKE bit, because we perform %CR4
initialization in initializecpu(), and the function is called before
xsave_mask is read.  To not redo the whole early initialization
sequence for the corner case, this should be good enough.

Reported by:	jhb
Reviewed by:	jhb, markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D38219
2023-01-27 19:44:49 +02:00
Andrew Gallatin
9cb6ba29cb vm: centralize VM_BATCHQUEUE_SIZE definition
Remove the platform-specific definitions of VM_BATCHQUEUE_SIZE
for amd64 and powerpc64, and instead treat all 64-bit platforms
identically.  This has the effect of increasing the arm64
and riscv VM_BATCHQUEUE_SIZE to match that of other platforms.

Reviewed by: jhb, markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D37707
2023-01-21 14:30:00 -05:00
Robert Wing
27029bc08f vmm: fix use after free in ppt_detach()
The vmm module destroys the host_domain before unloading the ppt module
causing a use after free. This can happen when kldunload'ing vmm.

Reviewed by:	markj, jhb
Differential Revision:	https://reviews.freebsd.org/D38072
2023-01-20 11:25:27 +00:00
Robert Wing
c668e8173a vmm: take exclusive mem_segs_lock in vm_cleanup()
The consumers of vm_cleanup() are vm_reinit() and vm_destroy().

The vm_reinit() call path is, here vmmdev_ioctl() takes mem_segs_lock:
    vmmdev_ioctl()
    vm_reinit()
    vm_cleanup(destroy=false)

The call path for vm_destroy() is (mem_segs_lock not taken):
    sysctl_vmm_destroy()
    vmmdev_destroy()
    vm_destroy()
    vm_cleanup(destroy=true)

Fix this by taking mem_segs_lock in vm_cleanup() when destroy == true.

Reviewed by:	corvink, markj, jhb
Fixes:  67b69e76e8 ("vmm: Use an sx lock to protect the memory map.")
Differential Revision:	https://reviews.freebsd.org/D38071
2023-01-20 11:10:53 +00:00
Robert Wing
ccf32a68f8 vmm: take exclusive mem_segs_lock when (un)assigning ppt dev
PR:             268744
Reported by:    mmatalka@gmail.com
Reviewed by:	corvink, markj, jhb
Fixes:  67b69e76e8 ("vmm: Use an sx lock to protect the memory map.")
Differential Revision:	https://reviews.freebsd.org/D37962
2023-01-20 10:03:59 +00:00
Gordon Bergling
05187f2ffc amd64: Fix a common typo in source code comments
- s/comparision/comparison/

MFC after:	3 days
2023-01-19 14:27:18 +01:00
Alexander V. Chernikov
692e19cf51 netlink: add netlink to GENERIC@amd64
Netlink is a communication protocol defined in RFC 3549. It is async,
TLV-based protocol, providing 1-1 and 1-many communications between kernel
and userland. Netlink is currently used in Linux kernel to modify, read and
subscribe for nearly all networking states. Interface state, addresses, routes,
firewall, rules, fibs, etc, are controlled via Netlink.

Netlink support was added in D36002. It has got a number of improvements and
first customers since then:
* net/bird2 got netlink support, enabling route multipath in FreeBSD
* netlink-based devd notifications are being worked on ( D37574 ).
* linux(4) fully supports and depends on Netlink

Enabling Netlink in GENERIC targets two goals.
The first one is to provide stability for the third-party userland applications,
so they can rely on the fact that netlink always exists since 14.0 and potentially 13.2.
Loadable module makes life of the app delepers harder. For example, `net/bird2` can be
either build with netlink or rtsock support, but not both.

The second goal is to enable gradual conversion of the base userland tools
to use netlink(4) interfaces. Converting tools like netstat (D36529), route,
ifconfig one-by-one simplifies testing and addressing the feedback.
Othewise, switching all base to use netlink at once may be too big of a leap.

This change targets amd64, the other architectures will follow soon.

Differential Revision: https://reviews.freebsd.org/D37783
2023-01-13 10:22:40 +00:00
Konstantin Belousov
ad97b9bbfc amd64 pmap.h: make it easier to use the header for other consumers
Guard pmap_invlpg() definition with checks that only provide it when
both sys/pcpu.h and machine/cpufunc.h were already included.

Requested by:	Elliott Mitchell
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2023-01-06 01:30:29 +02:00
Konstantin Belousov
a2c08eba43 amd64: be more precise when enabling the AlderLake small core PCID workaround
In particular, do not enable the workaround if INVPCID is not supported
by the core.

Reported by:	"Chen, Alvin W" <Weike.Chen@Dell.com>
Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D37940
2023-01-06 01:30:29 +02:00
Konstantin Belousov
231d75568f Move INVLPG to pmap_quick_enter_page() from pmap_quick_remove_page().
If processor prefetches neighboring TLB entries to the one being accessed
(as some have been reported to do), then the spin lock does not prevent
the situation described in the "AMD64 Architecture Programmer's Manual
Volume 2: System Programming" rev. 3.23, "7.3.1 Special Coherency
Considerations".

Reported and reviewed by:	alc
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D37770
2023-01-01 00:09:46 +02:00
Konstantin Belousov
cde70e312c amd64: for small cores, use (big hammer) INVPCID_CTXGLOB instead of INVLPG
A hypothetical CPU bug makes invalidation of global PTEs using INVLPG
in pcid mode unreliable, it seems.  The workaround is applied for all
CPUs with small cores, since we do not know the scope of the issue, and
the right fix.

Reviewed by:	alc (previous version)
Discussed with:	emaste, markj
Tested by:	karels
PR:	261169, 266145
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D37770
2023-01-01 00:09:45 +02:00
Konstantin Belousov
45ac7755a7 amd64: identify small cores
Reviewed by:	alc
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D37770
2023-01-01 00:09:45 +02:00
Andrew Gallatin
1cac76c93f vm: reduce lock contention when processing vm batchqueues
Rather than waiting until the batchqueue is full to acquire the lock &
process the queue, we now start trying to acquire the lock using trylocks
when the batchqueue is 1/2 full. This removes almost all contention on the
vm pagequeue mutex for for our busy sendfile() based web workload.
It also greadly reduces the amount of time a network driver ithread
remains blocked on a mutex, and eliminates some packet drops under
heavy load.

So that the system does not loose the benefit of processing large
batchqueues, I've doubled the size of the batchqueues. This way, when
there is no contention, we process the same batch size as before.

This has been run for several months on a busy Netflix server, as well
as on my personal desktop.

Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D37305
2022-12-14 14:34:07 -05:00
Alan Cox
f0878da03b pmap: standardize promotion conditions between amd64 and arm64
On amd64, don't abort promotion due to a missing accessed bit in a
mapping before possibly write protecting that mapping.  Previously,
in some cases, we might not repromote after madvise(MADV_FREE) because
there was no write fault to trigger the repromotion.  Conversely, on
arm64, don't pointlessly, yet harmlessly, write protect physical pages
that aren't part of the physical superpage.

Don't count aborted promotions due to explicit promotion prohibition
(arm64) or hardware errata (amd64) as ordinary promotion failures.

Reviewed by:	kib, markj
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D36916
2022-12-12 11:32:50 -06:00
John Baldwin
af3b48e101 vmm: Free vCPUs when destroying them.
Reported by:	andrew
Reviewed by:	corvink, andrew, markj
Differential Revision:	https://reviews.freebsd.org/D37649
2022-12-09 10:27:05 -08:00
John Baldwin
d212d6ebb4 vmm: Avoid infinite loop in vcpu_lock_all error case.
Reported by:	Coverity (CIDs 1501060,1501071)
Reviewed by:	corvink, markj, emaste
Differential Revision:	https://reviews.freebsd.org/D37648
2022-12-09 10:26:49 -08:00
John Baldwin
91980db1be vmm: Don't lock a vCPU for VM_PPTDEV_MSI[X].
These are manipulating state in a ppt(4) device none of which is
vCPU-specific.  Mark the vcpu fields in the relevant ioctl structures
as unused, but don't remove them for now.

Reviewed by:	corvink, markj
Differential Revision:	https://reviews.freebsd.org/D37639
2022-12-09 10:26:23 -08:00
John Baldwin
62be9ffd82 vmm: VM_GET/SET_KERNEMU_DEV should run with the vCPU locked.
Reviewed by:	corvink, kib, markj
Differential Revision:	https://reviews.freebsd.org/D37638
2022-12-09 10:25:30 -08:00
John Baldwin
1f6db5d6b5 vmm: Remove stale comment for vm_rendezvous.
Support for rendezvous outside of a vcpu context (vcpuid of -1) was
removed in commit 949f0f47a4, and the vm, vcpuid argument pair was
replaced by a single struct vcpu pointer in commit d8be3d523d.

Reported by:	andrew
2022-11-30 13:06:46 -08:00
Bjoern A. Zeeb
4a8e4d1546 net80211: fix IEEE80211_DEBUG_REFCNT builds
Remove the KPI/KBI changes from ieee80211_node.h and always use the
macros to pass in __func__ and __LINE__ to the functions.
The actual implementations are prefixed by "_" rather than suffixed
by "_debug" as they no longer are "debug"-specific.

Some of the select functions were not actually using the passed in
func, line options; however they are calling other functions which
use them.  Directly call the internal implementation in those cases
passing the arguments on.

Use a file-local __debrefcnt_used define to mark the arguments __unused
in cases when we compile without IEEE80211_DEBUG_REFCNT and hope the
toolchain is intelligent enough to not pass them at all in those cases.

Also _ieee80211_free_node() now has a conflict so make the previous
_ieee80211_free_node() the new __ieee80211_free_node().

Add IEEE80211_DEBUG_REFCNT to the NOTES file on amd64 to keep exercising
the option.

Sponsored by:	The FreeBSD Foundation
X-MFC:		never
Discussed on:	freebsd-wireless
Reviewed by:	adrian
Differential Revision: https://reviews.freebsd.org/D37529
2022-11-29 21:20:37 +00:00
Corvin Köhne
7c326ab5bb
vmm: don't lock a mtx in the icr_low write handler
x2apic accesses are handled by a wrmsr exit. This handler is called in a
critical section. So, we can't lock a mtx in the icr_low handler.

Reported by:		kp, pho
Tested by:		kp, pho
Approved by:		manu (mentor)
Fixes:			c0f35dbf19 vmm: Use a cpuset_t for vCPUs waiting for STARTUP IPIs.
MFC after:		1 week
MFC with:		c0f35dbf19
Sponsored by:		Beckhoff Automation GmbH & Co. KG
Differential Revision:	https://reviews.freebsd.org/D37452
2022-11-23 09:00:04 +01:00
Corvin Köhne
fde8ce8892
vmm: remove unneccessary rendezvous assertion
When a vcpu sees that a rendezvous is in progress, it exits and tries to
handle the rendezvous. The vcpu doesn't check if it's part of the
rendezvous or not. If the vcpu isn't part of the rendezvous, the
rendezvous could be done before it reaches the assertion. This will
cause a panic.

The assertion isn't needed at all because vm_handle_rendezvous properly
handles a spurious rendezvous. So, we can just remove it.

PR:			267779
Reviewed by:		jhb, markj
Tested by:		bz
Approved by:		manu (mentor)
MFC after:		1 week
Sponsored by:		Beckhoff Automation GmbH & Co. KG
Differential Revision:	https://reviews.freebsd.org/D37417
2022-11-21 08:19:36 +01:00
Dmitry Chagin
2ee1a18d51 vmm: Fix build w/o KDTRACE_HOOKS.
Reviewed by:		imp
Differential revision:	https://reviews.freebsd.org/D37446
2022-11-20 18:00:55 +03:00
Cy Schubert
d487cba33d vmm: Fix non-INVARIANTS build
Reported by:	O. Hartmann <freebsd@walstatt-de.de>
Reviewed by:	jhb
Fixes:		58eefc67a1
Differential Revision:	https://reviews.freebsd.org/D37444
2022-11-18 13:20:13 -08:00
Mark Johnston
ca6b48f080 vmm: Restore the correct vm_inject_*() prototypes
Fixes:	80cb5d845b ("vmm: Pass vcpu instead of vm and vcpuid...")
Reviewed by:	jhb
Differential Revision:	https://reviews.freebsd.org/D37443
2022-11-18 14:11:48 -05:00