the pipe is locked and shouldn't be.
initialize pipe->pipe_mtxp to NULL when creating pipes in order not
to trip the above assertions.
swap pipe lock with giant around calls to pipe_destroy_write_buffer()
pipe_destroy_write_buffer issue noticed by: jhb
fully protect p_ucred yet so Giant is needed until all the p_ucred
locking is done. This is the original reason td_ucred was not used
immediately after its addition. Unfortunately, not using td_ucred is
not enough to avoid problems. Since p_ucred could be stale, we could
actually be dereferencing a stale pointer to dink with the refcount, so
we really need Giant to avoid foot-shooting. This allows td_ucred to
be safely used as well.
In order to determine what to page out, the vm_daemon checks
reference bits on all pages belonging to all processes. Unfortunately,
the algorithm used reacted badly with shared pages; each shared page
would be checked once per process sharing it; this caused an O(N^2)
growth of tlb invalidations. The algorithm has been changed so that
each page will be checked only 16 times.
Prior to this change, a fork/sleepbomb of 1300 processes could cause
the vm_daemon to take over 60 seconds to complete, effectively
freezing the system for that time period. With this change
in place, the vm_daemon completes in less than a second. Any system
with hundreds of processes sharing pages should benefit from this change.
Note that the vm_daemon is only run when the system is under extreme
memory pressure. It is likely that many people with loaded systems saw
no symptoms of this problem until they reached the point where swapping
began.
Special thanks go to dillon, peter, and Chuck Cranor, who helped me
get up to speed with vm internals.
PR: 33542, 20393
Reviewed by: dillon
MFC after: 1 week
in most machines of the Sun Ultra series. This is a port of the NetBSD
driver which I enhanced to make use of the gather functionality and the
configurable RX buffer offset to avoid copying all received/sent packet
(instead, packets will be directly DMAd from mbuf chains and into mbuf
clusters now).
device drivers for bus system with other endinesses than the CPU (using
interfaces compatible to NetBSD):
- bwap16() and bswap32(). These have optimized implementations on some
architectures; for those that don't, there exist generic implementations.
- macros to convert from a certain byte order to host byte order and vice
versa, using a naming scheme like le16toh(), htole16().
These are implemented using the bswap functions.
- stream bus space access functions, which do not perform a byte order
conversion (while the normal access functions would if the bus endianess
differs from the CPU endianess).
htons(), htonl(), ntohs() and ntohl() are implemented using the new
functions above for kernel usage. None of the above interfaces is currently
exported to user land.
Make use of the new functions in a few places where local implementations
of the same functionality existed.
Reviewed by: mike, bde
Tested on alpha by: mike
as arguments. The correct hostname is copied into the buffer
while having the prison's lock acquired in a jailed process'
case.
Reviewed by: jhb, rwatson
There is some unresolved badness that has been eluding me, particularly
affecting uniprocessor kernels. Turning off PG_G helped (which is a bad
sign) but didn't solve it entirely. Userland programs still crashed.
1/ conditionalise (#if 0) function that is not used.
Unused code left in place for netBSD compatibility.
2/ Recode loop to convince gcc that it does initialise a variable
(use do-while instead of for() so gcc knows that we always go through
at least once. Feel free to check my logic.
Both ends of the pipe share a pool_mutex, this makes allocation
and deadlock avoidance easy.
Remove some un-needed FILE_LOCK ops while I'm here.
There are some issues wrt to select and the f{s,g}etown code that
we'll have to deal with, I think we may also need to move the calls
to vfs_timestamp outside of the sections covered by PIPE_LOCK.
work loads. It tapers off after that as gcc's working set generally just fits.
compiling bin/csh:
TSB_PAGES = 2
213.33 real 77.59 user 110.01 sys
TSB_PAGES = 4
116.43 real 75.78 user 19.16 sys
TSB_PAGES = 8
119.27 real 76.38 user 18.12 sys
Testing by: tmm
boot and run (and indeed I am committing from it) instead of exploding
during the int 0x15 call from inside the atkbd driver to get the keyboard
repeat rates.
code into cardbus and s/pci/cardbus. This exposes a few pci_*
functions that are now static.
This work is similar to work Justin posted to the mobile list about a
year or two ago, which I have neglected since then.
This is a subset of his current work with the multiple inheritance
newbus architecutre. When completed, that will eliminate the need for
pci/pci_private.h.
Similar work is needed for the cardbus_cis and pccard_cis code as well.
spares (the size of the field was changed from u_short to u_int to
reflect what it really ends up being). Accordingly, change users of
xucred to set and check this field as appropriate. In the kernel,
this is being done inside the new cru2x() routine which takes a
`struct ucred' and fills out a `struct xucred' according to the
former. This also has the pleasant sideaffect of removing some
duplicate code.
Reviewed by: rwatson
shootdowns in a couple of key places. Do the same for i386. This also
hides some physical addresses from higher levels and has it use the
generic vm_page_t's instead. This will help for PAE down the road.
Obtained from: jake (MI code, suggestions for MD part)
due to them being faster in certain cases. Therefore we need to save
and restore the v8 %y register around traps in kernel mode as well as
traps in usermode.
Tested by: obrien, tmm
enabled in critical sections and streamline critical_enter() and
critical_exit().
This commit allows an architecture to leave interrupts enabled inside
critical sections if it so wishes. Architectures that do not wish to do
this are not effected by this change.
This commit implements the feature for the I386 architecture and provides
a sysctl, debug.critical_mode, which defaults to 1 (use the feature). For
now you can turn the sysctl on and off at any time in order to test the
architectural changes or track down bugs.
This commit is just the first stage. Some areas of the code, specifically
the MACHINE_CRITICAL_ENTER #ifdef'd code, is strictly temporary and will
be cleaned up in the STAGE-2 commit when the critical_*() functions are
moved entirely into MD files.
The following changes have been made:
* critical_enter() and critical_exit() for I386 now simply increment
and decrement curthread->td_critnest. They no longer disable
hard interrupts. When critical_exit() decrements the counter to
0 it effectively calls a routine to deal with whatever interrupts
were deferred during the time the code was operating in a critical
section.
Other architectures are unaffected.
* fork_exit() has been conditionalized to remove MD assumptions for
the new code. Old code will still use the old MD assumptions
in regards to hard interrupt disablement. In STAGE-2 this will
be turned into a subroutine call into MD code rather then hardcoded
in MI code.
The new code places the burden of entering the critical section
in the trampoline code where it belongs.
* I386: interrupts are now enabled while we are in a critical section.
The interrupt vector code has been adjusted to deal with the fact.
If it detects that we are in a critical section it currently defers
the interrupt by adding the appropriate bit to an interrupt mask.
* In order to accomplish the deferral, icu_lock is required. This
is i386-specific. Thus icu_lock can only be obtained by mainline
i386 code while interrupts are hard disabled. This change has been
made.
* Because interrupts may or may not be hard disabled during a
context switch, cpu_switch() can no longer simply assume that
PSL_I will be in a consistent state. Therefore, it now saves and
restores eflags.
* FAST INTERRUPT PROVISION. Fast interrupts are currently deferred.
The intention is to eventually allow them to operate either while
we are in a critical section or, if we are able to restrict the
use of sched_lock, while we are not holding the sched_lock.
* ICU and APIC vector assembly for I386 cleaned up. The ICU code
has been cleaned up to match the APIC code in regards to format
and macro availability. Additionally, the code has been adjusted
to deal with deferred interrupts.
* Deferred interrupts use a per-cpu boolean int_pending, and
masks ipending, spending, and fpending. Being per-cpu variables
it is not currently necessary to lock; bus cycles modifying them.
Note that the same mechanism will enable preemption to be
incorporated as a true software interrupt without having to
further hack up the critical nesting code.
* Note: the old critical_enter() code in kern/kern_switch.c is
currently #ifdef to be compatible with both the old and new
methodology. In STAGE-2 it will be moved entirely to MD code.
Performance issues:
One of the purposes of this commit is to enhance critical section
performance, specifically to greatly reduce bus overhead to allow
the critical section code to be used to protect per-cpu caches.
These caches, such as Jeff's slab allocator work, can potentially
operate very quickly making the effective savings of the new
critical section code's performance very significant.
The second purpose of this commit is to allow architectures to
enable certain interrupts while in a critical section. Specifically,
the intention is to eventually allow certain FAST interrupts to
operate rather then defer.
The third purpose of this commit is to begin to clean up the
critical_enter()/critical_exit()/cpu_critical_enter()/
cpu_critical_exit() API which currently has serious cross pollution
in MI code (in fork_exit() and ast() for example).
The fourth purpose of this commit is to provide a framework that
allows kernel-preempting software interrupts to be implemented
cleanly. This is currently used for two forward interrupts in I386.
Other architectures will have the choice of using this infrastructure
or building the functionality directly into critical_enter()/
critical_exit().
Finally, this commit is designed to greatly improve the flexibility
of various architectures to manage critical section handling,
software interrupts, preemption, and other highly integrated
architecture-specific details.
until we do some testing to see what's best. This gives a massive reduction
in system time for processes with a relatively large working set. The size
of the tsb directly affects the rss size that a user process can keep mapped.
When it starts to get full replacements occur and the process takes a lot of
soft vm faults. Increasing the default from 1 page to 2 gives the following
before and after numbers for compiling vfs_bio.c:
before:
14.27 real 6.56 user 5.69 sys
after:
8.57 real 6.11 user 1.62 sys
This should make self hosted builds more tolerable.
header and push it up any attached bpf devices on the parent interface.
This makes hardware vlan decoding more like the normal software path.
Tested by: cjtt@employees.org
MFC after: 2 weeks
via sysctl's. The old #defines, MAX_GIF_NEST and XBONEHACK are
currently supported for backwards compatability, but will probably be
removed at some point in the future.
on for a while:
- fine grained TLB shootdown for SMP on i386
- ranged TLB shootdowns.. eg: specify a range of pages to shoot down with
a single IPI, since the IPI is very expensive. Adjust some callers
that used to trigger this inside tight loops to do a ranged shootdown
at the end instead.
- PG_G support for SMP on i386 (options ENABLE_PG_G)
- defer PG_G activation till after we decide what we are going to do with
PSE and the 4MB pages at the start of the kernel. This should solve
some rumored strangeness about stale PG_G entries getting stuck
underneath the 4MB pages.
- add some instrumentation for the fine TLB shootdown
- convert some asm instruction wrappers from functions to inlines. gcc
seems to do a fair bit better with this.
- [temporarily!] pessimize the tlb shootdown IPI handlers. I will fix
this again shortly.
This has been working fairly well for me for a while, but I have tweaked
it again prior to commit since my last major testing round. The only
outstanding problem that I know of is PG_G related, which is why there
is an option for it (not on by default for SMP). I have seen a world
speedups by a few percent (as much as 4 or 5% in one case) but I have
*not* accurately measured this - I am a bit sceptical of these numbers.
I don't believe anyone is quite using the sparc64 kernel sources in CVS
yet -- things aren't just quite ready (but almost). So this commit should
be OK to make.
- fix the warnings, they are there for a reason!
- add -DNO_ERROR to your make(1) command.
- add 'makeoptions NO_WERROR=true' to your kernel config.
- add 'nowerror' to conf/files* that have warnings that should be fixed
due to tracking 3rd party vendor code.
- add 'nowerror' to conf/files* where the warning is false due to a
compiler bug and fixing it with brute force would be too expensive.
There are some very sloppy warnings in our kernel build, come on folks!
'make release' uses -DNO_WERROR intentionally.
window to the user stack while in a nested kernel trap. We do this for
entry to the kernel from user mode, but if we get an interrupt in kernel
mode while there are still user windows in the cpu, and we attempt to spill
to the user stack, we may take too many nested traps and overflow the trap
stack, causing a red state exception. This is needed by upcoming changes
to allow the user tsb to not be locked in the tlb.
Reviewed by: tmm
the inter-value histogram for 2000 samples. If the width is 3 or less
for 10 consequtive samples, we trust the counter to be good, otherwise
we use the *_safe() method.
This method may be too strict, but the worst which can happen is that
we take the performance hit of the *_safe() method when we should not.
Make the *_safe() method more discriminating by mandating that the three
samples do not span more than 15 ticks on the counter.
Disable the PCI-ident based probing as a means to recognize good
counters.
Inspiration from: dillon and msmith
were destined for a broadcast IP address. All TCP packets with a
broadcast destination must be ignored. The system only ignored packets
that were _link-layer_ broadcasts or multicast. We need to check the
IP address too since it is quite possible for a broadcast IP address
to come in with a unicast link-layer address.
Note that the check existed prior to CSRG revision 7.35, but was
removed. This commit effectively backs out that nine-year-old change.
PR: misc/35022
virtual page number in a much more convenient way; all in one piece. This
greatly simplifies the comparison for a matching tte, and allows the fault
handlers to be much simpler due to not having to load wierd masks.
Rewrite the tlb fault handlers to account for the new format. These are also
written to allow faults on the user tsb inside of the fault handlers; the
kernel fault handler must be aware of this and not clobber the other's
registers. The faults do not yet occur due to other support that is needed
(and still under my desk).
Bug fixes from: tmm
deep in <stand.h> to eventually include <time.h> to declare the user
version.
This is not quite the right place to declare it, but <stand.h> would
be worse because time() is very MD so it isn't in libstand.
Many places in the boot sources still get the user version using only
1 layer of pollution (#include <sys/time.h>. Some pollute themselves
directly (#include <time.h>). But the boot Makefiles are too broken
to enable warnings for redeclarations.
but never accept'ed, so they must be destroyed. Originally, unp_drop()
detected this situation by checking if so->so_head is non-NULL.
However, since revision 1.54 of uipc_socket.c (Feb 1999), so->so_head
is set to NULL before calling soabort(), so any unix-domain sockets
waiting to be accept'ed are leaked if the server socket is closed.
Resolve this by moving the socket destruction code into uipc_abort()
itself, and making it unconditional (the other caller of unp_drop()
never needs the socket to be destroyed). Use unp_detach() to avoid
the original code duplication when destroying the socket.
PR: kern/17895
Reviewed by: dwmalone (an earlier version of the patch)
MFC after: 1 week
latch the acpi timer, resulting in weird deltas. The problem is severe
enough to adversely effect the timecounter code.
Default to the 'safe' version of the get-timecount function. The probe
will override it if a known-good chipset is found. This is temporary
until a more complete solution is found.
Reviewed by: phk
our feet when we look inside timecounter structures.
Make the "sync_other" code more robust by never overwriting the
tc_next field.
Add counters for the bin[up]time functions.
Call tc_windup() in tc_init() and switch_timecounter() to make sure
we all the fields set right.
Most of the contents are commented out as they are as-yet untested.
However, I wanted the contents to match our other arches, so that when
people make changes to {i386,alpha,ia64}, they will also make the same
changes here.
AIOGCAP ioctl reports software-emulated formats. It defaults to on. People
who use performance-sensitive audio software and do not want it to pick a
software-emulated audio format instead of one supported by their hardware
should turn it off.
This unbreaks isdnphone(1) on systems with PCM-only sound cards.
Approved by: cg
pmap_qenter and pmap_qremove in preference to pmap_kenter/pmap_kremove.
The former maps in multiple pages at a time, and so can do a ranged
flush. Don't assume that pmap_kenter and pmap_kremove will flush the tlb,
even though they still do. It will not once the MI code is updated to use
pmap_qenter and pmap_qremove.
will be used to reduce the number of tlb shootdown ipis in an smp system
by sending one ipi for a whole range of pages, instead of one per page.
Munge the context demap operations slightly to support demapping a non-primary
context.
If we don't do this here there's a 1 instruction race where an interrupt
could come in and crash the user process due to having no stack.
2. Pass %fsr to the user trap handler in %l4. Since %fsr can only be loaded
from or stored to memory, we need to do some contortions and temporarily
save it to the alternate global stack.
3. Reload the pcb and pcpu registers for traps in kernel mode, for sanity.
Submitted by: tmm (1, 2)
New locks are:
- pgrpsess_lock which locks the whole pgrps and sessions,
- pg_mtx which protects the pgrp members, and
- s_mtx which protects the session members.
Please refer to sys/proc.h for the coverage of these locks.
Changes on the pgrp/session interface:
- pgfind() needs the pgrpsess_lock held.
- The caller of enterpgrp() is responsible to allocate a new pgrp and
session.
- Call enterthispgrp() in order to enter an existing pgrp.
- pgsignal() requires a pgrp lock held.
Reviewed by: jhb, alfred
Tested on: cvsup.jp.FreeBSD.org
(which is a quad-CPU machine running -current)
Use ACPI_SUCCESS/ACPI_FAILURE consistently.
The AcpiGetInto* interfaces are obsoleted by ACPI_ALLOCATE_BUFFER.
Convert to using a kthread rather than timeout() to avoid problems
with the interpreter sleeping.
Use ACPI_SUCCESS/ACPI_FAILURE consistently.
The AcpiGetInto* interfaces are obsoleted by ACPI_ALLOCATE_BUFFER.
Use _ADR as well as _BBN to get our bus number.
Use ACPI_SUCCESS/ACPI_FAILURE consistently.
The ACPI global lock acquire takes a timeout value. I'm not sure what
we should do about timeouts on it; a deadlock against this lock is
catastrophic.
Use ACPI_SUCCESS/ACPI_FAILURE consistently.
The AcpiGetInto* interfaces are obsoleted by ACPI_ALLOCATE_BUFFER.
Kill off the timeouts that used to read _BIF and _BST. These are
invoked when the battery is actually read. timeout() is dangerous
in combination with ACPI, as the interpreter can block.
This driver still needs more work.
Use ACPI_SUCCESS/ACPI_FAILURE consistently.
The AcpiGetInto* interfaces are obsoleted by ACPI_ALLOCATE_BUFFER.
Add AcpiBatteryIsPresent helper to determine whether a battery device
is inserted.
Add ACPI_ALL_DRIVERS to the list of debug layers, now that we own the
namespace for this.
Pr:
watchpoint support for debugging (under LOADER_DEBUG). Claim the
physical and virtual addresses used to map the kernel from the prom;
we map it ourselves behind the scenes though. Add a reboot command.
Submitted by: tmm
While in userland, keep the thread's ucred reference in a shadow
field so that the usual place to store it is NULL.
If DIAGNOSTIC is not set, the thread ucred is kept valid until the next
kernel entry, at which time it is checked against the process cred
and possibly corrected. Produces a BIG speedup in
kernels with INVARIANTS set. (A previous commit corrected it
for the non INVARIANTS case already)
Reviewed by: dillon@freebsd.org
Remove bowrite(), it is now unused.
This is the first step in getting entirely rid of BIO_ORDERED which is
a generally accepted evil thing.
Approved by: mckusick
so that this is safe even if VARIABLE is longer than kern.argmax.
There is another instance of CFILES which might need the same treatment,
and might be noticed when doing a "make links".
The same has to be done in RELENG_4 (on some different file).
Noticed-by: picobsd cross-compiling LINT
Suggested-by: Alfred (bright@mu.org), des@freebsd.org
MFC-after: 3 days
counter type, as threatened in rev.1.8 (the density doesn't need to
be recorded since it can be derived from other fields). This doesn't
affect binary compatibility, but new utilities won't be able to depend
on the contents of this field because libc/gmon/gmon.c was broken --
it wrote garbage to the spare fields.
Added a history counter type field to struct gmonparam. This breaks
binary compatibility a little, since kgmon wanted to read the whole
struct. Fixed kgmon to only depend on reading the critical earlier
parts of the struct. This should also fix 6+ year old breakage of
binary compatibility when the profrate field was added.
Only initialize the new field in struct gmon for now, so that the
compatibility code for this (in kgmon) gets tested. The compatibility
code has to guesstimate the value. The new field in struct gmonparam
is for the kernel to initialize so that kgmon doesn't have to guess.
- P_INMEM checks in all the functions. P_INMEM must be checked because
PHOLD() is broken. The old bits had bogus locking (using sched_lock)
to lock P_INMEM. After removing the P_INMEM checks, we were left with
just the bogus locking.
- large comments. They were too large, but better than nothing.
Remove obfuscations that were gained in transition in rev.1.76:
- PROC_REG_ACTION() is even more of an obfuscation than PROC_ACTION().
The change copies procfs_machdep.c rev.1.22 of i386/procfs_machdep.c
verbatim except for "fixing" the old-style function headers and adjusting
function names and comments. It doesn't remove the bogus locking.
Approved by: des
OUT status. We are, apparently, required to force the f/w to log back in
if we want to try and talk to that disk again. This means either issuing
a LOGIN LOCAL LOOP PORT mailbox command, or by issuing a LIP. I've elected
to issue a LIP because this has a better chance of waking up the disk which
clearly just crashed and burned.
These should not occur at all. If they do, they should be darned rare.
MFC after: 1 week
It doesn't actually do it yet though. This adds a flag to config so
that we can exclude certain vendor files from this even when the rest
of the kernel has it on. make -DNO_WERROR would also bypass all of it.
so that after the first time we can follow the pointer instead
of having to scan the list.
This was the intended behaviour from day one.
PR: 34639
MFC-after: 3 days
