1
0
mirror of https://git.FreeBSD.org/src.git synced 2024-12-23 11:18:54 +00:00
Commit Graph

140 Commits

Author SHA1 Message Date
Gleb Smirnoff
8ec07310fa These files were getting sys/malloc.h and vm/uma.h with header pollution
via sys/mbuf.h
2016-02-01 17:41:21 +00:00
Sean Bruno
b834dcea9a Switch em(4) to the extended RX descriptor format. This matches the
e1000/e1000e split in linux.

Split rxbuffer and txbuffer apart to support the new RX descriptor format
structures. Move rxbuffer manipulation to em_setup_rxdesc() to unify the
new behavior changes.

Add a RSSKEYLEN macro for help in generating the RSSKEY data structures
in the card.

Change em_receive_checksum() to process the new rxdescriptor format
status bit.

MFC after:	2 weeks
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D3447
2016-01-07 16:42:48 +00:00
Kevin Lo
ddb1359877 Fix typo (s/harware/hardware/) 2015-12-25 14:51:36 +00:00
Adrian Chadd
15b1492c9b Don't call enable_all_rings if the adapter has been freed.
This is a subtle use-after-free race that results in some very undesirable
hang behaviour.

Reviewed by:	pkelsey
Obtained from:	Kip Macy, NextBSD (91a9bd1dbb)
2015-09-07 23:16:39 +00:00
Luigi Rizzo
847adfb7b3 add a use count so the netmap module cannot be unloaded while in use. 2015-07-19 18:07:25 +00:00
Luigi Rizzo
10b8ef3d6a properly destroy persistent vale ports 2015-07-19 18:06:30 +00:00
Luigi Rizzo
9694aad375 do not free NULL if pipe allocation fails 2015-07-19 18:05:49 +00:00
Luigi Rizzo
05f7605789 release a reference when stopping a monitor 2015-07-19 18:04:51 +00:00
Luigi Rizzo
85fe4e7c6b small documentation update 2015-07-19 17:54:42 +00:00
Patrick Kelsey
8aa7fdbd78 Add netmap support for ixgbe SRIOV VFs (that is, to if_ixv).
Differential Revision: https://reviews.freebsd.org/D2923
Reviewed by: erj, gnn
Approved by: jmallett (mentor)
Sponsored by: Norse Corp, Inc.
2015-07-15 01:02:01 +00:00
Luigi Rizzo
5f94000ee4 set the refcount for the structure (dropped by mistake in the last commit). 2015-07-13 10:23:52 +00:00
Luigi Rizzo
8fd44c9395 staticize functions only used in netmap.c
(detected by jenkins run with gcc 4.9)

Update documentation on the use of netmap_priv_d,
rename the refcount and use the same structure in
FreeBSD and linux

No functional changes.
2015-07-10 16:05:24 +00:00
Luigi Rizzo
847bf38369 Sync netmap sources with the version in our private tree.
This commit contains large contributions from Giuseppe Lettieri and
Stefano Garzarella, is partly supported by grants from Verisign and Cisco,
and brings in the following:

- fix zerocopy monitor ports and introduce copying monitor ports
  (the latter are lower performance but give access to all traffic
  in parallel with the application)

- exclusive open mode, useful to implement solutions that recover
  from crashes of the main netmap client (suggested by Patrick Kelsey)

- revised memory allocator in preparation for the 'passthrough mode'
  (ptnetmap) recently presented at bsdcan. ptnetmap is described in
        S. Garzarella, G. Lettieri, L. Rizzo;
        Virtual device passthrough for high speed VM networking,
        ACM/IEEE ANCS 2015, Oakland (CA) May 2015
        http://info.iet.unipi.it/~luigi/research.html

- fix rx CRC handing on ixl

- add module dependencies for netmap when building drivers as modules

- minor simplifications to device-specific routines (*txsync, *rxsync)

- general code cleanup (remove unused variables, introduce macros
  to access rings and remove duplicate code,

Applications do not need to be recompiled, unless of course
they want to use the new features (monitors and exclusive open).

Those willing to try this code on stable/10 can just update the
sys/dev/netmap/*, sys/net/netmap* with the version in HEAD
and apply the small patches to individual device drivers.

MFC after:	1 month
Sponsored by:	(partly) Verisign, Cisco
2015-07-10 05:51:36 +00:00
Sean Bruno
23c9098b2a Change EM_MULTIQUEUE to a real kernconf entry and enable support for
up to 2 rx/tx queues for the 82574.

Program the 82574 to enable 5 msix vectors, assign 1 to each rx queue,
1 to each tx queue and 1 to the link handler.

Inspired by DragonFlyBSD, enable some RSS logic for handling tx queue
handling/processing.

Move multiqueue handler functions so that they line up better in a diff
review to if_igb.c

Always enqueue tx work to be done in em_mq_start, if unable to acquire
the TX lock, then this will be processed in the background later by the
taskqueue.  Remove mbuf argument from em_start_mq_locked() as the work
is always enqueued.  (stolen from igb)

Setup TARC, TXDCTL and RXDCTL registers for better performance and stability
in multiqueue and singlequeue implementations. Handle Intel errata  3 and
generic multiqueue behavior with the initialization of TARC(0) and TARC(1)

Bind interrupt threads to cpus in order.  (stolen from igb)

Add 2 new DDB functions, one to display the queue(s) and their settings and
one to reset the adapter.  Primarily used for debugging.

In the multiqueue configuration, bump RXD and TXD ring size to max for the
adapter (4096).  Setup an RDTR of 64 and an RADV of 128 in multiqueue configuration
to cut down on the number of interrupts.  RADV was arbitrarily set to 2x RDTR
and can be adjusted as needed.

Cleanup the display in top a bit to make it clearer where the taskqueue threads
are running and what they should be doing.

Ensure that both queues are processed by em_local_timer() by writing them both
to the IMS register to generate soft interrupts.

Ensure that an soft interrupt is generated when em_msix_link() is run so that
any races between assertion of the link/status interrupt and a rx/tx interrupt
are handled.

Document existing tuneables: hw.em.eee_setting, hw.em.msix, hw.em.smart_pwr_down, hw.em.sbp

Document use of hw.em.num_queues and the new kernel option EM_MULTIQUEUE

Thanks to Intel for their continued support of FreeBSD.

Reviewed by:	erj jfv hiren gnn wblock
Obtained from:	Intel Corporation
MFC after:	2 weeks
Relnotes:	Yes
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D1994
2015-06-03 18:01:09 +00:00
Patrick Kelsey
dd4fcbc594 When a netmap process terminates without the full set of buffers it
was granted via rings and ni_bufs_list_head represented in those rings
and lists (e.g., via SIGKILL), those buffers are no longer available
for subsequent users for the lifetime of the system. To mitigate this
resource leak, reset the allocator state when the last ref to that
allocator is released.

Note that this only recovers leaked resources for an allocator when
there are no longer any users of that allocator, so there remain
circumstances in which leaked allocator resources may not ever be
recovered - consider a set of multiple netmap processes that are all
using the same allocator (say, the global allocator) where members of
that set may be killed and restarted over time but at any given point
there is one member of that set running.

Based on intial work by adrian@.

Reviewed by: Giuseppe Lettieri (g.lettieri@iet.unipi.it), luigi
Approved by: jmallett (mentor)
MFC after: 1 week
Sponsored by: Norse Corp, Inc.
2015-05-15 15:36:57 +00:00
Rui Paulo
d82f9014fa netmap: improve the netmap attach message on FreeBSD.
MFC after:	1 week
2015-04-11 06:20:46 +00:00
Bjoern A. Zeeb
69cfd6a666 Make ix_crcstrip a public symbol for the moment; it probably is not
the right solution but I will leave it to experts to untangle this
problem to properly stop the build failures.

At the moment only if_ix.c includes dev/netmap/ixgbe_netmap.h which is
good as ixgbe_netmap.h defines a couple of (file) static variables--thus
local to if_ix.c.
static int ix_crcstrip however now also got checked from ix_txrx.c
(as an extern) and should not be visible there.  In fact we do see
powerpc and powerpc64 build failures because of this.  It is unclear
to me why on other (clang built?) architectures this does not lead
to a reference of an undefined symbol and similar build breakage.
2015-03-24 09:46:47 +00:00
Luigi Rizzo
bc8b78d393 Add native netmap support to ixl.
Preliminary tests indicate 32 Mpps on tx, 24 Mpps on rx
with source and receiver on two different ports of the same 40G card.
Optimizations are likely possible.
The code follows closely the one for ixgbe so i do not
expect stability issues.

Hardware kindly supplied by Intel.

Reviewed by:	Jack Vogel
MFC after:	1 week
2015-02-24 06:20:50 +00:00
Luigi Rizzo
735c8d9528 add MODULE_VERSION, needed to track module dependencies
MFC after:	3 days
2015-02-23 07:28:31 +00:00
Luigi Rizzo
6641c68bcd two minor changes from the master netmap version:
1. handle errors from nm_config(), if any (none of the FreeBSD drivers
   currently returns an error on this function, so this change
   is a no-op at this time
2. use a full memory barrier on ioctls
2015-02-14 19:03:11 +00:00
Luigi Rizzo
c929ca72c9 whitespace change:
clarify the role of MAKEDEV_ETERNAL_KLD, and remove an old
#ifdef __FreeBSD__ since the code is valid on all platforms.
2015-02-14 18:59:31 +00:00
Adrian Chadd
11c0b69c08 Change the permissions from 0660 to 0600.
Otherwise people in wheel can do things with netmap, including
but not limited to promisc transmit/receive.

Approved by:	luigi
MFC after:	1 week
2015-01-24 19:49:27 +00:00
Hans Petter Selasky
c25290420e Start process of removing the use of the deprecated "M_FLOWID" flag
from the FreeBSD network code. The flag is still kept around in the
"sys/mbuf.h" header file, but does no longer have any users. Instead
the "m_pkthdr.rsstype" field in the mbuf structure is now used to
decide the meaning of the "m_pkthdr.flowid" field. To modify the
"m_pkthdr.rsstype" field please use the existing "M_HASHTYPE_XXX"
macros as defined in the "sys/mbuf.h" header file.

This patch introduces new behaviour in the transmit direction.
Previously network drivers checked if "M_FLOWID" was set in "m_flags"
before using the "m_pkthdr.flowid" field. This check has now now been
replaced by checking if "M_HASHTYPE_GET(m)" is different from
"M_HASHTYPE_NONE". In the future more hashtypes will be added, for
example hashtypes for hardware dedicated flows.

"M_HASHTYPE_OPAQUE" indicates that the "m_pkthdr.flowid" value is
valid and has no particular type. This change removes the need for an
"if" statement in TCP transmit code checking for the presence of a
valid flowid value. The "if" statement mentioned above is now a direct
variable assignment which is then later checked by the respective
network drivers like before.

Additional notes:
- The SCTP code changes will be committed as a separate patch.
- Removal of the "M_FLOWID" flag will also be done separately.
- The FreeBSD version has been bumped.

MFC after:	1 month
Sponsored by:	Mellanox Technologies
2014-12-01 11:45:24 +00:00
Luigi Rizzo
0e73f29ae2 add support for private knote lock (reduces lock contention),
adapting OS_selrecord accordingly.
Problem and fix suggested by adrian and jmg
2014-11-13 00:40:34 +00:00
Luigi Rizzo
ad15cc59e9 we need full barriers here 2014-11-13 00:14:25 +00:00
Luigi Rizzo
039dd540f5 in the Linux section, properly define the NMG_LOCK type.
Also import WITH_GENERIC in preparation to adding fine-grained
options to disable specific netmap components.
2014-11-11 00:13:28 +00:00
Luigi Rizzo
204f91dd3a - fix typo: use ring size from the rx ring, not the tx one (they should be
the same, but just in case);
- reuse the previously computed len-1 value
2014-11-11 00:10:44 +00:00
Luigi Rizzo
6435a0dc1b fix a typo 2014-11-10 21:00:23 +00:00
Luigi Rizzo
4e93beff92 initialize *color if passed as an argument 2014-11-10 20:25:33 +00:00
Luigi Rizzo
db5cb21105 sync a comment with our internal repo 2014-11-10 20:19:58 +00:00
Luigi Rizzo
b3d3758852 fix a panic when passing ifioctl from a netmap file descriptor to
the underlying device. This needs to be merged to 10.1

Reported by: Patrick Kelsey
MFC after:	3 days
2014-09-25 16:22:32 +00:00
Luigi Rizzo
7f154b713a adapt the code to different freebsd versions.
Not necessary to MFC
2014-09-25 15:57:57 +00:00
Gleb Smirnoff
c8dfaf382f Mechanically convert to if_inc_counter(). 2014-09-19 03:51:26 +00:00
Gleb Smirnoff
997d2d833f Provide pointer from struct ifnet to struct netmap_adapter,
instead of abusing spare field.
2014-08-31 11:33:19 +00:00
Navdeep Parhar
9721a22d4a Change netmap's global lock to sx instead of a mutex.
Reviewed by:	luigi@
MFC after:	1 day
2014-08-20 23:37:44 +00:00
Luigi Rizzo
1460a86867 staticize two functions, and use proper format for a struct sglist
(reported by bz)
2014-08-17 10:25:27 +00:00
Luigi Rizzo
4bf50f18eb Update to the current version of netmap.
Mostly bugfixes or features developed in the past 6 months,
so this is a 10.1 candidate.

Basically no user API changes (some bugfixes in sys/net/netmap_user.h).

In detail:

1. netmap support for virtio-net, including in netmap mode.
  Under bhyve and with a netmap backend [2] we reach over 1Mpps
  with standard APIs (e.g. libpcap), and 5-8 Mpps in netmap mode.

2. (kernel) add support for multiple memory allocators, so we can
  better partition physical and virtual interfaces giving access
  to separate users. The most visible effect is one additional
  argument to the various kernel functions to compute buffer
  addresses. All netmap-supported drivers are affected, but changes
  are mechanical and trivial

3. (kernel) simplify the prototype for *txsync() and *rxsync()
  driver methods. All netmap drivers affected, changes mostly mechanical.

4. add support for netmap-monitor ports. Think of it as a mirroring
  port on a physical switch: a netmap monitor port replicates traffic
  present on the main port. Restrictions apply. Drive carefully.

5. if_lem.c: support for various paravirtualization features,
  experimental and disabled by default.
  Most of these are described in our ANCS'13 paper [1].
  Paravirtualized support in netmap mode is new, and beats the
  numbers in the paper by a large factor (under qemu-kvm,
  we measured gues-host throughput up to 10-12 Mpps).

A lot of refactoring and additional documentation in the files
in sys/dev/netmap, but apart from #2 and #3 above, almost nothing
of this stuff is visible to other kernel parts.

Example programs in tools/tools/netmap have been updated with bugfixes
and to support more of the existing features.

This is meant to go into 10.1 so we plan an MFC before the Aug.22 deadline.

A lot of this code has been contributed by my colleagues at UNIPI,
including Giuseppe Lettieri, Vincenzo Maffione, Stefano Garzarella.

MFC after:	3 days.
2014-08-16 15:00:01 +00:00
Gleb Smirnoff
fcc34a238c Fix style bug: rename the refcount field of m_ext to ext_cnt, to match
other members.

Sponsored by:	Nginx, Inc.
2014-07-11 14:34:29 +00:00
Luigi Rizzo
63a3395e5d change the netmap mbuf destructor so the same code works also on FreeBSD 9.
For head and 10 this change has no effect, but on stable/9 it would cause
panics when using emulated netmap on top of a standard device driver.
2014-06-10 16:06:59 +00:00
Luigi Rizzo
348c44a5be Fixes from Fanco Ficthner on transparent mode
* The way rings are updated changed with the last API bump.
  Also sync ->head when moving slots in netmap_sw_to_nic().

* Remove a crashing selrecord() call.

* Unclog the logic surrounding netmap_rxsync_from_host().

* Add timestamping to RX host ring.

* Remove a couple of obsolete comments.

Submitted by:	Franco Fichtner
MFC after:	3 days
Sponsored by:	Packetwerk
2014-06-09 15:46:11 +00:00
Luigi Rizzo
46aa1303f3 sync the code with the one in stable/10
(wrap the if_t compatibilty function into a __FreeBSD_version
conditional block)
2014-06-09 15:44:31 +00:00
Luigi Rizzo
e4166283fb better handling of netmap emulation over standard device drivers:
plug a potential mbuf leak, and detect bogus drivers that
return ENOBUFS even when the packet has been queued.

MFC after:	3 days
2014-06-06 18:36:02 +00:00
Luigi Rizzo
997b054cf1 introduce mbq_lock() and mbq_unlock() for the mbq,
so it is easier to buil the same code on linux
(this generalizes the change in svn 267142)

MFC after:	3 days
2014-06-06 18:02:32 +00:00
Luigi Rizzo
0dc809c034 move netmap_getna() to a freebsd-specific file 2014-06-06 16:23:08 +00:00
Luigi Rizzo
89cc25561c align comments with the ones in our development trunk 2014-06-06 14:58:25 +00:00
Luigi Rizzo
d8e1c53b15 rate limit some error messages 2014-06-06 14:57:40 +00:00
Luigi Rizzo
5899a007ae remove two debugging messages, align comments with the code
in our development trunk
2014-06-06 14:57:16 +00:00
Luigi Rizzo
e31c6ec7e2 add checks for invalid buffer pointers and lengths 2014-06-06 10:50:14 +00:00
Luigi Rizzo
441ab64f52 prevent a panic when the netdev/ifp is not set in attach
(internal  c63a7b85)

MFC after:	3 days
2014-06-06 10:40:20 +00:00
Andrey Zonov
dc8a95e62b Use mtx_lock_spin/mtx_unlock_spin primitives on spin lock
Reviewed by:	luigi
MFC after:	1 week
2014-06-06 00:24:04 +00:00