mirror/freebsd
1
0
Fork 0
mirror of https://git.FreeBSD.org/src.git synced 2024-12-21 11:13:30 +00:00
Code Issues Releases Activity
d777904f05
freebsd/sys/dev/ixgbe/ixgbe.c

5664 lines
159 KiB
C
Raw Normal View History

This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/******************************************************************************
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
Copyright (c) 2001-2012, Intel Corporation
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of the Intel Corporation nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
******************************************************************************/
/*$FreeBSD$*/
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
#ifdef HAVE_KERNEL_OPTION_HEADERS
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
#include "opt_inet.h"
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
#include "opt_inet6.h"
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
#endif
A couple late breaking bugs that testing have turned up. - change include style so build in kernel try OR standalone work. - Limit HWCSUM - I was led to believe that it would work with RSS, but our testing had odd issues which suggests this is false. - A fatfinger error in the ioctl code made ifconfig up not work. Approved by: re
2007-07-12 19:04:11 +00:00
#include "ixgbe.h"
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*********************************************************************
* Set this to one to display debug statistics
*********************************************************************/
int ixgbe_display_debug_stats = 0;
/*********************************************************************
* Driver version
*********************************************************************/
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
char ixgbe_driver_version[] = "2.5.0 - 10";
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*********************************************************************
* PCI Device ID Table
*
* Used by probe to select devices to load on
* Last field stores an index into ixgbe_strings
* Last entry must be all 0s
*
* { Vendor ID, Device ID, SubVendor ID, SubDevice ID, String Index }
*********************************************************************/
static ixgbe_vendor_info_t ixgbe_vendor_info_array[] =
{
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82598AF_DUAL_PORT, 0, 0, 0},
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82598AF_SINGLE_PORT, 0, 0, 0},
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82598EB_CX4, 0, 0, 0},
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82598AT, 0, 0, 0},
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82598AT2, 0, 0, 0},
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82598, 0, 0, 0},
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82598_DA_DUAL_PORT, 0, 0, 0},
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82598_CX4_DUAL_PORT, 0, 0, 0},
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82598EB_XF_LR, 0, 0, 0},
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82598_SR_DUAL_PORT_EM, 0, 0, 0},
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82598EB_SFP_LOM, 0, 0, 0},
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82599_KX4, 0, 0, 0},
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82599_KX4_MEZZ, 0, 0, 0},
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82599_SFP, 0, 0, 0},
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82599_XAUI_LOM, 0, 0, 0},
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82599_CX4, 0, 0, 0},
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82599_T3_LOM, 0, 0, 0},
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82599_COMBO_BACKPLANE, 0, 0, 0},
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82599_BACKPLANE_FCOE, 0, 0, 0},
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82599_SFP_SF2, 0, 0, 0},
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82599_SFP_FCOE, 0, 0, 0},
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_82599EN_SFP, 0, 0, 0},
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_X540T1, 0, 0, 0},
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
{IXGBE_INTEL_VENDOR_ID, IXGBE_DEV_ID_X540T, 0, 0, 0},
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* required last entry */
{0, 0, 0, 0, 0}
};
/*********************************************************************
* Table of branding strings
*********************************************************************/
static char *ixgbe_strings[] = {
"Intel(R) PRO/10GbE PCI-Express Network Driver"
};
/*********************************************************************
* Function prototypes
*********************************************************************/
static int ixgbe_probe(device_t);
static int ixgbe_attach(device_t);
static int ixgbe_detach(device_t);
static int ixgbe_shutdown(device_t);
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#ifdef IXGBE_LEGACY_TX
static void ixgbe_start(struct ifnet *);
static void ixgbe_start_locked(struct tx_ring *, struct ifnet *);
#else /* ! IXGBE_LEGACY_TX */
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
static int ixgbe_mq_start(struct ifnet *, struct mbuf *);
static int ixgbe_mq_start_locked(struct ifnet *,
struct tx_ring *, struct mbuf *);
static void ixgbe_qflush(struct ifnet *);
Merge similar fixes from 223198 from igb to ixgbe: - Use a dedicated task to handle deferred transmits from the if_transmit method instead of reusing the existing per-queue interrupt task. Reusing the per-queue interrupt task could result in both an interrupt thread and the taskqueue thread trying to handle received packets on a single queue resulting in out-of-order packet processing and lock contention. - Don't define ixgbe_start() at all where if_transmit is used. Tested by: Vijay Singh Reviewed by: jfv MFC after: 2 weeks
2012-09-26 18:11:43 +00:00
static void ixgbe_deferred_mq_start(void *, int);
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#endif /* IXGBE_LEGACY_TX */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static int ixgbe_ioctl(struct ifnet *, u_long, caddr_t);
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
static void ixgbe_init(void *);
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
static void ixgbe_init_locked(struct adapter *);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static void ixgbe_stop(void *);
static void ixgbe_media_status(struct ifnet *, struct ifmediareq *);
static int ixgbe_media_change(struct ifnet *);
static void ixgbe_identify_hardware(struct adapter *);
static int ixgbe_allocate_pci_resources(struct adapter *);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
static int ixgbe_allocate_msix(struct adapter *);
static int ixgbe_allocate_legacy(struct adapter *);
static int ixgbe_allocate_queues(struct adapter *);
static int ixgbe_setup_msix(struct adapter *);
static void ixgbe_free_pci_resources(struct adapter *);
A couple fixes got clobbered, putting them back.
2010-11-26 23:57:13 +00:00
static void ixgbe_local_timer(void *);
static int ixgbe_setup_interface(device_t, struct adapter *);
static void ixgbe_config_link(struct adapter *);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static int ixgbe_allocate_transmit_buffers(struct tx_ring *);
static int ixgbe_setup_transmit_structures(struct adapter *);
static void ixgbe_setup_transmit_ring(struct tx_ring *);
static void ixgbe_initialize_transmit_units(struct adapter *);
static void ixgbe_free_transmit_structures(struct adapter *);
static void ixgbe_free_transmit_buffers(struct tx_ring *);
static int ixgbe_allocate_receive_buffers(struct rx_ring *);
static int ixgbe_setup_receive_structures(struct adapter *);
static int ixgbe_setup_receive_ring(struct rx_ring *);
static void ixgbe_initialize_receive_units(struct adapter *);
static void ixgbe_free_receive_structures(struct adapter *);
static void ixgbe_free_receive_buffers(struct rx_ring *);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
static void ixgbe_setup_hw_rsc(struct rx_ring *);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static void ixgbe_enable_intr(struct adapter *);
static void ixgbe_disable_intr(struct adapter *);
static void ixgbe_update_stats_counters(struct adapter *);
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
static bool ixgbe_txeof(struct tx_ring *);
Patch #7 This is primarily about processing limit control. - add a limit for both RX and TX, change the default to 256 - change the sysctl usage to be common, and now to be called during init for each ring. - the TX limit is not yet used, but the changes in the last patch in this series uses the value. - the motivation behind these changes is to improve data locality in the final code. - rxeof interface changes since it now gets limit from the ring struct
2012-11-30 23:28:01 +00:00
static bool ixgbe_rxeof(struct ix_queue *);
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
static void ixgbe_rx_checksum(u32, struct mbuf *, u32);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static void ixgbe_set_promisc(struct adapter *);
static void ixgbe_set_multi(struct adapter *);
static void ixgbe_update_link_status(struct adapter *);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
static void ixgbe_refresh_mbufs(struct rx_ring *, int);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
static int ixgbe_xmit(struct tx_ring *, struct mbuf **);
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
static int ixgbe_set_flowcntl(SYSCTL_HANDLER_ARGS);
BAH, I apologize, the wrong version of the code got fat fingered in place, this is the correct version that actually works... <sheepish grin> MFC: in a week
2010-06-30 01:10:08 +00:00
static int ixgbe_set_advertise(SYSCTL_HANDLER_ARGS);
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
static int ixgbe_set_thermal_test(SYSCTL_HANDLER_ARGS);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static int ixgbe_dma_malloc(struct adapter *, bus_size_t,
struct ixgbe_dma_alloc *, int);
static void ixgbe_dma_free(struct adapter *, struct ixgbe_dma_alloc *);
Patch #7 This is primarily about processing limit control. - add a limit for both RX and TX, change the default to 256 - change the sysctl usage to be common, and now to be called during init for each ring. - the TX limit is not yet used, but the changes in the last patch in this series uses the value. - the motivation behind these changes is to improve data locality in the final code. - rxeof interface changes since it now gets limit from the ring struct
2012-11-30 23:28:01 +00:00
static void ixgbe_add_process_limit(struct adapter *, const char *,
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
const char *, u16 *, u16);
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
static int ixgbe_tx_ctx_setup(struct tx_ring *,
struct mbuf *, u32 *, u32 *);
static int ixgbe_tso_setup(struct tx_ring *,
struct mbuf *, u32 *, u32 *);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
static void ixgbe_set_ivar(struct adapter *, u8, u8, s8);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static void ixgbe_configure_ivars(struct adapter *);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
static u8 * ixgbe_mc_array_itr(struct ixgbe_hw *, u8 **, u32 *);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
static void ixgbe_setup_vlan_hw_support(struct adapter *);
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
static void ixgbe_register_vlan(void *, struct ifnet *, u16);
static void ixgbe_unregister_vlan(void *, struct ifnet *, u16);
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
static void ixgbe_add_hw_stats(struct adapter *adapter);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
static __inline void ixgbe_rx_discard(struct rx_ring *, int);
static __inline void ixgbe_rx_input(struct rx_ring *, struct ifnet *,
struct mbuf *, u32);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
A few important fixes: - Testing TSO6 has led me to discover that HW RSC is a problematic feature, it is ONLY designed to work with IPv4 in the first place, and if IP forwarding is done it can't be disabled as LRO in the stack, also initial testing we've done at Intel shows an equal performance using TSO[46] on the TX and LRO on RX, if you ran older code on 82599 or later hardware you actually could have detrimental performance for this reason. So I am disabling the feature by default and all our adapters will now use LRO instead. - If you have flow control off and multiple queues it was possible when the buffer of one queue becomes full that all RX movement is stalled, to eliminate this problem a feature bit is now set that will allow packets to be dropped when full rather than stall. Note, the default is to have flow control on, and this keeps this from happening. - Because of the recent fixes in the stack, LRO is now auto-disabled when problematic, so I have decided to enable it by default in the capabilities in the driver. - There are some 1G modules used by some customers, a couple small tweaks to properly support those in the media code. - A note: we have now done some testing of TSO6 and using LRO with IPv6 and it all works great!! Seeing line rate in both directions in best cases. Thanks bz for your excellent work!!
2012-10-31 23:50:36 +00:00
static void ixgbe_enable_rx_drop(struct adapter *);
static void ixgbe_disable_rx_drop(struct adapter *);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/* Support for pluggable optic modules */
static bool ixgbe_sfp_probe(struct adapter *);
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
static void ixgbe_setup_optics(struct adapter *);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* Legacy (single vector interrupt handler */
static void ixgbe_legacy_irq(void *);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* The MSI/X Interrupt handlers */
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
static void ixgbe_msix_que(void *);
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
static void ixgbe_msix_link(void *);
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
/* Deferred interrupt tasklets */
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
static void ixgbe_handle_que(void *, int);
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
static void ixgbe_handle_link(void *, int);
static void ixgbe_handle_msf(void *, int);
static void ixgbe_handle_mod(void *, int);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
#ifdef IXGBE_FDIR
static void ixgbe_atr(struct tx_ring *, struct mbuf *);
static void ixgbe_reinit_fdir(void *, int);
#endif
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*********************************************************************
* FreeBSD Device Interface Entry Points
*********************************************************************/
static device_method_t ixgbe_methods[] = {
/* Device interface */
DEVMETHOD(device_probe, ixgbe_probe),
DEVMETHOD(device_attach, ixgbe_attach),
DEVMETHOD(device_detach, ixgbe_detach),
DEVMETHOD(device_shutdown, ixgbe_shutdown),
{0, 0}
};
static driver_t ixgbe_driver = {
"ix", ixgbe_methods, sizeof(struct adapter),
};
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
devclass_t ixgbe_devclass;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
DRIVER_MODULE(ixgbe, pci, ixgbe_driver, ixgbe_devclass, 0, 0);
MODULE_DEPEND(ixgbe, pci, 1, 1, 1);
MODULE_DEPEND(ixgbe, ether, 1, 1, 1);
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
/*
** TUNEABLE PARAMETERS:
*/
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/*
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
** AIM: Adaptive Interrupt Moderation
** which means that the interrupt rate
** is varied over time based on the
** traffic for that interrupt vector
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
*/
static int ixgbe_enable_aim = TRUE;
TUNABLE_INT("hw.ixgbe.enable_aim", &ixgbe_enable_aim);
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
static int ixgbe_max_interrupt_rate = (4000000 / IXGBE_LOW_LATENCY);
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
TUNABLE_INT("hw.ixgbe.max_interrupt_rate", &ixgbe_max_interrupt_rate);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* How many packets rxeof tries to clean at a time */
Patch #7 This is primarily about processing limit control. - add a limit for both RX and TX, change the default to 256 - change the sysctl usage to be common, and now to be called during init for each ring. - the TX limit is not yet used, but the changes in the last patch in this series uses the value. - the motivation behind these changes is to improve data locality in the final code. - rxeof interface changes since it now gets limit from the ring struct
2012-11-30 23:28:01 +00:00
static int ixgbe_rx_process_limit = 256;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
TUNABLE_INT("hw.ixgbe.rx_process_limit", &ixgbe_rx_process_limit);
Patch #7 This is primarily about processing limit control. - add a limit for both RX and TX, change the default to 256 - change the sysctl usage to be common, and now to be called during init for each ring. - the TX limit is not yet used, but the changes in the last patch in this series uses the value. - the motivation behind these changes is to improve data locality in the final code. - rxeof interface changes since it now gets limit from the ring struct
2012-11-30 23:28:01 +00:00
/* How many packets txeof tries to clean at a time */
static int ixgbe_tx_process_limit = 256;
TUNABLE_INT("hw.ixgbe.tx_process_limit", &ixgbe_tx_process_limit);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/*
** Smart speed setting, default to on
** this only works as a compile option
** right now as its during attach, set
** this to 'ixgbe_smart_speed_off' to
** disable.
*/
static int ixgbe_smart_speed = ixgbe_smart_speed_on;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/*
* MSIX should be the default for best performance,
* but this allows it to be forced off for testing.
*/
static int ixgbe_enable_msix = 1;
TUNABLE_INT("hw.ixgbe.enable_msix", &ixgbe_enable_msix);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/*
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
* Number of Queues, can be set to 0,
* it then autoconfigures based on the
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
* number of cpus with a max of 8. This
* can be overriden manually here.
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
*/
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
static int ixgbe_num_queues = 0;
TUNABLE_INT("hw.ixgbe.num_queues", &ixgbe_num_queues);
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
/*
** Number of TX descriptors per ring,
** setting higher than RX as this seems
** the better performing choice.
*/
static int ixgbe_txd = PERFORM_TXD;
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
TUNABLE_INT("hw.ixgbe.txd", &ixgbe_txd);
/* Number of RX descriptors per ring */
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
static int ixgbe_rxd = PERFORM_RXD;
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
TUNABLE_INT("hw.ixgbe.rxd", &ixgbe_rxd);
A few important fixes: - Testing TSO6 has led me to discover that HW RSC is a problematic feature, it is ONLY designed to work with IPv4 in the first place, and if IP forwarding is done it can't be disabled as LRO in the stack, also initial testing we've done at Intel shows an equal performance using TSO[46] on the TX and LRO on RX, if you ran older code on 82599 or later hardware you actually could have detrimental performance for this reason. So I am disabling the feature by default and all our adapters will now use LRO instead. - If you have flow control off and multiple queues it was possible when the buffer of one queue becomes full that all RX movement is stalled, to eliminate this problem a feature bit is now set that will allow packets to be dropped when full rather than stall. Note, the default is to have flow control on, and this keeps this from happening. - Because of the recent fixes in the stack, LRO is now auto-disabled when problematic, so I have decided to enable it by default in the capabilities in the driver. - There are some 1G modules used by some customers, a couple small tweaks to properly support those in the media code. - A note: we have now done some testing of TSO6 and using LRO with IPv6 and it all works great!! Seeing line rate in both directions in best cases. Thanks bz for your excellent work!!
2012-10-31 23:50:36 +00:00
/*
** HW RSC control:
** this feature only works with
** IPv4, and only on 82599 and later.
** Also this will cause IP forwarding to
** fail and that can't be controlled by
** the stack as LRO can. For all these
** reasons I've deemed it best to leave
** this off and not bother with a tuneable
** interface, this would need to be compiled
** to enable.
*/
static bool ixgbe_rsc_enable = FALSE;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* Keep running tab on them for sanity check */
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
static int ixgbe_total_ports;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
#ifdef IXGBE_FDIR
/*
** For Flow Director: this is the
** number of TX packets we sample
** for the filter pool, this means
** every 20th packet will be probed.
**
** This feature can be disabled by
** setting this to 0.
*/
static int atr_sample_rate = 20;
/*
** Flow Director actually 'steals'
** part of the packet buffer as its
** filter pool, this variable controls
** how much it uses:
** 0 = 64K, 1 = 128K, 2 = 256K
*/
static int fdir_pballoc = 1;
#endif
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
#ifdef DEV_NETMAP
/*
* The #ifdef DEV_NETMAP / #endif blocks in this file are meant to
* be a reference on how to implement netmap support in a driver.
* Additional comments are in ixgbe_netmap.h .
*
fix a typo in a comment
2012-05-17 14:36:19 +00:00
* <dev/netmap/ixgbe_netmap.h> contains functions for netmap support
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
* that extend the standard driver.
*/
#include <dev/netmap/ixgbe_netmap.h>
#endif /* DEV_NETMAP */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*********************************************************************
* Device identification routine
*
* ixgbe_probe determines if the driver should be loaded on
* adapter based on PCI vendor/device id of the adapter.
*
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
* return BUS_PROBE_DEFAULT on success, positive on failure
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
*********************************************************************/
static int
ixgbe_probe(device_t dev)
{
ixgbe_vendor_info_t *ent;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
u16 pci_vendor_id = 0;
u16 pci_device_id = 0;
u16 pci_subvendor_id = 0;
u16 pci_subdevice_id = 0;
char adapter_name[256];
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
INIT_DEBUGOUT("ixgbe_probe: begin");
pci_vendor_id = pci_get_vendor(dev);
if (pci_vendor_id != IXGBE_INTEL_VENDOR_ID)
return (ENXIO);
pci_device_id = pci_get_device(dev);
pci_subvendor_id = pci_get_subvendor(dev);
pci_subdevice_id = pci_get_subdevice(dev);
ent = ixgbe_vendor_info_array;
while (ent->vendor_id != 0) {
if ((pci_vendor_id == ent->vendor_id) &&
(pci_device_id == ent->device_id) &&
((pci_subvendor_id == ent->subvendor_id) ||
(ent->subvendor_id == 0)) &&
((pci_subdevice_id == ent->subdevice_id) ||
(ent->subdevice_id == 0))) {
sprintf(adapter_name, "%s, Version - %s",
ixgbe_strings[ent->index],
ixgbe_driver_version);
device_set_desc_copy(dev, adapter_name);
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
++ixgbe_total_ports;
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
return (BUS_PROBE_DEFAULT);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
ent++;
}
return (ENXIO);
}
/*********************************************************************
* Device initialization routine
*
* The attach entry point is called when the driver is being loaded.
* This routine identifies the type of hardware, allocates all resources
* and initializes the hardware.
*
* return 0 on success, positive on failure
*********************************************************************/
static int
ixgbe_attach(device_t dev)
{
struct adapter *adapter;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
struct ixgbe_hw *hw;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
int error = 0;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
u16 csum;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
u32 ctrl_ext;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
INIT_DEBUGOUT("ixgbe_attach: begin");
/* Allocate, clear, and link in our adapter structure */
adapter = device_get_softc(dev);
adapter->dev = adapter->osdep.dev = dev;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
hw = &adapter->hw;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* Core Lock Init*/
IXGBE_CORE_LOCK_INIT(adapter, device_get_nameunit(dev));
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* SYSCTL APIs */
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
SYSCTL_ADD_PROC(device_get_sysctl_ctx(dev),
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
SYSCTL_CHILDREN(device_get_sysctl_tree(dev)),
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
OID_AUTO, "fc", CTLTYPE_INT | CTLFLAG_RW,
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
adapter, 0, ixgbe_set_flowcntl, "I", "Flow Control");
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
SYSCTL_ADD_INT(device_get_sysctl_ctx(dev),
SYSCTL_CHILDREN(device_get_sysctl_tree(dev)),
OID_AUTO, "enable_aim", CTLTYPE_INT|CTLFLAG_RW,
&ixgbe_enable_aim, 1, "Interrupt Moderation");
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
/*
** Allow a kind of speed control by forcing the autoneg
** advertised speed list to only a certain value, this
** supports 1G on 82599 devices, and 100Mb on x540.
*/
SYSCTL_ADD_PROC(device_get_sysctl_ctx(dev),
SYSCTL_CHILDREN(device_get_sysctl_tree(dev)),
OID_AUTO, "advertise_speed", CTLTYPE_INT | CTLFLAG_RW,
adapter, 0, ixgbe_set_advertise, "I", "Link Speed");
SYSCTL_ADD_PROC(device_get_sysctl_ctx(dev),
SYSCTL_CHILDREN(device_get_sysctl_tree(dev)),
OID_AUTO, "ts", CTLTYPE_INT | CTLFLAG_RW, adapter,
0, ixgbe_set_thermal_test, "I", "Thermal Test");
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Set up the timer callout */
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
callout_init_mtx(&adapter->timer, &adapter->core_mtx, 0);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Determine hardware revision */
ixgbe_identify_hardware(adapter);
/* Do base PCI setup - map BAR0 */
if (ixgbe_allocate_pci_resources(adapter)) {
device_printf(dev, "Allocation of PCI resources failed\n");
error = ENXIO;
goto err_out;
}
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
/* Do descriptor calc and sanity checks */
if (((ixgbe_txd * sizeof(union ixgbe_adv_tx_desc)) % DBA_ALIGN) != 0 ||
ixgbe_txd < MIN_TXD || ixgbe_txd > MAX_TXD) {
device_printf(dev, "TXD config issue, using default!\n");
adapter->num_tx_desc = DEFAULT_TXD;
} else
adapter->num_tx_desc = ixgbe_txd;
/*
** With many RX rings it is easy to exceed the
** system mbuf allocation. Tuning nmbclusters
** can alleviate this.
*/
Multiqueue RX is not correctly enabled on the new 82599 adapter, the SRRCTL register needs to be setup per queue. Approved by: re
2009-07-01 16:13:01 +00:00
if (nmbclusters > 0 ) {
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
int s;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
s = (ixgbe_rxd * adapter->num_queues) * ixgbe_total_ports;
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
if (s > nmbclusters) {
device_printf(dev, "RX Descriptors exceed "
"system mbuf max, using default instead!\n");
ixgbe_rxd = DEFAULT_RXD;
}
}
if (((ixgbe_rxd * sizeof(union ixgbe_adv_rx_desc)) % DBA_ALIGN) != 0 ||
ixgbe_rxd < MIN_TXD || ixgbe_rxd > MAX_TXD) {
device_printf(dev, "RXD config issue, using default!\n");
adapter->num_rx_desc = DEFAULT_RXD;
} else
adapter->num_rx_desc = ixgbe_rxd;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Allocate our TX/RX Queues */
if (ixgbe_allocate_queues(adapter)) {
error = ENOMEM;
goto err_out;
}
A couple fixes got clobbered, putting them back.
2010-11-26 23:57:13 +00:00
/* Allocate multicast array memory. */
adapter->mta = malloc(sizeof(u8) * IXGBE_ETH_LENGTH_OF_ADDRESS *
MAX_NUM_MULTICAST_ADDRESSES, M_DEVBUF, M_NOWAIT);
if (adapter->mta == NULL) {
device_printf(dev, "Can not allocate multicast setup array\n");
error = ENOMEM;
goto err_late;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Initialize the shared code */
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
error = ixgbe_init_shared_code(hw);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
if (error == IXGBE_ERR_SFP_NOT_PRESENT) {
/*
** No optics in this port, set up
** so the timer routine will probe
** for later insertion.
*/
adapter->sfp_probe = TRUE;
error = 0;
} else if (error == IXGBE_ERR_SFP_NOT_SUPPORTED) {
device_printf(dev,"Unsupported SFP+ module detected!\n");
error = EIO;
goto err_late;
} else if (error) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
device_printf(dev,"Unable to initialize the shared code\n");
error = EIO;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
goto err_late;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* Make sure we have a good EEPROM before we read from it */
if (ixgbe_validate_eeprom_checksum(&adapter->hw, &csum) < 0) {
device_printf(dev,"The EEPROM Checksum Is Not Valid\n");
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
error = EIO;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
goto err_late;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
error = ixgbe_init_hw(hw);
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
switch (error) {
case IXGBE_ERR_EEPROM_VERSION:
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
device_printf(dev, "This device is a pre-production adapter/"
"LOM. Please be aware there may be issues associated "
"with your hardware.\n If you are experiencing problems "
"please contact your Intel or hardware representative "
"who provided you with this hardware.\n");
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
break;
case IXGBE_ERR_SFP_NOT_SUPPORTED:
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
device_printf(dev,"Unsupported SFP+ Module\n");
error = EIO;
goto err_late;
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
case IXGBE_ERR_SFP_NOT_PRESENT:
device_printf(dev,"No SFP+ Module found\n");
/* falls thru */
default:
break;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
}
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
/* Detect and set physical type */
ixgbe_setup_optics(adapter);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
if ((adapter->msix > 1) && (ixgbe_enable_msix))
error = ixgbe_allocate_msix(adapter);
else
error = ixgbe_allocate_legacy(adapter);
if (error)
goto err_late;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Setup OS specific network interface */
A couple fixes got clobbered, putting them back.
2010-11-26 23:57:13 +00:00
if (ixgbe_setup_interface(dev, adapter) != 0)
goto err_late;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Initialize statistics */
ixgbe_update_stats_counters(adapter);
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
/* Register for VLAN events */
adapter->vlan_attach = EVENTHANDLER_REGISTER(vlan_config,
Improvement on the last change, this gives a precise way to tell the one and only interface that a vlan event is for. Thanks to John Baldwin for the patch. Approved by: re
2009-07-24 21:35:52 +00:00
ixgbe_register_vlan, adapter, EVENTHANDLER_PRI_FIRST);
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
adapter->vlan_detach = EVENTHANDLER_REGISTER(vlan_unconfig,
Improvement on the last change, this gives a precise way to tell the one and only interface that a vlan event is for. Thanks to John Baldwin for the patch. Approved by: re
2009-07-24 21:35:52 +00:00
ixgbe_unregister_vlan, adapter, EVENTHANDLER_PRI_FIRST);
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* Print PCIE bus type/speed/width info */
ixgbe_get_bus_info(hw);
device_printf(dev,"PCI Express Bus: Speed %s %s\n",
((hw->bus.speed == ixgbe_bus_speed_5000) ? "5.0Gb/s":
(hw->bus.speed == ixgbe_bus_speed_2500) ? "2.5Gb/s":"Unknown"),
(hw->bus.width == ixgbe_bus_width_pcie_x8) ? "Width x8" :
(hw->bus.width == ixgbe_bus_width_pcie_x4) ? "Width x4" :
(hw->bus.width == ixgbe_bus_width_pcie_x1) ? "Width x1" :
("Unknown"));
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if ((hw->bus.width <= ixgbe_bus_width_pcie_x4) &&
(hw->bus.speed == ixgbe_bus_speed_2500)) {
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
device_printf(dev, "PCI-Express bandwidth available"
" for this card\n is not sufficient for"
" optimal performance.\n");
device_printf(dev, "For optimal performance a x8 "
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
"PCIE, or x4 PCIE 2 slot is required.\n");
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* let hardware know driver is loaded */
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
ctrl_ext = IXGBE_READ_REG(hw, IXGBE_CTRL_EXT);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ctrl_ext |= IXGBE_CTRL_EXT_DRV_LOAD;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
IXGBE_WRITE_REG(hw, IXGBE_CTRL_EXT, ctrl_ext);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
ixgbe_add_hw_stats(adapter);
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
#ifdef DEV_NETMAP
ixgbe_netmap_attach(adapter);
#endif /* DEV_NETMAP */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
INIT_DEBUGOUT("ixgbe_attach: end");
return (0);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
err_late:
ixgbe_free_transmit_structures(adapter);
ixgbe_free_receive_structures(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
err_out:
A couple fixes got clobbered, putting them back.
2010-11-26 23:57:13 +00:00
if (adapter->ifp != NULL)
if_free(adapter->ifp);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_free_pci_resources(adapter);
A couple fixes got clobbered, putting them back.
2010-11-26 23:57:13 +00:00
free(adapter->mta, M_DEVBUF);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return (error);
}
/*********************************************************************
* Device removal routine
*
* The detach entry point is called when the driver is being removed.
* This routine stops the adapter and deallocates all the resources
* that were allocated for driver operation.
*
* return 0 on success, positive on failure
*********************************************************************/
static int
ixgbe_detach(device_t dev)
{
struct adapter *adapter = device_get_softc(dev);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct ix_queue *que = adapter->queues;
Merge similar fixes from 223198 from igb to ixgbe: - Use a dedicated task to handle deferred transmits from the if_transmit method instead of reusing the existing per-queue interrupt task. Reusing the per-queue interrupt task could result in both an interrupt thread and the taskqueue thread trying to handle received packets on a single queue resulting in out-of-order packet processing and lock contention. - Don't define ixgbe_start() at all where if_transmit is used. Tested by: Vijay Singh Reviewed by: jfv MFC after: 2 weeks
2012-09-26 18:11:43 +00:00
struct tx_ring *txr = adapter->tx_rings;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
u32 ctrl_ext;
INIT_DEBUGOUT("ixgbe_detach: begin");
/* Make sure VLANS are not using driver */
if (adapter->ifp->if_vlantrunk != NULL) {
device_printf(dev,"Vlan in use, detach first\n");
return (EBUSY);
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_LOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_stop(adapter);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_UNLOCK(adapter);
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
Merge similar fixes from 223198 from igb to ixgbe: - Use a dedicated task to handle deferred transmits from the if_transmit method instead of reusing the existing per-queue interrupt task. Reusing the per-queue interrupt task could result in both an interrupt thread and the taskqueue thread trying to handle received packets on a single queue resulting in out-of-order packet processing and lock contention. - Don't define ixgbe_start() at all where if_transmit is used. Tested by: Vijay Singh Reviewed by: jfv MFC after: 2 weeks
2012-09-26 18:11:43 +00:00
for (int i = 0; i < adapter->num_queues; i++, que++, txr++) {
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if (que->tq) {
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#ifdef IXGBE_LEGACY_TX
Merge similar fixes from 223198 from igb to ixgbe: - Use a dedicated task to handle deferred transmits from the if_transmit method instead of reusing the existing per-queue interrupt task. Reusing the per-queue interrupt task could result in both an interrupt thread and the taskqueue thread trying to handle received packets on a single queue resulting in out-of-order packet processing and lock contention. - Don't define ixgbe_start() at all where if_transmit is used. Tested by: Vijay Singh Reviewed by: jfv MFC after: 2 weeks
2012-09-26 18:11:43 +00:00
taskqueue_drain(que->tq, &txr->txq_task);
#endif
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
taskqueue_drain(que->tq, &que->que_task);
taskqueue_free(que->tq);
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
}
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
/* Drain the Link queue */
if (adapter->tq) {
taskqueue_drain(adapter->tq, &adapter->link_task);
taskqueue_drain(adapter->tq, &adapter->mod_task);
taskqueue_drain(adapter->tq, &adapter->msf_task);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
#ifdef IXGBE_FDIR
taskqueue_drain(adapter->tq, &adapter->fdir_task);
#endif
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
taskqueue_free(adapter->tq);
}
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/* let hardware know driver is unloading */
ctrl_ext = IXGBE_READ_REG(&adapter->hw, IXGBE_CTRL_EXT);
ctrl_ext &= ~IXGBE_CTRL_EXT_DRV_LOAD;
IXGBE_WRITE_REG(&adapter->hw, IXGBE_CTRL_EXT, ctrl_ext);
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
/* Unregister VLAN events */
if (adapter->vlan_attach != NULL)
EVENTHANDLER_DEREGISTER(vlan_config, adapter->vlan_attach);
if (adapter->vlan_detach != NULL)
EVENTHANDLER_DEREGISTER(vlan_unconfig, adapter->vlan_detach);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ether_ifdetach(adapter->ifp);
callout_drain(&adapter->timer);
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
#ifdef DEV_NETMAP
netmap_detach(adapter->ifp);
#endif /* DEV_NETMAP */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_free_pci_resources(adapter);
bus_generic_detach(dev);
if_free(adapter->ifp);
ixgbe_free_transmit_structures(adapter);
ixgbe_free_receive_structures(adapter);
A couple fixes got clobbered, putting them back.
2010-11-26 23:57:13 +00:00
free(adapter->mta, M_DEVBUF);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_LOCK_DESTROY(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return (0);
}
/*********************************************************************
*
* Shutdown entry point
*
**********************************************************************/
static int
ixgbe_shutdown(device_t dev)
{
struct adapter *adapter = device_get_softc(dev);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_LOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_stop(adapter);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_UNLOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return (0);
}
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#ifdef IXGBE_LEGACY_TX
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*********************************************************************
* Transmit entry point
*
* ixgbe_start is called by the stack to initiate a transmit.
* The driver will remain in this routine as long as there are
* packets to transmit and transmit resources are available.
* In case resources are not available stack is notified and
* the packet is requeued.
**********************************************************************/
static void
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
ixgbe_start_locked(struct tx_ring *txr, struct ifnet * ifp)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
struct mbuf *m_head;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
struct adapter *adapter = txr->adapter;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_TX_LOCK_ASSERT(txr);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Patch #2 - remove OACTIVE and DEPLETED notions from the multiqueue code, this functionality has proven to be more trouble than it was worth. Thanks to Gleb for a second critical look over my code and help in the patches!
2012-11-30 22:33:21 +00:00
if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
if (!adapter->link_active)
return;
while (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) {
Patch #2 - remove OACTIVE and DEPLETED notions from the multiqueue code, this functionality has proven to be more trouble than it was worth. Thanks to Gleb for a second critical look over my code and help in the patches!
2012-11-30 22:33:21 +00:00
if (txr->tx_avail <= IXGBE_QUEUE_MIN_FREE)
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
break;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
IFQ_DRV_DEQUEUE(&ifp->if_snd, m_head);
if (m_head == NULL)
break;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
if (ixgbe_xmit(txr, &m_head)) {
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
if (m_head != NULL)
IFQ_DRV_PREPEND(&ifp->if_snd, m_head);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
break;
}
/* Send a copy of the frame to the BPF listener */
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
ETHER_BPF_MTAP(ifp, m_head);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* Set watchdog on */
Remove the tx queue selection based on the cpu whe no flowid is present, this was causing some bad reordering, now just use 0. Also, add a few watchdog bits, and tx handler bits that were corrected in igb.
2010-04-16 16:33:05 +00:00
txr->watchdog_time = ticks;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
txr->queue_status = IXGBE_QUEUE_WORKING;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
return;
}
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/*
* Legacy TX start - called by the stack, this
* always uses the first tx ring, and should
* not be used with multiqueue tx enabled.
*/
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static void
ixgbe_start(struct ifnet *ifp)
{
struct adapter *adapter = ifp->if_softc;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
struct tx_ring *txr = adapter->tx_rings;
if (ifp->if_drv_flags & IFF_DRV_RUNNING) {
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
IXGBE_TX_LOCK(txr);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
ixgbe_start_locked(txr, ifp);
IXGBE_TX_UNLOCK(txr);
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
}
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#else /* ! IXGBE_LEGACY_TX */
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/*
** Multiqueue Transmit driver
**
*/
static int
ixgbe_mq_start(struct ifnet *ifp, struct mbuf *m)
{
struct adapter *adapter = ifp->if_softc;
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
struct ix_queue *que;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
struct tx_ring *txr;
int i = 0, err = 0;
/* Which queue to use */
if ((m->m_flags & M_FLOWID) != 0)
i = m->m_pkthdr.flowid % adapter->num_queues;
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
else
i = curcpu % adapter->num_queues;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
txr = &adapter->tx_rings[i];
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
que = &adapter->queues[i];
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
Patch #2 - remove OACTIVE and DEPLETED notions from the multiqueue code, this functionality has proven to be more trouble than it was worth. Thanks to Gleb for a second critical look over my code and help in the patches!
2012-11-30 22:33:21 +00:00
if (IXGBE_TX_TRYLOCK(txr)) {
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
err = ixgbe_mq_start_locked(ifp, txr, m);
IXGBE_TX_UNLOCK(txr);
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
} else {
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
err = drbr_enqueue(ifp, txr->br, m);
Merge similar fixes from 223198 from igb to ixgbe: - Use a dedicated task to handle deferred transmits from the if_transmit method instead of reusing the existing per-queue interrupt task. Reusing the per-queue interrupt task could result in both an interrupt thread and the taskqueue thread trying to handle received packets on a single queue resulting in out-of-order packet processing and lock contention. - Don't define ixgbe_start() at all where if_transmit is used. Tested by: Vijay Singh Reviewed by: jfv MFC after: 2 weeks
2012-09-26 18:11:43 +00:00
taskqueue_enqueue(que->tq, &txr->txq_task);
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
}
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
return (err);
}
static int
ixgbe_mq_start_locked(struct ifnet *ifp, struct tx_ring *txr, struct mbuf *m)
{
struct adapter *adapter = txr->adapter;
struct mbuf *next;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
int enqueued, err = 0;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
if (((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) ||
adapter->link_active == 0) {
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if (m != NULL)
err = drbr_enqueue(ifp, txr->br, m);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
return (err);
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
enqueued = 0;
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
if (m == NULL) {
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
next = drbr_dequeue(ifp, txr->br);
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
} else if (drbr_needs_enqueue(ifp, txr->br)) {
if ((err = drbr_enqueue(ifp, txr->br, m)) != 0)
return (err);
next = drbr_dequeue(ifp, txr->br);
} else
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
next = m;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/* Process the queue */
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
while (next != NULL) {
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
if ((err = ixgbe_xmit(txr, &next)) != 0) {
if (next != NULL)
err = drbr_enqueue(ifp, txr->br, next);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
break;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
enqueued++;
/* Send a copy of the frame to the BPF listener */
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
ETHER_BPF_MTAP(ifp, next);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0)
break;
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
if (txr->tx_avail < IXGBE_TX_OP_THRESHOLD)
ixgbe_txeof(txr);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
next = drbr_dequeue(ifp, txr->br);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
Remove the tx queue selection based on the cpu whe no flowid is present, this was causing some bad reordering, now just use 0. Also, add a few watchdog bits, and tx handler bits that were corrected in igb.
2010-04-16 16:33:05 +00:00
if (enqueued > 0) {
/* Set watchdog on */
Patch #2 - remove OACTIVE and DEPLETED notions from the multiqueue code, this functionality has proven to be more trouble than it was worth. Thanks to Gleb for a second critical look over my code and help in the patches!
2012-11-30 22:33:21 +00:00
txr->queue_status = IXGBE_QUEUE_WORKING;
Remove the tx queue selection based on the cpu whe no flowid is present, this was causing some bad reordering, now just use 0. Also, add a few watchdog bits, and tx handler bits that were corrected in igb.
2010-04-16 16:33:05 +00:00
txr->watchdog_time = ticks;
}
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
if (txr->tx_avail < IXGBE_TX_CLEANUP_THRESHOLD)
ixgbe_txeof(txr);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
return (err);
}
Merge similar fixes from 223198 from igb to ixgbe: - Use a dedicated task to handle deferred transmits from the if_transmit method instead of reusing the existing per-queue interrupt task. Reusing the per-queue interrupt task could result in both an interrupt thread and the taskqueue thread trying to handle received packets on a single queue resulting in out-of-order packet processing and lock contention. - Don't define ixgbe_start() at all where if_transmit is used. Tested by: Vijay Singh Reviewed by: jfv MFC after: 2 weeks
2012-09-26 18:11:43 +00:00
/*
* Called from a taskqueue to drain queued transmit packets.
*/
static void
ixgbe_deferred_mq_start(void *arg, int pending)
{
struct tx_ring *txr = arg;
struct adapter *adapter = txr->adapter;
struct ifnet *ifp = adapter->ifp;
IXGBE_TX_LOCK(txr);
if (!drbr_empty(ifp, txr->br))
ixgbe_mq_start_locked(ifp, txr, NULL);
IXGBE_TX_UNLOCK(txr);
}
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/*
** Flush all ring buffers
*/
static void
ixgbe_qflush(struct ifnet *ifp)
{
struct adapter *adapter = ifp->if_softc;
struct tx_ring *txr = adapter->tx_rings;
struct mbuf *m;
for (int i = 0; i < adapter->num_queues; i++, txr++) {
IXGBE_TX_LOCK(txr);
while ((m = buf_ring_dequeue_sc(txr->br)) != NULL)
m_freem(m);
IXGBE_TX_UNLOCK(txr);
}
if_qflush(ifp);
}
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#endif /* IXGBE_LEGACY_TX */
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*********************************************************************
* Ioctl entry point
*
* ixgbe_ioctl is called when the user wants to configure the
* interface.
*
* return 0 on success, positive on failure
**********************************************************************/
static int
ixgbe_ioctl(struct ifnet * ifp, u_long command, caddr_t data)
{
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct adapter *adapter = ifp->if_softc;
Patch #3 - Add a new ioctl to access SFP+ module diagnostic data via the I2C routines in shared code.
2012-11-30 22:41:32 +00:00
struct ixgbe_hw *hw = &adapter->hw;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct ifreq *ifr = (struct ifreq *) data;
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
#if defined(INET) || defined(INET6)
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
struct ifaddr *ifa = (struct ifaddr *)data;
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
bool avoid_reset = FALSE;
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
#endif
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
int error = 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
switch (command) {
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
case SIOCSIFADDR:
#ifdef INET
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
if (ifa->ifa_addr->sa_family == AF_INET)
avoid_reset = TRUE;
#endif
#ifdef INET6
if (ifa->ifa_addr->sa_family == AF_INET6)
avoid_reset = TRUE;
#endif
#if defined(INET) || defined(INET6)
/*
** Calling init results in link renegotiation,
** so we avoid doing it when possible.
*/
if (avoid_reset) {
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
ifp->if_flags |= IFF_UP;
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
if (!(ifp->if_drv_flags & IFF_DRV_RUNNING))
ixgbe_init(adapter);
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
if (!(ifp->if_flags & IFF_NOARP))
arp_ifinit(ifp, ifa);
} else
error = ether_ioctl(ifp, command, data);
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
#endif
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
break;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
case SIOCSIFMTU:
IOCTL_DEBUGOUT("ioctl: SIOCSIFMTU (Set Interface MTU)");
if (ifr->ifr_mtu > IXGBE_MAX_FRAME_SIZE - ETHER_HDR_LEN) {
error = EINVAL;
} else {
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_LOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ifp->if_mtu = ifr->ifr_mtu;
adapter->max_frame_size =
ifp->if_mtu + ETHER_HDR_LEN + ETHER_CRC_LEN;
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
ixgbe_init_locked(adapter);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_UNLOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
break;
case SIOCSIFFLAGS:
IOCTL_DEBUGOUT("ioctl: SIOCSIFFLAGS (Set Interface Flags)");
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_LOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (ifp->if_flags & IFF_UP) {
A couple late breaking bugs that testing have turned up. - change include style so build in kernel try OR standalone work. - Limit HWCSUM - I was led to believe that it would work with RSS, but our testing had odd issues which suggests this is false. - A fatfinger error in the ioctl code made ifconfig up not work. Approved by: re
2007-07-12 19:04:11 +00:00
if ((ifp->if_drv_flags & IFF_DRV_RUNNING)) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if ((ifp->if_flags ^ adapter->if_flags) &
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
(IFF_PROMISC | IFF_ALLMULTI)) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_set_promisc(adapter);
}
} else
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
ixgbe_init_locked(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
} else
if (ifp->if_drv_flags & IFF_DRV_RUNNING)
ixgbe_stop(adapter);
adapter->if_flags = ifp->if_flags;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_UNLOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
break;
case SIOCADDMULTI:
case SIOCDELMULTI:
IOCTL_DEBUGOUT("ioctl: SIOC(ADD|DEL)MULTI");
if (ifp->if_drv_flags & IFF_DRV_RUNNING) {
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_LOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_disable_intr(adapter);
ixgbe_set_multi(adapter);
ixgbe_enable_intr(adapter);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_UNLOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
break;
case SIOCSIFMEDIA:
case SIOCGIFMEDIA:
IOCTL_DEBUGOUT("ioctl: SIOCxIFMEDIA (Get/Set Interface Media)");
error = ifmedia_ioctl(ifp, ifr, &adapter->media, command);
break;
case SIOCSIFCAP:
{
int mask = ifr->ifr_reqcap ^ ifp->if_capenable;
IOCTL_DEBUGOUT("ioctl: SIOCSIFCAP (Set Capabilities)");
if (mask & IFCAP_HWCSUM)
ifp->if_capenable ^= IFCAP_HWCSUM;
if (mask & IFCAP_TSO4)
ifp->if_capenable ^= IFCAP_TSO4;
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
if (mask & IFCAP_TSO6)
ifp->if_capenable ^= IFCAP_TSO6;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if (mask & IFCAP_LRO)
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
ifp->if_capenable ^= IFCAP_LRO;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (mask & IFCAP_VLAN_HWTAGGING)
ifp->if_capenable ^= IFCAP_VLAN_HWTAGGING;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
if (mask & IFCAP_VLAN_HWFILTER)
ifp->if_capenable ^= IFCAP_VLAN_HWFILTER;
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
if (mask & IFCAP_VLAN_HWTSO)
ifp->if_capenable ^= IFCAP_VLAN_HWTSO;
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
if (ifp->if_drv_flags & IFF_DRV_RUNNING) {
IXGBE_CORE_LOCK(adapter);
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
ixgbe_init_locked(adapter);
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
IXGBE_CORE_UNLOCK(adapter);
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
VLAN_CAPABILITIES(ifp);
break;
}
Patch #3 - Add a new ioctl to access SFP+ module diagnostic data via the I2C routines in shared code.
2012-11-30 22:41:32 +00:00
case SIOCGI2C:
{
struct ixgbe_i2c_req i2c;
IOCTL_DEBUGOUT("ioctl: SIOCGI2C (Get I2C Data)");
error = copyin(ifr->ifr_data, &i2c, sizeof(i2c));
if (error)
break;
if ((i2c.dev_addr != 0xA0) || (i2c.dev_addr != 0xA2)){
error = EINVAL;
break;
}
hw->phy.ops.read_i2c_byte(hw, i2c.offset,
i2c.dev_addr, i2c.data);
error = copyout(&i2c, ifr->ifr_data, sizeof(i2c));
break;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
default:
IOCTL_DEBUGOUT1("ioctl: UNKNOWN (0x%X)\n", (int)command);
error = ether_ioctl(ifp, command, data);
break;
}
return (error);
}
/*********************************************************************
* Init entry point
*
* This routine is used in two ways. It is used by the stack as
* init entry point in network interface structure. It is also used
* by the driver as a hw/sw initialization routine to get to a
* consistent state.
*
* return 0 on success, positive on failure
**********************************************************************/
#define IXGBE_MHADD_MFS_SHIFT 16
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
static void
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_init_locked(struct adapter *adapter)
{
struct ifnet *ifp = adapter->ifp;
device_t dev = adapter->dev;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct ixgbe_hw *hw = &adapter->hw;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
u32 k, txdctl, mhadd, gpie;
u32 rxdctl, rxctrl;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
mtx_assert(&adapter->core_mtx, MA_OWNED);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
INIT_DEBUGOUT("ixgbe_init: begin");
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
hw->adapter_stopped = FALSE;
ixgbe_stop_adapter(hw);
callout_stop(&adapter->timer);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
/* reprogram the RAR[0] in case user changed it. */
ixgbe_set_rar(hw, 0, adapter->hw.mac.addr, 0, IXGBE_RAH_AV);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Get the latest mac address, User can use a LAA */
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
bcopy(IF_LLADDR(adapter->ifp), hw->mac.addr,
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
IXGBE_ETH_LENGTH_OF_ADDRESS);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
ixgbe_set_rar(hw, 0, hw->mac.addr, 0, 1);
hw->addr_ctrl.rar_used_count = 1;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
/* Set the various hardware offload abilities */
ifp->if_hwassist = 0;
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
if (ifp->if_capenable & IFCAP_TSO)
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
ifp->if_hwassist |= CSUM_TSO;
if (ifp->if_capenable & IFCAP_TXCSUM) {
ifp->if_hwassist |= (CSUM_TCP | CSUM_UDP);
#if __FreeBSD_version >= 800000
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
if (hw->mac.type != ixgbe_mac_82598EB)
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
ifp->if_hwassist |= CSUM_SCTP;
#endif
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Prepare transmit descriptors and buffers */
if (ixgbe_setup_transmit_structures(adapter)) {
device_printf(dev,"Could not setup transmit structures\n");
ixgbe_stop(adapter);
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
return;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
ixgbe_init_hw(hw);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_initialize_transmit_units(adapter);
/* Setup Multicast table */
ixgbe_set_multi(adapter);
/*
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
** Determine the correct mbuf pool
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
** for doing jumbo frames
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
*/
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
if (adapter->max_frame_size <= 2048)
adapter->rx_mbuf_sz = MCLBYTES;
else if (adapter->max_frame_size <= 4096)
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
adapter->rx_mbuf_sz = MJUMPAGESIZE;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
else if (adapter->max_frame_size <= 9216)
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
adapter->rx_mbuf_sz = MJUM9BYTES;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
else
adapter->rx_mbuf_sz = MJUM16BYTES;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Prepare receive descriptors and buffers */
if (ixgbe_setup_receive_structures(adapter)) {
device_printf(dev,"Could not setup receive structures\n");
ixgbe_stop(adapter);
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
return;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
/* Configure RX settings */
ixgbe_initialize_receive_units(adapter);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
gpie = IXGBE_READ_REG(&adapter->hw, IXGBE_GPIE);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* Enable Fan Failure Interrupt */
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
gpie |= IXGBE_SDP1_GPIEN;
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
/* Add for Module detection */
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
if (hw->mac.type == ixgbe_mac_82599EB)
gpie |= IXGBE_SDP2_GPIEN;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
/* Thermal Failure Detection */
if (hw->mac.type == ixgbe_mac_X540)
gpie |= IXGBE_SDP0_GPIEN;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
if (adapter->msix > 1) {
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* Enable Enhanced MSIX mode */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
gpie |= IXGBE_GPIE_MSIX_MODE;
gpie |= IXGBE_GPIE_EIAME | IXGBE_GPIE_PBA_SUPPORT |
IXGBE_GPIE_OCD;
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
IXGBE_WRITE_REG(hw, IXGBE_GPIE, gpie);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Set MTU size */
if (ifp->if_mtu > ETHERMTU) {
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
mhadd = IXGBE_READ_REG(hw, IXGBE_MHADD);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
mhadd &= ~IXGBE_MHADD_MFS_MASK;
mhadd |= adapter->max_frame_size << IXGBE_MHADD_MFS_SHIFT;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
IXGBE_WRITE_REG(hw, IXGBE_MHADD, mhadd);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
/* Now enable all the queues */
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
for (int i = 0; i < adapter->num_queues; i++) {
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
txdctl = IXGBE_READ_REG(hw, IXGBE_TXDCTL(i));
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
txdctl |= IXGBE_TXDCTL_ENABLE;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* Set WTHRESH to 8, burst writeback */
txdctl |= (8 << 16);
Enable prefetching of descriptors on the TX ring, using the same values as in the Intel driver 3.8.21 for linux. The fact that it is standard in the above driver suggests that it has no bad side effects. But of course there must be a reason for enabling features, not just "it does not harm", so here it is a good one: Prefetching enables full line rate even using a single queue (14.88 Mpps, compared to ~12 Mpps without prefetch). This in turn is terribly useful when one wants to schedule traffic. For obvious reasons the difference is only visible with netmap or other high speed solutions, but presumably the advantage should be in the order of a fraction of a microsecond when starting transmission on an empty queue. Discussed with Jack Vogel. MFC after: 1 week
2012-04-11 15:02:14 +00:00
/*
* When the internal queue falls below PTHRESH (32),
* start prefetching as long as there are at least
* HTHRESH (1) buffers ready. The values are taken
* from the Intel linux driver 3.8.21.
* Prefetching enables tx line rate even with 1 queue.
*/
Correct typo(?) and actually set PTHRESH to 32 and not 16 as per Intel Linux driver 3.8.21. MFC after: 1 week
2012-06-07 22:57:26 +00:00
txdctl |= (32 << 0) | (1 << 8);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
IXGBE_WRITE_REG(hw, IXGBE_TXDCTL(i), txdctl);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
for (int i = 0; i < adapter->num_queues; i++) {
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
rxdctl = IXGBE_READ_REG(hw, IXGBE_RXDCTL(i));
if (hw->mac.type == ixgbe_mac_82598EB) {
/*
** PTHRESH = 21
** HTHRESH = 4
** WTHRESH = 8
*/
rxdctl &= ~0x3FFFFF;
rxdctl |= 0x080420;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
rxdctl |= IXGBE_RXDCTL_ENABLE;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
IXGBE_WRITE_REG(hw, IXGBE_RXDCTL(i), rxdctl);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
for (k = 0; k < 10; k++) {
if (IXGBE_READ_REG(hw, IXGBE_RXDCTL(i)) &
IXGBE_RXDCTL_ENABLE)
break;
else
msec_delay(1);
}
wmb();
netmap-related changes: 1. correct the initialization of RDT when there is an ixgbe_init() while a netmap client is active. This code was previously in ixgbe_initialize_receive_units() but RDT is overwritten shortly afterwards in ixgbe_init_locked() 2. add code (not active yet) to disable CRCSTRIP while in netmap mode. From all evidence i could gather, it seems that when the 82599 has to write a data block that is not a full cache line, it first reads the line (64 bytes) and then writes back the updated version. This hurts reception of min-sized frames, which are only 60 bytes if the CRC is stripped: i could never get above 11Mpps (received from one queue) with CRCSTRIP enabled, whyle 64+4-byte packets reach 14.2 Mpps (the theoretical maximum). Leaving the CRC in gets us 14.88Mpps for 60+4 byte frames, (and penalizes 64+4). The min-size case is important not just because it looks good in benchmarks, but also because this is the size of pure acks. Note we cannot leave CRCSTRIP on by default because it is incompatible with some other features (LRO etc.)
2012-01-19 09:36:19 +00:00
#ifdef DEV_NETMAP
/*
* In netmap mode, we must preserve the buffers made
* available to userspace before the if_init()
* (this is true by default on the TX side, because
* init makes all buffers available to userspace).
*
* netmap_reset() and the device specific routines
* (e.g. ixgbe_setup_receive_rings()) map these
* buffers at the end of the NIC ring, so here we
* must set the RDT (tail) register to make sure
* they are not overwritten.
*
* In this driver the NIC ring starts at RDH = 0,
* RDT points to the last slot available for reception (?),
* so RDT = num_rx_desc - 1 means the whole ring is available.
*/
if (ifp->if_capenable & IFCAP_NETMAP) {
struct netmap_adapter *na = NA(adapter->ifp);
struct netmap_kring *kring = &na->rx_rings[i];
int t = na->num_rx_desc - 1 - kring->nr_hwavail;
IXGBE_WRITE_REG(hw, IXGBE_RDT(i), t);
} else
#endif /* DEV_NETMAP */
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
IXGBE_WRITE_REG(hw, IXGBE_RDT(i), adapter->num_rx_desc - 1);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
/* Set up VLAN support and filter */
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
ixgbe_setup_vlan_hw_support(adapter);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/* Enable Receive engine */
rxctrl = IXGBE_READ_REG(hw, IXGBE_RXCTRL);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
if (hw->mac.type == ixgbe_mac_82598EB)
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
rxctrl |= IXGBE_RXCTRL_DMBYPS;
rxctrl |= IXGBE_RXCTRL_RXEN;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
ixgbe_enable_rx_dma(hw, rxctrl);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
callout_reset(&adapter->timer, hz, ixgbe_local_timer, adapter);
/* Set up MSI/X routing */
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
if (ixgbe_enable_msix) {
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
ixgbe_configure_ivars(adapter);
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
/* Set up auto-mask */
if (hw->mac.type == ixgbe_mac_82598EB)
IXGBE_WRITE_REG(hw, IXGBE_EIAM, IXGBE_EICS_RTX_QUEUE);
else {
IXGBE_WRITE_REG(hw, IXGBE_EIAM_EX(0), 0xFFFFFFFF);
IXGBE_WRITE_REG(hw, IXGBE_EIAM_EX(1), 0xFFFFFFFF);
}
} else { /* Simple settings for Legacy/MSI */
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
ixgbe_set_ivar(adapter, 0, 0, 0);
ixgbe_set_ivar(adapter, 0, 0, 1);
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
IXGBE_WRITE_REG(hw, IXGBE_EIAM, IXGBE_EICS_RTX_QUEUE);
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
#ifdef IXGBE_FDIR
/* Init Flow director */
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
if (hw->mac.type != ixgbe_mac_82598EB) {
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
u32 hdrm = 32 << fdir_pballoc;
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
hw->mac.ops.setup_rxpba(hw, 0, hdrm, PBA_STRATEGY_EQUAL);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
ixgbe_init_fdir_signature_82599(&adapter->hw, fdir_pballoc);
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
#endif
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
/*
** Check on any SFP devices that
** need to be kick-started
*/
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
if (hw->phy.type == ixgbe_phy_none) {
int err = hw->phy.ops.identify(hw);
if (err == IXGBE_ERR_SFP_NOT_SUPPORTED) {
device_printf(dev,
"Unsupported SFP+ module type was detected.\n");
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
return;
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
}
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
A few minor fixes: - add a moderation value to the Link vector - allow disabling HW RSC on the 82599 if LRO is not enabled. - correct error in the stats code - change optic type on the 82598 DA device Thanks to Andrew Boyer for the changes.
2010-05-14 22:00:37 +00:00
/* Set moderation on the Link interrupt */
IXGBE_WRITE_REG(hw, IXGBE_EITR(adapter->linkvec), IXGBE_LINK_ITR);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* Config/Enable Link */
ixgbe_config_link(adapter);
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
/* Hardware Packet Buffer & Flow Control setup */
{
u32 rxpb, frame, size, tmp;
frame = adapter->max_frame_size;
/* Calculate High Water */
if (hw->mac.type == ixgbe_mac_X540)
tmp = IXGBE_DV_X540(frame, frame);
else
tmp = IXGBE_DV(frame, frame);
size = IXGBE_BT2KB(tmp);
rxpb = IXGBE_READ_REG(hw, IXGBE_RXPBSIZE(0)) >> 10;
hw->fc.high_water[0] = rxpb - size;
/* Now calculate Low Water */
if (hw->mac.type == ixgbe_mac_X540)
tmp = IXGBE_LOW_DV_X540(frame);
else
tmp = IXGBE_LOW_DV(frame);
hw->fc.low_water[0] = IXGBE_BT2KB(tmp);
adapter->fc = hw->fc.requested_mode = ixgbe_fc_full;
hw->fc.pause_time = IXGBE_FC_PAUSE;
hw->fc.send_xon = TRUE;
}
/* Initialize the FC settings */
ixgbe_start_hw(hw);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* And now turn on interrupts */
ixgbe_enable_intr(adapter);
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Now inform the stack we're ready */
ifp->if_drv_flags |= IFF_DRV_RUNNING;
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
return;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
static void
ixgbe_init(void *arg)
{
struct adapter *adapter = arg;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_LOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_init_locked(adapter);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_UNLOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/*
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
**
** MSIX Interrupt Handlers and Tasklets
**
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
*/
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
static inline void
ixgbe_enable_queue(struct adapter *adapter, u32 vector)
{
struct ixgbe_hw *hw = &adapter->hw;
u64 queue = (u64)(1 << vector);
u32 mask;
if (hw->mac.type == ixgbe_mac_82598EB) {
mask = (IXGBE_EIMS_RTX_QUEUE & queue);
IXGBE_WRITE_REG(hw, IXGBE_EIMS, mask);
} else {
mask = (queue & 0xFFFFFFFF);
if (mask)
IXGBE_WRITE_REG(hw, IXGBE_EIMS_EX(0), mask);
mask = (queue >> 32);
if (mask)
IXGBE_WRITE_REG(hw, IXGBE_EIMS_EX(1), mask);
}
}
static inline void
ixgbe_disable_queue(struct adapter *adapter, u32 vector)
{
struct ixgbe_hw *hw = &adapter->hw;
u64 queue = (u64)(1 << vector);
u32 mask;
if (hw->mac.type == ixgbe_mac_82598EB) {
mask = (IXGBE_EIMS_RTX_QUEUE & queue);
IXGBE_WRITE_REG(hw, IXGBE_EIMC, mask);
} else {
mask = (queue & 0xFFFFFFFF);
if (mask)
IXGBE_WRITE_REG(hw, IXGBE_EIMC_EX(0), mask);
mask = (queue >> 32);
if (mask)
IXGBE_WRITE_REG(hw, IXGBE_EIMC_EX(1), mask);
}
}
static inline void
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
ixgbe_rearm_queues(struct adapter *adapter, u64 queues)
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
{
u32 mask;
if (adapter->hw.mac.type == ixgbe_mac_82598EB) {
mask = (IXGBE_EIMS_RTX_QUEUE & queues);
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EICS, mask);
} else {
mask = (queues & 0xFFFFFFFF);
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EICS_EX(0), mask);
mask = (queues >> 32);
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EICS_EX(1), mask);
}
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
static void
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
ixgbe_handle_que(void *context, int pending)
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
{
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct ix_queue *que = context;
struct adapter *adapter = que->adapter;
struct tx_ring *txr = que->txr;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
struct ifnet *ifp = adapter->ifp;
Remove the tx queue selection based on the cpu whe no flowid is present, this was causing some bad reordering, now just use 0. Also, add a few watchdog bits, and tx handler bits that were corrected in igb.
2010-04-16 16:33:05 +00:00
bool more;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
if (ifp->if_drv_flags & IFF_DRV_RUNNING) {
Patch #7 This is primarily about processing limit control. - add a limit for both RX and TX, change the default to 256 - change the sysctl usage to be common, and now to be called during init for each ring. - the TX limit is not yet used, but the changes in the last patch in this series uses the value. - the motivation behind these changes is to improve data locality in the final code. - rxeof interface changes since it now gets limit from the ring struct
2012-11-30 23:28:01 +00:00
more = ixgbe_rxeof(que);
Remove the tx queue selection based on the cpu whe no flowid is present, this was causing some bad reordering, now just use 0. Also, add a few watchdog bits, and tx handler bits that were corrected in igb.
2010-04-16 16:33:05 +00:00
IXGBE_TX_LOCK(txr);
ixgbe_txeof(txr);
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#ifndef IXGBE_LEGACY_TX
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
if (!drbr_empty(ifp, txr->br))
ixgbe_mq_start_locked(ifp, txr, NULL);
#else
if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd))
ixgbe_start_locked(txr, ifp);
#endif
Remove the tx queue selection based on the cpu whe no flowid is present, this was causing some bad reordering, now just use 0. Also, add a few watchdog bits, and tx handler bits that were corrected in igb.
2010-04-16 16:33:05 +00:00
IXGBE_TX_UNLOCK(txr);
Before it gets lost in the noise. Put a bandaid to prevent ixgbe(4) from completely locking up the system under high load. Our platform has a few CPU cores and a single active ixgbe(4) port with 4 queues. Under high enough traffic load, at about 7.5GBs and 700,000 packets/sec (outbound), the entire system would deadlock. What we found was that each CPU was in an endless loop on a different ix taskqueue thread. The OACTIVE flag had gotten set on each queue, and the ixgbe_handle_queue() function was continuously rescheduling itself via the taskqueue_enqueue. Since all CPUs were busy with their taskqueue threads, the ixgbe_local_timer() function couldn't run to clear the OACTIVE flag. Submitted by: scottl MFC after: 1 week
2012-06-05 18:48:02 +00:00
if (more) {
Remove the tx queue selection based on the cpu whe no flowid is present, this was causing some bad reordering, now just use 0. Also, add a few watchdog bits, and tx handler bits that were corrected in igb.
2010-04-16 16:33:05 +00:00
taskqueue_enqueue(que->tq, &que->que_task);
return;
}
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
}
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/* Reenable this interrupt */
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
ixgbe_enable_queue(adapter, que->msix);
Remove the tx queue selection based on the cpu whe no flowid is present, this was causing some bad reordering, now just use 0. Also, add a few watchdog bits, and tx handler bits that were corrected in igb.
2010-04-16 16:33:05 +00:00
return;
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*********************************************************************
*
* Legacy Interrupt Service routine
*
**********************************************************************/
static void
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
ixgbe_legacy_irq(void *arg)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct ix_queue *que = arg;
struct adapter *adapter = que->adapter;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
struct ixgbe_hw *hw = &adapter->hw;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
struct tx_ring *txr = adapter->tx_rings;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
bool more_tx, more_rx;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
u32 reg_eicr, loop = MAX_LOOP;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
reg_eicr = IXGBE_READ_REG(hw, IXGBE_EICR);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
++que->irqs;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
if (reg_eicr == 0) {
ixgbe_enable_intr(adapter);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
return;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Patch #7 This is primarily about processing limit control. - add a limit for both RX and TX, change the default to 256 - change the sysctl usage to be common, and now to be called during init for each ring. - the TX limit is not yet used, but the changes in the last patch in this series uses the value. - the motivation behind these changes is to improve data locality in the final code. - rxeof interface changes since it now gets limit from the ring struct
2012-11-30 23:28:01 +00:00
more_rx = ixgbe_rxeof(que);
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
IXGBE_TX_LOCK(txr);
do {
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
more_tx = ixgbe_txeof(txr);
} while (loop-- && more_tx);
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
IXGBE_TX_UNLOCK(txr);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if (more_rx || more_tx)
taskqueue_enqueue(que->tq, &que->que_task);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* Check for fan failure */
if ((hw->phy.media_type == ixgbe_media_type_copper) &&
(reg_eicr & IXGBE_EICR_GPI_SDP1)) {
device_printf(adapter->dev, "\nCRITICAL: FAN FAILURE!! "
"REPLACE IMMEDIATELY!!\n");
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
IXGBE_WRITE_REG(hw, IXGBE_EIMS, IXGBE_EICR_GPI_SDP1);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
}
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
/* Link status change */
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if (reg_eicr & IXGBE_EICR_LSC)
taskqueue_enqueue(adapter->tq, &adapter->link_task);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
ixgbe_enable_intr(adapter);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
return;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
/*********************************************************************
*
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
* MSIX Queue Interrupt Service routine
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
*
**********************************************************************/
void
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
ixgbe_msix_que(void *arg)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct ix_queue *que = arg;
struct adapter *adapter = que->adapter;
struct tx_ring *txr = que->txr;
struct rx_ring *rxr = que->rxr;
bool more_tx, more_rx;
u32 newitr = 0;
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
ixgbe_disable_queue(adapter, que->msix);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
++que->irqs;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Patch #7 This is primarily about processing limit control. - add a limit for both RX and TX, change the default to 256 - change the sysctl usage to be common, and now to be called during init for each ring. - the TX limit is not yet used, but the changes in the last patch in this series uses the value. - the motivation behind these changes is to improve data locality in the final code. - rxeof interface changes since it now gets limit from the ring struct
2012-11-30 23:28:01 +00:00
more_rx = ixgbe_rxeof(que);
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
IXGBE_TX_LOCK(txr);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
more_tx = ixgbe_txeof(txr);
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
/*
** Make certain that if the stack
** has anything queued the task gets
** scheduled to handle it.
*/
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#ifdef IXGBE_LEGACY_TX
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
if (!IFQ_DRV_IS_EMPTY(&adapter->ifp->if_snd))
#else
if (!drbr_empty(adapter->ifp, txr->br))
#endif
more_tx = 1;
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
IXGBE_TX_UNLOCK(txr);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
/* Do AIM now? */
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if (ixgbe_enable_aim == FALSE)
goto no_calc;
/*
** Do Adaptive Interrupt Moderation:
** - Write out last calculated setting
** - Calculate based on average size over
** the last interval.
*/
if (que->eitr_setting)
IXGBE_WRITE_REG(&adapter->hw,
IXGBE_EITR(que->msix), que->eitr_setting);
que->eitr_setting = 0;
/* Idle, do nothing */
if ((txr->bytes == 0) && (rxr->bytes == 0))
goto no_calc;
if ((txr->bytes) && (txr->packets))
newitr = txr->bytes/txr->packets;
if ((rxr->bytes) && (rxr->packets))
newitr = max(newitr,
(rxr->bytes / rxr->packets));
newitr += 24; /* account for hardware frame, crc */
/* set an upper boundary */
newitr = min(newitr, 3000);
/* Be nice to the mid range */
if ((newitr > 300) && (newitr < 1200))
newitr = (newitr / 3);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
else
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
newitr = (newitr / 2);
if (adapter->hw.mac.type == ixgbe_mac_82598EB)
newitr |= newitr << 16;
else
newitr |= IXGBE_EITR_CNT_WDIS;
/* save for next interrupt */
que->eitr_setting = newitr;
/* Reset state */
txr->bytes = 0;
txr->packets = 0;
rxr->bytes = 0;
rxr->packets = 0;
no_calc:
if (more_tx || more_rx)
taskqueue_enqueue(que->tq, &que->que_task);
else /* Reenable this interrupt */
ixgbe_enable_queue(adapter, que->msix);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
return;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
static void
ixgbe_msix_link(void *arg)
{
struct adapter *adapter = arg;
struct ixgbe_hw *hw = &adapter->hw;
u32 reg_eicr;
++adapter->link_irq;
/* First get the cause */
reg_eicr = IXGBE_READ_REG(hw, IXGBE_EICS);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/* Clear interrupt with write */
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
IXGBE_WRITE_REG(hw, IXGBE_EICR, reg_eicr);
/* Link status change */
if (reg_eicr & IXGBE_EICR_LSC)
taskqueue_enqueue(adapter->tq, &adapter->link_task);
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
if (adapter->hw.mac.type != ixgbe_mac_82598EB) {
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
#ifdef IXGBE_FDIR
if (reg_eicr & IXGBE_EICR_FLOW_DIR) {
/* This is probably overkill :) */
if (!atomic_cmpset_int(&adapter->fdir_reinit, 0, 1))
return;
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
/* Disable the interrupt */
IXGBE_WRITE_REG(hw, IXGBE_EIMC, IXGBE_EICR_FLOW_DIR);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
taskqueue_enqueue(adapter->tq, &adapter->fdir_task);
} else
#endif
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
if (reg_eicr & IXGBE_EICR_ECC) {
device_printf(adapter->dev, "\nCRITICAL: ECC ERROR!! "
"Please Reboot!!\n");
IXGBE_WRITE_REG(hw, IXGBE_EICR, IXGBE_EICR_ECC);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
} else
if (reg_eicr & IXGBE_EICR_GPI_SDP1) {
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
/* Clear the interrupt */
IXGBE_WRITE_REG(hw, IXGBE_EICR, IXGBE_EICR_GPI_SDP1);
taskqueue_enqueue(adapter->tq, &adapter->msf_task);
} else if (reg_eicr & IXGBE_EICR_GPI_SDP2) {
/* Clear the interrupt */
IXGBE_WRITE_REG(hw, IXGBE_EICR, IXGBE_EICR_GPI_SDP2);
taskqueue_enqueue(adapter->tq, &adapter->mod_task);
}
}
/* Check for fan failure */
if ((hw->device_id == IXGBE_DEV_ID_82598AT) &&
(reg_eicr & IXGBE_EICR_GPI_SDP1)) {
device_printf(adapter->dev, "\nCRITICAL: FAN FAILURE!! "
"REPLACE IMMEDIATELY!!\n");
IXGBE_WRITE_REG(hw, IXGBE_EICR, IXGBE_EICR_GPI_SDP1);
}
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
/* Check for over temp condition */
if ((hw->mac.type == ixgbe_mac_X540) &&
(reg_eicr & IXGBE_EICR_GPI_SDP0)) {
device_printf(adapter->dev, "\nCRITICAL: OVER TEMP!! "
"PHY IS SHUT DOWN!!\n");
device_printf(adapter->dev, "System shutdown required\n");
IXGBE_WRITE_REG(hw, IXGBE_EICR, IXGBE_EICR_GPI_SDP0);
}
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EIMS, IXGBE_EIMS_OTHER);
return;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*********************************************************************
*
* Media Ioctl callback
*
* This routine is called whenever the user queries the status of
* the interface using ifconfig.
*
**********************************************************************/
static void
ixgbe_media_status(struct ifnet * ifp, struct ifmediareq * ifmr)
{
struct adapter *adapter = ifp->if_softc;
INIT_DEBUGOUT("ixgbe_media_status: begin");
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_CORE_LOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_update_link_status(adapter);
ifmr->ifm_status = IFM_AVALID;
ifmr->ifm_active = IFM_ETHER;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
if (!adapter->link_active) {
IXGBE_CORE_UNLOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ifmr->ifm_status |= IFM_ACTIVE;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
switch (adapter->link_speed) {
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
case IXGBE_LINK_SPEED_100_FULL:
ifmr->ifm_active |= IFM_100_TX | IFM_FDX;
break;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
case IXGBE_LINK_SPEED_1GB_FULL:
A few important fixes: - Testing TSO6 has led me to discover that HW RSC is a problematic feature, it is ONLY designed to work with IPv4 in the first place, and if IP forwarding is done it can't be disabled as LRO in the stack, also initial testing we've done at Intel shows an equal performance using TSO[46] on the TX and LRO on RX, if you ran older code on 82599 or later hardware you actually could have detrimental performance for this reason. So I am disabling the feature by default and all our adapters will now use LRO instead. - If you have flow control off and multiple queues it was possible when the buffer of one queue becomes full that all RX movement is stalled, to eliminate this problem a feature bit is now set that will allow packets to be dropped when full rather than stall. Note, the default is to have flow control on, and this keeps this from happening. - Because of the recent fixes in the stack, LRO is now auto-disabled when problematic, so I have decided to enable it by default in the capabilities in the driver. - There are some 1G modules used by some customers, a couple small tweaks to properly support those in the media code. - A note: we have now done some testing of TSO6 and using LRO with IPv6 and it all works great!! Seeing line rate in both directions in best cases. Thanks bz for your excellent work!!
2012-10-31 23:50:36 +00:00
ifmr->ifm_active |= adapter->optics | IFM_FDX;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
break;
case IXGBE_LINK_SPEED_10GB_FULL:
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
ifmr->ifm_active |= adapter->optics | IFM_FDX;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
break;
}
IXGBE_CORE_UNLOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
}
/*********************************************************************
*
* Media Ioctl callback
*
* This routine is called when the user changes speed/duplex using
* media/mediopt option with ifconfig.
*
**********************************************************************/
static int
ixgbe_media_change(struct ifnet * ifp)
{
struct adapter *adapter = ifp->if_softc;
struct ifmedia *ifm = &adapter->media;
INIT_DEBUGOUT("ixgbe_media_change: begin");
if (IFM_TYPE(ifm->ifm_media) != IFM_ETHER)
return (EINVAL);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
switch (IFM_SUBTYPE(ifm->ifm_media)) {
case IFM_AUTO:
adapter->hw.phy.autoneg_advertised =
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
IXGBE_LINK_SPEED_100_FULL |
IXGBE_LINK_SPEED_1GB_FULL |
IXGBE_LINK_SPEED_10GB_FULL;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
break;
default:
device_printf(adapter->dev, "Only auto media type\n");
return (EINVAL);
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return (0);
}
/*********************************************************************
*
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
* This routine maps the mbufs to tx descriptors, allowing the
* TX engine to transmit the packets.
* - return 0 on success, positive on failure
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
*
**********************************************************************/
static int
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
ixgbe_xmit(struct tx_ring *txr, struct mbuf **m_headp)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
struct adapter *adapter = txr->adapter;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
u32 olinfo_status = 0, cmd_type_len;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
int i, j, error, nsegs;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
int first, last = 0;
Patch #9 Performance - improve the tx dma failure path, similar to a change done in igb long ago.
2012-11-30 23:54:57 +00:00
bool remap = TRUE;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
struct mbuf *m_head;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
bus_dma_segment_t segs[adapter->num_segs];
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
bus_dmamap_t map;
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
struct ixgbe_tx_buf *txbuf;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
union ixgbe_adv_tx_desc *txd = NULL;
m_head = *m_headp;
/* Basic descriptor defines */
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
cmd_type_len = (IXGBE_ADVTXD_DTYP_DATA |
IXGBE_ADVTXD_DCMD_IFCS | IXGBE_ADVTXD_DCMD_DEXT);
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (m_head->m_flags & M_VLANTAG)
cmd_type_len |= IXGBE_ADVTXD_DCMD_VLE;
/*
* Important to capture the first descriptor
* used because it will contain the index of
* the one we tell the hardware to report back
*/
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
first = txr->next_avail_desc;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
txbuf = &txr->tx_buffers[first];
map = txbuf->map;
/*
* Map the packet for DMA.
*/
Patch #9 Performance - improve the tx dma failure path, similar to a change done in igb long ago.
2012-11-30 23:54:57 +00:00
retry:
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
error = bus_dmamap_load_mbuf_sg(txr->txtag, map,
*m_headp, segs, &nsegs, BUS_DMA_NOWAIT);
Patch #9 Performance - improve the tx dma failure path, similar to a change done in igb long ago.
2012-11-30 23:54:57 +00:00
if (__predict_false(error)) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
struct mbuf *m;
Patch #9 Performance - improve the tx dma failure path, similar to a change done in igb long ago.
2012-11-30 23:54:57 +00:00
switch (error) {
case EFBIG:
/* Try it again? - one try */
if (remap == TRUE) {
remap = FALSE;
m = m_defrag(*m_headp, M_NOWAIT);
if (m == NULL) {
adapter->mbuf_defrag_failed++;
m_freem(*m_headp);
*m_headp = NULL;
return (ENOBUFS);
}
*m_headp = m;
goto retry;
} else
return (error);
case ENOMEM:
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
txr->no_tx_dma_setup++;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return (error);
Patch #9 Performance - improve the tx dma failure path, similar to a change done in igb long ago.
2012-11-30 23:54:57 +00:00
default:
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
txr->no_tx_dma_setup++;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
m_freem(*m_headp);
*m_headp = NULL;
return (error);
}
}
/* Make certain there are enough descriptors */
if (nsegs > txr->tx_avail - 2) {
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
txr->no_desc_avail++;
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
bus_dmamap_unload(txr->txtag, map);
return (ENOBUFS);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
m_head = *m_headp;
/*
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
** Set up the appropriate offload context
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
** this will consume the first descriptor
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
*/
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
error = ixgbe_tx_ctx_setup(txr, m_head, &cmd_type_len, &olinfo_status);
if (__predict_false(error)) {
if (error == ENOBUFS)
*m_headp = NULL;
return (error);
}
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
#ifdef IXGBE_FDIR
/* Do the flow director magic */
if ((txr->atr_sample) && (!adapter->fdir_reinit)) {
++txr->atr_count;
if (txr->atr_count >= atr_sample_rate) {
ixgbe_atr(txr, m_head);
txr->atr_count = 0;
}
}
#endif
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
i = txr->next_avail_desc;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
for (j = 0; j < nsegs; j++) {
bus_size_t seglen;
bus_addr_t segaddr;
txbuf = &txr->tx_buffers[i];
txd = &txr->tx_base[i];
seglen = segs[j].ds_len;
segaddr = htole64(segs[j].ds_addr);
txd->read.buffer_addr = segaddr;
txd->read.cmd_type_len = htole32(txr->txd_cmd |
cmd_type_len |seglen);
txd->read.olinfo_status = htole32(olinfo_status);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
last = i; /* descriptor that will get completion IRQ */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
if (++i == txr->num_desc)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
i = 0;
txbuf->m_head = NULL;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
txbuf->eop_index = -1;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
txd->read.cmd_type_len |=
htole32(IXGBE_TXD_CMD_EOP | IXGBE_TXD_CMD_RS);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
txr->tx_avail -= nsegs;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
txr->next_avail_desc = i;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
txbuf->m_head = m_head;
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
/*
** Here we swap the map so the last descriptor,
** which gets the completion interrupt has the
** real map, and the first descriptor gets the
** unused map from this descriptor.
*/
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
txr->tx_buffers[first].map = txbuf->map;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
txbuf->map = map;
bus_dmamap_sync(txr->txtag, map, BUS_DMASYNC_PREWRITE);
/* Set the index of the descriptor that will be marked done */
txbuf = &txr->tx_buffers[first];
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
txbuf->eop_index = last;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
bus_dmamap_sync(txr->txdma.dma_tag, txr->txdma.dma_map,
BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE);
/*
* Advance the Transmit Descriptor Tail (Tdt), this tells the
* hardware that this frame is available to transmit.
*/
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
++txr->total_packets;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
IXGBE_WRITE_REG(&adapter->hw, IXGBE_TDT(txr->me), i);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return (0);
}
static void
ixgbe_set_promisc(struct adapter *adapter)
{
u_int32_t reg_rctl;
struct ifnet *ifp = adapter->ifp;
reg_rctl = IXGBE_READ_REG(&adapter->hw, IXGBE_FCTRL);
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
reg_rctl &= (~IXGBE_FCTRL_UPE);
reg_rctl &= (~IXGBE_FCTRL_MPE);
IXGBE_WRITE_REG(&adapter->hw, IXGBE_FCTRL, reg_rctl);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (ifp->if_flags & IFF_PROMISC) {
reg_rctl |= (IXGBE_FCTRL_UPE | IXGBE_FCTRL_MPE);
IXGBE_WRITE_REG(&adapter->hw, IXGBE_FCTRL, reg_rctl);
} else if (ifp->if_flags & IFF_ALLMULTI) {
reg_rctl |= IXGBE_FCTRL_MPE;
reg_rctl &= ~IXGBE_FCTRL_UPE;
IXGBE_WRITE_REG(&adapter->hw, IXGBE_FCTRL, reg_rctl);
}
return;
}
/*********************************************************************
* Multicast Update
*
* This routine is called whenever multicast address list is updated.
*
**********************************************************************/
#define IXGBE_RAR_ENTRIES 16
static void
ixgbe_set_multi(struct adapter *adapter)
{
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
u32 fctrl;
A couple fixes got clobbered, putting them back.
2010-11-26 23:57:13 +00:00
u8 *mta;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
u8 *update_ptr;
struct ifmultiaddr *ifma;
int mcnt = 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
struct ifnet *ifp = adapter->ifp;
IOCTL_DEBUGOUT("ixgbe_set_multi: begin");
A couple fixes got clobbered, putting them back.
2010-11-26 23:57:13 +00:00
mta = adapter->mta;
bzero(mta, sizeof(u8) * IXGBE_ETH_LENGTH_OF_ADDRESS *
MAX_NUM_MULTICAST_ADDRESSES);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
fctrl = IXGBE_READ_REG(&adapter->hw, IXGBE_FCTRL);
fctrl |= (IXGBE_FCTRL_UPE | IXGBE_FCTRL_MPE);
if (ifp->if_flags & IFF_PROMISC)
fctrl |= (IXGBE_FCTRL_UPE | IXGBE_FCTRL_MPE);
else if (ifp->if_flags & IFF_ALLMULTI) {
fctrl |= IXGBE_FCTRL_MPE;
fctrl &= ~IXGBE_FCTRL_UPE;
} else
fctrl &= ~(IXGBE_FCTRL_UPE | IXGBE_FCTRL_MPE);
IXGBE_WRITE_REG(&adapter->hw, IXGBE_FCTRL, fctrl);
A couple of changes: first, make header split on by default and remove from its dependency on LRO, my tests have shown that its always beneficial, even when doing bridging. Second, fix up a few problems in the statistics code, the adapter dependencies had gotten lost so some code that should only run on 82599 was always running, this resulted in bogus flow control numbers on 82598.
2009-08-24 20:41:51 +00:00
#if __FreeBSD_version < 800000
IF_ADDR_LOCK(ifp);
#else
Use if_maddr_rlock()/if_maddr_runlock() rather than IF_ADDR_LOCK()/ IF_ADDR_UNLOCK() across network device drivers when accessing the per-interface multicast address list, if_multiaddrs. This will allow us to change the locking strategy without affecting our driver programming interface or binary interface. For two wireless drivers, remove unnecessary locking, since they don't actually access the multicast address list. Approved by: re (kib) MFC after: 6 weeks
2009-06-26 11:45:06 +00:00
if_maddr_rlock(ifp);
A couple of changes: first, make header split on by default and remove from its dependency on LRO, my tests have shown that its always beneficial, even when doing bridging. Second, fix up a few problems in the statistics code, the adapter dependencies had gotten lost so some code that should only run on 82599 was always running, this resulted in bogus flow control numbers on 82598.
2009-08-24 20:41:51 +00:00
#endif
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
TAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link) {
if (ifma->ifma_addr->sa_family != AF_LINK)
continue;
bcopy(LLADDR((struct sockaddr_dl *) ifma->ifma_addr),
&mta[mcnt * IXGBE_ETH_LENGTH_OF_ADDRESS],
IXGBE_ETH_LENGTH_OF_ADDRESS);
mcnt++;
}
A couple of changes: first, make header split on by default and remove from its dependency on LRO, my tests have shown that its always beneficial, even when doing bridging. Second, fix up a few problems in the statistics code, the adapter dependencies had gotten lost so some code that should only run on 82599 was always running, this resulted in bogus flow control numbers on 82598.
2009-08-24 20:41:51 +00:00
#if __FreeBSD_version < 800000
IF_ADDR_UNLOCK(ifp);
#else
Use if_maddr_rlock()/if_maddr_runlock() rather than IF_ADDR_LOCK()/ IF_ADDR_UNLOCK() across network device drivers when accessing the per-interface multicast address list, if_multiaddrs. This will allow us to change the locking strategy without affecting our driver programming interface or binary interface. For two wireless drivers, remove unnecessary locking, since they don't actually access the multicast address list. Approved by: re (kib) MFC after: 6 weeks
2009-06-26 11:45:06 +00:00
if_maddr_runlock(ifp);
A couple of changes: first, make header split on by default and remove from its dependency on LRO, my tests have shown that its always beneficial, even when doing bridging. Second, fix up a few problems in the statistics code, the adapter dependencies had gotten lost so some code that should only run on 82599 was always running, this resulted in bogus flow control numbers on 82598.
2009-08-24 20:41:51 +00:00
#endif
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
update_ptr = mta;
ixgbe_update_mc_addr_list(&adapter->hw,
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
update_ptr, mcnt, ixgbe_mc_array_itr, TRUE);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/*
* This is an iterator function now needed by the multicast
* shared code. It simply feeds the shared code routine the
* addresses in the array of ixgbe_set_multi() one by one.
*/
static u8 *
ixgbe_mc_array_itr(struct ixgbe_hw *hw, u8 **update_ptr, u32 *vmdq)
{
u8 *addr = *update_ptr;
u8 *newptr;
*vmdq = 0;
newptr = addr + IXGBE_ETH_LENGTH_OF_ADDRESS;
*update_ptr = newptr;
return addr;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*********************************************************************
* Timer routine
*
* This routine checks for link status,updates statistics,
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
* and runs the watchdog check.
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
*
**********************************************************************/
static void
ixgbe_local_timer(void *arg)
{
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
struct adapter *adapter = arg;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
device_t dev = adapter->dev;
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
struct ix_queue *que = adapter->queues;
struct tx_ring *txr = adapter->tx_rings;
Patch #2 - remove OACTIVE and DEPLETED notions from the multiqueue code, this functionality has proven to be more trouble than it was worth. Thanks to Gleb for a second critical look over my code and help in the patches!
2012-11-30 22:33:21 +00:00
int hung = 0, paused = 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
mtx_assert(&adapter->core_mtx, MA_OWNED);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/* Check for pluggable optics */
if (adapter->sfp_probe)
if (!ixgbe_sfp_probe(adapter))
goto out; /* Nothing to do */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_update_link_status(adapter);
ixgbe_update_stats_counters(adapter);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
/*
* If the interface has been paused
* then don't do the watchdog check
*/
if (IXGBE_READ_REG(&adapter->hw, IXGBE_TFCS) & IXGBE_TFCS_TXOFF)
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
paused = 1;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
** Check the TX queues status
** - watchdog only if all queues show hung
*/
for (int i = 0; i < adapter->num_queues; i++, que++, txr++) {
Patch #2 - remove OACTIVE and DEPLETED notions from the multiqueue code, this functionality has proven to be more trouble than it was worth. Thanks to Gleb for a second critical look over my code and help in the patches!
2012-11-30 22:33:21 +00:00
if ((txr->queue_status == IXGBE_QUEUE_HUNG) &&
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
(paused == 0))
++hung;
Patch #2 - remove OACTIVE and DEPLETED notions from the multiqueue code, this functionality has proven to be more trouble than it was worth. Thanks to Gleb for a second critical look over my code and help in the patches!
2012-11-30 22:33:21 +00:00
else if (txr->queue_status == IXGBE_QUEUE_WORKING)
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
taskqueue_enqueue(que->tq, &que->que_task);
}
/* Only truely watchdog if all queues show hung */
if (hung == adapter->num_queues)
goto watchdog;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
out:
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
ixgbe_rearm_queues(adapter, adapter->que_mask);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
callout_reset(&adapter->timer, hz, ixgbe_local_timer, adapter);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
return;
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
watchdog:
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
device_printf(adapter->dev, "Watchdog timeout -- resetting\n");
device_printf(dev,"Queue(%d) tdh = %d, hw tdt = %d\n", txr->me,
IXGBE_READ_REG(&adapter->hw, IXGBE_TDH(txr->me)),
IXGBE_READ_REG(&adapter->hw, IXGBE_TDT(txr->me)));
device_printf(dev,"TX(%d) desc avail = %d,"
"Next TX to Clean = %d\n",
txr->me, txr->tx_avail, txr->next_to_clean);
adapter->ifp->if_drv_flags &= ~IFF_DRV_RUNNING;
adapter->watchdog_events++;
ixgbe_init_locked(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
/*
** Note: this routine updates the OS on the link state
** the real check of the hardware only happens with
** a link interrupt.
*/
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static void
ixgbe_update_link_status(struct adapter *adapter)
{
struct ifnet *ifp = adapter->ifp;
device_t dev = adapter->dev;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
if (adapter->link_up){
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (adapter->link_active == FALSE) {
if (bootverbose)
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
device_printf(dev,"Link is up %d Gbps %s \n",
((adapter->link_speed == 128)? 10:1),
"Full Duplex");
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
adapter->link_active = TRUE;
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
/* Update any Flow Control changes */
ixgbe_fc_enable(&adapter->hw);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if_link_state_change(ifp, LINK_STATE_UP);
}
} else { /* Link down */
if (adapter->link_active == TRUE) {
if (bootverbose)
device_printf(dev,"Link is Down\n");
if_link_state_change(ifp, LINK_STATE_DOWN);
adapter->link_active = FALSE;
}
}
return;
}
/*********************************************************************
*
* This routine disables all traffic on the adapter by issuing a
* global reset on the MAC and deallocates TX/RX buffers.
*
**********************************************************************/
static void
ixgbe_stop(void *arg)
{
struct ifnet *ifp;
struct adapter *adapter = arg;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
struct ixgbe_hw *hw = &adapter->hw;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ifp = adapter->ifp;
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
mtx_assert(&adapter->core_mtx, MA_OWNED);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
INIT_DEBUGOUT("ixgbe_stop: begin\n");
ixgbe_disable_intr(adapter);
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
callout_stop(&adapter->timer);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
/* Let the stack know...*/
ifp->if_drv_flags &= ~IFF_DRV_RUNNING;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
ixgbe_reset_hw(hw);
hw->adapter_stopped = FALSE;
ixgbe_stop_adapter(hw);
/* Turn off the laser */
if (hw->phy.multispeed_fiber)
ixgbe_disable_tx_laser(hw);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
/* reprogram the RAR[0] in case user changed it. */
ixgbe_set_rar(&adapter->hw, 0, adapter->hw.mac.addr, 0, IXGBE_RAH_AV);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
}
/*********************************************************************
*
* Determine hardware revision.
*
**********************************************************************/
static void
ixgbe_identify_hardware(struct adapter *adapter)
{
device_t dev = adapter->dev;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
struct ixgbe_hw *hw = &adapter->hw;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Save off the information about this board */
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
hw->vendor_id = pci_get_vendor(dev);
hw->device_id = pci_get_device(dev);
hw->revision_id = pci_read_config(dev, PCIR_REVID, 1);
hw->subsystem_vendor_id =
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
pci_read_config(dev, PCIR_SUBVEND_0, 2);
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
hw->subsystem_device_id =
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
pci_read_config(dev, PCIR_SUBDEV_0, 2);
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
/* We need this here to set the num_segs below */
ixgbe_set_mac_type(hw);
/* Pick up the 82599 and VF settings */
if (hw->mac.type != ixgbe_mac_82598EB) {
hw->phy.smart_speed = ixgbe_smart_speed;
adapter->num_segs = IXGBE_82599_SCATTER;
} else
adapter->num_segs = IXGBE_82598_SCATTER;
return;
}
/*********************************************************************
*
* Determine optic type
*
**********************************************************************/
static void
ixgbe_setup_optics(struct adapter *adapter)
{
struct ixgbe_hw *hw = &adapter->hw;
int layer;
layer = ixgbe_get_supported_physical_layer(hw);
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_T) {
adapter->optics = IFM_10G_T;
return;
}
if (layer & IXGBE_PHYSICAL_LAYER_1000BASE_T) {
adapter->optics = IFM_1000_T;
return;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
}
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
A few important fixes: - Testing TSO6 has led me to discover that HW RSC is a problematic feature, it is ONLY designed to work with IPv4 in the first place, and if IP forwarding is done it can't be disabled as LRO in the stack, also initial testing we've done at Intel shows an equal performance using TSO[46] on the TX and LRO on RX, if you ran older code on 82599 or later hardware you actually could have detrimental performance for this reason. So I am disabling the feature by default and all our adapters will now use LRO instead. - If you have flow control off and multiple queues it was possible when the buffer of one queue becomes full that all RX movement is stalled, to eliminate this problem a feature bit is now set that will allow packets to be dropped when full rather than stall. Note, the default is to have flow control on, and this keeps this from happening. - Because of the recent fixes in the stack, LRO is now auto-disabled when problematic, so I have decided to enable it by default in the capabilities in the driver. - There are some 1G modules used by some customers, a couple small tweaks to properly support those in the media code. - A note: we have now done some testing of TSO6 and using LRO with IPv6 and it all works great!! Seeing line rate in both directions in best cases. Thanks bz for your excellent work!!
2012-10-31 23:50:36 +00:00
if (layer & IXGBE_PHYSICAL_LAYER_1000BASE_SX) {
adapter->optics = IFM_1000_SX;
return;
}
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
if (layer & (IXGBE_PHYSICAL_LAYER_10GBASE_LR |
IXGBE_PHYSICAL_LAYER_10GBASE_LRM)) {
adapter->optics = IFM_10G_LR;
return;
}
if (layer & IXGBE_PHYSICAL_LAYER_10GBASE_SR) {
adapter->optics = IFM_10G_SR;
return;
}
if (layer & IXGBE_PHYSICAL_LAYER_SFP_PLUS_CU) {
adapter->optics = IFM_10G_TWINAX;
return;
}
if (layer & (IXGBE_PHYSICAL_LAYER_10GBASE_KX4 |
IXGBE_PHYSICAL_LAYER_10GBASE_CX4)) {
adapter->optics = IFM_10G_CX4;
return;
}
/* If we get here just set the default */
adapter->optics = IFM_ETHER | IFM_AUTO;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
}
/*********************************************************************
*
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
* Setup the Legacy or MSI Interrupt handler
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
*
**********************************************************************/
static int
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
ixgbe_allocate_legacy(struct adapter *adapter)
{
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
device_t dev = adapter->dev;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct ix_queue *que = adapter->queues;
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#ifndef IXGBE_LEGACY_TX
Merge similar fixes from 223198 from igb to ixgbe: - Use a dedicated task to handle deferred transmits from the if_transmit method instead of reusing the existing per-queue interrupt task. Reusing the per-queue interrupt task could result in both an interrupt thread and the taskqueue thread trying to handle received packets on a single queue resulting in out-of-order packet processing and lock contention. - Don't define ixgbe_start() at all where if_transmit is used. Tested by: Vijay Singh Reviewed by: jfv MFC after: 2 weeks
2012-09-26 18:11:43 +00:00
struct tx_ring *txr = adapter->tx_rings;
#endif
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
int error, rid = 0;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/* MSI RID at 1 */
if (adapter->msix == 1)
rid = 1;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* We allocate a single interrupt resource */
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
adapter->res = bus_alloc_resource_any(dev,
SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE);
if (adapter->res == NULL) {
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
device_printf(dev, "Unable to allocate bus resource: "
"interrupt\n");
return (ENXIO);
}
/*
* Try allocating a fast interrupt and the associated deferred
* processing contexts.
*/
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#ifndef IXGBE_LEGACY_TX
Merge similar fixes from 223198 from igb to ixgbe: - Use a dedicated task to handle deferred transmits from the if_transmit method instead of reusing the existing per-queue interrupt task. Reusing the per-queue interrupt task could result in both an interrupt thread and the taskqueue thread trying to handle received packets on a single queue resulting in out-of-order packet processing and lock contention. - Don't define ixgbe_start() at all where if_transmit is used. Tested by: Vijay Singh Reviewed by: jfv MFC after: 2 weeks
2012-09-26 18:11:43 +00:00
TASK_INIT(&txr->txq_task, 0, ixgbe_deferred_mq_start, txr);
#endif
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
TASK_INIT(&que->que_task, 0, ixgbe_handle_que, que);
que->tq = taskqueue_create_fast("ixgbe_que", M_NOWAIT,
taskqueue_thread_enqueue, &que->tq);
taskqueue_start_threads(&que->tq, 1, PI_NET, "%s ixq",
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
device_get_nameunit(adapter->dev));
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/* Tasklets for Link, SFP and Multispeed Fiber */
TASK_INIT(&adapter->link_task, 0, ixgbe_handle_link, adapter);
TASK_INIT(&adapter->mod_task, 0, ixgbe_handle_mod, adapter);
TASK_INIT(&adapter->msf_task, 0, ixgbe_handle_msf, adapter);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
#ifdef IXGBE_FDIR
TASK_INIT(&adapter->fdir_task, 0, ixgbe_reinit_fdir, adapter);
#endif
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
adapter->tq = taskqueue_create_fast("ixgbe_link", M_NOWAIT,
taskqueue_thread_enqueue, &adapter->tq);
taskqueue_start_threads(&adapter->tq, 1, PI_NET, "%s linkq",
device_get_nameunit(adapter->dev));
if ((error = bus_setup_intr(dev, adapter->res,
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
INTR_TYPE_NET | INTR_MPSAFE, NULL, ixgbe_legacy_irq,
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
que, &adapter->tag)) != 0) {
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
device_printf(dev, "Failed to register fast interrupt "
"handler: %d\n", error);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
taskqueue_free(que->tq);
taskqueue_free(adapter->tq);
que->tq = NULL;
adapter->tq = NULL;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
return (error);
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
/* For simplicity in the handlers */
adapter->que_mask = IXGBE_EIMS_ENABLE_MASK;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
return (0);
}
/*********************************************************************
*
* Setup MSIX Interrupt resources and handlers
*
**********************************************************************/
static int
ixgbe_allocate_msix(struct adapter *adapter)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
device_t dev = adapter->dev;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct ix_queue *que = adapter->queues;
Merge similar fixes from 223198 from igb to ixgbe: - Use a dedicated task to handle deferred transmits from the if_transmit method instead of reusing the existing per-queue interrupt task. Reusing the per-queue interrupt task could result in both an interrupt thread and the taskqueue thread trying to handle received packets on a single queue resulting in out-of-order packet processing and lock contention. - Don't define ixgbe_start() at all where if_transmit is used. Tested by: Vijay Singh Reviewed by: jfv MFC after: 2 weeks
2012-09-26 18:11:43 +00:00
struct tx_ring *txr = adapter->tx_rings;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
int error, rid, vector = 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Merge similar fixes from 223198 from igb to ixgbe: - Use a dedicated task to handle deferred transmits from the if_transmit method instead of reusing the existing per-queue interrupt task. Reusing the per-queue interrupt task could result in both an interrupt thread and the taskqueue thread trying to handle received packets on a single queue resulting in out-of-order packet processing and lock contention. - Don't define ixgbe_start() at all where if_transmit is used. Tested by: Vijay Singh Reviewed by: jfv MFC after: 2 weeks
2012-09-26 18:11:43 +00:00
for (int i = 0; i < adapter->num_queues; i++, vector++, que++, txr++) {
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
rid = vector + 1;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
que->res = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid,
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
RF_SHAREABLE | RF_ACTIVE);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if (que->res == NULL) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
device_printf(dev,"Unable to allocate"
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
" bus resource: que interrupt [%d]\n", vector);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return (ENXIO);
}
/* Set the handler function */
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
error = bus_setup_intr(dev, que->res,
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
INTR_TYPE_NET | INTR_MPSAFE, NULL,
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
ixgbe_msix_que, que, &que->tag);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (error) {
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
que->res = NULL;
device_printf(dev, "Failed to register QUE handler");
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return (error);
}
Remove a disable_queue from the beginning of the interrupt handler, automask handles it. Also, add in msix vector descriptions. MFC for 8.1 asap
2010-06-11 19:03:59 +00:00
#if __FreeBSD_version >= 800504
bus_describe_intr(dev, que->res, que->tag, "que %d", i);
#endif
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
que->msix = vector;
adapter->que_mask |= (u64)(1 << que->msix);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/*
** Bind the msix vector, and thus the
** ring to the corresponding cpu.
*/
Decided to limit the interrupt bind to multiqueue config as done in igb.
2009-06-25 18:40:27 +00:00
if (adapter->num_queues > 1)
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
bus_bind_intr(dev, que->res, i);
Change intr_bind to bus_bind_intr, thanks to John Baldwin for pointing out this simplification.
2009-06-25 17:16:26 +00:00
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#ifdef IXGBE_LEGACY_TX
Merge similar fixes from 223198 from igb to ixgbe: - Use a dedicated task to handle deferred transmits from the if_transmit method instead of reusing the existing per-queue interrupt task. Reusing the per-queue interrupt task could result in both an interrupt thread and the taskqueue thread trying to handle received packets on a single queue resulting in out-of-order packet processing and lock contention. - Don't define ixgbe_start() at all where if_transmit is used. Tested by: Vijay Singh Reviewed by: jfv MFC after: 2 weeks
2012-09-26 18:11:43 +00:00
TASK_INIT(&txr->txq_task, 0, ixgbe_deferred_mq_start, txr);
#endif
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
TASK_INIT(&que->que_task, 0, ixgbe_handle_que, que);
que->tq = taskqueue_create_fast("ixgbe_que", M_NOWAIT,
taskqueue_thread_enqueue, &que->tq);
taskqueue_start_threads(&que->tq, 1, PI_NET, "%s que",
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
device_get_nameunit(adapter->dev));
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
/* and Link */
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
rid = vector + 1;
adapter->res = bus_alloc_resource_any(dev,
SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE);
if (!adapter->res) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
device_printf(dev,"Unable to allocate"
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
" bus resource: Link interrupt [%d]\n", rid);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return (ENXIO);
}
/* Set the link handler function */
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
error = bus_setup_intr(dev, adapter->res,
INTR_TYPE_NET | INTR_MPSAFE, NULL,
ixgbe_msix_link, adapter, &adapter->tag);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (error) {
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
adapter->res = NULL;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
device_printf(dev, "Failed to register LINK handler");
return (error);
}
Remove a disable_queue from the beginning of the interrupt handler, automask handles it. Also, add in msix vector descriptions. MFC for 8.1 asap
2010-06-11 19:03:59 +00:00
#if __FreeBSD_version >= 800504
bus_describe_intr(dev, adapter->res, adapter->tag, "link");
#endif
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
adapter->linkvec = vector;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
/* Tasklets for Link, SFP and Multispeed Fiber */
TASK_INIT(&adapter->link_task, 0, ixgbe_handle_link, adapter);
TASK_INIT(&adapter->mod_task, 0, ixgbe_handle_mod, adapter);
TASK_INIT(&adapter->msf_task, 0, ixgbe_handle_msf, adapter);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
#ifdef IXGBE_FDIR
TASK_INIT(&adapter->fdir_task, 0, ixgbe_reinit_fdir, adapter);
#endif
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
adapter->tq = taskqueue_create_fast("ixgbe_link", M_NOWAIT,
taskqueue_thread_enqueue, &adapter->tq);
taskqueue_start_threads(&adapter->tq, 1, PI_NET, "%s linkq",
device_get_nameunit(adapter->dev));
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return (0);
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/*
* Setup Either MSI/X or MSI
*/
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static int
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
ixgbe_setup_msix(struct adapter *adapter)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
device_t dev = adapter->dev;
int rid, want, queues, msgs;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/* Override by tuneable */
if (ixgbe_enable_msix == 0)
goto msi;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* First try MSI/X */
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
rid = PCIR_BAR(MSIX_82598_BAR);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
adapter->msix_mem = bus_alloc_resource_any(dev,
SYS_RES_MEMORY, &rid, RF_ACTIVE);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
if (!adapter->msix_mem) {
rid += 4; /* 82599 maps in higher BAR */
adapter->msix_mem = bus_alloc_resource_any(dev,
SYS_RES_MEMORY, &rid, RF_ACTIVE);
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
if (!adapter->msix_mem) {
/* May not be enabled */
device_printf(adapter->dev,
"Unable to map MSIX table \n");
goto msi;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
msgs = pci_msix_count(dev);
if (msgs == 0) { /* system has msix disabled */
bus_release_resource(dev, SYS_RES_MEMORY,
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
rid, adapter->msix_mem);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
adapter->msix_mem = NULL;
goto msi;
}
/* Figure out a reasonable auto config value */
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
queues = (mp_ncpus > (msgs-1)) ? (msgs-1) : mp_ncpus;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
if (ixgbe_num_queues != 0)
queues = ixgbe_num_queues;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
/* Set max queues to 8 when autoconfiguring */
else if ((ixgbe_num_queues == 0) && (queues > 8))
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
queues = 8;
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/*
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
** Want one vector (RX/TX pair) per queue
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
** plus an additional for Link.
*/
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
want = queues + 1;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
if (msgs >= want)
msgs = want;
else {
device_printf(adapter->dev,
"MSIX Configuration Problem, "
"%d vectors but %d queues wanted!\n",
msgs, want);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
return (0); /* Will go to Legacy setup */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
if ((msgs) && pci_alloc_msix(dev, &msgs) == 0) {
device_printf(adapter->dev,
"Using MSIX interrupts with %d vectors\n", msgs);
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
adapter->num_queues = queues;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
return (msgs);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
msi:
msgs = pci_msi_count(dev);
if (msgs == 1 && pci_alloc_msi(dev, &msgs) == 0)
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
device_printf(adapter->dev,"Using an MSI interrupt\n");
else
device_printf(adapter->dev,"Using a Legacy interrupt\n");
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
return (msgs);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static int
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
ixgbe_allocate_pci_resources(struct adapter *adapter)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
int rid;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
device_t dev = adapter->dev;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
rid = PCIR_BAR(0);
adapter->pci_mem = bus_alloc_resource_any(dev, SYS_RES_MEMORY,
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
&rid, RF_ACTIVE);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
if (!(adapter->pci_mem)) {
device_printf(dev,"Unable to allocate bus resource: memory\n");
return (ENXIO);
}
adapter->osdep.mem_bus_space_tag =
rman_get_bustag(adapter->pci_mem);
adapter->osdep.mem_bus_space_handle =
rman_get_bushandle(adapter->pci_mem);
adapter->hw.hw_addr = (u8 *) &adapter->osdep.mem_bus_space_handle;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* Legacy defaults */
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
adapter->num_queues = 1;
adapter->hw.back = &adapter->osdep;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/*
** Now setup MSI or MSI/X, should
** return us the number of supported
** vectors. (Will be 1 for MSI)
*/
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
adapter->msix = ixgbe_setup_msix(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return (0);
}
static void
ixgbe_free_pci_resources(struct adapter * adapter)
{
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct ix_queue *que = adapter->queues;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
device_t dev = adapter->dev;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
int rid, memrid;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
if (adapter->hw.mac.type == ixgbe_mac_82598EB)
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
memrid = PCIR_BAR(MSIX_82598_BAR);
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
else
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
memrid = PCIR_BAR(MSIX_82599_BAR);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
** There is a slight possibility of a failure mode
** in attach that will result in entering this function
** before interrupt resources have been initialized, and
** in that case we do not want to execute the loops below
** We can detect this reliably by the state of the adapter
** res pointer.
*/
if (adapter->res == NULL)
goto mem;
/*
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
** Release all msix queue resources:
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
*/
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
for (int i = 0; i < adapter->num_queues; i++, que++) {
rid = que->msix + 1;
if (que->tag != NULL) {
bus_teardown_intr(dev, que->res, que->tag);
que->tag = NULL;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if (que->res != NULL)
bus_release_resource(dev, SYS_RES_IRQ, rid, que->res);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/* Clean the Legacy or Link interrupt last */
if (adapter->linkvec) /* we are doing MSIX */
rid = adapter->linkvec + 1;
else
(adapter->msix != 0) ? (rid = 1):(rid = 0);
if (adapter->tag != NULL) {
bus_teardown_intr(dev, adapter->res, adapter->tag);
adapter->tag = NULL;
}
if (adapter->res != NULL)
bus_release_resource(dev, SYS_RES_IRQ, rid, adapter->res);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
mem:
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
if (adapter->msix)
pci_release_msi(dev);
if (adapter->msix_mem != NULL)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
bus_release_resource(dev, SYS_RES_MEMORY,
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
memrid, adapter->msix_mem);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
if (adapter->pci_mem != NULL)
bus_release_resource(dev, SYS_RES_MEMORY,
PCIR_BAR(0), adapter->pci_mem);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
}
/*********************************************************************
*
* Setup networking device structure and register an interface.
*
**********************************************************************/
A couple fixes got clobbered, putting them back.
2010-11-26 23:57:13 +00:00
static int
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_setup_interface(device_t dev, struct adapter *adapter)
{
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
struct ixgbe_hw *hw = &adapter->hw;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
struct ifnet *ifp;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
INIT_DEBUGOUT("ixgbe_setup_interface: begin");
ifp = adapter->ifp = if_alloc(IFT_ETHER);
A couple fixes got clobbered, putting them back.
2010-11-26 23:57:13 +00:00
if (ifp == NULL) {
device_printf(dev, "can not allocate ifnet structure\n");
return (-1);
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if_initname(ifp, device_get_name(dev), device_get_unit(dev));
provide helper if_initbaudrate() to set if_baudrate_pf and if_baudrate_pf. again, use ixgbe(4) as an example of how to use new helper function. Reviewed by: jhb MFC after: 1 week
2012-10-17 19:24:13 +00:00
if_initbaudrate(ifp, IF_Gbps(10));
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ifp->if_init = ixgbe_init;
ifp->if_softc = adapter;
ifp->if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST;
ifp->if_ioctl = ixgbe_ioctl;
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#ifndef IXGBE_LEGACY_TX
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
ifp->if_transmit = ixgbe_mq_start;
ifp->if_qflush = ixgbe_qflush;
Merge similar fixes from 223198 from igb to ixgbe: - Use a dedicated task to handle deferred transmits from the if_transmit method instead of reusing the existing per-queue interrupt task. Reusing the per-queue interrupt task could result in both an interrupt thread and the taskqueue thread trying to handle received packets on a single queue resulting in out-of-order packet processing and lock contention. - Don't define ixgbe_start() at all where if_transmit is used. Tested by: Vijay Singh Reviewed by: jfv MFC after: 2 weeks
2012-09-26 18:11:43 +00:00
#else
ifp->if_start = ixgbe_start;
IFQ_SET_MAXLEN(&ifp->if_snd, adapter->num_tx_desc - 2);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
#endif
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ether_ifattach(ifp, adapter->hw.mac.addr);
adapter->max_frame_size =
ifp->if_mtu + ETHER_HDR_LEN + ETHER_CRC_LEN;
/*
* Tell the upper layer(s) we support long frames.
*/
ifp->if_data.ifi_hdrlen = sizeof(struct ether_vlan_header);
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
ifp->if_capabilities |= IFCAP_HWCSUM | IFCAP_TSO | IFCAP_VLAN_HWCSUM;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
ifp->if_capabilities |= IFCAP_JUMBO_MTU;
A few important fixes: - Testing TSO6 has led me to discover that HW RSC is a problematic feature, it is ONLY designed to work with IPv4 in the first place, and if IP forwarding is done it can't be disabled as LRO in the stack, also initial testing we've done at Intel shows an equal performance using TSO[46] on the TX and LRO on RX, if you ran older code on 82599 or later hardware you actually could have detrimental performance for this reason. So I am disabling the feature by default and all our adapters will now use LRO instead. - If you have flow control off and multiple queues it was possible when the buffer of one queue becomes full that all RX movement is stalled, to eliminate this problem a feature bit is now set that will allow packets to be dropped when full rather than stall. Note, the default is to have flow control on, and this keeps this from happening. - Because of the recent fixes in the stack, LRO is now auto-disabled when problematic, so I have decided to enable it by default in the capabilities in the driver. - There are some 1G modules used by some customers, a couple small tweaks to properly support those in the media code. - A note: we have now done some testing of TSO6 and using LRO with IPv6 and it all works great!! Seeing line rate in both directions in best cases. Thanks bz for your excellent work!!
2012-10-31 23:50:36 +00:00
ifp->if_capabilities |= IFCAP_LRO;
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
ifp->if_capabilities |= IFCAP_VLAN_HWTAGGING
| IFCAP_VLAN_HWTSO
| IFCAP_VLAN_MTU;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ifp->if_capenable = ifp->if_capabilities;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
/*
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
** Don't turn this on by default, if vlans are
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
** created on another pseudo device (eg. lagg)
** then vlan events are not passed thru, breaking
** operation, but with HW FILTER off it works. If
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
** using vlans directly on the ixgbe driver you can
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
** enable this and get full hardware tag filtering.
*/
ifp->if_capabilities |= IFCAP_VLAN_HWFILTER;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*
* Specify the media types supported by this adapter and register
* callbacks to update media and link information
*/
ifmedia_init(&adapter->media, IFM_IMASK, ixgbe_media_change,
ixgbe_media_status);
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
ifmedia_add(&adapter->media, IFM_ETHER | adapter->optics, 0, NULL);
ifmedia_set(&adapter->media, IFM_ETHER | adapter->optics);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
if (hw->device_id == IXGBE_DEV_ID_82598AT) {
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
ifmedia_add(&adapter->media,
IFM_ETHER | IFM_1000_T | IFM_FDX, 0, NULL);
ifmedia_add(&adapter->media,
IFM_ETHER | IFM_1000_T, 0, NULL);
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ifmedia_add(&adapter->media, IFM_ETHER | IFM_AUTO, 0, NULL);
ifmedia_set(&adapter->media, IFM_ETHER | IFM_AUTO);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
A couple fixes got clobbered, putting them back.
2010-11-26 23:57:13 +00:00
return (0);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
}
static void
ixgbe_config_link(struct adapter *adapter)
{
struct ixgbe_hw *hw = &adapter->hw;
u32 autoneg, err = 0;
bool sfp, negotiate;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
sfp = ixgbe_is_sfp(hw);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
if (sfp) {
if (hw->phy.multispeed_fiber) {
hw->mac.ops.setup_sfp(hw);
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
ixgbe_enable_tx_laser(hw);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
taskqueue_enqueue(adapter->tq, &adapter->msf_task);
} else
taskqueue_enqueue(adapter->tq, &adapter->mod_task);
} else {
if (hw->mac.ops.check_link)
err = ixgbe_check_link(hw, &autoneg,
&adapter->link_up, FALSE);
if (err)
goto out;
autoneg = hw->phy.autoneg_advertised;
if ((!autoneg) && (hw->mac.ops.get_link_capabilities))
err = hw->mac.ops.get_link_capabilities(hw,
&autoneg, &negotiate);
if (err)
goto out;
if (hw->mac.ops.setup_link)
err = hw->mac.ops.setup_link(hw, autoneg,
negotiate, adapter->link_up);
}
out:
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
}
/********************************************************************
* Manage DMA'able memory.
*******************************************************************/
static void
ixgbe_dmamap_cb(void *arg, bus_dma_segment_t * segs, int nseg, int error)
{
if (error)
return;
*(bus_addr_t *) arg = segs->ds_addr;
return;
}
static int
ixgbe_dma_malloc(struct adapter *adapter, bus_size_t size,
struct ixgbe_dma_alloc *dma, int mapflags)
{
device_t dev = adapter->dev;
int r;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
r = bus_dma_tag_create(bus_get_dma_tag(adapter->dev), /* parent */
DBA_ALIGN, 0, /* alignment, bounds */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
BUS_SPACE_MAXADDR, /* lowaddr */
BUS_SPACE_MAXADDR, /* highaddr */
NULL, NULL, /* filter, filterarg */
size, /* maxsize */
1, /* nsegments */
size, /* maxsegsize */
BUS_DMA_ALLOCNOW, /* flags */
NULL, /* lockfunc */
NULL, /* lockfuncarg */
&dma->dma_tag);
if (r != 0) {
device_printf(dev,"ixgbe_dma_malloc: bus_dma_tag_create failed; "
"error %u\n", r);
goto fail_0;
}
r = bus_dmamem_alloc(dma->dma_tag, (void **)&dma->dma_vaddr,
BUS_DMA_NOWAIT, &dma->dma_map);
if (r != 0) {
device_printf(dev,"ixgbe_dma_malloc: bus_dmamem_alloc failed; "
"error %u\n", r);
goto fail_1;
}
r = bus_dmamap_load(dma->dma_tag, dma->dma_map, dma->dma_vaddr,
size,
ixgbe_dmamap_cb,
&dma->dma_paddr,
mapflags | BUS_DMA_NOWAIT);
if (r != 0) {
device_printf(dev,"ixgbe_dma_malloc: bus_dmamap_load failed; "
"error %u\n", r);
goto fail_2;
}
dma->dma_size = size;
return (0);
fail_2:
bus_dmamem_free(dma->dma_tag, dma->dma_vaddr, dma->dma_map);
fail_1:
bus_dma_tag_destroy(dma->dma_tag);
fail_0:
dma->dma_map = NULL;
dma->dma_tag = NULL;
return (r);
}
static void
ixgbe_dma_free(struct adapter *adapter, struct ixgbe_dma_alloc *dma)
{
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
bus_dmamap_sync(dma->dma_tag, dma->dma_map,
BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
bus_dmamap_unload(dma->dma_tag, dma->dma_map);
bus_dmamem_free(dma->dma_tag, dma->dma_vaddr, dma->dma_map);
bus_dma_tag_destroy(dma->dma_tag);
}
/*********************************************************************
*
* Allocate memory for the transmit and receive rings, and then
* the descriptors associated with each, called only once at attach.
*
**********************************************************************/
static int
ixgbe_allocate_queues(struct adapter *adapter)
{
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
device_t dev = adapter->dev;
struct ix_queue *que;
struct tx_ring *txr;
struct rx_ring *rxr;
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
int rsize, tsize, error = IXGBE_SUCCESS;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
int txconf = 0, rxconf = 0;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
/* First allocate the top level queue structs */
if (!(adapter->queues =
(struct ix_queue *) malloc(sizeof(struct ix_queue) *
adapter->num_queues, M_DEVBUF, M_NOWAIT | M_ZERO))) {
device_printf(dev, "Unable to allocate queue memory\n");
error = ENOMEM;
goto fail;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* First allocate the TX ring struct memory */
if (!(adapter->tx_rings =
(struct tx_ring *) malloc(sizeof(struct tx_ring) *
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
adapter->num_queues, M_DEVBUF, M_NOWAIT | M_ZERO))) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
device_printf(dev, "Unable to allocate TX ring memory\n");
error = ENOMEM;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
goto tx_fail;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
/* Next allocate the RX */
if (!(adapter->rx_rings =
(struct rx_ring *) malloc(sizeof(struct rx_ring) *
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
adapter->num_queues, M_DEVBUF, M_NOWAIT | M_ZERO))) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
device_printf(dev, "Unable to allocate RX ring memory\n");
error = ENOMEM;
goto rx_fail;
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* For the ring itself */
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
tsize = roundup2(adapter->num_tx_desc *
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
sizeof(union ixgbe_adv_tx_desc), DBA_ALIGN);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*
* Now set up the TX queues, txconf is needed to handle the
* possibility that things fail midcourse and we need to
* undo memory gracefully
*/
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
for (int i = 0; i < adapter->num_queues; i++, txconf++) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Set up some basics */
txr = &adapter->tx_rings[i];
txr->adapter = adapter;
txr->me = i;
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
txr->num_desc = adapter->num_tx_desc;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* Initialize the TX side lock */
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
snprintf(txr->mtx_name, sizeof(txr->mtx_name), "%s:tx(%d)",
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
device_get_nameunit(dev), txr->me);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
mtx_init(&txr->tx_mtx, txr->mtx_name, NULL, MTX_DEF);
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
if (ixgbe_dma_malloc(adapter, tsize,
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
&txr->txdma, BUS_DMA_NOWAIT)) {
device_printf(dev,
"Unable to allocate TX Descriptor memory\n");
error = ENOMEM;
goto err_tx_desc;
}
txr->tx_base = (union ixgbe_adv_tx_desc *)txr->txdma.dma_vaddr;
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
bzero((void *)txr->tx_base, tsize);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Now allocate transmit buffers for the ring */
if (ixgbe_allocate_transmit_buffers(txr)) {
device_printf(dev,
"Critical Failure setting up transmit buffers\n");
error = ENOMEM;
goto err_tx_desc;
}
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#ifndef IXGBE_LEGACY_TX
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/* Allocate a buf ring */
txr->br = buf_ring_alloc(IXGBE_BR_SIZE, M_DEVBUF,
M_WAITOK, &txr->tx_mtx);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if (txr->br == NULL) {
device_printf(dev,
"Critical Failure setting up buf ring\n");
error = ENOMEM;
goto err_tx_desc;
}
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
#endif
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
/*
* Next the RX queues...
*/
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
rsize = roundup2(adapter->num_rx_desc *
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
sizeof(union ixgbe_adv_rx_desc), DBA_ALIGN);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
for (int i = 0; i < adapter->num_queues; i++, rxconf++) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
rxr = &adapter->rx_rings[i];
/* Set up some basics */
rxr->adapter = adapter;
rxr->me = i;
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
rxr->num_desc = adapter->num_rx_desc;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/* Initialize the RX side lock */
snprintf(rxr->mtx_name, sizeof(rxr->mtx_name), "%s:rx(%d)",
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
device_get_nameunit(dev), rxr->me);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
mtx_init(&rxr->rx_mtx, rxr->mtx_name, NULL, MTX_DEF);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (ixgbe_dma_malloc(adapter, rsize,
&rxr->rxdma, BUS_DMA_NOWAIT)) {
device_printf(dev,
"Unable to allocate RxDescriptor memory\n");
error = ENOMEM;
goto err_rx_desc;
}
rxr->rx_base = (union ixgbe_adv_rx_desc *)rxr->rxdma.dma_vaddr;
bzero((void *)rxr->rx_base, rsize);
/* Allocate receive buffers for the ring*/
if (ixgbe_allocate_receive_buffers(rxr)) {
device_printf(dev,
"Critical Failure setting up receive buffers\n");
error = ENOMEM;
goto err_rx_desc;
}
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
/*
** Finally set up the queue holding structs
*/
for (int i = 0; i < adapter->num_queues; i++) {
que = &adapter->queues[i];
que->adapter = adapter;
que->txr = &adapter->tx_rings[i];
que->rxr = &adapter->rx_rings[i];
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return (0);
err_rx_desc:
for (rxr = adapter->rx_rings; rxconf > 0; rxr++, rxconf--)
ixgbe_dma_free(adapter, &rxr->rxdma);
err_tx_desc:
for (txr = adapter->tx_rings; txconf > 0; txr++, txconf--)
ixgbe_dma_free(adapter, &txr->txdma);
free(adapter->rx_rings, M_DEVBUF);
rx_fail:
free(adapter->tx_rings, M_DEVBUF);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
tx_fail:
free(adapter->queues, M_DEVBUF);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
fail:
return (error);
}
/*********************************************************************
*
* Allocate memory for tx_buffer structures. The tx_buffer stores all
* the information needed to transmit a packet on the wire. This is
* called only once at attach, setup is done every reset.
*
**********************************************************************/
static int
ixgbe_allocate_transmit_buffers(struct tx_ring *txr)
{
struct adapter *adapter = txr->adapter;
device_t dev = adapter->dev;
struct ixgbe_tx_buf *txbuf;
int error, i;
/*
* Setup DMA descriptor areas.
*/
More conversions of drivers to use the PCI parent DMA tag.
2012-03-12 18:15:08 +00:00
if ((error = bus_dma_tag_create(
bus_get_dma_tag(adapter->dev), /* parent */
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
1, 0, /* alignment, bounds */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
BUS_SPACE_MAXADDR, /* lowaddr */
BUS_SPACE_MAXADDR, /* highaddr */
NULL, NULL, /* filter, filterarg */
IXGBE_TSO_SIZE, /* maxsize */
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
adapter->num_segs, /* nsegments */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
PAGE_SIZE, /* maxsegsize */
0, /* flags */
NULL, /* lockfunc */
NULL, /* lockfuncarg */
&txr->txtag))) {
device_printf(dev,"Unable to allocate TX DMA tag\n");
goto fail;
}
if (!(txr->tx_buffers =
(struct ixgbe_tx_buf *) malloc(sizeof(struct ixgbe_tx_buf) *
adapter->num_tx_desc, M_DEVBUF, M_NOWAIT | M_ZERO))) {
device_printf(dev, "Unable to allocate tx_buffer memory\n");
error = ENOMEM;
goto fail;
}
/* Create the descriptor buffer dma maps */
txbuf = txr->tx_buffers;
for (i = 0; i < adapter->num_tx_desc; i++, txbuf++) {
error = bus_dmamap_create(txr->txtag, 0, &txbuf->map);
if (error != 0) {
device_printf(dev, "Unable to create TX DMA map\n");
goto fail;
}
}
return 0;
fail:
/* We free all, it handles case where we are in the middle */
ixgbe_free_transmit_structures(adapter);
return (error);
}
/*********************************************************************
*
* Initialize a transmit ring.
*
**********************************************************************/
static void
ixgbe_setup_transmit_ring(struct tx_ring *txr)
{
struct adapter *adapter = txr->adapter;
struct ixgbe_tx_buf *txbuf;
int i;
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
#ifdef DEV_NETMAP
struct netmap_adapter *na = NA(adapter->ifp);
struct netmap_slot *slot;
#endif /* DEV_NETMAP */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Clear the old ring contents */
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
IXGBE_TX_LOCK(txr);
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
#ifdef DEV_NETMAP
/*
* (under lock): if in netmap mode, do some consistency
* checks and set slot to entry 0 of the netmap ring.
*/
slot = netmap_reset(na, NR_TX, txr->me, 0);
#endif /* DEV_NETMAP */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
bzero((void *)txr->tx_base,
(sizeof(union ixgbe_adv_tx_desc)) * adapter->num_tx_desc);
/* Reset indices */
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
txr->next_avail_desc = 0;
txr->next_to_clean = 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Free any existing tx buffers. */
txbuf = txr->tx_buffers;
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
for (i = 0; i < txr->num_desc; i++, txbuf++) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (txbuf->m_head != NULL) {
bus_dmamap_sync(txr->txtag, txbuf->map,
BUS_DMASYNC_POSTWRITE);
bus_dmamap_unload(txr->txtag, txbuf->map);
m_freem(txbuf->m_head);
txbuf->m_head = NULL;
}
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
#ifdef DEV_NETMAP
/*
* In netmap mode, set the map for the packet buffer.
* NOTE: Some drivers (not this one) also need to set
* the physical buffer address in the NIC ring.
* Slots in the netmap ring (indexed by "si") are
* kring->nkr_hwofs positions "ahead" wrt the
* corresponding slot in the NIC ring. In some drivers
(This commit only touches code within the DEV_NETMAP blocks) Introduce some functions to map NIC ring indexes into netmap ring indexes and vice versa. This way we can implement the bound checks only in one place (and hopefully in a correct way). On passing, make the code and comments more uniform across the various drivers.
2012-02-15 23:13:29 +00:00
* (not here) nkr_hwofs can be negative. Function
A bunch of netmap fixes: USERSPACE: 1. add support for devices with different number of rx and tx queues; 2. add better support for zero-copy operation, adding an extra field to the netmap ring to indicate how many buffers we have already processed but not yet released (with help from Eddie Kohler); 3. The two changes above unfortunately require an API change, so while at it add a version field and some spares to the ioctl() argument to help detect mismatches. 4. update the manual page for the two changes above; 5. update sample applications in tools/tools/netmap KERNEL: 1. simplify the internal structures moving the global wait queues to the 'struct netmap_adapter'; 2. simplify the functions that map kring<->nic ring indexes 3. normalize device-specific code, helps mainteinance; 4. start exploring the impact of micro-optimizations (prefetch etc.) in the ixgbe driver. Use 'legacy' descriptors on the tx ring and prefetch slots gives about 20% speedup at 900 MHz. Another 7-10% would come from removing the explict calls to bus_dmamap* in the core (they are effectively NOPs in this case, but it takes expensive load of the per-buffer dma maps to figure out that they are all NULL. Rx performance not investigated. I am postponing the MFC so i can import a few more improvements before merging.
2012-02-27 19:05:01 +00:00
* netmap_idx_n2k() handles wraparounds properly.
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
*/
if (slot) {
A bunch of netmap fixes: USERSPACE: 1. add support for devices with different number of rx and tx queues; 2. add better support for zero-copy operation, adding an extra field to the netmap ring to indicate how many buffers we have already processed but not yet released (with help from Eddie Kohler); 3. The two changes above unfortunately require an API change, so while at it add a version field and some spares to the ioctl() argument to help detect mismatches. 4. update the manual page for the two changes above; 5. update sample applications in tools/tools/netmap KERNEL: 1. simplify the internal structures moving the global wait queues to the 'struct netmap_adapter'; 2. simplify the functions that map kring<->nic ring indexes 3. normalize device-specific code, helps mainteinance; 4. start exploring the impact of micro-optimizations (prefetch etc.) in the ixgbe driver. Use 'legacy' descriptors on the tx ring and prefetch slots gives about 20% speedup at 900 MHz. Another 7-10% would come from removing the explict calls to bus_dmamap* in the core (they are effectively NOPs in this case, but it takes expensive load of the per-buffer dma maps to figure out that they are all NULL. Rx performance not investigated. I am postponing the MFC so i can import a few more improvements before merging.
2012-02-27 19:05:01 +00:00
int si = netmap_idx_n2k(&na->tx_rings[txr->me], i);
small code cleanup in preparation for future modifications in the memory allocator used by netmap. No functional change, two small bug fixes: - in if_re.c add a missing bus_dmamap_sync() - in netmap.c comment out a spurious free() in an error handling block
2012-01-10 19:57:23 +00:00
netmap_load_map(txr->txtag, txbuf->map, NMB(slot + si));
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
}
#endif /* DEV_NETMAP */
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
/* Clear the EOP index */
txbuf->eop_index = -1;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
#ifdef IXGBE_FDIR
/* Set the rate at which we sample packets */
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
if (adapter->hw.mac.type != ixgbe_mac_82598EB)
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
txr->atr_sample = atr_sample_rate;
#endif
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Set number of descriptors available */
txr->tx_avail = adapter->num_tx_desc;
bus_dmamap_sync(txr->txdma.dma_tag, txr->txdma.dma_map,
BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
IXGBE_TX_UNLOCK(txr);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
/*********************************************************************
*
* Initialize all transmit rings.
*
**********************************************************************/
static int
ixgbe_setup_transmit_structures(struct adapter *adapter)
{
struct tx_ring *txr = adapter->tx_rings;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
for (int i = 0; i < adapter->num_queues; i++, txr++)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_setup_transmit_ring(txr);
return (0);
}
/*********************************************************************
*
* Enable transmit unit.
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
*
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
**********************************************************************/
static void
ixgbe_initialize_transmit_units(struct adapter *adapter)
{
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
struct tx_ring *txr = adapter->tx_rings;
struct ixgbe_hw *hw = &adapter->hw;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Setup the Base and Length of the Tx Descriptor Ring */
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
for (int i = 0; i < adapter->num_queues; i++, txr++) {
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
u64 tdba = txr->txdma.dma_paddr;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
u32 txctrl;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
Patch #7 This is primarily about processing limit control. - add a limit for both RX and TX, change the default to 256 - change the sysctl usage to be common, and now to be called during init for each ring. - the TX limit is not yet used, but the changes in the last patch in this series uses the value. - the motivation behind these changes is to improve data locality in the final code. - rxeof interface changes since it now gets limit from the ring struct
2012-11-30 23:28:01 +00:00
/* Sysctl for limiting work done in tx clean */
ixgbe_add_process_limit(adapter, "tx_processing_limit",
"max number of packets to process", &txr->process_limit,
ixgbe_tx_process_limit);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_WRITE_REG(hw, IXGBE_TDBAL(i),
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
(tdba & 0x00000000ffffffffULL));
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_WRITE_REG(hw, IXGBE_TDBAH(i), (tdba >> 32));
IXGBE_WRITE_REG(hw, IXGBE_TDLEN(i),
adapter->num_tx_desc * sizeof(struct ixgbe_legacy_tx_desc));
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* Setup the HW Tx Head and Tail descriptor pointers */
IXGBE_WRITE_REG(hw, IXGBE_TDH(i), 0);
IXGBE_WRITE_REG(hw, IXGBE_TDT(i), 0);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* Setup Transmit Descriptor Cmd Settings */
txr->txd_cmd = IXGBE_TXD_CMD_IFCS;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
txr->queue_status = IXGBE_QUEUE_IDLE;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* Disable Head Writeback */
switch (hw->mac.type) {
case ixgbe_mac_82598EB:
txctrl = IXGBE_READ_REG(hw, IXGBE_DCA_TXCTRL(i));
break;
case ixgbe_mac_82599EB:
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
case ixgbe_mac_X540:
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
default:
txctrl = IXGBE_READ_REG(hw, IXGBE_DCA_TXCTRL_82599(i));
break;
}
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
txctrl &= ~IXGBE_DCA_TXCTRL_DESC_WRO_EN;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
switch (hw->mac.type) {
case ixgbe_mac_82598EB:
IXGBE_WRITE_REG(hw, IXGBE_DCA_TXCTRL(i), txctrl);
break;
case ixgbe_mac_82599EB:
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
case ixgbe_mac_X540:
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
default:
IXGBE_WRITE_REG(hw, IXGBE_DCA_TXCTRL_82599(i), txctrl);
break;
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
if (hw->mac.type != ixgbe_mac_82598EB) {
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
u32 dmatxctl, rttdcs;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
dmatxctl = IXGBE_READ_REG(hw, IXGBE_DMATXCTL);
dmatxctl |= IXGBE_DMATXCTL_TE;
IXGBE_WRITE_REG(hw, IXGBE_DMATXCTL, dmatxctl);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* Disable arbiter to set MTQC */
rttdcs = IXGBE_READ_REG(hw, IXGBE_RTTDCS);
rttdcs |= IXGBE_RTTDCS_ARBDIS;
IXGBE_WRITE_REG(hw, IXGBE_RTTDCS, rttdcs);
IXGBE_WRITE_REG(hw, IXGBE_MTQC, IXGBE_MTQC_64Q_1PB);
rttdcs &= ~IXGBE_RTTDCS_ARBDIS;
IXGBE_WRITE_REG(hw, IXGBE_RTTDCS, rttdcs);
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
}
/*********************************************************************
*
* Free all transmit rings.
*
**********************************************************************/
static void
ixgbe_free_transmit_structures(struct adapter *adapter)
{
struct tx_ring *txr = adapter->tx_rings;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
for (int i = 0; i < adapter->num_queues; i++, txr++) {
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_TX_LOCK(txr);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_free_transmit_buffers(txr);
ixgbe_dma_free(adapter, &txr->txdma);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_TX_UNLOCK(txr);
IXGBE_TX_LOCK_DESTROY(txr);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
free(adapter->tx_rings, M_DEVBUF);
}
/*********************************************************************
*
* Free transmit ring related data structures.
*
**********************************************************************/
static void
ixgbe_free_transmit_buffers(struct tx_ring *txr)
{
struct adapter *adapter = txr->adapter;
struct ixgbe_tx_buf *tx_buffer;
int i;
INIT_DEBUGOUT("free_transmit_ring: begin");
if (txr->tx_buffers == NULL)
return;
tx_buffer = txr->tx_buffers;
for (i = 0; i < adapter->num_tx_desc; i++, tx_buffer++) {
if (tx_buffer->m_head != NULL) {
bus_dmamap_sync(txr->txtag, tx_buffer->map,
BUS_DMASYNC_POSTWRITE);
bus_dmamap_unload(txr->txtag,
tx_buffer->map);
m_freem(tx_buffer->m_head);
tx_buffer->m_head = NULL;
if (tx_buffer->map != NULL) {
bus_dmamap_destroy(txr->txtag,
tx_buffer->map);
tx_buffer->map = NULL;
}
} else if (tx_buffer->map != NULL) {
bus_dmamap_unload(txr->txtag,
tx_buffer->map);
bus_dmamap_destroy(txr->txtag,
tx_buffer->map);
tx_buffer->map = NULL;
}
}
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
#ifdef IXGBE_LEGACY_TX
This delta fixes two bugs: - When a vlan event occurs a check was not made that the event was actually for the interface, thus resulting in a panic. All three drivers have this vulnerability. Add a check for this condition. - Secondly, there was a duplicate buf_ring free in the em driver resulting in a panic on unload. Remove. Approved by: re
2009-07-24 16:57:49 +00:00
if (txr->br != NULL)
buf_ring_free(txr->br, M_DEVBUF);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
#endif
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (txr->tx_buffers != NULL) {
free(txr->tx_buffers, M_DEVBUF);
txr->tx_buffers = NULL;
}
if (txr->txtag != NULL) {
bus_dma_tag_destroy(txr->txtag);
txr->txtag = NULL;
}
return;
}
/*********************************************************************
*
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
* Advanced Context Descriptor setup for VLAN, CSUM or TSO
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
*
**********************************************************************/
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
static int
ixgbe_tx_ctx_setup(struct tx_ring *txr, struct mbuf *mp,
u32 *cmd_type_len, u32 *olinfo_status)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
struct ixgbe_adv_tx_context_desc *TXD;
struct ether_vlan_header *eh;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
struct ip *ip;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
struct ip6_hdr *ip6;
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
u32 vlan_macip_lens = 0, type_tucmd_mlhl = 0;
int ehdrlen, ip_hlen = 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
u16 etype;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
u8 ipproto = 0;
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
int offload = TRUE;
int ctxd = txr->next_avail_desc;
u16 vtag = 0;
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
/* First check if TSO is to be used */
if (mp->m_pkthdr.csum_flags & CSUM_TSO)
return (ixgbe_tso_setup(txr, mp, cmd_type_len, olinfo_status));
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
if ((mp->m_pkthdr.csum_flags & CSUM_OFFLOAD) == 0)
offload = FALSE;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
/* Indicate the whole packet as payload when not doing TSO */
*olinfo_status |= mp->m_pkthdr.len << IXGBE_ADVTXD_PAYLEN_SHIFT;
/* Now ready a context descriptor */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
TXD = (struct ixgbe_adv_tx_context_desc *) &txr->tx_base[ctxd];
/*
** In advanced descriptors the vlan tag must
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
** be placed into the context descriptor. Hence
** we need to make one even if not doing offloads.
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
*/
if (mp->m_flags & M_VLANTAG) {
vtag = htole16(mp->m_pkthdr.ether_vtag);
vlan_macip_lens |= (vtag << IXGBE_ADVTXD_VLAN_SHIFT);
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
} else if (offload == FALSE) /* ... no offload to do */
return (0);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*
* Determine where frame payload starts.
* Jump over vlan headers if already present,
* helpful for QinQ too.
*/
eh = mtod(mp, struct ether_vlan_header *);
if (eh->evl_encap_proto == htons(ETHERTYPE_VLAN)) {
etype = ntohs(eh->evl_proto);
ehdrlen = ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN;
} else {
etype = ntohs(eh->evl_encap_proto);
ehdrlen = ETHER_HDR_LEN;
}
/* Set the ether header length */
vlan_macip_lens |= ehdrlen << IXGBE_ADVTXD_MACLEN_SHIFT;
switch (etype) {
case ETHERTYPE_IP:
ip = (struct ip *)(mp->m_data + ehdrlen);
ip_hlen = ip->ip_hl << 2;
ipproto = ip->ip_p;
type_tucmd_mlhl |= IXGBE_ADVTXD_TUCMD_IPV4;
break;
case ETHERTYPE_IPV6:
ip6 = (struct ip6_hdr *)(mp->m_data + ehdrlen);
ip_hlen = sizeof(struct ip6_hdr);
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
/* XXX-BZ this will go badly in case of ext hdrs. */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ipproto = ip6->ip6_nxt;
type_tucmd_mlhl |= IXGBE_ADVTXD_TUCMD_IPV6;
break;
default:
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
offload = FALSE;
break;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
vlan_macip_lens |= ip_hlen;
type_tucmd_mlhl |= IXGBE_ADVTXD_DCMD_DEXT | IXGBE_ADVTXD_DTYP_CTXT;
switch (ipproto) {
case IPPROTO_TCP:
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
if (mp->m_pkthdr.csum_flags & CSUM_TCP)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
type_tucmd_mlhl |= IXGBE_ADVTXD_TUCMD_L4T_TCP;
break;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
case IPPROTO_UDP:
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
if (mp->m_pkthdr.csum_flags & CSUM_UDP)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
type_tucmd_mlhl |= IXGBE_ADVTXD_TUCMD_L4T_UDP;
break;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
#if __FreeBSD_version >= 800000
case IPPROTO_SCTP:
if (mp->m_pkthdr.csum_flags & CSUM_SCTP)
type_tucmd_mlhl |= IXGBE_ADVTXD_TUCMD_L4T_SCTP;
break;
#endif
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
default:
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
offload = FALSE;
break;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
if (offload) /* For the TX descriptor setup */
*olinfo_status |= IXGBE_TXD_POPTS_TXSM << 8;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Now copy bits into descriptor */
TXD->vlan_macip_lens |= htole32(vlan_macip_lens);
TXD->type_tucmd_mlhl |= htole32(type_tucmd_mlhl);
TXD->seqnum_seed = htole32(0);
TXD->mss_l4len_idx = htole32(0);
/* We've consumed the first desc, adjust counters */
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
if (++ctxd == txr->num_desc)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ctxd = 0;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
txr->next_avail_desc = ctxd;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
--txr->tx_avail;
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
return (0);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
/**********************************************************************
*
* Setup work for hardware segmentation offload (TSO) on
* adapters using advanced tx descriptors
*
**********************************************************************/
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
static int
ixgbe_tso_setup(struct tx_ring *txr, struct mbuf *mp,
u32 *cmd_type_len, u32 *olinfo_status)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
struct ixgbe_adv_tx_context_desc *TXD;
u32 vlan_macip_lens = 0, type_tucmd_mlhl = 0;
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
u32 mss_l4len_idx = 0, paylen;
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
u16 vtag = 0, eh_type;
int ctxd, ehdrlen, ip_hlen, tcp_hlen;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
struct ether_vlan_header *eh;
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
#ifdef INET6
struct ip6_hdr *ip6;
#endif
#ifdef INET
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
struct ip *ip;
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
#endif
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
struct tcphdr *th;
/*
* Determine where frame payload starts.
* Jump over vlan headers if already present
*/
eh = mtod(mp, struct ether_vlan_header *);
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
if (eh->evl_encap_proto == htons(ETHERTYPE_VLAN)) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ehdrlen = ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN;
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
eh_type = eh->evl_proto;
} else {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ehdrlen = ETHER_HDR_LEN;
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
eh_type = eh->evl_encap_proto;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
switch (ntohs(eh_type)) {
#ifdef INET6
case ETHERTYPE_IPV6:
ip6 = (struct ip6_hdr *)(mp->m_data + ehdrlen);
/* XXX-BZ For now we do not pretend to support ext. hdrs. */
if (ip6->ip6_nxt != IPPROTO_TCP)
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
return (ENXIO);
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
ip_hlen = sizeof(struct ip6_hdr);
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
ip6 = (struct ip6_hdr *)(mp->m_data + ehdrlen);
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
th = (struct tcphdr *)((caddr_t)ip6 + ip_hlen);
th->th_sum = in6_cksum_pseudo(ip6, 0, IPPROTO_TCP, 0);
type_tucmd_mlhl |= IXGBE_ADVTXD_TUCMD_IPV6;
break;
#endif
#ifdef INET
case ETHERTYPE_IP:
ip = (struct ip *)(mp->m_data + ehdrlen);
if (ip->ip_p != IPPROTO_TCP)
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
return (ENXIO);
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
ip->ip_sum = 0;
ip_hlen = ip->ip_hl << 2;
th = (struct tcphdr *)((caddr_t)ip + ip_hlen);
th->th_sum = in_pseudo(ip->ip_src.s_addr,
ip->ip_dst.s_addr, htons(IPPROTO_TCP));
type_tucmd_mlhl |= IXGBE_ADVTXD_TUCMD_IPV4;
/* Tell transmit desc to also do IPv4 checksum. */
*olinfo_status |= IXGBE_TXD_POPTS_IXSM << 8;
break;
#endif
default:
panic("%s: CSUM_TSO but no supported IP version (0x%04x)",
__func__, ntohs(eh_type));
break;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
ctxd = txr->next_avail_desc;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
TXD = (struct ixgbe_adv_tx_context_desc *) &txr->tx_base[ctxd];
tcp_hlen = th->th_off << 2;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* This is used in the transmit desc in encap */
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
paylen = mp->m_pkthdr.len - ehdrlen - ip_hlen - tcp_hlen;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* VLAN MACLEN IPLEN */
if (mp->m_flags & M_VLANTAG) {
vtag = htole16(mp->m_pkthdr.ether_vtag);
vlan_macip_lens |= (vtag << IXGBE_ADVTXD_VLAN_SHIFT);
}
vlan_macip_lens |= ehdrlen << IXGBE_ADVTXD_MACLEN_SHIFT;
vlan_macip_lens |= ip_hlen;
TXD->vlan_macip_lens |= htole32(vlan_macip_lens);
/* ADV DTYPE TUCMD */
type_tucmd_mlhl |= IXGBE_ADVTXD_DCMD_DEXT | IXGBE_ADVTXD_DTYP_CTXT;
type_tucmd_mlhl |= IXGBE_ADVTXD_TUCMD_L4T_TCP;
TXD->type_tucmd_mlhl |= htole32(type_tucmd_mlhl);
/* MSS L4LEN IDX */
mss_l4len_idx |= (mp->m_pkthdr.tso_segsz << IXGBE_ADVTXD_MSS_SHIFT);
mss_l4len_idx |= (tcp_hlen << IXGBE_ADVTXD_L4LEN_SHIFT);
TXD->mss_l4len_idx = htole32(mss_l4len_idx);
TXD->seqnum_seed = htole32(0);
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
if (++ctxd == txr->num_desc)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ctxd = 0;
txr->tx_avail--;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
txr->next_avail_desc = ctxd;
Patch #10 Performance - this changes the protocol offload interface and code in the TX path,making it tighter and hopefully more efficient.
2012-12-01 00:03:58 +00:00
*cmd_type_len |= IXGBE_ADVTXD_DCMD_TSE;
*olinfo_status |= IXGBE_TXD_POPTS_TXSM << 8;
*olinfo_status |= paylen << IXGBE_ADVTXD_PAYLEN_SHIFT;
++txr->tso_tx;
return (0);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
#ifdef IXGBE_FDIR
/*
** This routine parses packet headers so that Flow
** Director can make a hashed filter table entry
** allowing traffic flows to be identified and kept
** on the same cpu. This would be a performance
** hit, but we only do it at IXGBE_FDIR_RATE of
** packets.
*/
static void
ixgbe_atr(struct tx_ring *txr, struct mbuf *mp)
{
struct adapter *adapter = txr->adapter;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct ix_queue *que;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
struct ip *ip;
struct tcphdr *th;
struct udphdr *uh;
struct ether_vlan_header *eh;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
union ixgbe_atr_hash_dword input = {.dword = 0};
union ixgbe_atr_hash_dword common = {.dword = 0};
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
int ehdrlen, ip_hlen;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
u16 etype;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
eh = mtod(mp, struct ether_vlan_header *);
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
if (eh->evl_encap_proto == htons(ETHERTYPE_VLAN)) {
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
ehdrlen = ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
etype = eh->evl_proto;
} else {
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
ehdrlen = ETHER_HDR_LEN;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
etype = eh->evl_encap_proto;
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* Only handling IPv4 */
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
if (etype != htons(ETHERTYPE_IP))
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
return;
ip = (struct ip *)(mp->m_data + ehdrlen);
ip_hlen = ip->ip_hl << 2;
/* check if we're UDP or TCP */
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
switch (ip->ip_p) {
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
case IPPROTO_TCP:
th = (struct tcphdr *)((caddr_t)ip + ip_hlen);
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
/* src and dst are inverted */
common.port.dst ^= th->th_sport;
common.port.src ^= th->th_dport;
input.formatted.flow_type ^= IXGBE_ATR_FLOW_TYPE_TCPV4;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
break;
case IPPROTO_UDP:
uh = (struct udphdr *)((caddr_t)ip + ip_hlen);
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
/* src and dst are inverted */
common.port.dst ^= uh->uh_sport;
common.port.src ^= uh->uh_dport;
input.formatted.flow_type ^= IXGBE_ATR_FLOW_TYPE_UDPV4;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
break;
default:
return;
}
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
input.formatted.vlan_id = htobe16(mp->m_pkthdr.ether_vtag);
if (mp->m_pkthdr.ether_vtag)
common.flex_bytes ^= htons(ETHERTYPE_VLAN);
else
common.flex_bytes ^= etype;
common.ip ^= ip->ip_src.s_addr ^ ip->ip_dst.s_addr;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
que = &adapter->queues[txr->me];
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
/*
** This assumes the Rx queue and Tx
** queue are bound to the same CPU
*/
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
ixgbe_fdir_add_signature_filter_82599(&adapter->hw,
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
input, common, que->msix);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
}
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
#endif /* IXGBE_FDIR */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/**********************************************************************
*
* Examine each tx_buffer in the used queue. If the hardware is done
* processing the packet then free associated resources. The
* tx_buffer is put back on the free queue.
*
**********************************************************************/
Consistently use types in ixgbe driver code: - {ixgbe,ixv}_header_split is passed to TUNABLE_INT, so delcare it int, not bool. - {ixgbe,ixv}_tx_ctx_setup() returns a boolean value, so declare it bool, not int. - {ixgbe,ixv}_tso_setup() returns a bool, so declare it bool, not boolean_t. - {ixgbe,ixv}_txeof() returns a bool, so declare it bool, not boolean_t. - Do not re-define bool if the symbol already exists. MFC after: 2 weeks Sponsored by: Isilon Systems, LLC
2011-12-12 18:27:28 +00:00
static bool
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_txeof(struct tx_ring *txr)
{
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
struct adapter *adapter = txr->adapter;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
struct ifnet *ifp = adapter->ifp;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
u32 first, last, done, processed;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
struct ixgbe_tx_buf *tx_buffer;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
struct ixgbe_legacy_tx_desc *tx_desc, *eop_desc;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
mtx_assert(&txr->tx_mtx, MA_OWNED);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
#ifdef DEV_NETMAP
if (ifp->if_capenable & IFCAP_NETMAP) {
struct netmap_adapter *na = NA(ifp);
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
struct netmap_kring *kring = &na->tx_rings[txr->me];
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
tx_desc = (struct ixgbe_legacy_tx_desc *)txr->tx_base;
bus_dmamap_sync(txr->txdma.dma_tag, txr->txdma.dma_map,
BUS_DMASYNC_POSTREAD);
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
/*
* In netmap mode, all the work is done in the context
* of the client thread. Interrupt handlers only wake up
* clients, which may be sleeping on individual rings
* or on a global resource for all rings.
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
* To implement tx interrupt mitigation, we wake up the client
* thread roughly every half ring, even if the NIC interrupts
* more frequently. This is implemented as follows:
* - ixgbe_txsync() sets kring->nr_kflags with the index of
* the slot that should wake up the thread (nkr_num_slots
* means the user thread should not be woken up);
* - the driver ignores tx interrupts unless netmap_mitigate=0
* or the slot has the DD bit set.
*
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
* When the driver has separate locks, we need to
* release and re-acquire txlock to avoid deadlocks.
* XXX see if we can find a better way.
*/
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
if (!netmap_mitigate ||
(kring->nr_kflags < kring->nkr_num_slots &&
Patch #6 Whitespace cleanup, and removal of some very old defines (at Gleb's request). Also, change the defines around the old transmit code to IXGBE_LEGACY_TX, I do this to make it possible to define this regardless of the OS level (it is not defined by default). There are also a couple changed comments for clarity.
2012-11-30 23:13:56 +00:00
tx_desc[kring->nr_kflags].upper.fields.status &
IXGBE_TXD_STAT_DD)) {
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
kring->nr_kflags = kring->nkr_num_slots;
selwakeuppri(&na->tx_rings[txr->me].si, PI_NET);
IXGBE_TX_UNLOCK(txr);
IXGBE_CORE_LOCK(adapter);
A bunch of netmap fixes: USERSPACE: 1. add support for devices with different number of rx and tx queues; 2. add better support for zero-copy operation, adding an extra field to the netmap ring to indicate how many buffers we have already processed but not yet released (with help from Eddie Kohler); 3. The two changes above unfortunately require an API change, so while at it add a version field and some spares to the ioctl() argument to help detect mismatches. 4. update the manual page for the two changes above; 5. update sample applications in tools/tools/netmap KERNEL: 1. simplify the internal structures moving the global wait queues to the 'struct netmap_adapter'; 2. simplify the functions that map kring<->nic ring indexes 3. normalize device-specific code, helps mainteinance; 4. start exploring the impact of micro-optimizations (prefetch etc.) in the ixgbe driver. Use 'legacy' descriptors on the tx ring and prefetch slots gives about 20% speedup at 900 MHz. Another 7-10% would come from removing the explict calls to bus_dmamap* in the core (they are effectively NOPs in this case, but it takes expensive load of the per-buffer dma maps to figure out that they are all NULL. Rx performance not investigated. I am postponing the MFC so i can import a few more improvements before merging.
2012-02-27 19:05:01 +00:00
selwakeuppri(&na->tx_si, PI_NET);
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
IXGBE_CORE_UNLOCK(adapter);
IXGBE_TX_LOCK(txr);
}
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
return FALSE;
}
#endif /* DEV_NETMAP */
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
if (txr->tx_avail == txr->num_desc) {
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
txr->queue_status = IXGBE_QUEUE_IDLE;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return FALSE;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
processed = 0;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
first = txr->next_to_clean;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
tx_buffer = &txr->tx_buffers[first];
/* For cleanup we just use legacy struct */
tx_desc = (struct ixgbe_legacy_tx_desc *)&txr->tx_base[first];
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
last = tx_buffer->eop_index;
if (last == -1)
return FALSE;
eop_desc = (struct ixgbe_legacy_tx_desc *)&txr->tx_base[last];
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
/*
** Get the index of the first descriptor
** BEYOND the EOP and call that 'done'.
** I do this so the comparison in the
** inner while loop below can be simple
*/
if (++last == adapter->num_tx_desc) last = 0;
done = last;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
bus_dmamap_sync(txr->txdma.dma_tag, txr->txdma.dma_map,
BUS_DMASYNC_POSTREAD);
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
/*
** Only the EOP descriptor of a packet now has the DD
** bit set, this is what we look for...
*/
while (eop_desc->upper.fields.status & IXGBE_TXD_STAT_DD) {
/* We clean the range of the packet */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
while (first != done) {
tx_desc->upper.data = 0;
tx_desc->lower.data = 0;
tx_desc->buffer_addr = 0;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
++txr->tx_avail;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
++processed;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (tx_buffer->m_head) {
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
txr->bytes +=
tx_buffer->m_head->m_pkthdr.len;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
bus_dmamap_sync(txr->txtag,
tx_buffer->map,
BUS_DMASYNC_POSTWRITE);
bus_dmamap_unload(txr->txtag,
tx_buffer->map);
m_freem(tx_buffer->m_head);
tx_buffer->m_head = NULL;
tx_buffer->map = NULL;
}
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
tx_buffer->eop_index = -1;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
txr->watchdog_time = ticks;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (++first == adapter->num_tx_desc)
first = 0;
tx_buffer = &txr->tx_buffers[first];
tx_desc =
(struct ixgbe_legacy_tx_desc *)&txr->tx_base[first];
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
++txr->packets;
++ifp->if_opackets;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* See if there is more work now */
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
last = tx_buffer->eop_index;
if (last != -1) {
eop_desc =
(struct ixgbe_legacy_tx_desc *)&txr->tx_base[last];
/* Get next done point */
if (++last == adapter->num_tx_desc) last = 0;
done = last;
} else
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
break;
}
bus_dmamap_sync(txr->txdma.dma_tag, txr->txdma.dma_map,
BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
txr->next_to_clean = first;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
/*
** Watchdog calculation, we know there's
** work outstanding or the first return
** would have been taken, so none processed
** for too long indicates a hang.
*/
if ((!processed) && ((ticks - txr->watchdog_time) > IXGBE_WATCHDOG))
txr->queue_status = IXGBE_QUEUE_HUNG;
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
if (txr->tx_avail == txr->num_desc) {
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
txr->queue_status = IXGBE_QUEUE_IDLE;
return (FALSE);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
return TRUE;
}
/*********************************************************************
*
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
* Refresh mbuf buffers for RX descriptor rings
* - now keeps its own state so discards due to resource
* exhaustion are unnecessary, if an mbuf cannot be obtained
* it just returns, keeping its placeholder, thus it can simply
* be recalled to try again.
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
*
**********************************************************************/
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
static void
ixgbe_refresh_mbufs(struct rx_ring *rxr, int limit)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
struct adapter *adapter = rxr->adapter;
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
bus_dma_segment_t seg[1];
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
struct ixgbe_rx_buf *rxbuf;
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
struct mbuf *mp;
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
int i, j, nsegs, error;
bool refreshed = FALSE;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
i = j = rxr->next_to_refresh;
/* Control the loop with one beyond */
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
if (++j == rxr->num_desc)
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
j = 0;
while (j != limit) {
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
rxbuf = &rxr->rx_buffers[i];
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
if (rxbuf->buf == NULL) {
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
mp = m_getjcl(M_DONTWAIT, MT_DATA,
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
M_PKTHDR, rxr->mbuf_sz);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
if (mp == NULL)
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
goto update;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
} else
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
mp = rxbuf->buf;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
mp->m_pkthdr.len = mp->m_len = rxr->mbuf_sz;
Heavily optimize the case of small RX packets of 160 bytes or less. For this case, allocate a plain mbuf and copy the frame into it, then send the copy up the stack, leaving the original mbuf+cluster in place in the receive ring for immediate re-use. This saves a trip through 2 of the 3 zones of the compound mbuf allocator, a trip through busdma, and a trip through the 1 of the 3 mbuf destructors. For our load at Netflix, this can lower CPU consumption by as much as 20%. The copy algorithm is based on investigative work from Luigi Rizzo earlier in the year. Reviewed by: jfv Obtained from: Netflix
2012-08-31 10:07:38 +00:00
/* If we're dealing with an mbuf that was copied rather
* than replaced, there's no need to go through busdma.
*/
if ((rxbuf->flags & IXGBE_RX_COPY) == 0) {
/* Get the memory mapping */
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
error = bus_dmamap_load_mbuf_sg(rxr->tag,
rxbuf->map, mp, seg, &nsegs, BUS_DMA_NOWAIT);
Heavily optimize the case of small RX packets of 160 bytes or less. For this case, allocate a plain mbuf and copy the frame into it, then send the copy up the stack, leaving the original mbuf+cluster in place in the receive ring for immediate re-use. This saves a trip through 2 of the 3 zones of the compound mbuf allocator, a trip through busdma, and a trip through the 1 of the 3 mbuf destructors. For our load at Netflix, this can lower CPU consumption by as much as 20%. The copy algorithm is based on investigative work from Luigi Rizzo earlier in the year. Reviewed by: jfv Obtained from: Netflix
2012-08-31 10:07:38 +00:00
if (error != 0) {
printf("Refresh mbufs: payload dmamap load"
" failure - %d\n", error);
m_free(mp);
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
rxbuf->buf = NULL;
Heavily optimize the case of small RX packets of 160 bytes or less. For this case, allocate a plain mbuf and copy the frame into it, then send the copy up the stack, leaving the original mbuf+cluster in place in the receive ring for immediate re-use. This saves a trip through 2 of the 3 zones of the compound mbuf allocator, a trip through busdma, and a trip through the 1 of the 3 mbuf destructors. For our load at Netflix, this can lower CPU consumption by as much as 20%. The copy algorithm is based on investigative work from Luigi Rizzo earlier in the year. Reviewed by: jfv Obtained from: Netflix
2012-08-31 10:07:38 +00:00
goto update;
}
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
rxbuf->buf = mp;
bus_dmamap_sync(rxr->tag, rxbuf->map,
Heavily optimize the case of small RX packets of 160 bytes or less. For this case, allocate a plain mbuf and copy the frame into it, then send the copy up the stack, leaving the original mbuf+cluster in place in the receive ring for immediate re-use. This saves a trip through 2 of the 3 zones of the compound mbuf allocator, a trip through busdma, and a trip through the 1 of the 3 mbuf destructors. For our load at Netflix, this can lower CPU consumption by as much as 20%. The copy algorithm is based on investigative work from Luigi Rizzo earlier in the year. Reviewed by: jfv Obtained from: Netflix
2012-08-31 10:07:38 +00:00
BUS_DMASYNC_PREREAD);
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
rxbuf->addr = rxr->rx_base[i].read.pkt_addr =
htole64(seg[0].ds_addr);
Heavily optimize the case of small RX packets of 160 bytes or less. For this case, allocate a plain mbuf and copy the frame into it, then send the copy up the stack, leaving the original mbuf+cluster in place in the receive ring for immediate re-use. This saves a trip through 2 of the 3 zones of the compound mbuf allocator, a trip through busdma, and a trip through the 1 of the 3 mbuf destructors. For our load at Netflix, this can lower CPU consumption by as much as 20%. The copy algorithm is based on investigative work from Luigi Rizzo earlier in the year. Reviewed by: jfv Obtained from: Netflix
2012-08-31 10:07:38 +00:00
} else {
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
rxr->rx_base[i].read.pkt_addr = rxbuf->addr;
Heavily optimize the case of small RX packets of 160 bytes or less. For this case, allocate a plain mbuf and copy the frame into it, then send the copy up the stack, leaving the original mbuf+cluster in place in the receive ring for immediate re-use. This saves a trip through 2 of the 3 zones of the compound mbuf allocator, a trip through busdma, and a trip through the 1 of the 3 mbuf destructors. For our load at Netflix, this can lower CPU consumption by as much as 20%. The copy algorithm is based on investigative work from Luigi Rizzo earlier in the year. Reviewed by: jfv Obtained from: Netflix
2012-08-31 10:07:38 +00:00
rxbuf->flags &= ~IXGBE_RX_COPY;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
}
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
refreshed = TRUE;
/* Next is precalculated */
i = j;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
rxr->next_to_refresh = i;
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
if (++j == rxr->num_desc)
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
j = 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
update:
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
if (refreshed) /* Update hardware tail index */
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
IXGBE_WRITE_REG(&adapter->hw,
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
IXGBE_RDT(rxr->me), rxr->next_to_refresh);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
return;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
/*********************************************************************
*
* Allocate memory for rx_buffer structures. Since we use one
* rx_buffer per received packet, the maximum number of rx_buffer's
* that we'll need is equal to the number of receive descriptors
* that we've allocated.
*
**********************************************************************/
static int
ixgbe_allocate_receive_buffers(struct rx_ring *rxr)
{
struct adapter *adapter = rxr->adapter;
device_t dev = adapter->dev;
struct ixgbe_rx_buf *rxbuf;
int i, bsize, error;
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
bsize = sizeof(struct ixgbe_rx_buf) * rxr->num_desc;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (!(rxr->rx_buffers =
(struct ixgbe_rx_buf *) malloc(bsize,
M_DEVBUF, M_NOWAIT | M_ZERO))) {
device_printf(dev, "Unable to allocate rx_buffer memory\n");
error = ENOMEM;
goto fail;
}
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
if ((error = bus_dma_tag_create(bus_get_dma_tag(dev), /* parent */
1, 0, /* alignment, bounds */
BUS_SPACE_MAXADDR, /* lowaddr */
BUS_SPACE_MAXADDR, /* highaddr */
NULL, NULL, /* filter, filterarg */
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
MJUM16BYTES, /* maxsize */
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
1, /* nsegments */
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
MJUM16BYTES, /* maxsegsize */
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
0, /* flags */
NULL, /* lockfunc */
NULL, /* lockfuncarg */
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
&rxr->tag))) {
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
device_printf(dev, "Unable to create RX DMA tag\n");
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
goto fail;
}
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
for (i = 0; i < rxr->num_desc; i++, rxbuf++) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
rxbuf = &rxr->rx_buffers[i];
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
error = bus_dmamap_create(rxr->tag,
BUS_DMA_NOWAIT, &rxbuf->map);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (error) {
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
device_printf(dev, "Unable to create RX dma map\n");
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
goto fail;
}
}
return (0);
fail:
/* Frees all, but can handle partial completion */
ixgbe_free_receive_structures(adapter);
return (error);
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/*
** Used to detect a descriptor that has
** been merged by Hardware RSC.
*/
static inline u32
ixgbe_rsc_count(union ixgbe_adv_rx_desc *rx)
{
return (le32toh(rx->wb.lower.lo_dword.data) &
IXGBE_RXDADV_RSCCNT_MASK) >> IXGBE_RXDADV_RSCCNT_SHIFT;
}
/*********************************************************************
*
* Initialize Hardware RSC (LRO) feature on 82599
* for an RX ring, this is toggled by the LRO capability
* even though it is transparent to the stack.
*
A few important fixes: - Testing TSO6 has led me to discover that HW RSC is a problematic feature, it is ONLY designed to work with IPv4 in the first place, and if IP forwarding is done it can't be disabled as LRO in the stack, also initial testing we've done at Intel shows an equal performance using TSO[46] on the TX and LRO on RX, if you ran older code on 82599 or later hardware you actually could have detrimental performance for this reason. So I am disabling the feature by default and all our adapters will now use LRO instead. - If you have flow control off and multiple queues it was possible when the buffer of one queue becomes full that all RX movement is stalled, to eliminate this problem a feature bit is now set that will allow packets to be dropped when full rather than stall. Note, the default is to have flow control on, and this keeps this from happening. - Because of the recent fixes in the stack, LRO is now auto-disabled when problematic, so I have decided to enable it by default in the capabilities in the driver. - There are some 1G modules used by some customers, a couple small tweaks to properly support those in the media code. - A note: we have now done some testing of TSO6 and using LRO with IPv6 and it all works great!! Seeing line rate in both directions in best cases. Thanks bz for your excellent work!!
2012-10-31 23:50:36 +00:00
* NOTE: since this HW feature only works with IPV4 and
* our testing has shown soft LRO to be as effective
* I have decided to disable this by default.
*
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
**********************************************************************/
static void
ixgbe_setup_hw_rsc(struct rx_ring *rxr)
{
struct adapter *adapter = rxr->adapter;
struct ixgbe_hw *hw = &adapter->hw;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
u32 rscctrl, rdrxctl;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
A few important fixes: - Testing TSO6 has led me to discover that HW RSC is a problematic feature, it is ONLY designed to work with IPv4 in the first place, and if IP forwarding is done it can't be disabled as LRO in the stack, also initial testing we've done at Intel shows an equal performance using TSO[46] on the TX and LRO on RX, if you ran older code on 82599 or later hardware you actually could have detrimental performance for this reason. So I am disabling the feature by default and all our adapters will now use LRO instead. - If you have flow control off and multiple queues it was possible when the buffer of one queue becomes full that all RX movement is stalled, to eliminate this problem a feature bit is now set that will allow packets to be dropped when full rather than stall. Note, the default is to have flow control on, and this keeps this from happening. - Because of the recent fixes in the stack, LRO is now auto-disabled when problematic, so I have decided to enable it by default in the capabilities in the driver. - There are some 1G modules used by some customers, a couple small tweaks to properly support those in the media code. - A note: we have now done some testing of TSO6 and using LRO with IPv6 and it all works great!! Seeing line rate in both directions in best cases. Thanks bz for your excellent work!!
2012-10-31 23:50:36 +00:00
/* If turning LRO/RSC off we need to disable it */
if ((adapter->ifp->if_capenable & IFCAP_LRO) == 0) {
rscctrl = IXGBE_READ_REG(hw, IXGBE_RSCCTL(rxr->me));
rscctrl &= ~IXGBE_RSCCTL_RSCEN;
return;
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
rdrxctl = IXGBE_READ_REG(hw, IXGBE_RDRXCTL);
rdrxctl &= ~IXGBE_RDRXCTL_RSCFRSTSIZE;
Properly disable crc stripping when operating in netmap mode. Contrarily to what i wrote in my previous commit, the 82599 does include the CRC in the length. The operating mode is reset in ixgbe_init_locked() and so we need to hook into the places where the two registers (HLREG0 and RDRXCTL) are modified.
2012-04-13 16:42:54 +00:00
#ifdef DEV_NETMAP /* crcstrip is optional in netmap */
if (adapter->ifp->if_capenable & IFCAP_NETMAP && !ix_crcstrip)
#endif /* DEV_NETMAP */
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
rdrxctl |= IXGBE_RDRXCTL_CRCSTRIP;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
rdrxctl |= IXGBE_RDRXCTL_RSCACKC;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
IXGBE_WRITE_REG(hw, IXGBE_RDRXCTL, rdrxctl);
rscctrl = IXGBE_READ_REG(hw, IXGBE_RSCCTL(rxr->me));
rscctrl |= IXGBE_RSCCTL_RSCEN;
/*
** Limit the total number of descriptors that
** can be combined, so it does not exceed 64K
*/
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
if (rxr->mbuf_sz == MCLBYTES)
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
rscctrl |= IXGBE_RSCCTL_MAXDESC_16;
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
else if (rxr->mbuf_sz == MJUMPAGESIZE)
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
rscctrl |= IXGBE_RSCCTL_MAXDESC_8;
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
else if (rxr->mbuf_sz == MJUM9BYTES)
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
rscctrl |= IXGBE_RSCCTL_MAXDESC_4;
else /* Using 16K cluster */
rscctrl |= IXGBE_RSCCTL_MAXDESC_1;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
IXGBE_WRITE_REG(hw, IXGBE_RSCCTL(rxr->me), rscctrl);
/* Enable TCP header recognition */
IXGBE_WRITE_REG(hw, IXGBE_PSRTYPE(0),
(IXGBE_READ_REG(hw, IXGBE_PSRTYPE(0)) |
IXGBE_PSRTYPE_TCPHDR));
/* Disable RSC for ACK packets */
IXGBE_WRITE_REG(hw, IXGBE_RSCDBU,
(IXGBE_RSCDBU_RSCACKDIS | IXGBE_READ_REG(hw, IXGBE_RSCDBU)));
rxr->hw_rsc = TRUE;
}
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
static void
ixgbe_free_receive_ring(struct rx_ring *rxr)
{
struct ixgbe_rx_buf *rxbuf;
int i;
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
for (i = 0; i < rxr->num_desc; i++) {
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
rxbuf = &rxr->rx_buffers[i];
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
if (rxbuf->buf != NULL) {
bus_dmamap_sync(rxr->tag, rxbuf->map,
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
BUS_DMASYNC_POSTREAD);
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
bus_dmamap_unload(rxr->tag, rxbuf->map);
rxbuf->buf->m_flags |= M_PKTHDR;
m_freem(rxbuf->buf);
rxbuf->buf = NULL;
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
}
}
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*********************************************************************
*
* Initialize a receive ring and its buffers.
*
**********************************************************************/
static int
ixgbe_setup_receive_ring(struct rx_ring *rxr)
{
struct adapter *adapter;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
struct ifnet *ifp;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
device_t dev;
struct ixgbe_rx_buf *rxbuf;
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
bus_dma_segment_t seg[1];
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
struct lro_ctrl *lro = &rxr->lro;
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
int rsize, nsegs, error = 0;
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
#ifdef DEV_NETMAP
struct netmap_adapter *na = NA(rxr->adapter->ifp);
struct netmap_slot *slot;
#endif /* DEV_NETMAP */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
adapter = rxr->adapter;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
ifp = adapter->ifp;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
dev = adapter->dev;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Clear the ring contents */
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
IXGBE_RX_LOCK(rxr);
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
#ifdef DEV_NETMAP
/* same as in ixgbe_setup_transmit_ring() */
slot = netmap_reset(na, NR_RX, rxr->me, 0);
#endif /* DEV_NETMAP */
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
rsize = roundup2(adapter->num_rx_desc *
sizeof(union ixgbe_adv_rx_desc), DBA_ALIGN);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
bzero((void *)rxr->rx_base, rsize);
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
/* Cache the size */
rxr->mbuf_sz = adapter->rx_mbuf_sz;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
/* Free current RX buffer structs and their mbufs */
ixgbe_free_receive_ring(rxr);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* Now replenish the mbufs */
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
for (int j = 0; j != rxr->num_desc; ++j) {
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
struct mbuf *mp;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
rxbuf = &rxr->rx_buffers[j];
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
#ifdef DEV_NETMAP
/*
* In netmap mode, fill the map and set the buffer
* address in the NIC ring, considering the offset
* between the netmap and NIC rings (see comment in
* ixgbe_setup_transmit_ring() ). No need to allocate
* an mbuf, so end the block with a continue;
*/
if (slot) {
A bunch of netmap fixes: USERSPACE: 1. add support for devices with different number of rx and tx queues; 2. add better support for zero-copy operation, adding an extra field to the netmap ring to indicate how many buffers we have already processed but not yet released (with help from Eddie Kohler); 3. The two changes above unfortunately require an API change, so while at it add a version field and some spares to the ioctl() argument to help detect mismatches. 4. update the manual page for the two changes above; 5. update sample applications in tools/tools/netmap KERNEL: 1. simplify the internal structures moving the global wait queues to the 'struct netmap_adapter'; 2. simplify the functions that map kring<->nic ring indexes 3. normalize device-specific code, helps mainteinance; 4. start exploring the impact of micro-optimizations (prefetch etc.) in the ixgbe driver. Use 'legacy' descriptors on the tx ring and prefetch slots gives about 20% speedup at 900 MHz. Another 7-10% would come from removing the explict calls to bus_dmamap* in the core (they are effectively NOPs in this case, but it takes expensive load of the per-buffer dma maps to figure out that they are all NULL. Rx performance not investigated. I am postponing the MFC so i can import a few more improvements before merging.
2012-02-27 19:05:01 +00:00
int sj = netmap_idx_n2k(&na->rx_rings[rxr->me], j);
small code cleanup in preparation for future modifications in the memory allocator used by netmap. No functional change, two small bug fixes: - in if_re.c add a missing bus_dmamap_sync() - in netmap.c comment out a spurious free() in an error handling block
2012-01-10 19:57:23 +00:00
uint64_t paddr;
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
void *addr;
small code cleanup in preparation for future modifications in the memory allocator used by netmap. No functional change, two small bug fixes: - in if_re.c add a missing bus_dmamap_sync() - in netmap.c comment out a spurious free() in an error handling block
2012-01-10 19:57:23 +00:00
addr = PNMB(slot + sj, &paddr);
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
netmap_load_map(rxr->tag, rxbuf->map, addr);
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
/* Update descriptor */
small code cleanup in preparation for future modifications in the memory allocator used by netmap. No functional change, two small bug fixes: - in if_re.c add a missing bus_dmamap_sync() - in netmap.c comment out a spurious free() in an error handling block
2012-01-10 19:57:23 +00:00
rxr->rx_base[j].read.pkt_addr = htole64(paddr);
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
continue;
}
#endif /* DEV_NETMAP */
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
rxbuf->buf = m_getjcl(M_NOWAIT, MT_DATA,
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
M_PKTHDR, adapter->rx_mbuf_sz);
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
if (rxbuf->buf == NULL) {
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
error = ENOBUFS;
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
goto fail;
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
}
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
mp = rxbuf->buf;
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
mp->m_pkthdr.len = mp->m_len = rxr->mbuf_sz;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* Get the memory mapping */
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
error = bus_dmamap_load_mbuf_sg(rxr->tag,
rxbuf->map, mp, seg,
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
&nsegs, BUS_DMA_NOWAIT);
if (error != 0)
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
goto fail;
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
bus_dmamap_sync(rxr->tag,
rxbuf->map, BUS_DMASYNC_PREREAD);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* Update descriptor */
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
rxr->rx_base[j].read.pkt_addr = htole64(seg[0].ds_addr);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Setup our descriptor indices */
rxr->next_to_check = 0;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
rxr->next_to_refresh = 0;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
rxr->lro_enabled = FALSE;
Heavily optimize the case of small RX packets of 160 bytes or less. For this case, allocate a plain mbuf and copy the frame into it, then send the copy up the stack, leaving the original mbuf+cluster in place in the receive ring for immediate re-use. This saves a trip through 2 of the 3 zones of the compound mbuf allocator, a trip through busdma, and a trip through the 1 of the 3 mbuf destructors. For our load at Netflix, this can lower CPU consumption by as much as 20%. The copy algorithm is based on investigative work from Luigi Rizzo earlier in the year. Reviewed by: jfv Obtained from: Netflix
2012-08-31 10:07:38 +00:00
rxr->rx_copies = 0;
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
rxr->rx_bytes = 0;
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
rxr->discard = FALSE;
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
rxr->vtag_strip = FALSE;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
bus_dmamap_sync(rxr->rxdma.dma_tag, rxr->rxdma.dma_map,
BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/*
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
** Now set up the LRO interface:
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
*/
A few important fixes: - Testing TSO6 has led me to discover that HW RSC is a problematic feature, it is ONLY designed to work with IPv4 in the first place, and if IP forwarding is done it can't be disabled as LRO in the stack, also initial testing we've done at Intel shows an equal performance using TSO[46] on the TX and LRO on RX, if you ran older code on 82599 or later hardware you actually could have detrimental performance for this reason. So I am disabling the feature by default and all our adapters will now use LRO instead. - If you have flow control off and multiple queues it was possible when the buffer of one queue becomes full that all RX movement is stalled, to eliminate this problem a feature bit is now set that will allow packets to be dropped when full rather than stall. Note, the default is to have flow control on, and this keeps this from happening. - Because of the recent fixes in the stack, LRO is now auto-disabled when problematic, so I have decided to enable it by default in the capabilities in the driver. - There are some 1G modules used by some customers, a couple small tweaks to properly support those in the media code. - A note: we have now done some testing of TSO6 and using LRO with IPv6 and it all works great!! Seeing line rate in both directions in best cases. Thanks bz for your excellent work!!
2012-10-31 23:50:36 +00:00
if (ixgbe_rsc_enable)
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
ixgbe_setup_hw_rsc(rxr);
else if (ifp->if_capenable & IFCAP_LRO) {
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
int err = tcp_lro_init(lro);
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
if (err) {
device_printf(dev, "LRO Initialization failed!\n");
goto fail;
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
INIT_DEBUGOUT("RX Soft LRO Initialized\n");
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
rxr->lro_enabled = TRUE;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
lro->ifp = adapter->ifp;
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
IXGBE_RX_UNLOCK(rxr);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return (0);
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
fail:
ixgbe_free_receive_ring(rxr);
IXGBE_RX_UNLOCK(rxr);
return (error);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
/*********************************************************************
*
* Initialize all receive rings.
*
**********************************************************************/
static int
ixgbe_setup_receive_structures(struct adapter *adapter)
{
struct rx_ring *rxr = adapter->rx_rings;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
int j;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
for (j = 0; j < adapter->num_queues; j++, rxr++)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (ixgbe_setup_receive_ring(rxr))
goto fail;
return (0);
fail:
/*
* Free RX buffers allocated so far, we will only handle
* the rings that completed, the failing case will have
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
* cleaned up for itself. 'j' failed, so its the terminus.
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
*/
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
for (int i = 0; i < j; ++i) {
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
rxr = &adapter->rx_rings[i];
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
ixgbe_free_receive_ring(rxr);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
return (ENOBUFS);
}
/*********************************************************************
*
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
* Setup receive registers and features.
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
*
**********************************************************************/
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
#define IXGBE_SRRCTL_BSIZEHDRSIZE_SHIFT 2
The maximum read size of incoming packets is done in 1024-byte increments. The current code was rounding down the maximum frame size instead of routing up, resulting in a read size of 1024 bytes, in the non-jumbo frame case, and splitting the packets across multiple mbufs. Consequently the above problem exposed another issue, which is when packets were splitted across multiple mbufs, and all of the mbufs in the chain have the M_PKTHDR flag set. Submitted by: original patch by Ray Ruvinskiy at BlueCoat dot com Reviewed by: jfv, kmacy, rwatson Approved by: re (rwatson) MFC after: 5 days
2011-09-05 17:54:19 +00:00
#define BSIZEPKT_ROUNDUP ((1<<IXGBE_SRRCTL_BSIZEPKT_SHIFT)-1)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static void
ixgbe_initialize_receive_units(struct adapter *adapter)
{
struct rx_ring *rxr = adapter->rx_rings;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
struct ixgbe_hw *hw = &adapter->hw;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
struct ifnet *ifp = adapter->ifp;
Multiqueue RX is not correctly enabled on the new 82599 adapter, the SRRCTL register needs to be setup per queue. Approved by: re
2009-07-01 16:13:01 +00:00
u32 bufsz, rxctrl, fctrl, srrctl, rxcsum;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
u32 reta, mrqc = 0, hlreg, random[10];
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*
* Make sure receives are disabled while
* setting up the descriptor ring
*/
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
rxctrl = IXGBE_READ_REG(hw, IXGBE_RXCTRL);
IXGBE_WRITE_REG(hw, IXGBE_RXCTRL,
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
rxctrl & ~IXGBE_RXCTRL_RXEN);
/* Enable broadcasts */
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
fctrl = IXGBE_READ_REG(hw, IXGBE_FCTRL);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
fctrl |= IXGBE_FCTRL_BAM;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
fctrl |= IXGBE_FCTRL_DPF;
fctrl |= IXGBE_FCTRL_PMCF;
IXGBE_WRITE_REG(hw, IXGBE_FCTRL, fctrl);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/* Set for Jumbo Frames? */
Multiqueue RX is not correctly enabled on the new 82599 adapter, the SRRCTL register needs to be setup per queue. Approved by: re
2009-07-01 16:13:01 +00:00
hlreg = IXGBE_READ_REG(hw, IXGBE_HLREG0);
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
if (ifp->if_mtu > ETHERMTU)
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
hlreg |= IXGBE_HLREG0_JUMBOEN;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
else
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
hlreg &= ~IXGBE_HLREG0_JUMBOEN;
Properly disable crc stripping when operating in netmap mode. Contrarily to what i wrote in my previous commit, the 82599 does include the CRC in the length. The operating mode is reset in ixgbe_init_locked() and so we need to hook into the places where the two registers (HLREG0 and RDRXCTL) are modified.
2012-04-13 16:42:54 +00:00
#ifdef DEV_NETMAP
/* crcstrip is conditional in netmap (in RDRXCTL too ?) */
if (ifp->if_capenable & IFCAP_NETMAP && !ix_crcstrip)
hlreg &= ~IXGBE_HLREG0_RXCRCSTRP;
else
hlreg |= IXGBE_HLREG0_RXCRCSTRP;
#endif /* DEV_NETMAP */
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
IXGBE_WRITE_REG(hw, IXGBE_HLREG0, hlreg);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
bufsz = (adapter->rx_mbuf_sz +
BSIZEPKT_ROUNDUP) >> IXGBE_SRRCTL_BSIZEPKT_SHIFT;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
for (int i = 0; i < adapter->num_queues; i++, rxr++) {
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
u64 rdba = rxr->rxdma.dma_paddr;
Multiqueue RX is not correctly enabled on the new 82599 adapter, the SRRCTL register needs to be setup per queue. Approved by: re
2009-07-01 16:13:01 +00:00
Patch #7 This is primarily about processing limit control. - add a limit for both RX and TX, change the default to 256 - change the sysctl usage to be common, and now to be called during init for each ring. - the TX limit is not yet used, but the changes in the last patch in this series uses the value. - the motivation behind these changes is to improve data locality in the final code. - rxeof interface changes since it now gets limit from the ring struct
2012-11-30 23:28:01 +00:00
/* Sysctl for limiting work done in rx clean */
ixgbe_add_process_limit(adapter, "rx_processing_limit",
"max number of packets to process", &rxr->process_limit,
ixgbe_rx_process_limit);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Setup the Base and Length of the Rx Descriptor Ring */
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
IXGBE_WRITE_REG(hw, IXGBE_RDBAL(i),
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
(rdba & 0x00000000ffffffffULL));
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
IXGBE_WRITE_REG(hw, IXGBE_RDBAH(i), (rdba >> 32));
IXGBE_WRITE_REG(hw, IXGBE_RDLEN(i),
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
adapter->num_rx_desc * sizeof(union ixgbe_adv_rx_desc));
Multiqueue RX is not correctly enabled on the new 82599 adapter, the SRRCTL register needs to be setup per queue. Approved by: re
2009-07-01 16:13:01 +00:00
/* Set up the SRRCTL register */
srrctl = IXGBE_READ_REG(hw, IXGBE_SRRCTL(i));
srrctl &= ~IXGBE_SRRCTL_BSIZEHDR_MASK;
srrctl &= ~IXGBE_SRRCTL_BSIZEPKT_MASK;
srrctl |= bufsz;
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
srrctl |= IXGBE_SRRCTL_DESCTYPE_ADV_ONEBUF;
Multiqueue RX is not correctly enabled on the new 82599 adapter, the SRRCTL register needs to be setup per queue. Approved by: re
2009-07-01 16:13:01 +00:00
IXGBE_WRITE_REG(hw, IXGBE_SRRCTL(i), srrctl);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Setup the HW Rx Head and Tail Descriptor Pointers */
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
IXGBE_WRITE_REG(hw, IXGBE_RDH(i), 0);
IXGBE_WRITE_REG(hw, IXGBE_RDT(i), 0);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
if (adapter->hw.mac.type != ixgbe_mac_82598EB) {
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
u32 psrtype = IXGBE_PSRTYPE_TCPHDR |
IXGBE_PSRTYPE_UDPHDR |
IXGBE_PSRTYPE_IPV4HDR |
IXGBE_PSRTYPE_IPV6HDR;
IXGBE_WRITE_REG(hw, IXGBE_PSRTYPE(0), psrtype);
}
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
rxcsum = IXGBE_READ_REG(hw, IXGBE_RXCSUM);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/* Setup RSS */
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
if (adapter->num_queues > 1) {
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
int i, j;
reta = 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* set up random bits */
arc4rand(&random, sizeof(random), 0);
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Set up the redirection table */
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
for (i = 0, j = 0; i < 128; i++, j++) {
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
if (j == adapter->num_queues) j = 0;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
reta = (reta << 8) | (j * 0x11);
if ((i & 3) == 3)
IXGBE_WRITE_REG(hw, IXGBE_RETA(i >> 2), reta);
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Now fill our hash function seeds */
for (int i = 0; i < 10; i++)
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
IXGBE_WRITE_REG(hw, IXGBE_RSSRK(i), random[i]);
/* Perform hash on these packet types */
Multiqueue RX is not correctly enabled on the new 82599 adapter, the SRRCTL register needs to be setup per queue. Approved by: re
2009-07-01 16:13:01 +00:00
mrqc = IXGBE_MRQC_RSSEN
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
| IXGBE_MRQC_RSS_FIELD_IPV4
| IXGBE_MRQC_RSS_FIELD_IPV4_TCP
| IXGBE_MRQC_RSS_FIELD_IPV4_UDP
| IXGBE_MRQC_RSS_FIELD_IPV6_EX_TCP
| IXGBE_MRQC_RSS_FIELD_IPV6_EX
| IXGBE_MRQC_RSS_FIELD_IPV6
| IXGBE_MRQC_RSS_FIELD_IPV6_TCP
| IXGBE_MRQC_RSS_FIELD_IPV6_UDP
| IXGBE_MRQC_RSS_FIELD_IPV6_EX_UDP;
IXGBE_WRITE_REG(hw, IXGBE_MRQC, mrqc);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* RSS and RX IPP Checksum are mutually exclusive */
rxcsum |= IXGBE_RXCSUM_PCSD;
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
if (ifp->if_capenable & IFCAP_RXCSUM)
rxcsum |= IXGBE_RXCSUM_PCSD;
if (!(rxcsum & IXGBE_RXCSUM_PCSD))
rxcsum |= IXGBE_RXCSUM_IPPCSE;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
IXGBE_WRITE_REG(hw, IXGBE_RXCSUM, rxcsum);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
}
/*********************************************************************
*
* Free all receive rings.
*
**********************************************************************/
static void
ixgbe_free_receive_structures(struct adapter *adapter)
{
struct rx_ring *rxr = adapter->rx_rings;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
for (int i = 0; i < adapter->num_queues; i++, rxr++) {
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
struct lro_ctrl *lro = &rxr->lro;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_free_receive_buffers(rxr);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* Free LRO memory */
tcp_lro_free(lro);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Free the ring memory as well */
ixgbe_dma_free(adapter, &rxr->rxdma);
}
free(adapter->rx_rings, M_DEVBUF);
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*********************************************************************
*
* Free receive ring data structures
*
**********************************************************************/
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
static void
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ixgbe_free_receive_buffers(struct rx_ring *rxr)
{
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
struct adapter *adapter = rxr->adapter;
struct ixgbe_rx_buf *rxbuf;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
INIT_DEBUGOUT("free_receive_structures: begin");
/* Cleanup any existing buffers */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (rxr->rx_buffers != NULL) {
for (int i = 0; i < adapter->num_rx_desc; i++) {
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
rxbuf = &rxr->rx_buffers[i];
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
if (rxbuf->buf != NULL) {
bus_dmamap_sync(rxr->tag, rxbuf->map,
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
BUS_DMASYNC_POSTREAD);
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
bus_dmamap_unload(rxr->tag, rxbuf->map);
rxbuf->buf->m_flags |= M_PKTHDR;
m_freem(rxbuf->buf);
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
}
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
rxbuf->buf = NULL;
if (rxbuf->map != NULL) {
bus_dmamap_destroy(rxr->tag, rxbuf->map);
rxbuf->map = NULL;
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
}
}
if (rxr->rx_buffers != NULL) {
free(rxr->rx_buffers, M_DEVBUF);
rxr->rx_buffers = NULL;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
}
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
if (rxr->tag != NULL) {
bus_dma_tag_destroy(rxr->tag);
rxr->tag = NULL;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
A few changes: When not defining header split do not allocate mbufs, this can be a BIG savings in the mbuf memory pool. Also keep seperate dma maps for the header and payload pieces when doing header split. The basis of this code was a patch done a while ago by yongari, thank you :) A number of white space changes. MFC: in a few days
2010-05-19 00:03:48 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
static __inline void
ixgbe_rx_input(struct rx_ring *rxr, struct ifnet *ifp, struct mbuf *m, u32 ptype)
{
/*
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
* ATM LRO is only for IP/TCP packets and TCP checksum of the packet
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
* should be computed by hardware. Also it should not have VLAN tag in
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
* ethernet header. In case of IPv6 we do not yet support ext. hdrs.
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
*/
if (rxr->lro_enabled &&
(ifp->if_capenable & IFCAP_VLAN_HWTAGGING) != 0 &&
(ptype & IXGBE_RXDADV_PKTTYPE_ETQF) == 0 &&
MFp4 bz_ipv6_fast: Add TSO6 and LRO/IPv6 support. Fix the module Makefile to at least properly inlcude opt_inet6.h and allow builds without INET or INET6. Sponsored by: The FreeBSD Foundation Sponsored by: iXsystems Reviewed by: gnn (as part of the whole) MFC After: 3 days
2012-05-25 03:02:56 +00:00
((ptype & (IXGBE_RXDADV_PKTTYPE_IPV4 | IXGBE_RXDADV_PKTTYPE_TCP)) ==
(IXGBE_RXDADV_PKTTYPE_IPV4 | IXGBE_RXDADV_PKTTYPE_TCP) ||
(ptype & (IXGBE_RXDADV_PKTTYPE_IPV6 | IXGBE_RXDADV_PKTTYPE_TCP)) ==
(IXGBE_RXDADV_PKTTYPE_IPV6 | IXGBE_RXDADV_PKTTYPE_TCP)) &&
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
(m->m_pkthdr.csum_flags & (CSUM_DATA_VALID | CSUM_PSEUDO_HDR)) ==
(CSUM_DATA_VALID | CSUM_PSEUDO_HDR)) {
/*
* Send to the stack if:
** - LRO not enabled, or
** - no LRO resources, or
** - lro enqueue fails
*/
if (rxr->lro.lro_cnt != 0)
if (tcp_lro_rx(&rxr->lro, m, 0) == 0)
return;
}
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
IXGBE_RX_UNLOCK(rxr);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
(*ifp->if_input)(ifp, m);
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
IXGBE_RX_LOCK(rxr);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
}
static __inline void
ixgbe_rx_discard(struct rx_ring *rxr, int i)
{
struct ixgbe_rx_buf *rbuf;
rbuf = &rxr->rx_buffers[i];
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
if (rbuf->fmp != NULL) {/* Partial chain ? */
rbuf->fmp->m_flags |= M_PKTHDR;
m_freem(rbuf->fmp);
rbuf->fmp = NULL;
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
/*
** With advanced descriptors the writeback
** clobbers the buffer addrs, so its easier
** to just free the existing mbufs and take
** the normal refresh path to get new buffers
** and mapping.
*/
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
if (rbuf->buf) {
m_free(rbuf->buf);
rbuf->buf = NULL;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
return;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*********************************************************************
*
* This routine executes in interrupt context. It replenishes
* the mbufs in the descriptor and sends data which has been
* dma'ed into host memory to upper layer.
*
* We loop at most count times if count is > 0, or until done if
* count < 0.
*
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
* Return TRUE for more work, FALSE for all clean.
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
*********************************************************************/
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
static bool
Patch #7 This is primarily about processing limit control. - add a limit for both RX and TX, change the default to 256 - change the sysctl usage to be common, and now to be called during init for each ring. - the TX limit is not yet used, but the changes in the last patch in this series uses the value. - the motivation behind these changes is to improve data locality in the final code. - rxeof interface changes since it now gets limit from the ring struct
2012-11-30 23:28:01 +00:00
ixgbe_rxeof(struct ix_queue *que)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct adapter *adapter = que->adapter;
struct rx_ring *rxr = que->rxr;
struct ifnet *ifp = adapter->ifp;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
struct lro_ctrl *lro = &rxr->lro;
struct lro_entry *queued;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
int i, nextp, processed = 0;
u32 staterr = 0;
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
u16 count = rxr->process_limit;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
union ixgbe_adv_rx_desc *cur;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct ixgbe_rx_buf *rbuf, *nbuf;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_RX_LOCK(rxr);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
#ifdef DEV_NETMAP
if (ifp->if_capenable & IFCAP_NETMAP) {
/*
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
* Same as the txeof routine: only wakeup clients on intr.
* NKR_PENDINTR in nr_kflags is used to implement interrupt
* mitigation (ixgbe_rxsync() will not look for new packets
* unless NKR_PENDINTR is set).
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
*/
struct netmap_adapter *na = NA(ifp);
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
na->rx_rings[rxr->me].nr_kflags |= NKR_PENDINTR;
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
selwakeuppri(&na->rx_rings[rxr->me].si, PI_NET);
IXGBE_RX_UNLOCK(rxr);
IXGBE_CORE_LOCK(adapter);
A bunch of netmap fixes: USERSPACE: 1. add support for devices with different number of rx and tx queues; 2. add better support for zero-copy operation, adding an extra field to the netmap ring to indicate how many buffers we have already processed but not yet released (with help from Eddie Kohler); 3. The two changes above unfortunately require an API change, so while at it add a version field and some spares to the ioctl() argument to help detect mismatches. 4. update the manual page for the two changes above; 5. update sample applications in tools/tools/netmap KERNEL: 1. simplify the internal structures moving the global wait queues to the 'struct netmap_adapter'; 2. simplify the functions that map kring<->nic ring indexes 3. normalize device-specific code, helps mainteinance; 4. start exploring the impact of micro-optimizations (prefetch etc.) in the ixgbe driver. Use 'legacy' descriptors on the tx ring and prefetch slots gives about 20% speedup at 900 MHz. Another 7-10% would come from removing the explict calls to bus_dmamap* in the core (they are effectively NOPs in this case, but it takes expensive load of the per-buffer dma maps to figure out that they are all NULL. Rx performance not investigated. I am postponing the MFC so i can import a few more improvements before merging.
2012-02-27 19:05:01 +00:00
selwakeuppri(&na->rx_si, PI_NET);
1. Fix the handling of link reset while in netmap more. A link reset now is completely transparent for the netmap client: even if the NIC resets its own ring (e.g. restarting from 0), the client will not see any change in the current rx/tx positions, because the driver will keep track of the offset between the two. 2. make the device-specific code more uniform across different drivers There were some inconsistencies in the implementation of the netmap support routines, now drivers have been aligned to a common code structure. 3. import netmap support for ixgbe . This is implemented as a very small patch for ixgbe.c (233 lines, 11 chunks, mostly comments: in total the patch has only 54 lines of new code) , as most of the code is in an external file sys/dev/netmap/ixgbe_netmap.h , following some initial comments from Jack Vogel about making changes less intrusive. (Note, i have emailed Jack multiple times asking if he had comments on this structure of the code; i got no reply so i assume he is fine with it). Support for other drivers (em, lem, re, igb) will come later. "ixgbe" is now the reference driver for netmap support. Both the external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should serve as a reference for other device drivers. Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap, the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz on an i7-860 with 4 cores and 82599 card. Haven't tried yet more aggressive optimizations such as adding 'prefetch' instructions in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
IXGBE_CORE_UNLOCK(adapter);
return (FALSE);
}
#endif /* DEV_NETMAP */
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
for (i = rxr->next_to_check; count != 0;) {
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
struct mbuf *sendmp, *mp;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
u32 rsc, ptype;
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
u16 len;
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
u16 vtag = 0;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
bool eop;
Fixes for panic experienced in test at Intel, when doing bidirectional stress traffic on 82598. Also a couple bug fixes from Michael Tuexen, thank you!! Add a workaround into the header so that 8 REL can use the driver (adds local copy of ALTQ fix). MFC: in a few days
2010-06-03 00:00:45 +00:00
/* Sync the ring. */
bus_dmamap_sync(rxr->rxdma.dma_tag, rxr->rxdma.dma_map,
BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
cur = &rxr->rx_base[i];
staterr = le32toh(cur->wb.upper.status_error);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if ((staterr & IXGBE_RXD_STAT_DD) == 0)
break;
if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0)
break;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
count--;
sendmp = NULL;
nbuf = NULL;
rsc = 0;
cur->wb.upper.status_error = 0;
rbuf = &rxr->rx_buffers[i];
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
mp = rbuf->buf;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
len = le16toh(cur->wb.upper.length);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
ptype = le32toh(cur->wb.lower.lo_dword.data) &
IXGBE_RXDADV_PKTTYPE_MASK;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
eop = ((staterr & IXGBE_RXD_STAT_EOP) != 0);
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
/* Make sure bad packets are discarded */
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if (((staterr & IXGBE_RXDADV_ERR_FRAME_ERR_MASK) != 0) ||
(rxr->discard)) {
ifp->if_ierrors++;
rxr->rx_discarded++;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
if (eop)
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
rxr->discard = FALSE;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
else
rxr->discard = TRUE;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
ixgbe_rx_discard(rxr, i);
goto next_desc;
}
/*
** On 82599 which supports a hardware
** LRO (called HW RSC), packets need
** not be fragmented across sequential
** descriptors, rather the next descriptor
** is indicated in bits of the descriptor.
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
** This also means that we might proceses
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
** more than one packet at a time, something
** that has never been true before, it
** required eliminating global chain pointers
** in favor of what we are doing here. -jfv
*/
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
if (!eop) {
/*
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
** Figure out the next descriptor
** of this frame.
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
*/
if (rxr->hw_rsc == TRUE) {
rsc = ixgbe_rsc_count(cur);
rxr->rsc_num += (rsc - 1);
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if (rsc) { /* Get hardware index */
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
nextp = ((staterr &
IXGBE_RXDADV_NEXTP_MASK) >>
IXGBE_RXDADV_NEXTP_SHIFT);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
} else { /* Just sequential */
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
nextp = i + 1;
if (nextp == adapter->num_rx_desc)
nextp = 0;
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
nbuf = &rxr->rx_buffers[nextp];
prefetch(nbuf);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
}
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/*
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
** Rather than using the fmp/lmp global pointers
** we now keep the head of a packet chain in the
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
** buffer struct and pass this along from one
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
** descriptor to the next, until we get EOP.
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
*/
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
mp->m_len = len;
/*
** See if there is a stored head
** that determines what we are
*/
sendmp = rbuf->fmp;
if (sendmp != NULL) { /* secondary frag */
rbuf->buf = rbuf->fmp = NULL;
mp->m_flags &= ~M_PKTHDR;
sendmp->m_pkthdr.len += mp->m_len;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
} else {
/*
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
* Optimize. This might be a small packet,
* maybe just a TCP ACK. Do a fast copy that
* is cache aligned into a new mbuf, and
* leave the old mbuf+cluster for re-use.
*/
if (eop && len <= IXGBE_RX_COPY_LEN) {
sendmp = m_gethdr(M_NOWAIT, MT_DATA);
if (sendmp != NULL) {
sendmp->m_data +=
IXGBE_RX_COPY_ALIGN;
ixgbe_bcopy(mp->m_data,
sendmp->m_data, len);
sendmp->m_len = len;
rxr->rx_copies++;
rbuf->flags |= IXGBE_RX_COPY;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
}
}
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
if (sendmp == NULL) {
rbuf->buf = rbuf->fmp = NULL;
sendmp = mp;
}
/* first desc of a non-ps chain */
sendmp->m_flags |= M_PKTHDR;
sendmp->m_pkthdr.len = mp->m_len;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
++processed;
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
/* Pass the head pointer on */
if (eop == 0) {
nbuf->fmp = sendmp;
sendmp = NULL;
mp->m_next = nbuf->buf;
} else { /* Sending this frame */
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
sendmp->m_pkthdr.rcvif = ifp;
ifp->if_ipackets++;
rxr->rx_packets++;
/* capture data for AIM */
rxr->bytes += sendmp->m_pkthdr.len;
rxr->rx_bytes += sendmp->m_pkthdr.len;
First of a series of 11 patches leading to new ixgbe version 2.5.0 This removes the header split and supporting code from the driver.
2012-11-30 22:19:18 +00:00
/* Process vlan info */
if ((rxr->vtag_strip) &&
(staterr & IXGBE_RXD_STAT_VP))
vtag = le16toh(cur->wb.upper.vlan);
if (vtag) {
sendmp->m_pkthdr.ether_vtag = vtag;
sendmp->m_flags |= M_VLANTAG;
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if ((ifp->if_capenable & IFCAP_RXCSUM) != 0)
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
ixgbe_rx_checksum(staterr, sendmp, ptype);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
#if __FreeBSD_version >= 800000
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
sendmp->m_pkthdr.flowid = que->msix;
sendmp->m_flags |= M_FLOWID;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
#endif
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
next_desc:
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
bus_dmamap_sync(rxr->rxdma.dma_tag, rxr->rxdma.dma_map,
BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
/* Advance our pointers to the next descriptor. */
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
if (++i == rxr->num_desc)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
i = 0;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
/* Now send to the stack or do LRO */
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
if (sendmp != NULL) {
rxr->next_to_check = i;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
ixgbe_rx_input(rxr, ifp, sendmp, ptype);
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
i = rxr->next_to_check;
}
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
/* Every 8 descriptors we go to refresh mbufs */
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
if (processed == 8) {
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
ixgbe_refresh_mbufs(rxr, i);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
processed = 0;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
/* Refresh any remaining buf structs */
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
if (ixgbe_rx_unrefreshed(rxr))
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
ixgbe_refresh_mbufs(rxr, i);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
rxr->next_to_check = i;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/*
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
* Flush any outstanding LRO work
*/
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
while ((queued = SLIST_FIRST(&lro->lro_active)) != NULL) {
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
SLIST_REMOVE_HEAD(&lro->lro_active, next);
tcp_lro_flush(lro, queued);
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
IXGBE_RX_UNLOCK(rxr);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/*
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
** We still have cleaning to do?
** Schedule another interrupt if so.
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
*/
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
if ((staterr & IXGBE_RXD_STAT_DD) != 0) {
ixgbe_rearm_queues(adapter, (u64)(1 << que->msix));
return (TRUE);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
return (FALSE);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*********************************************************************
*
* Verify that the hardware indicated that the checksum is valid.
* Inform the stack about the status of checksum so that stack
* doesn't spend time verifying the checksum.
*
*********************************************************************/
static void
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
ixgbe_rx_checksum(u32 staterr, struct mbuf * mp, u32 ptype)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
u16 status = (u16) staterr;
u8 errors = (u8) (staterr >> 24);
bool sctp = FALSE;
if ((ptype & IXGBE_RXDADV_PKTTYPE_ETQF) == 0 &&
(ptype & IXGBE_RXDADV_PKTTYPE_SCTP) != 0)
sctp = TRUE;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (status & IXGBE_RXD_STAT_IPCS) {
if (!(errors & IXGBE_RXD_ERR_IPE)) {
/* IP Checksum Good */
mp->m_pkthdr.csum_flags = CSUM_IP_CHECKED;
mp->m_pkthdr.csum_flags |= CSUM_IP_VALID;
} else
mp->m_pkthdr.csum_flags = 0;
}
if (status & IXGBE_RXD_STAT_L4CS) {
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
u16 type = (CSUM_DATA_VALID | CSUM_PSEUDO_HDR);
#if __FreeBSD_version >= 800000
if (sctp)
type = CSUM_SCTP_VALID;
#endif
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (!(errors & IXGBE_RXD_ERR_TCPE)) {
Thanks to Michael Tuexen for adding SCTP support for 82599, also for finding a one character bug that kept TSO from working. Sometimes with direct attach cables a failure can occur in init, the old method of calling detach was broken, there is no way to return an error to the system from init, so I have changed it to return failure thru the ioctl. And, have fixed the ALTQ code changes of Max Laier, sorry Max :)
2010-03-30 19:09:18 +00:00
mp->m_pkthdr.csum_flags |= type;
if (!sctp)
mp->m_pkthdr.csum_data = htons(0xffff);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
}
return;
}
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
/*
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
** This routine is run via an vlan config EVENT,
** it enables us to use the HW Filter table since
** we can get the vlan id. This just creates the
** entry in the soft version of the VFTA, init will
** repopulate the real table.
*/
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static void
Improvement on the last change, this gives a precise way to tell the one and only interface that a vlan event is for. Thanks to John Baldwin for the patch. Approved by: re
2009-07-24 21:35:52 +00:00
ixgbe_register_vlan(void *arg, struct ifnet *ifp, u16 vtag)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
struct adapter *adapter = ifp->if_softc;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
u16 index, bit;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Improvement on the last change, this gives a precise way to tell the one and only interface that a vlan event is for. Thanks to John Baldwin for the patch. Approved by: re
2009-07-24 21:35:52 +00:00
if (ifp->if_softc != arg) /* Not our event */
This delta fixes two bugs: - When a vlan event occurs a check was not made that the event was actually for the interface, thus resulting in a panic. All three drivers have this vulnerability. Add a check for this condition. - Secondly, there was a duplicate buf_ring free in the em driver resulting in a panic on unload. Remove. Approved by: re
2009-07-24 16:57:49 +00:00
return;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
if ((vtag == 0) || (vtag > 4095)) /* Invalid */
return;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
IXGBE_CORE_LOCK(adapter);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
index = (vtag >> 5) & 0x7F;
bit = vtag & 0x1F;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
adapter->shadow_vfta[index] |= (1 << bit);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
++adapter->num_vlans;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
ixgbe_init_locked(adapter);
IXGBE_CORE_UNLOCK(adapter);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
/*
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
** This routine is run via an vlan
** unconfig EVENT, remove our entry
** in the soft vfta.
*/
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
static void
Improvement on the last change, this gives a precise way to tell the one and only interface that a vlan event is for. Thanks to John Baldwin for the patch. Approved by: re
2009-07-24 21:35:52 +00:00
ixgbe_unregister_vlan(void *arg, struct ifnet *ifp, u16 vtag)
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
{
struct adapter *adapter = ifp->if_softc;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
u16 index, bit;
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
Improvement on the last change, this gives a precise way to tell the one and only interface that a vlan event is for. Thanks to John Baldwin for the patch. Approved by: re
2009-07-24 21:35:52 +00:00
if (ifp->if_softc != arg)
This delta fixes two bugs: - When a vlan event occurs a check was not made that the event was actually for the interface, thus resulting in a panic. All three drivers have this vulnerability. Add a check for this condition. - Secondly, there was a duplicate buf_ring free in the em driver resulting in a panic on unload. Remove. Approved by: re
2009-07-24 16:57:49 +00:00
return;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
if ((vtag == 0) || (vtag > 4095)) /* Invalid */
return;
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
IXGBE_CORE_LOCK(adapter);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
index = (vtag >> 5) & 0x7F;
bit = vtag & 0x1F;
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
adapter->shadow_vfta[index] &= ~(1 << bit);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
--adapter->num_vlans;
/* Re-init to load the changes */
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
ixgbe_init_locked(adapter);
IXGBE_CORE_UNLOCK(adapter);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
}
static void
ixgbe_setup_vlan_hw_support(struct adapter *adapter)
{
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
struct ifnet *ifp = adapter->ifp;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
struct ixgbe_hw *hw = &adapter->hw;
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
struct rx_ring *rxr;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
u32 ctrl;
/*
** We get here thru init_locked, meaning
** a soft reset, this has already cleared
** the VFTA and other state, so if there
** have been no vlan's registered do nothing.
*/
if (adapter->num_vlans == 0)
return;
/*
** A soft reset zero's out the VFTA, so
** we need to repopulate it now.
*/
for (int i = 0; i < IXGBE_VFTA_SIZE; i++)
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
if (adapter->shadow_vfta[i] != 0)
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
IXGBE_WRITE_REG(hw, IXGBE_VFTA(i),
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
adapter->shadow_vfta[i]);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
ctrl = IXGBE_READ_REG(hw, IXGBE_VLNCTRL);
Update ixgbe driver to verion 2.3.6 - This adds a VM SRIOV interface, ixv, it is however transparent to the user, it links with the ixgbe.ko, but when ixgbe is loaded in a virtualized guest with SRIOV configured this will be detected. - Sync shared code to latest - Many bug fixes and improvements, thanks to everyone who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
/* Enable the Filter Table if enabled */
if (ifp->if_capenable & IFCAP_VLAN_HWFILTER) {
ctrl &= ~IXGBE_VLNCTRL_CFIEN;
ctrl |= IXGBE_VLNCTRL_VFE;
}
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
if (hw->mac.type == ixgbe_mac_82598EB)
ctrl |= IXGBE_VLNCTRL_VME;
IXGBE_WRITE_REG(hw, IXGBE_VLNCTRL, ctrl);
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
/* Setup the queues for vlans */
for (int i = 0; i < adapter->num_queues; i++) {
rxr = &adapter->rx_rings[i];
/* On 82599 the VLAN enable is per/queue in RXDCTL */
if (hw->mac.type != ixgbe_mac_82598EB) {
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
ctrl = IXGBE_READ_REG(hw, IXGBE_RXDCTL(i));
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
ctrl |= IXGBE_RXDCTL_VME;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
IXGBE_WRITE_REG(hw, IXGBE_RXDCTL(i), ctrl);
}
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
rxr->vtag_strip = TRUE;
}
This updates the ixgbe driver to Intel internal version 1.4.7 Shared code changes, core driver fixes, vlan event/filter support Also Kip Macy's fix to allow any number of queues, thanks Kip!
2008-07-30 18:15:18 +00:00
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static void
ixgbe_enable_intr(struct adapter *adapter)
{
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
struct ixgbe_hw *hw = &adapter->hw;
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct ix_queue *que = adapter->queues;
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
u32 mask = (IXGBE_EIMS_ENABLE_MASK & ~IXGBE_EIMS_RTX_QUEUE);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* Enable Fan Failure detection */
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
if (hw->device_id == IXGBE_DEV_ID_82598AT)
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
mask |= IXGBE_EIMS_GPI_SDP1;
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
else {
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
mask |= IXGBE_EIMS_ECC;
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
mask |= IXGBE_EIMS_GPI_SDP0;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
mask |= IXGBE_EIMS_GPI_SDP1;
mask |= IXGBE_EIMS_GPI_SDP2;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
#ifdef IXGBE_FDIR
mask |= IXGBE_EIMS_FLOW_DIR;
#endif
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
}
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
IXGBE_WRITE_REG(hw, IXGBE_EIMS, mask);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* With RSS we use auto clear */
if (adapter->msix_mem) {
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
mask = IXGBE_EIMS_ENABLE_MASK;
A couple fixes got clobbered, putting them back.
2010-11-26 23:57:13 +00:00
/* Don't autoclear Link */
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
mask &= ~IXGBE_EIMS_OTHER;
mask &= ~IXGBE_EIMS_LSC;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
IXGBE_WRITE_REG(hw, IXGBE_EIAC, mask);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
/*
A couple fixes got clobbered, putting them back.
2010-11-26 23:57:13 +00:00
** Now enable all queues, this is done separately to
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
** allow for handling the extended (beyond 32) MSIX
** vectors that can be used by 82599
*/
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
for (int i = 0; i < adapter->num_queues; i++, que++)
ixgbe_enable_queue(adapter, que->msix);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_WRITE_FLUSH(hw);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
}
static void
ixgbe_disable_intr(struct adapter *adapter)
{
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
if (adapter->msix_mem)
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EIAC, 0);
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
if (adapter->hw.mac.type == ixgbe_mac_82598EB) {
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EIMC, ~0);
} else {
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EIMC, 0xFFFF0000);
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EIMC_EX(0), ~0);
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EIMC_EX(1), ~0);
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
IXGBE_WRITE_FLUSH(&adapter->hw);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
return;
}
u16
ixgbe_read_pci_cfg(struct ixgbe_hw *hw, u32 reg)
{
u16 value;
value = pci_read_config(((struct ixgbe_osdep *)hw->back)->dev,
reg, 2);
return (value);
}
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
void
ixgbe_write_pci_cfg(struct ixgbe_hw *hw, u32 reg, u16 value)
{
pci_write_config(((struct ixgbe_osdep *)hw->back)->dev,
reg, value, 2);
return;
}
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
/*
** Setup the correct IVAR register for a particular MSIX interrupt
** (yes this is all very magic and confusing :)
** - entry is the register array entry
** - vector is the MSIX vector for this queue
** - type is RX/TX/MISC
*/
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static void
Update for the Intel 10G driver, this adds support for newest hardware, adds multiqueue tx interface, infrastructure cleanup to allow up to 32 MSIX vectors on newer Nehalem systems. Bug fixes, etc.
2009-06-24 18:27:07 +00:00
ixgbe_set_ivar(struct adapter *adapter, u8 entry, u8 vector, s8 type)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
struct ixgbe_hw *hw = &adapter->hw;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
u32 ivar, index;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
vector |= IXGBE_IVAR_ALLOC_VAL;
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
switch (hw->mac.type) {
case ixgbe_mac_82598EB:
if (type == -1)
entry = IXGBE_IVAR_OTHER_CAUSES_INDEX;
else
entry += (type * 64);
index = (entry >> 2) & 0x1F;
ivar = IXGBE_READ_REG(hw, IXGBE_IVAR(index));
ivar &= ~(0xFF << (8 * (entry & 0x3)));
ivar |= (vector << (8 * (entry & 0x3)));
IXGBE_WRITE_REG(&adapter->hw, IXGBE_IVAR(index), ivar);
break;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
case ixgbe_mac_82599EB:
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
case ixgbe_mac_X540:
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
if (type == -1) { /* MISC IVAR */
index = (entry & 1) * 8;
ivar = IXGBE_READ_REG(hw, IXGBE_IVAR_MISC);
ivar &= ~(0xFF << index);
ivar |= (vector << index);
IXGBE_WRITE_REG(hw, IXGBE_IVAR_MISC, ivar);
} else { /* RX/TX IVARS */
index = (16 * (entry & 1)) + (8 * type);
ivar = IXGBE_READ_REG(hw, IXGBE_IVAR(entry >> 1));
ivar &= ~(0xFF << index);
ivar |= (vector << index);
IXGBE_WRITE_REG(hw, IXGBE_IVAR(entry >> 1), ivar);
}
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
default:
break;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
static void
ixgbe_configure_ivars(struct adapter *adapter)
{
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
struct ix_queue *que = adapter->queues;
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
u32 newitr;
if (ixgbe_max_interrupt_rate > 0)
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
newitr = (4000000 / ixgbe_max_interrupt_rate) & 0x0FF8;
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
else
newitr = 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
for (int i = 0; i < adapter->num_queues; i++, que++) {
/* First the RX queue entry */
ixgbe_set_ivar(adapter, i, que->msix, 0);
/* ... and the TX */
ixgbe_set_ivar(adapter, i, que->msix, 1);
/* Set an Initial EITR value */
IXGBE_WRITE_REG(&adapter->hw,
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
IXGBE_EITR(que->msix), newitr);
Update the driver to Intel version 2.1.6 - add some new hardware support for 82599 - Big change to interrupt architecture, it now uses a queue which contains an RX/TX pair as the recipient of the interrupt. This will reduce overall system interrupts/msix usage. - Improved RX mbuf handling: the old get_buf routine is no longer synchronized with rxeof, this allows the elimination of packet discards due to mbuf allocation failure. - Much simplified and improved AIM code, it now happens in the queue interrupt context and takes into account both the traffic on the RX AND TX side. - variety of small tweaks, like ring size, that have been seen as performance improvements. - Thanks to those that provided feedback or suggested changes, I hope I've caught all of them.
2010-03-27 00:21:40 +00:00
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* For the Link interrupt */
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
ixgbe_set_ivar(adapter, 1, adapter->linkvec, -1);
}
/*
** ixgbe_sfp_probe - called in the local timer to
** determine if a port had optics inserted.
*/
static bool ixgbe_sfp_probe(struct adapter *adapter)
{
struct ixgbe_hw *hw = &adapter->hw;
device_t dev = adapter->dev;
bool result = FALSE;
if ((hw->phy.type == ixgbe_phy_nl) &&
(hw->phy.sfp_type == ixgbe_sfp_type_not_present)) {
s32 ret = hw->phy.ops.identify_sfp(hw);
if (ret)
goto out;
ret = hw->phy.ops.reset(hw);
if (ret == IXGBE_ERR_SFP_NOT_SUPPORTED) {
device_printf(dev,"Unsupported SFP+ module detected!");
printf(" Reload driver with supported module.\n");
adapter->sfp_probe = FALSE;
goto out;
} else
device_printf(dev,"SFP+ module detected!\n");
/* We now have supported optics */
adapter->sfp_probe = FALSE;
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
/* Set the optics type so system reports correctly */
ixgbe_setup_optics(adapter);
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
result = TRUE;
}
out:
return (result);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
/*
** Tasklet handler for MSIX Link interrupts
** - do outside interrupt since it might sleep
*/
static void
ixgbe_handle_link(void *context, int pending)
{
struct adapter *adapter = context;
ixgbe_check_link(&adapter->hw,
&adapter->link_speed, &adapter->link_up, 0);
ixgbe_update_link_status(adapter);
}
/*
** Tasklet for handling SFP module interrupts
*/
static void
ixgbe_handle_mod(void *context, int pending)
{
struct adapter *adapter = context;
struct ixgbe_hw *hw = &adapter->hw;
device_t dev = adapter->dev;
u32 err;
err = hw->phy.ops.identify_sfp(hw);
if (err == IXGBE_ERR_SFP_NOT_SUPPORTED) {
device_printf(dev,
"Unsupported SFP+ module type was detected.\n");
return;
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
err = hw->mac.ops.setup_sfp(hw);
if (err == IXGBE_ERR_SFP_NOT_SUPPORTED) {
device_printf(dev,
"Setup failure - unsupported SFP+ module type.\n");
return;
}
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
taskqueue_enqueue(adapter->tq, &adapter->msf_task);
return;
}
/*
** Tasklet for handling MSF (multispeed fiber) interrupts
*/
static void
ixgbe_handle_msf(void *context, int pending)
{
struct adapter *adapter = context;
struct ixgbe_hw *hw = &adapter->hw;
u32 autoneg;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
bool negotiate;
autoneg = hw->phy.autoneg_advertised;
if ((!autoneg) && (hw->mac.ops.get_link_capabilities))
hw->mac.ops.get_link_capabilities(hw, &autoneg, &negotiate);
if (hw->mac.ops.setup_link)
hw->mac.ops.setup_link(hw, autoneg, negotiate, TRUE);
return;
}
#ifdef IXGBE_FDIR
/*
** Tasklet for reinitializing the Flow Director filter table
*/
static void
ixgbe_reinit_fdir(void *context, int pending)
{
struct adapter *adapter = context;
Remove extraneous semicolons, no functional changes. Submitted by: Marc Balmer <marc@msys.ch> MFC after: 1 week
2010-01-07 21:01:37 +00:00
struct ifnet *ifp = adapter->ifp;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
if (adapter->fdir_reinit != 1) /* Shouldn't happen */
return;
ixgbe_reinit_fdir_tables_82599(&adapter->hw);
adapter->fdir_reinit = 0;
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
/* re-enable flow director interrupts */
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EIMS, IXGBE_EIMS_FLOW_DIR);
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* Restart the interface */
ifp->if_drv_flags |= IFF_DRV_RUNNING;
Update the ixgbe driver to 1.7.4, this includes support for the new 82599 adapter family, adds header split, and many small fixes. The driver should now be added to the GENERIC kernel. MFC after: 2 weeks
2009-04-10 00:22:48 +00:00
return;
}
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
#endif
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/**********************************************************************
*
* Update the board statistics counters.
*
**********************************************************************/
static void
ixgbe_update_stats_counters(struct adapter *adapter)
{
Remove extraneous semicolons, no functional changes. Submitted by: Marc Balmer <marc@msys.ch> MFC after: 1 week
2010-01-07 21:01:37 +00:00
struct ifnet *ifp = adapter->ifp;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
struct ixgbe_hw *hw = &adapter->hw;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
u32 missed_rx = 0, bprc, lxon, lxoff, total;
A couple of changes: first, make header split on by default and remove from its dependency on LRO, my tests have shown that its always beneficial, even when doing bridging. Second, fix up a few problems in the statistics code, the adapter dependencies had gotten lost so some code that should only run on 82599 was always running, this resulted in bogus flow control numbers on 82598.
2009-08-24 20:41:51 +00:00
u64 total_missed_rx = 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
adapter->stats.crcerrs += IXGBE_READ_REG(hw, IXGBE_CRCERRS);
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
adapter->stats.illerrc += IXGBE_READ_REG(hw, IXGBE_ILLERRC);
adapter->stats.errbc += IXGBE_READ_REG(hw, IXGBE_ERRBC);
adapter->stats.mspdc += IXGBE_READ_REG(hw, IXGBE_MSPDC);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
for (int i = 0; i < 8; i++) {
Stats missed packet handling was still not quite right, thanks to Dmitrij Tejblum for the correction, need a variable with scope only within the for loop for all queues. MFC: 3 days
2009-09-11 00:00:23 +00:00
u32 mp;
mp = IXGBE_READ_REG(hw, IXGBE_MPC(i));
A couple of changes: first, make header split on by default and remove from its dependency on LRO, my tests have shown that its always beneficial, even when doing bridging. Second, fix up a few problems in the statistics code, the adapter dependencies had gotten lost so some code that should only run on 82599 was always running, this resulted in bogus flow control numbers on 82598.
2009-08-24 20:41:51 +00:00
/* missed_rx tallies misses for the gprc workaround */
Stats missed packet handling was still not quite right, thanks to Dmitrij Tejblum for the correction, need a variable with scope only within the for loop for all queues. MFC: 3 days
2009-09-11 00:00:23 +00:00
missed_rx += mp;
Update driver to Intel version 2.0.7: This adds new feature support for the 82599, a hardware assist to LRO, doing this required a large revamp to the RX cleanup code because the descriptor ring may not be processed out of order, this necessitated the elimination of global pointers. Additionally, the RX routine now does not refresh mbufs on every descriptor, rather it will do a range, and then update the hardware pointer at that time. These are performance oriented changes. The TX side now has a cleaner simpler watchdog algorithm as well, in TX cleanup a read of ticks is stored, that can then be compared in local_timer to determine if there is a hang. Various other cleanups along the way, thanks to all who have provided input and testing.
2009-12-07 21:30:54 +00:00
/* global total per queue */
Stats missed packet handling was still not quite right, thanks to Dmitrij Tejblum for the correction, need a variable with scope only within the for loop for all queues. MFC: 3 days
2009-09-11 00:00:23 +00:00
adapter->stats.mpc[i] += mp;
A couple of changes: first, make header split on by default and remove from its dependency on LRO, my tests have shown that its always beneficial, even when doing bridging. Second, fix up a few problems in the statistics code, the adapter dependencies had gotten lost so some code that should only run on 82599 was always running, this resulted in bogus flow control numbers on 82598.
2009-08-24 20:41:51 +00:00
/* Running comprehensive total for stats display */
total_missed_rx += adapter->stats.mpc[i];
if (hw->mac.type == ixgbe_mac_82598EB)
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
adapter->stats.rnbc[i] +=
IXGBE_READ_REG(hw, IXGBE_RNBC(i));
adapter->stats.pxontxc[i] +=
IXGBE_READ_REG(hw, IXGBE_PXONTXC(i));
adapter->stats.pxonrxc[i] +=
IXGBE_READ_REG(hw, IXGBE_PXONRXC(i));
adapter->stats.pxofftxc[i] +=
IXGBE_READ_REG(hw, IXGBE_PXOFFTXC(i));
adapter->stats.pxoffrxc[i] +=
IXGBE_READ_REG(hw, IXGBE_PXOFFRXC(i));
adapter->stats.pxon2offc[i] +=
IXGBE_READ_REG(hw, IXGBE_PXON2OFFCNT(i));
}
for (int i = 0; i < 16; i++) {
adapter->stats.qprc[i] += IXGBE_READ_REG(hw, IXGBE_QPRC(i));
adapter->stats.qptc[i] += IXGBE_READ_REG(hw, IXGBE_QPTC(i));
adapter->stats.qbrc[i] += IXGBE_READ_REG(hw, IXGBE_QBRC(i));
adapter->stats.qbrc[i] +=
((u64)IXGBE_READ_REG(hw, IXGBE_QBRC(i)) << 32);
adapter->stats.qbtc[i] += IXGBE_READ_REG(hw, IXGBE_QBTC(i));
adapter->stats.qbtc[i] +=
((u64)IXGBE_READ_REG(hw, IXGBE_QBTC(i)) << 32);
adapter->stats.qprdc[i] += IXGBE_READ_REG(hw, IXGBE_QPRDC(i));
}
adapter->stats.mlfc += IXGBE_READ_REG(hw, IXGBE_MLFC);
adapter->stats.mrfc += IXGBE_READ_REG(hw, IXGBE_MRFC);
adapter->stats.rlec += IXGBE_READ_REG(hw, IXGBE_RLEC);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
/* Hardware workaround, gprc counts missed packets */
adapter->stats.gprc += IXGBE_READ_REG(hw, IXGBE_GPRC);
adapter->stats.gprc -= missed_rx;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
if (hw->mac.type != ixgbe_mac_82598EB) {
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
adapter->stats.gorc += IXGBE_READ_REG(hw, IXGBE_GORCL) +
((u64)IXGBE_READ_REG(hw, IXGBE_GORCH) << 32);
adapter->stats.gotc += IXGBE_READ_REG(hw, IXGBE_GOTCL) +
((u64)IXGBE_READ_REG(hw, IXGBE_GOTCH) << 32);
adapter->stats.tor += IXGBE_READ_REG(hw, IXGBE_TORL) +
((u64)IXGBE_READ_REG(hw, IXGBE_TORH) << 32);
A couple of changes: first, make header split on by default and remove from its dependency on LRO, my tests have shown that its always beneficial, even when doing bridging. Second, fix up a few problems in the statistics code, the adapter dependencies had gotten lost so some code that should only run on 82599 was always running, this resulted in bogus flow control numbers on 82598.
2009-08-24 20:41:51 +00:00
adapter->stats.lxonrxc += IXGBE_READ_REG(hw, IXGBE_LXONRXCNT);
adapter->stats.lxoffrxc += IXGBE_READ_REG(hw, IXGBE_LXOFFRXCNT);
} else {
adapter->stats.lxonrxc += IXGBE_READ_REG(hw, IXGBE_LXONRXC);
adapter->stats.lxoffrxc += IXGBE_READ_REG(hw, IXGBE_LXOFFRXC);
/* 82598 only has a counter in the high register */
adapter->stats.gorc += IXGBE_READ_REG(hw, IXGBE_GORCH);
A few minor fixes: - add a moderation value to the Link vector - allow disabling HW RSC on the 82599 if LRO is not enabled. - correct error in the stats code - change optic type on the 82598 DA device Thanks to Andrew Boyer for the changes.
2010-05-14 22:00:37 +00:00
adapter->stats.gotc += IXGBE_READ_REG(hw, IXGBE_GOTCH);
A couple of changes: first, make header split on by default and remove from its dependency on LRO, my tests have shown that its always beneficial, even when doing bridging. Second, fix up a few problems in the statistics code, the adapter dependencies had gotten lost so some code that should only run on 82599 was always running, this resulted in bogus flow control numbers on 82598.
2009-08-24 20:41:51 +00:00
adapter->stats.tor += IXGBE_READ_REG(hw, IXGBE_TORH);
}
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/*
* Workaround: mprc hardware is incorrectly counting
* broadcasts, so for now we subtract those.
*/
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
bprc = IXGBE_READ_REG(hw, IXGBE_BPRC);
adapter->stats.bprc += bprc;
adapter->stats.mprc += IXGBE_READ_REG(hw, IXGBE_MPRC);
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
if (hw->mac.type == ixgbe_mac_82598EB)
adapter->stats.mprc -= bprc;
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
adapter->stats.prc64 += IXGBE_READ_REG(hw, IXGBE_PRC64);
adapter->stats.prc127 += IXGBE_READ_REG(hw, IXGBE_PRC127);
adapter->stats.prc255 += IXGBE_READ_REG(hw, IXGBE_PRC255);
adapter->stats.prc511 += IXGBE_READ_REG(hw, IXGBE_PRC511);
adapter->stats.prc1023 += IXGBE_READ_REG(hw, IXGBE_PRC1023);
adapter->stats.prc1522 += IXGBE_READ_REG(hw, IXGBE_PRC1522);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
lxon = IXGBE_READ_REG(hw, IXGBE_LXONTXC);
adapter->stats.lxontxc += lxon;
lxoff = IXGBE_READ_REG(hw, IXGBE_LXOFFTXC);
adapter->stats.lxofftxc += lxoff;
total = lxon + lxoff;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
adapter->stats.gptc += IXGBE_READ_REG(hw, IXGBE_GPTC);
This is driver version 1.4.4 of the Intel ixgbe driver. -It has new hardware support -It uses a new method of TX cleanup called Head Write Back -It includes the provisional generic TCP LRO feature contributed by Myricom and made general purpose by me. This should move into the stack upon approval but for this driver drop its in here. -Also bug fixes and etc... MFC in a week if no serious issues arise.
2008-05-16 18:46:30 +00:00
adapter->stats.mptc += IXGBE_READ_REG(hw, IXGBE_MPTC);
adapter->stats.ptc64 += IXGBE_READ_REG(hw, IXGBE_PTC64);
adapter->stats.gptc -= total;
adapter->stats.mptc -= total;
adapter->stats.ptc64 -= total;
adapter->stats.gotc -= total * ETHER_MIN_LEN;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
adapter->stats.ruc += IXGBE_READ_REG(hw, IXGBE_RUC);
adapter->stats.rfc += IXGBE_READ_REG(hw, IXGBE_RFC);
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
adapter->stats.roc += IXGBE_READ_REG(hw, IXGBE_ROC);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
adapter->stats.rjc += IXGBE_READ_REG(hw, IXGBE_RJC);
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
adapter->stats.mngprc += IXGBE_READ_REG(hw, IXGBE_MNGPRC);
adapter->stats.mngpdc += IXGBE_READ_REG(hw, IXGBE_MNGPDC);
adapter->stats.mngptc += IXGBE_READ_REG(hw, IXGBE_MNGPTC);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
adapter->stats.tpr += IXGBE_READ_REG(hw, IXGBE_TPR);
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
adapter->stats.tpt += IXGBE_READ_REG(hw, IXGBE_TPT);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
adapter->stats.ptc127 += IXGBE_READ_REG(hw, IXGBE_PTC127);
adapter->stats.ptc255 += IXGBE_READ_REG(hw, IXGBE_PTC255);
adapter->stats.ptc511 += IXGBE_READ_REG(hw, IXGBE_PTC511);
adapter->stats.ptc1023 += IXGBE_READ_REG(hw, IXGBE_PTC1023);
adapter->stats.ptc1522 += IXGBE_READ_REG(hw, IXGBE_PTC1522);
adapter->stats.bptc += IXGBE_READ_REG(hw, IXGBE_BPTC);
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
adapter->stats.xec += IXGBE_READ_REG(hw, IXGBE_XEC);
adapter->stats.fccrc += IXGBE_READ_REG(hw, IXGBE_FCCRC);
adapter->stats.fclast += IXGBE_READ_REG(hw, IXGBE_FCLAST);
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
/* Only read FCOE on 82599 */
- Add the RX refresh changes from igb to ixgbe - Also a couple minor tweaks to the TX code from the same source. - Add the INET ioctl code which has been missing from this driver, and which caused IP aliases to reset the interface. - Last, some minor logic changes that just reflect upcoming hardware support, but have no other functional effect now. MFC after a week
2011-04-25 23:34:21 +00:00
if (hw->mac.type != ixgbe_mac_82598EB) {
Update driver to version 2.3.8: CRITICAL FIX - with stats changes the older 82598 will panic and trash the stack on driver load, FCOE registers ONLY exist in 82599 and must not be read otherwise. kern/153951 - to correct incorrect media type on adapters with pluggable modules I have eliminated the old static table in favor of a new dynamic shared code routine. This also has the benefit of detecting changes when a different module is inserted. Performance/enhancement to the Flow Director code from my linux coworker (the developer of the code). Fixes from Michael Tuexen - a data corruption problem on the 82599 (CRITICAL), fix so the buf size correctly adjusts as the cluster changes, and max descriptors are set properly. Also added 16K clusters for those REALLY big jumbos :) In the RX path, the RX LOCK was not being released, and this causes LOR problems. Add the code that igb already has. Sync with in house shared code, this was necessary for the Flow Director fix. MFC in 2 days
2011-01-19 19:36:27 +00:00
adapter->stats.fcoerpdc += IXGBE_READ_REG(hw, IXGBE_FCOERPDC);
adapter->stats.fcoeprc += IXGBE_READ_REG(hw, IXGBE_FCOEPRC);
adapter->stats.fcoeptc += IXGBE_READ_REG(hw, IXGBE_FCOEPTC);
adapter->stats.fcoedwrc += IXGBE_READ_REG(hw, IXGBE_FCOEDWRC);
adapter->stats.fcoedwtc += IXGBE_READ_REG(hw, IXGBE_FCOEDWTC);
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
/* Fill out the OS statistics structure */
ifp->if_ipackets = adapter->stats.gprc;
ifp->if_opackets = adapter->stats.gptc;
ifp->if_ibytes = adapter->stats.gorc;
ifp->if_obytes = adapter->stats.gotc;
ifp->if_imcasts = adapter->stats.mprc;
The drbr(9) API appeared to be so unclear, that most drivers in tree used it incorrectly, which lead to inaccurate overrated if_obytes accounting. The drbr(9) used to update ifnet stats on drbr_enqueue(), which is not accurate since enqueuing doesn't imply successful processing by driver. Dequeuing neither mean that. Most drivers also called drbr_stats_update() which did accounting again, leading to doubled if_obytes statistics. And in case of severe transmitting, when a packet could be several times enqueued and dequeued it could have been accounted several times. o Thus, make drbr(9) API thinner. Now drbr(9) merely chooses between ALTQ queueing or buf_ring(9) queueing. - It doesn't touch the buf_ring stats any more. - It doesn't touch ifnet stats anymore. - drbr_stats_update() no longer exists. o buf_ring(9) handles its stats itself: - It handles br_drops itself. - br_prod_bytes stats are dropped. Rationale: no one ever reads them but update of a common counter on every packet negatively affects performance due to excessive cache invalidation. - buf_ring_enqueue_bytes() reduced to buf_ring_enqueue(), since we no longer account bytes. o Drivers handle their stats theirselves: if_obytes, if_omcasts. o mlx4(4), igb(4), em(4), vxge(4), oce(4) and ixv(4) no longer use drbr_stats_update(), and update ifnet stats theirselves. o bxe(4) was the most correct driver, it didn't call drbr_stats_update(), thus it was the only driver accurate under moderate load. Now it also maintains stats itself. o ixgbe(4) had already taken stats from hardware, so just - drop software stats updating. - take multicast packet count from hardware as well. o mxge(4) just no longer needs NO_SLOW_STATS define. o cxgb(4), cxgbe(4) need no change, since they obtain stats from hardware. Reviewed by: jfv, gnn
2012-09-28 18:28:27 +00:00
ifp->if_omcasts = adapter->stats.mptc;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
ifp->if_collisions = 0;
/* Rx Errors */
A couple of changes: first, make header split on by default and remove from its dependency on LRO, my tests have shown that its always beneficial, even when doing bridging. Second, fix up a few problems in the statistics code, the adapter dependencies had gotten lost so some code that should only run on 82599 was always running, this resulted in bogus flow control numbers on 82598.
2009-08-24 20:41:51 +00:00
ifp->if_ierrors = total_missed_rx + adapter->stats.crcerrs +
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
adapter->stats.rlec;
}
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
/** ixgbe_sysctl_tdh_handler - Handler function
* Retrieves the TDH value from the hardware
*/
static int
ixgbe_sysctl_tdh_handler(SYSCTL_HANDLER_ARGS)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
int error;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
struct tx_ring *txr = ((struct tx_ring *)oidp->oid_arg1);
if (!txr) return 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
unsigned val = IXGBE_READ_REG(&txr->adapter->hw, IXGBE_TDH(txr->me));
error = sysctl_handle_int(oidp, &val, 0, req);
if (error || !req->newptr)
return error;
return 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
/** ixgbe_sysctl_tdt_handler - Handler function
* Retrieves the TDT value from the hardware
*/
static int
ixgbe_sysctl_tdt_handler(SYSCTL_HANDLER_ARGS)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
int error;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
struct tx_ring *txr = ((struct tx_ring *)oidp->oid_arg1);
if (!txr) return 0;
unsigned val = IXGBE_READ_REG(&txr->adapter->hw, IXGBE_TDT(txr->me));
error = sysctl_handle_int(oidp, &val, 0, req);
if (error || !req->newptr)
return error;
return 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
/** ixgbe_sysctl_rdh_handler - Handler function
* Retrieves the RDH value from the hardware
*/
static int
ixgbe_sysctl_rdh_handler(SYSCTL_HANDLER_ARGS)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
int error;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
struct rx_ring *rxr = ((struct rx_ring *)oidp->oid_arg1);
if (!rxr) return 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
unsigned val = IXGBE_READ_REG(&rxr->adapter->hw, IXGBE_RDH(rxr->me));
error = sysctl_handle_int(oidp, &val, 0, req);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (error || !req->newptr)
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
return error;
return 0;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
/** ixgbe_sysctl_rdt_handler - Handler function
* Retrieves the RDT value from the hardware
*/
static int
ixgbe_sysctl_rdt_handler(SYSCTL_HANDLER_ARGS)
{
int error;
struct rx_ring *rxr = ((struct rx_ring *)oidp->oid_arg1);
if (!rxr) return 0;
unsigned val = IXGBE_READ_REG(&rxr->adapter->hw, IXGBE_RDT(rxr->me));
error = sysctl_handle_int(oidp, &val, 0, req);
if (error || !req->newptr)
return error;
return 0;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
static int
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
ixgbe_sysctl_interrupt_rate_handler(SYSCTL_HANDLER_ARGS)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
int error;
struct ix_queue *que = ((struct ix_queue *)oidp->oid_arg1);
unsigned int reg, usec, rate;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
reg = IXGBE_READ_REG(&que->adapter->hw, IXGBE_EITR(que->msix));
usec = ((reg & 0x0FF8) >> 3);
if (usec > 0)
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
rate = 500000 / usec;
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
else
rate = 0;
error = sysctl_handle_int(oidp, &rate, 0, req);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
if (error || !req->newptr)
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
return error;
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
reg &= ~0xfff; /* default, no limitation */
ixgbe_max_interrupt_rate = 0;
if (rate > 0 && rate < 500000) {
if (rate < 1000)
rate = 1000;
ixgbe_max_interrupt_rate = rate;
reg |= ((4000000/rate) & 0xff8 );
}
IXGBE_WRITE_REG(&que->adapter->hw, IXGBE_EITR(que->msix), reg);
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
return 0;
}
/*
* Add sysctl variables, one per statistic, to the system.
*/
static void
ixgbe_add_hw_stats(struct adapter *adapter)
{
device_t dev = adapter->dev;
struct tx_ring *txr = adapter->tx_rings;
struct rx_ring *rxr = adapter->rx_rings;
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
struct sysctl_ctx_list *ctx = device_get_sysctl_ctx(dev);
struct sysctl_oid *tree = device_get_sysctl_tree(dev);
struct sysctl_oid_list *child = SYSCTL_CHILDREN(tree);
struct ixgbe_hw_stats *stats = &adapter->stats;
struct sysctl_oid *stat_node, *queue_node;
struct sysctl_oid_list *stat_list, *queue_list;
#define QUEUE_NAME_LEN 32
char namebuf[QUEUE_NAME_LEN];
/* Driver Statistics */
SYSCTL_ADD_ULONG(ctx, child, OID_AUTO, "dropped",
CTLFLAG_RD, &adapter->dropped_pkts,
"Driver dropped packets");
SYSCTL_ADD_ULONG(ctx, child, OID_AUTO, "mbuf_defrag_failed",
CTLFLAG_RD, &adapter->mbuf_defrag_failed,
"m_defrag() failed");
SYSCTL_ADD_ULONG(ctx, child, OID_AUTO, "watchdog_events",
CTLFLAG_RD, &adapter->watchdog_events,
"Watchdog timeouts");
SYSCTL_ADD_ULONG(ctx, child, OID_AUTO, "link_irq",
CTLFLAG_RD, &adapter->link_irq,
"Link MSIX IRQ Handled");
for (int i = 0; i < adapter->num_queues; i++, txr++) {
snprintf(namebuf, QUEUE_NAME_LEN, "queue%d", i);
queue_node = SYSCTL_ADD_NODE(ctx, child, OID_AUTO, namebuf,
CTLFLAG_RD, NULL, "Queue Name");
queue_list = SYSCTL_CHILDREN(queue_node);
SYSCTL_ADD_PROC(ctx, queue_list, OID_AUTO, "interrupt_rate",
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
CTLTYPE_UINT | CTLFLAG_RW, &adapter->queues[i],
Specify a CTLTYPE_FOO so that a future sysctl(8) change does not need to rely on the format string.
2011-01-18 21:14:23 +00:00
sizeof(&adapter->queues[i]),
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
ixgbe_sysctl_interrupt_rate_handler, "IU",
"Interrupt Rate");
ixgbe changes: - remove experimental code for disabling CRC - use the correct constant for conversion between interrupt rate and EITR values (the previous values were off by a factor of 2) - make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing individual values affects the queue immediately, and propagates to all interfaces at the next reinit. - add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt counts Netmap-related changes for ixgbe: - use the "new" format for TX descriptors in netmap mode. - pass interrupt mitigation delays to the user process doing poll() on a netmap file descriptor. On the RX side this means we will not check the ring more than once per interrupt. This gives the process a chance to sleep and process packets in larger batches, thus reducing CPU usage. On the TX side we take this even further: completed transmissions are reclaimed every half ring even if the NIC interrupts more often. This saves even more CPU without any additional tx delays. Generic Netmap-related changes: - align the netmap_kring to cache lines so that there is no false sharing (possibly useful for multiqueue NICs and MSIX interrupts, which are handled by different cores). It's a minor improvement but it does not cost anything. Reviewed by: Jack Vogel Approved by: Jack Vogel
2012-01-26 09:55:16 +00:00
SYSCTL_ADD_UQUAD(ctx, queue_list, OID_AUTO, "irqs",
CTLFLAG_RD, &(adapter->queues[i].irqs),
"irqs on this queue");
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
SYSCTL_ADD_PROC(ctx, queue_list, OID_AUTO, "txd_head",
Specify a CTLTYPE_FOO so that a future sysctl(8) change does not need to rely on the format string.
2011-01-18 21:14:23 +00:00
CTLTYPE_UINT | CTLFLAG_RD, txr, sizeof(txr),
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
ixgbe_sysctl_tdh_handler, "IU",
"Transmit Descriptor Head");
SYSCTL_ADD_PROC(ctx, queue_list, OID_AUTO, "txd_tail",
Specify a CTLTYPE_FOO so that a future sysctl(8) change does not need to rely on the format string.
2011-01-18 21:14:23 +00:00
CTLTYPE_UINT | CTLFLAG_RD, txr, sizeof(txr),
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
ixgbe_sysctl_tdt_handler, "IU",
"Transmit Descriptor Tail");
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
SYSCTL_ADD_ULONG(ctx, queue_list, OID_AUTO, "tso_tx",
CTLFLAG_RD, &txr->tso_tx,
"TSO");
SYSCTL_ADD_ULONG(ctx, queue_list, OID_AUTO, "no_tx_dma_setup",
CTLFLAG_RD, &txr->no_tx_dma_setup,
"Driver tx dma failure in xmit");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, queue_list, OID_AUTO, "no_desc_avail",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &txr->no_desc_avail,
"Queue No Descriptor Available");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, queue_list, OID_AUTO, "tx_packets",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &txr->total_packets,
"Queue Packets Transmitted");
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
for (int i = 0; i < adapter->num_queues; i++, rxr++) {
snprintf(namebuf, QUEUE_NAME_LEN, "queue%d", i);
queue_node = SYSCTL_ADD_NODE(ctx, child, OID_AUTO, namebuf,
CTLFLAG_RD, NULL, "Queue Name");
queue_list = SYSCTL_CHILDREN(queue_node);
struct lro_ctrl *lro = &rxr->lro;
snprintf(namebuf, QUEUE_NAME_LEN, "queue%d", i);
queue_node = SYSCTL_ADD_NODE(ctx, child, OID_AUTO, namebuf,
CTLFLAG_RD, NULL, "Queue Name");
queue_list = SYSCTL_CHILDREN(queue_node);
SYSCTL_ADD_PROC(ctx, queue_list, OID_AUTO, "rxd_head",
Specify a CTLTYPE_FOO so that a future sysctl(8) change does not need to rely on the format string.
2011-01-18 21:14:23 +00:00
CTLTYPE_UINT | CTLFLAG_RD, rxr, sizeof(rxr),
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
ixgbe_sysctl_rdh_handler, "IU",
"Receive Descriptor Head");
SYSCTL_ADD_PROC(ctx, queue_list, OID_AUTO, "rxd_tail",
Specify a CTLTYPE_FOO so that a future sysctl(8) change does not need to rely on the format string.
2011-01-18 21:14:23 +00:00
CTLTYPE_UINT | CTLFLAG_RD, rxr, sizeof(rxr),
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
ixgbe_sysctl_rdt_handler, "IU",
"Receive Descriptor Tail");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, queue_list, OID_AUTO, "rx_packets",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &rxr->rx_packets,
"Queue Packets Received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, queue_list, OID_AUTO, "rx_bytes",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &rxr->rx_bytes,
"Queue Bytes Received");
Heavily optimize the case of small RX packets of 160 bytes or less. For this case, allocate a plain mbuf and copy the frame into it, then send the copy up the stack, leaving the original mbuf+cluster in place in the receive ring for immediate re-use. This saves a trip through 2 of the 3 zones of the compound mbuf allocator, a trip through busdma, and a trip through the 1 of the 3 mbuf destructors. For our load at Netflix, this can lower CPU consumption by as much as 20%. The copy algorithm is based on investigative work from Luigi Rizzo earlier in the year. Reviewed by: jfv Obtained from: Netflix
2012-08-31 10:07:38 +00:00
SYSCTL_ADD_UQUAD(ctx, queue_list, OID_AUTO, "rx_copies",
CTLFLAG_RD, &rxr->rx_copies,
"Copied RX Frames");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_INT(ctx, queue_list, OID_AUTO, "lro_queued",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &lro->lro_queued, 0,
"LRO Queued");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_INT(ctx, queue_list, OID_AUTO, "lro_flushed",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &lro->lro_flushed, 0,
"LRO Flushed");
}
/* MAC stats get the own sub node */
stat_node = SYSCTL_ADD_NODE(ctx, child, OID_AUTO, "mac_stats",
CTLFLAG_RD, NULL, "MAC Statistics");
stat_list = SYSCTL_CHILDREN(stat_node);
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "crc_errs",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->crcerrs,
"CRC Errors");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "ill_errs",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->illerrc,
"Illegal Byte Errors");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "byte_errs",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->errbc,
"Byte Errors");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "short_discards",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->mspdc,
"MAC Short Packets Discarded");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "local_faults",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->mlfc,
"MAC Local Faults");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "remote_faults",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->mrfc,
"MAC Remote Faults");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "rec_len_errs",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->rlec,
"Receive Length Errors");
Patch #4 - this does two things, it removes a number of statistics, these are FCOE stats (fiber channel over ethernet), something that FreeBSD does not yet have, they were mistaken for flow control by the implementor I believe. Secondly, the real flow control stats are oddly named with a 'link' tag on the front, it was requested by my validation engineer to make these stats have the same name as the igb driver for clarity and that seemed reasonable to me.
2012-11-30 22:54:14 +00:00
/* Flow Control stats */
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "xon_txd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->lxontxc,
"Link XON Transmitted");
Patch #4 - this does two things, it removes a number of statistics, these are FCOE stats (fiber channel over ethernet), something that FreeBSD does not yet have, they were mistaken for flow control by the implementor I believe. Secondly, the real flow control stats are oddly named with a 'link' tag on the front, it was requested by my validation engineer to make these stats have the same name as the igb driver for clarity and that seemed reasonable to me.
2012-11-30 22:54:14 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "xon_recvd",
kern/153772 fix variable names. Thank you Andrew Boyer for catching these MFC in 3 days
2011-01-07 22:34:56 +00:00
CTLFLAG_RD, &stats->lxonrxc,
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
"Link XON Received");
Patch #4 - this does two things, it removes a number of statistics, these are FCOE stats (fiber channel over ethernet), something that FreeBSD does not yet have, they were mistaken for flow control by the implementor I believe. Secondly, the real flow control stats are oddly named with a 'link' tag on the front, it was requested by my validation engineer to make these stats have the same name as the igb driver for clarity and that seemed reasonable to me.
2012-11-30 22:54:14 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "xoff_txd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->lxofftxc,
"Link XOFF Transmitted");
Patch #4 - this does two things, it removes a number of statistics, these are FCOE stats (fiber channel over ethernet), something that FreeBSD does not yet have, they were mistaken for flow control by the implementor I believe. Secondly, the real flow control stats are oddly named with a 'link' tag on the front, it was requested by my validation engineer to make these stats have the same name as the igb driver for clarity and that seemed reasonable to me.
2012-11-30 22:54:14 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "xoff_recvd",
kern/153772 fix variable names. Thank you Andrew Boyer for catching these MFC in 3 days
2011-01-07 22:34:56 +00:00
CTLFLAG_RD, &stats->lxoffrxc,
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
"Link XOFF Received");
/* Packet Reception Stats */
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "total_octets_rcvd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->tor,
"Total Octets Received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "good_octets_rcvd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->gorc,
"Good Octets Received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "total_pkts_rcvd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->tpr,
"Total Packets Received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "good_pkts_rcvd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->gprc,
"Good Packets Received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "mcast_pkts_rcvd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->mprc,
"Multicast Packets Received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "bcast_pkts_rcvd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->bprc,
"Broadcast Packets Received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "rx_frames_64",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->prc64,
"64 byte frames received ");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "rx_frames_65_127",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->prc127,
"65-127 byte frames received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "rx_frames_128_255",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->prc255,
"128-255 byte frames received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "rx_frames_256_511",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->prc511,
"256-511 byte frames received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "rx_frames_512_1023",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->prc1023,
"512-1023 byte frames received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "rx_frames_1024_1522",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->prc1522,
"1023-1522 byte frames received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "recv_undersized",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->ruc,
"Receive Undersized");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "recv_fragmented",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->rfc,
"Fragmented Packets Received ");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "recv_oversized",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->roc,
"Oversized Packets Received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "recv_jabberd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->rjc,
"Received Jabber");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "management_pkts_rcvd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->mngprc,
"Management Packets Received");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "management_pkts_drpd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->mngptc,
"Management Packets Dropped");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "checksum_errs",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->xec,
"Checksum Errors");
/* Packet Transmission Stats */
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "good_octets_txd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->gotc,
"Good Octets Transmitted");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "total_pkts_txd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->tpt,
"Total Packets Transmitted");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "good_pkts_txd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->gptc,
"Good Packets Transmitted");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "bcast_pkts_txd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->bptc,
"Broadcast Packets Transmitted");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "mcast_pkts_txd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->mptc,
"Multicast Packets Transmitted");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "management_pkts_txd",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->mngptc,
"Management Packets Transmitted");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "tx_frames_64",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->ptc64,
"64 byte frames transmitted ");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "tx_frames_65_127",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->ptc127,
"65-127 byte frames transmitted");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "tx_frames_128_255",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->ptc255,
"128-255 byte frames transmitted");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "tx_frames_256_511",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->ptc511,
"256-511 byte frames transmitted");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "tx_frames_512_1023",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->ptc1023,
"512-1023 byte frames transmitted");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
SYSCTL_ADD_UQUAD(ctx, stat_list, OID_AUTO, "tx_frames_1024_1522",
Interrupt handler, and stats changes from Michael Tuexen, thanks Michael!
2010-11-27 01:34:09 +00:00
CTLFLAG_RD, &stats->ptc1522,
"1024-1522 byte frames transmitted");
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
/*
** Set flow control using sysctl:
** Flow control values:
** 0 - off
** 1 - rx pause
** 2 - tx pause
** 3 - full
*/
static int
ixgbe_set_flowcntl(SYSCTL_HANDLER_ARGS)
{
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
int error, last;
struct adapter *adapter = (struct adapter *) arg1;
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
last = adapter->fc;
error = sysctl_handle_int(oidp, &adapter->fc, 0, req);
if ((error) || (req->newptr == NULL))
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
return (error);
Don't bother to run the flowcontrol code if there is no change. Thanks to Andrew for the tweak.
2011-01-22 00:19:15 +00:00
/* Don't bother if it's not changed */
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
if (adapter->fc == last)
Don't bother to run the flowcontrol code if there is no change. Thanks to Andrew for the tweak.
2011-01-22 00:19:15 +00:00
return (0);
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
switch (adapter->fc) {
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
case ixgbe_fc_rx_pause:
case ixgbe_fc_tx_pause:
case ixgbe_fc_full:
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
adapter->hw.fc.requested_mode = adapter->fc;
A few important fixes: - Testing TSO6 has led me to discover that HW RSC is a problematic feature, it is ONLY designed to work with IPv4 in the first place, and if IP forwarding is done it can't be disabled as LRO in the stack, also initial testing we've done at Intel shows an equal performance using TSO[46] on the TX and LRO on RX, if you ran older code on 82599 or later hardware you actually could have detrimental performance for this reason. So I am disabling the feature by default and all our adapters will now use LRO instead. - If you have flow control off and multiple queues it was possible when the buffer of one queue becomes full that all RX movement is stalled, to eliminate this problem a feature bit is now set that will allow packets to be dropped when full rather than stall. Note, the default is to have flow control on, and this keeps this from happening. - Because of the recent fixes in the stack, LRO is now auto-disabled when problematic, so I have decided to enable it by default in the capabilities in the driver. - There are some 1G modules used by some customers, a couple small tweaks to properly support those in the media code. - A note: we have now done some testing of TSO6 and using LRO with IPv6 and it all works great!! Seeing line rate in both directions in best cases. Thanks bz for your excellent work!!
2012-10-31 23:50:36 +00:00
if (adapter->num_queues > 1)
ixgbe_disable_rx_drop(adapter);
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
break;
case ixgbe_fc_none:
default:
Updated ixgbe driver - version 1.6.2 -This version has header split, and as a result a number of aspects of the code have been improved/simplified. - Interrupt handling refined for performance - Many small bugs fixed along the way MFC after: ASAP - in time for 7.1
2008-11-26 23:41:18 +00:00
adapter->hw.fc.requested_mode = ixgbe_fc_none;
A few important fixes: - Testing TSO6 has led me to discover that HW RSC is a problematic feature, it is ONLY designed to work with IPv4 in the first place, and if IP forwarding is done it can't be disabled as LRO in the stack, also initial testing we've done at Intel shows an equal performance using TSO[46] on the TX and LRO on RX, if you ran older code on 82599 or later hardware you actually could have detrimental performance for this reason. So I am disabling the feature by default and all our adapters will now use LRO instead. - If you have flow control off and multiple queues it was possible when the buffer of one queue becomes full that all RX movement is stalled, to eliminate this problem a feature bit is now set that will allow packets to be dropped when full rather than stall. Note, the default is to have flow control on, and this keeps this from happening. - Because of the recent fixes in the stack, LRO is now auto-disabled when problematic, so I have decided to enable it by default in the capabilities in the driver. - There are some 1G modules used by some customers, a couple small tweaks to properly support those in the media code. - A note: we have now done some testing of TSO6 and using LRO with IPv6 and it all works great!! Seeing line rate in both directions in best cases. Thanks bz for your excellent work!!
2012-10-31 23:50:36 +00:00
if (adapter->num_queues > 1)
ixgbe_enable_rx_drop(adapter);
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
}
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
/* Don't autoneg if forcing a value */
adapter->hw.fc.disable_fc_autoneg = TRUE;
ixgbe_fc_enable(&adapter->hw);
This is an update to the new Intel 10G 82598 driver. The first drop was Beta, this code is expected to be the release version. Note that this driver code will build in either 6.2 or 7. If you use the code in 6.2 you will not get TSO or MSI/X support but it will function in a legacy mode. Approved by: re
2007-09-04 02:31:35 +00:00
return error;
}
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
static void
Patch #7 This is primarily about processing limit control. - add a limit for both RX and TX, change the default to 256 - change the sysctl usage to be common, and now to be called during init for each ring. - the TX limit is not yet used, but the changes in the last patch in this series uses the value. - the motivation behind these changes is to improve data locality in the final code. - rxeof interface changes since it now gets limit from the ring struct
2012-11-30 23:28:01 +00:00
ixgbe_add_process_limit(struct adapter *adapter, const char *name,
Patch #8 Performance changes - this one improves locality, moving some counters and data to the ring struct from the adapter struct, also compressing some data in the move.
2012-11-30 23:45:55 +00:00
const char *description, u16 *limit, u16 value)
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
{
*limit = value;
Patch #7 This is primarily about processing limit control. - add a limit for both RX and TX, change the default to 256 - change the sysctl usage to be common, and now to be called during init for each ring. - the TX limit is not yet used, but the changes in the last patch in this series uses the value. - the motivation behind these changes is to improve data locality in the final code. - rxeof interface changes since it now gets limit from the ring struct
2012-11-30 23:28:01 +00:00
SYSCTL_ADD_UINT(device_get_sysctl_ctx(adapter->dev),
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
SYSCTL_CHILDREN(device_get_sysctl_tree(adapter->dev)),
Patch #7 This is primarily about processing limit control. - add a limit for both RX and TX, change the default to 256 - change the sysctl usage to be common, and now to be called during init for each ring. - the TX limit is not yet used, but the changes in the last patch in this series uses the value. - the motivation behind these changes is to improve data locality in the final code. - rxeof interface changes since it now gets limit from the ring struct
2012-11-30 23:28:01 +00:00
OID_AUTO, name, CTLTYPE_UINT|CTLFLAG_RW, limit, value, description);
Add the actual source too :) Approved by: re
2007-07-11 23:03:16 +00:00
}
BAH, I apologize, the wrong version of the code got fat fingered in place, this is the correct version that actually works... <sheepish grin> MFC: in a week
2010-06-30 01:10:08 +00:00
/*
** Control link advertise speed:
** 1 - advertise only 1G
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
** 2 - advertise 100Mb
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
** 3 - advertise normal
BAH, I apologize, the wrong version of the code got fat fingered in place, this is the correct version that actually works... <sheepish grin> MFC: in a week
2010-06-30 01:10:08 +00:00
*/
static int
ixgbe_set_advertise(SYSCTL_HANDLER_ARGS)
{
First off: update the driver README, the old one was horribly crusty, and this still isn't perfect, but its at least a bit more recent. Secondly, a few improvements to the driver from Andrew Boyer, support hint to allow devices to not attach, add VLAN_HWTSO capability so vlans can use TSO, fix in the interrupt handler to make sure the stack TX queue is processed. Oh, and also make sure IPv6 does not cause a re-init in the ioctl routine. Thanks for your efforts Andrew! Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine was not correctly swapping the dma map from the first to the last descriptor in a multi-descriptor transmission, corrected this.
2011-06-02 00:34:57 +00:00
int error = 0;
BAH, I apologize, the wrong version of the code got fat fingered in place, this is the correct version that actually works... <sheepish grin> MFC: in a week
2010-06-30 01:10:08 +00:00
struct adapter *adapter;
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
device_t dev;
BAH, I apologize, the wrong version of the code got fat fingered in place, this is the correct version that actually works... <sheepish grin> MFC: in a week
2010-06-30 01:10:08 +00:00
struct ixgbe_hw *hw;
ixgbe_link_speed speed, last;
adapter = (struct adapter *) arg1;
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
dev = adapter->dev;
BAH, I apologize, the wrong version of the code got fat fingered in place, this is the correct version that actually works... <sheepish grin> MFC: in a week
2010-06-30 01:10:08 +00:00
hw = &adapter->hw;
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
last = adapter->advertise;
BAH, I apologize, the wrong version of the code got fat fingered in place, this is the correct version that actually works... <sheepish grin> MFC: in a week
2010-06-30 01:10:08 +00:00
error = sysctl_handle_int(oidp, &adapter->advertise, 0, req);
if ((error) || (adapter->advertise == -1))
return (error);
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
if (adapter->advertise == last) /* no change */
return (0);
BAH, I apologize, the wrong version of the code got fat fingered in place, this is the correct version that actually works... <sheepish grin> MFC: in a week
2010-06-30 01:10:08 +00:00
if (!((hw->phy.media_type == ixgbe_media_type_copper) ||
(hw->phy.multispeed_fiber)))
return (error);
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
if ((adapter->advertise == 2) && (hw->mac.type != ixgbe_mac_X540)) {
device_printf(dev, "Set Advertise: 100Mb on X540 only\n");
return (error);
}
BAH, I apologize, the wrong version of the code got fat fingered in place, this is the correct version that actually works... <sheepish grin> MFC: in a week
2010-06-30 01:10:08 +00:00
if (adapter->advertise == 1)
speed = IXGBE_LINK_SPEED_1GB_FULL;
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
else if (adapter->advertise == 2)
speed = IXGBE_LINK_SPEED_100_FULL;
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
else if (adapter->advertise == 3)
BAH, I apologize, the wrong version of the code got fat fingered in place, this is the correct version that actually works... <sheepish grin> MFC: in a week
2010-06-30 01:10:08 +00:00
speed = IXGBE_LINK_SPEED_1GB_FULL |
IXGBE_LINK_SPEED_10GB_FULL;
Update to the ixgbe driver: - Add a couple of new devices - Flow control changes in shared and core code - Bug fix to Flow Director for 82598 - Shared code sync to internal with required core change Thanks to those helping in the testing and improvements to this driver! MFC after:5 days
2012-07-05 20:51:44 +00:00
else /* bogus value */
BAH, I apologize, the wrong version of the code got fat fingered in place, this is the correct version that actually works... <sheepish grin> MFC: in a week
2010-06-30 01:10:08 +00:00
return (error);
hw->mac.autotry_restart = TRUE;
hw->mac.ops.setup_link(hw, speed, TRUE, TRUE);
return (error);
}
New hardware support: Intel X540 adapter support added. Some shared code reorganization along with the new adapter. Sync changes to OACTIVE in igb into this driver. Misc small fixes.
2012-01-30 16:42:02 +00:00
/*
** Thermal Shutdown Trigger
** - cause a Thermal Overtemp IRQ
*/
static int
ixgbe_set_thermal_test(SYSCTL_HANDLER_ARGS)
{
int error, fire = 0;
struct adapter *adapter = (struct adapter *) arg1;
struct ixgbe_hw *hw = &adapter->hw;
if (hw->mac.type != ixgbe_mac_X540)
return (0);
error = sysctl_handle_int(oidp, &fire, 0, req);
if ((error) || (req->newptr == NULL))
return (error);
if (fire) {
u32 reg = IXGBE_READ_REG(hw, IXGBE_EICS);
reg |= IXGBE_EICR_TS;
IXGBE_WRITE_REG(hw, IXGBE_EICS, reg);
}
return (0);
}
A few important fixes: - Testing TSO6 has led me to discover that HW RSC is a problematic feature, it is ONLY designed to work with IPv4 in the first place, and if IP forwarding is done it can't be disabled as LRO in the stack, also initial testing we've done at Intel shows an equal performance using TSO[46] on the TX and LRO on RX, if you ran older code on 82599 or later hardware you actually could have detrimental performance for this reason. So I am disabling the feature by default and all our adapters will now use LRO instead. - If you have flow control off and multiple queues it was possible when the buffer of one queue becomes full that all RX movement is stalled, to eliminate this problem a feature bit is now set that will allow packets to be dropped when full rather than stall. Note, the default is to have flow control on, and this keeps this from happening. - Because of the recent fixes in the stack, LRO is now auto-disabled when problematic, so I have decided to enable it by default in the capabilities in the driver. - There are some 1G modules used by some customers, a couple small tweaks to properly support those in the media code. - A note: we have now done some testing of TSO6 and using LRO with IPv6 and it all works great!! Seeing line rate in both directions in best cases. Thanks bz for your excellent work!!
2012-10-31 23:50:36 +00:00
/*
** Enable the hardware to drop packets when the buffer is
** full. This is useful when multiqueue,so that no single
** queue being full stalls the entire RX engine. We only
** enable this when Multiqueue AND when Flow Control is
** disabled.
*/
static void
ixgbe_enable_rx_drop(struct adapter *adapter)
{
struct ixgbe_hw *hw = &adapter->hw;
for (int i = 0; i < adapter->num_queues; i++) {
u32 srrctl = IXGBE_READ_REG(hw, IXGBE_SRRCTL(i));
srrctl |= IXGBE_SRRCTL_DROP_EN;
IXGBE_WRITE_REG(hw, IXGBE_SRRCTL(i), srrctl);
}
}
static void
ixgbe_disable_rx_drop(struct adapter *adapter)
{
struct ixgbe_hw *hw = &adapter->hw;
for (int i = 0; i < adapter->num_queues; i++) {
u32 srrctl = IXGBE_READ_REG(hw, IXGBE_SRRCTL(i));
srrctl &= ~IXGBE_SRRCTL_DROP_EN;
IXGBE_WRITE_REG(hw, IXGBE_SRRCTL(i), srrctl);
}
}
Reference in New Issue Copy Permalink