Hans Petter Selasky e936121d31 Add optimizing LRO wrapper: ... - Add optimizing LRO wrapper which pre-sorts all incoming packets according to the hash type and flowid. This prevents exhaustion of the LRO entries due to too many connections at the same time. Testing using a larger number of higher bandwidth TCP connections showed that the incoming ACK packet aggregation rate increased from ~1.3:1 to almost 3:1. Another test showed that for a number of TCP connections greater than 16 per hardware receive ring, where 8 TCP connections was the LRO active entry limit, there was a significant improvement in throughput due to being able to fully aggregate more than 8 TCP stream. For very few very high bandwidth TCP streams, the optimizing LRO wrapper will add CPU usage instead of reducing CPU usage. This is expected. Network drivers which want to use the optimizing LRO wrapper needs to call "tcp_lro_queue_mbuf()" instead of "tcp_lro_rx()" and "tcp_lro_flush_all()" instead of "tcp_lro_flush()". Further the LRO control structure must be initialized using "tcp_lro_init_args()" passing a non-zero number into the "lro_mbufs" argument. - Make LRO statistics 64-bit. Previously 32-bit integers were used for statistics which can be prone to wrap-around. Fix this while at it and update all SYSCTL's which expose LRO statistics. - Ensure all data is freed when destroying a LRO control structures, especially leftover LRO entries. - Reduce number of memory allocations needed when setting up a LRO control structure by precomputing the total amount of memory needed. - Add own memory allocation counter for LRO. - Bump the FreeBSD version to force recompilation of all KLDs due to change of the LRO control structure size. Sponsored by: Mellanox Technologies Reviewed by: gallatin, sbruno, rrs, gnn, transport Tested by: Netflix Differential Revision: https://reviews.freebsd.org/D4914		2016-01-19 15:33:28 +00:00
..
cc	Add an option to use rfc6675 based pipe/inflight bytes calculation in newreno.	2015-12-09 08:53:41 +00:00
khelp
libalias
tcp_stacks	Apply the changes from r293284 to one additional file.	2016-01-07 11:54:20 +00:00
accf_data.c
accf_dns.c
accf_http.c
cc.h
icmp6.h
icmp_var.h
if_atm.c
if_atm.h
if_ether.c	Remove second EVENTHANDLER_REGISTER slipped in r292978.	2016-01-01 10:15:06 +00:00
if_ether.h	Implement interface link header precomputation API.	2015-12-31 05:03:27 +00:00
igmp_var.h
igmp.c	In the same way fix the problem described in r291578 for IGMPv3.	2015-12-01 11:24:30 +00:00
igmp.h
in_cksum.c
in_debug.c
in_fib.c	Bring RADIX_MPATH support to new routing KPI to ease migration.	2016-01-11 08:45:28 +00:00
in_fib.h	Merge helper fib* functions used for basic lookups.	2015-12-08 10:50:03 +00:00
in_gif.c	Merge helper fib* functions used for basic lookups.	2015-12-08 10:50:03 +00:00
in_kdtrace.c	Fix style issues around existing SDT probes.	2015-12-16 23:39:27 +00:00
in_kdtrace.h	Fix style issues around existing SDT probes.	2015-12-16 23:39:27 +00:00
in_mcast.c	Make in_arpinput(), inp_lookup_mcast_ifp(), icmp_reflect(),	2015-12-09 11:14:27 +00:00
in_pcb.c	Remove sys/eventhandler.h from net/route.h	2016-01-09 09:34:39 +00:00
in_pcb.h	Use Jenkins hash for TCP syncache.	2015-09-05 10:15:19 +00:00
in_pcbgroup.c
in_proto.c
in_rmx.c	Remove now-unused wrappers for various routing functions.	2016-01-14 08:54:44 +00:00
in_rss.c	Rename rss_soft_m2cpuid() -> rss_soft_m2cpuid_v4() in preparation for	2015-08-29 06:58:30 +00:00
in_rss.h	Rename rss_soft_m2cpuid() -> rss_soft_m2cpuid_v4() in preparation for	2015-08-29 06:58:30 +00:00
in_systm.h
in_var.h	Remove now-unused wrappers for various routing functions.	2016-01-14 08:54:44 +00:00
in.c	Add rib_lookup_info() to provide API for retrieving individual route	2016-01-04 15:03:20 +00:00
in.h
ip6.h
ip_carp.c	Revert r292275 & r292379	2015-12-17 14:41:30 +00:00
ip_carp.h
ip_divert.c
ip_divert.h
ip_dummynet.h
ip_ecn.c
ip_ecn.h	Remove unneded #include "opt_inet.h".	2015-07-31 09:02:28 +00:00
ip_encap.c	Remove sys/eventhandler.h from net/route.h	2016-01-09 09:34:39 +00:00
ip_encap.h
ip_fastfwd.c	Replace the fastforward path with tryforward which does not require a	2015-11-05 07:26:32 +00:00
ip_fw.h	Implement ipfw internal olist command to list named objects.	2015-11-03 10:21:53 +00:00
ip_gre.c
ip_icmp.c	Make in_arpinput(), inp_lookup_mcast_ifp(), icmp_reflect(),	2015-12-09 11:14:27 +00:00
ip_icmp.h
ip_id.c
ip_input.c	Make in_arpinput(), inp_lookup_mcast_ifp(), icmp_reflect(),	2015-12-09 11:14:27 +00:00
ip_ipsec.c	Set the proper direction to check for policies in this one case.	2015-10-29 21:26:32 +00:00
ip_ipsec.h
ip_mroute.c	Remove now-unused wrappers for various routing functions.	2016-01-14 08:54:44 +00:00
ip_mroute.h
ip_options.c	Make in_arpinput(), inp_lookup_mcast_ifp(), icmp_reflect(),	2015-12-09 11:14:27 +00:00
ip_options.h
ip_output.c	Finish r275196: do not dereference rtentry in if_output() routines.	2016-01-09 16:34:37 +00:00
ip_reass.c
ip_var.h	Make in_arpinput(), inp_lookup_mcast_ifp(), icmp_reflect(),	2015-12-09 11:14:27 +00:00
ip.h	Since the IETF has redefined the meaning of the tos field to accommodate	2015-07-18 06:48:30 +00:00
pim_var.h
pim.h
raw_ip.c	Remove sys/eventhandler.h from net/route.h	2016-01-09 09:34:39 +00:00
sctp_asconf.c	Provide consistent error causes whenever an ABORT chunk is sent.	2015-07-27 22:35:54 +00:00
sctp_asconf.h
sctp_auth.c	Use __func__ instead of __FUNCTION__.	2015-10-19 11:17:54 +00:00
sctp_auth.h
sctp_bsd_addr.c
sctp_bsd_addr.h
sctp_cc_functions.c	Fix style issues around existing SDT probes.	2015-12-16 23:39:27 +00:00
sctp_constants.h	Ensure that ERROR chunks are always padded by implementing this	2015-09-11 13:54:33 +00:00
sctp_crc32.c
sctp_crc32.h
sctp_dtrace_declare.h
sctp_dtrace_define.h
sctp_header.h	Cleanup the handling of error causes for ERROR chunks. This fixes	2015-09-12 17:08:51 +00:00
sctp_indata.c	Stop processing of a SACK when the association has been aborted.	2015-12-21 18:52:02 +00:00
sctp_indata.h	Don't take the port numbers for packets containing ABORT chunks from	2015-08-02 16:07:30 +00:00
sctp_input.c	Retire sctp_validate_no_locks().	2015-12-10 11:49:32 +00:00
sctp_input.h	Add FIB support for SCTP.	2015-06-17 15:20:14 +00:00
sctp_lock_bsd.h
sctp_os_bsd.h	Replace sys/crypto/sha2/sha2.c with lib/libmd/sha512c.c	2015-12-27 17:33:59 +00:00
sctp_os.h
sctp_output.c	Don't implicitly terminate a user message when moving it to the	2015-12-25 18:11:40 +00:00
sctp_output.h	Fix an issue with MAC OS locking and also optimize the case	2015-07-24 14:09:03 +00:00
sctp_pcb.c	Fix a bug in INIT handling on accepted 1-to-1 style sockets when the	2016-01-15 00:26:15 +00:00
sctp_pcb.h	Retire sctp_validate_no_locks().	2015-12-10 11:49:32 +00:00
sctp_peeloff.c
sctp_peeloff.h
sctp_ss_functions.c
sctp_structs.h	Get struct sctp_net_route in sync with struct route again.	2016-01-04 20:34:40 +00:00
sctp_syscalls.c	Fix two KTRACE related bugs.	2015-06-19 21:55:12 +00:00
sctp_sysctl.c	Fix the exporting of SCTP association states to userland. Without this,	2015-08-29 09:14:32 +00:00
sctp_sysctl.h	Remove redundant 'man page'	2015-09-15 21:16:45 +00:00
sctp_timer.c	Ensure that outgoing streams get reset when they run dry.	2015-12-03 15:19:29 +00:00
sctp_timer.h
sctp_uio.h	The field sinfo_timetolive should have been sinfo_pr_value.	2015-11-06 14:00:26 +00:00
sctp_usrreq.c	Fail the SCTP_GET_ASSOC_NUMBER and SCTP_GET_ASSOC_ID_LIST	2016-01-14 11:25:28 +00:00
sctp_var.h	Stop the heartbeat timer when removing a net.	2015-06-14 17:48:44 +00:00
sctp.h	Cleanup the handling of error causes for ERROR chunks. This fixes	2015-09-12 17:08:51 +00:00
sctputil.c	Store the timer type for logging, because the timer can be freed	2016-01-13 14:28:12 +00:00
sctputil.h	Fix the allocation of outgoing streams:	2015-12-06 16:17:57 +00:00
siftr.c
tcp_debug.c
tcp_debug.h
tcp_fastopen.c	Implementation of server-side TCP Fast Open (TFO) [RFC7413].	2015-12-24 19:09:48 +00:00
tcp_fastopen.h	Implementation of server-side TCP Fast Open (TFO) [RFC7413].	2015-12-24 19:09:48 +00:00
tcp_fsm.h
tcp_hostcache.c	Clean up unused bandwidth entry in the TCP hostcache.	2015-12-11 06:22:58 +00:00
tcp_hostcache.h	Clean up unused bandwidth entry in the TCP hostcache.	2015-12-11 06:22:58 +00:00
tcp_input.c	Historically we have two fields in tcpcb to describe sender MSS: t_maxopd,	2016-01-07 00:14:42 +00:00
tcp_lro.c	Add optimizing LRO wrapper:	2016-01-19 15:33:28 +00:00
tcp_lro.h	Add optimizing LRO wrapper:	2016-01-19 15:33:28 +00:00
tcp_offload.c
tcp_offload.h
tcp_output.c	There is a bug in tcp_output()'s implementation of the TCP_SIGNATURE	2016-01-14 10:22:45 +00:00
tcp_pcap.c	Hopefully also unbreak VIMAGE kernels replacing the &V_... with	2015-10-15 01:44:32 +00:00
tcp_pcap.h	There are times when it would be really nice to have a record of the last few	2015-10-14 00:35:37 +00:00
tcp_reass.c	Remove sys/eventhandler.h from net/route.h	2016-01-09 09:34:39 +00:00
tcp_sack.c	First cut of the modularization of our TCP stack. Still	2015-12-16 00:56:45 +00:00
tcp_seq.h
tcp_subr.c	Remove sys/eventhandler.h from net/route.h	2016-01-09 09:34:39 +00:00
tcp_syncache.c	Implementation of server-side TCP Fast Open (TFO) [RFC7413].	2015-12-24 19:09:48 +00:00
tcp_syncache.h	Implementation of server-side TCP Fast Open (TFO) [RFC7413].	2015-12-24 19:09:48 +00:00
tcp_timer.c	Historically we have two fields in tcpcb to describe sender MSS: t_maxopd,	2016-01-07 00:14:42 +00:00
tcp_timer.h	Put r284245 back in place: If at first this fix was seen as a temporary	2015-08-30 13:44:39 +00:00
tcp_timewait.c	Decompose TCP INP_INFO lock to increase short-lived TCP connections scalability:	2015-08-03 12:13:54 +00:00
tcp_usrreq.c	Historically we have two fields in tcpcb to describe sender MSS: t_maxopd,	2016-01-07 00:14:42 +00:00
tcp_var.h	Historically we have two fields in tcpcb to describe sender MSS: t_maxopd,	2016-01-07 00:14:42 +00:00
tcp.h	Implementation of server-side TCP Fast Open (TFO) [RFC7413].	2015-12-24 19:09:48 +00:00
tcpip.h
toecore.c	(Temporarily) remove route_redirect_event eventhandler.	2016-01-09 06:26:40 +00:00
toecore.h
udp_usrreq.c	Fixed comment placement.	2015-12-24 13:57:43 +00:00
udp_var.h	get_inpcbinfo() and get_pcblist() are UDP local functions and	2015-08-27 15:27:41 +00:00
udp.h
udplite.h