linux-hardened/net/rxrpc
Paolo Abeni 7c13f97ffd udp: do fwd memory scheduling on dequeue
A new argument is added to __skb_recv_datagram to provide
an explicit skb destructor, invoked under the receive queue
lock.
The UDP protocol uses such argument to perform memory
reclaiming on dequeue, so that the UDP protocol does not
set anymore skb->desctructor.
Instead explicit memory reclaiming is performed at close() time and
when skbs are removed from the receive queue.
The in kernel UDP protocol users now need to call a
skb_recv_udp() variant instead of skb_recv_datagram() to
properly perform memory accounting on dequeue.

Overall, this allows acquiring only once the receive queue
lock on dequeue.

Tested using pktgen with random src port, 64 bytes packet,
wire-speed on a 10G link as sender and udp_sink as the receiver,
using an l4 tuple rxhash to stress the contention, and one or more
udp_sink instances with reuseport.

nr sinks	vanilla		patched
1		440		560
3		2150		2300
6		3650		3800
9		4450		4600
12		6250		6450

v1 -> v2:
 - do rmem and allocated memory scheduling under the receive lock
 - do bulk scheduling in first_packet_length() and in udp_destruct_sock()
 - avoid the typdef for the dequeue callback

Suggested-by: Eric Dumazet <edumazet@google.com>
Acked-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Acked-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-11-07 13:24:41 -05:00
..
af_rxrpc.c rxrpc: Accesses of rxrpc_local::service need to be RCU managed 2016-10-06 08:11:48 +01:00
ar-internal.h rxrpc: Need to produce an ACK for service op if op takes a long time 2016-10-06 08:11:50 +01:00
call_accept.c rxrpc: Fix warning by splitting rxrpc_send_call_packet() 2016-10-06 08:11:49 +01:00
call_event.c rxrpc: Need to produce an ACK for service op if op takes a long time 2016-10-06 08:11:50 +01:00
call_object.c rxrpc: Fix checker warning by not passing always-zero value to ERR_PTR() 2016-10-13 08:39:52 +01:00
conn_client.c rxrpc: When activating client conn channels, do state check inside lock 2016-09-29 22:57:47 +01:00
conn_event.c rxrpc: The offset field in struct rxrpc_skb_priv is unnecessary 2016-09-30 14:39:28 +01:00
conn_object.c rxrpc: Don't store the rxrpc header in the Tx queue sk_buffs 2016-09-22 01:23:50 +01:00
conn_service.c rxrpc: Add connection tracepoint and client conn state tracepoint 2016-09-17 11:24:03 +01:00
input.c udp: do fwd memory scheduling on dequeue 2016-11-07 13:24:41 -05:00
insecure.c rxrpc: Rewrite the data and ack handling code 2016-09-08 11:10:12 +01:00
Kconfig rxrpc: Add config to inject packet loss 2016-09-17 11:24:04 +01:00
key.c rxrpc: Use structs to hold connection params and protocol info 2016-06-22 09:09:59 +01:00
local_event.c rxrpc: The offset field in struct rxrpc_skb_priv is unnecessary 2016-09-30 14:39:28 +01:00
local_object.c rxrpc: Reduce the rxrpc_local::services list to a pointer 2016-09-29 22:57:47 +01:00
Makefile rxrpc: Split sendmsg from packet transmission code 2016-09-04 21:41:39 +01:00
misc.c rxrpc: Need to produce an ACK for service op if op takes a long time 2016-10-06 08:11:50 +01:00
output.c rxrpc: Don't request an ACK on the last DATA packet of a call's Tx phase 2016-10-06 08:11:51 +01:00
peer_event.c rxrpc: Add per-peer RTT tracker 2016-09-22 01:26:25 +01:00
peer_object.c rxrpc: Fix checking of error from ip6_route_output() 2016-10-13 08:43:17 +01:00
proc.c rxrpc: Add IPv6 support 2016-09-13 23:09:13 +01:00
recvmsg.c rxrpc: Need to produce an ACK for service op if op takes a long time 2016-10-06 08:11:50 +01:00
rxkad.c rxrpc: Fix warning by splitting rxrpc_send_call_packet() 2016-10-06 08:11:49 +01:00
security.c rxrpc: Reduce the rxrpc_local::services list to a pointer 2016-09-29 22:57:47 +01:00
sendmsg.c rxrpc: Need to produce an ACK for service op if op takes a long time 2016-10-06 08:11:50 +01:00
skbuff.c rxrpc: Make Tx loss-injection go through normal return and adjust tracing 2016-09-29 22:37:15 +01:00
sysctl.c rxrpc: Keep the call timeouts as ktimes rather than jiffies 2016-09-30 14:40:11 +01:00
utils.c rxrpc: Make IPv6 support conditional on CONFIG_IPV6 2016-09-17 03:58:45 -04:00