linux/include
Eric Dumazet b650bf0977 udp: remove busylock and add per NUMA queues
busylock was protecting UDP sockets against packet floods,
but unfortunately was not protecting the host itself.

Under stress, many cpus could spin while acquiring the busylock,
and NIC had to drop packets. Or packets would be dropped
in cpu backlog if RPS/RFS were in place.

This patch replaces the busylock by intermediate
lockless queues. (One queue per NUMA node).

This means that fewer number of cpus have to acquire
the UDP receive queue lock.

Most of the cpus can either:
- immediately drop the packet.
- or queue it in their NUMA aware lockless queue.

Then one of the cpu is chosen to process this lockless queue
in a batch.

The batch only contains packets that were cooked on the same
NUMA node, thus with very limited latency impact.

Tested:

DDOS targeting a victim UDP socket, on a platform with 6 NUMA nodes
(Intel(R) Xeon(R) 6985P-C)

Before:

nstat -n ; sleep 1 ; nstat | grep Udp
Udp6InDatagrams                 1004179            0.0
Udp6InErrors                    3117               0.0
Udp6RcvbufErrors                3117               0.0

After:
nstat -n ; sleep 1 ; nstat | grep Udp
Udp6InDatagrams                 1116633            0.0
Udp6InErrors                    14197275           0.0
Udp6RcvbufErrors                14197275           0.0

We can see this host can now proces 14.2 M more packets per second
while under attack, and the victim socket can receive 11 % more
packets.

I used a small bpftrace program measuring time (in us) spent in
__udp_enqueue_schedule_skb().

Before:

@udp_enqueue_us[398]:
[0]                24901 |@@@                                                 |
[1]                63512 |@@@@@@@@@                                           |
[2, 4)            344827 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[4, 8)            244673 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@                |
[8, 16)            54022 |@@@@@@@@                                            |
[16, 32)          222134 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@                   |
[32, 64)          232042 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@                  |
[64, 128)           4219 |                                                    |
[128, 256)           188 |                                                    |

After:

@udp_enqueue_us[398]:
[0]              5608855 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[1]              1111277 |@@@@@@@@@@                                          |
[2, 4)            501439 |@@@@                                                |
[4, 8)            102921 |                                                    |
[8, 16)            29895 |                                                    |
[16, 32)           43500 |                                                    |
[32, 64)           31552 |                                                    |
[64, 128)            979 |                                                    |
[128, 256)            13 |                                                    |

Note that the remaining bottleneck for this platform is in
udp_drops_inc() because we limited struct numa_drop_counters
to only two nodes so far.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Paolo Abeni <pabeni@redhat.com>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20250922104240.2182559-1-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2025-09-23 16:38:39 -07:00
..
acpi mailbox/pcc: support mailbox management of the shared buffer 2025-08-07 23:49:56 -05:00
asm-generic Deferred unwind changes for 6.17 2025-08-01 09:46:24 -07:00
clocksource
crypto This push fixes a regression that breaks hmac(sha3-224-s390). 2025-08-09 07:26:19 +03:00
cxl
drm drm/gpuvm: fix various typos in .c and .h gpuvm file 2025-08-25 21:48:50 +02:00
dt-bindings dt-bindings: net: pcs: renesas,rzn1-miic: Add RZ/T2H and RZ/N2H support 2025-09-15 17:44:35 -07:00
hyperv
keys
kunit linux_kselftest-kunit-6.17-rc1 2025-07-29 12:43:10 -07:00
kvm KVM: arm64: vgic-v3: Erase LPIs from xarray outside of raw spinlocks 2025-09-10 02:56:20 -07:00
linux udp: remove busylock and add per NUMA queues 2025-09-23 16:38:39 -07:00
math-emu
media
memory
misc
net udp: remove busylock and add per NUMA queues 2025-09-23 16:38:39 -07:00
pcmcia pcmcia: remove PCCARD_IODYN 2025-08-16 15:37:47 +02:00
ras
rdma
rv rv: Retry when da monitor detects race conditions 2025-07-28 16:47:34 -04:00
scsi
soc This is the usual collection of primarily clk driver updates. The big part of 2025-07-31 13:36:27 -07:00
sound ASoC: Fixes for v6.17 2025-08-21 09:02:28 +02:00
target
trace ipv4: Convert ->flowi4_tos to dscp_t. 2025-08-26 17:34:31 -07:00
uapi mptcp: use _BITUL() instead of (1 << x) 2025-09-22 11:51:25 -07:00
ufs
vdso
video
xen Significant patch series in this pull request: 2025-08-03 16:23:09 -07:00
Kbuild