github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-05-31 10:33:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

tcp: remove one ktime_get() from recvmsg() fast path

Browse Source 
Each time some payload is consumed by user space (recvmsg() and friends),
TCP calls tcp_rcv_space_adjust() to run DRS algorithm to check
if an increase of sk->sk_rcvbuf is needed.

This function is based on time sampling, and currently calls
tcp_mstamp_refresh(tp), which is a wrapper around ktime_get_ns().

ktime_get_ns() has a high cost on some platforms.
100+ cycles for rdtscp on AMD EPYC Turin for instance.

We do not have to refresh tp->tcp_mpstamp, using the last cached value
is enough. We only need to refresh it from __tcp_cleanup_rbuf()
if an ACK must be sent (this is a rare event).

Signed-off-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20251024120707.3516550-1-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
This commit is contained in:
Eric Dumazet 2025-10-24 12:07:07 +00:00 committed by Jakub Kicinski
parent 6f147c8328
commit 0ae1ac7335
2 changed files with 11 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
net/ipv4/tcp.c
View File

@ -1556,8 +1556,10 @@ void __tcp_cleanup_rbuf(struct sock *sk, int copied)
time_to_ack = true;
}
}
if (time_to_ack)
if (time_to_ack) {
tcp_mstamp_refresh(tp);
tcp_send_ack(sk);
}
}
void tcp_cleanup_rbuf(struct sock *sk, int copied)

10
net/ipv4/tcp_input.c
View File

@ -928,9 +928,15 @@ void tcp_rcv_space_adjust(struct sock *sk)
trace_tcp_rcv_space_adjust(sk);
tcp_mstamp_refresh(tp);
if (unlikely(!tp->rcv_rtt_est.rtt_us))
return;
/* We do not refresh tp->tcp_mstamp here.
* Some platforms have expensive ktime_get() implementations.
* Using the last cached value is enough for DRS.
*/
time = tcp_stamp_us_delta(tp->tcp_mstamp, tp->rcvq_space.time);
if (time < (tp->rcv_rtt_est.rtt_us >> 3) || tp->rcv_rtt_est.rtt_us == 0)
if (time < (tp->rcv_rtt_est.rtt_us >> 3))
return;
/* Number of bytes copied to user in last RTT */