github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-06-06 05:27:07 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

Merge branch 'net-smc-make-wr-buffer-count-configurable'

Browse Source 
Halil Pasic says:

====================
net/smc: make wr buffer count configurable

The current value of SMC_WR_BUF_CNT is 16 which leads to heavy
contention on the wr_tx_wait workqueue of the SMC-R linkgroup and its
spinlock when many connections are competing for the work request
buffers. Currently up to 256 connections per linkgroup are supported.

To make things worse when finally a buffer becomes available and
smc_wr_tx_put_slot() signals the linkgroup's wr_tx_wait wq, because
WQ_FLAG_EXCLUSIVE is not used all the waiters get woken up, most of the
time a single one can proceed, and the rest is contending on the
spinlock of the wq to go to sleep again.

Addressing this by simply bumping SMC_WR_BUF_CNT to 256 was deemed
risky, because the large-ish physically continuous allocation could fail
and lead to TCP fall-backs. For reference see this discussion thread on
"[PATCH net-next] net/smc: increase SMC_WR_BUF_CNT" (in archive
https://lists.openwall.net/netdev/2024/11/05/186), which concludes with
the agreement to try to come up with something smarter, which is what
this series aims for.

Additionally if for some reason it is known that heavy contention is not
to be expected going with something like 256 work request buffers is
wasteful. To address these concerns make the number of work requests
configurable, and introduce a back-off logic with handles -ENOMEM form
smc_wr_alloc_link_mem() gracefully.

v5: https://lore.kernel.org/netdev/20250929000001.1752206-1-pasic@linux.ibm.com/
v4: https://lore.kernel.org/netdev/20250927232144.3478161-1-pasic@linux.ibm.com/
v3: https://lore.kernel.org/netdev/20250921214440.325325-1-pasic@linux.ibm.com/
v2: https://lore.kernel.org/netdev/20250908220150.3329433-1-pasic@linux.ibm.com/
v1: https://lore.kernel.org/all/20250904211254.1057445-1-pasic@linux.ibm.com/
====================

Link: https://patch.msgid.link/20251027224856.2970019-1-pasic@linux.ibm.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
This commit is contained in:
Paolo Abeni 2025-10-30 13:31:45 +01:00
commit 7ea7694495
10 changed files with 121 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

40
Documentation/networking/smc-sysctl.rst
View File

@ -71,3 +71,43 @@ smcr_max_conns_per_lgr - INTEGER
acceptable value ranges from 16 to 255. Only for SMC-R v2.1 and later.
Default: 255
smcr_max_send_wr - INTEGER
So-called work request buffers are SMCR link (and RDMA queue pair) level
resources necessary for performing RDMA operations. Since up to 255
connections can share a link group and thus also a link and the number
of the work request buffers is decided when the link is allocated,
depending on the workload it can be a bottleneck in a sense that threads
have to wait for work request buffers to become available. Before the
introduction of this control the maximal number of work request buffers
available on the send path used to be hard coded to 16. With this control
it becomes configurable. The acceptable range is between 2 and 2048.
Please be aware that all the buffers need to be allocated as a physically
continuous array in which each element is a single buffer and has the size
of SMC_WR_BUF_SIZE (48) bytes. If the allocation fails, we keep retrying
with half of the buffer count until it is ether successful or (unlikely)
we dip below the old hard coded value which is 16 where we give up much
like before having this control.
Default: 16
smcr_max_recv_wr - INTEGER
So-called work request buffers are SMCR link (and RDMA queue pair) level
resources necessary for performing RDMA operations. Since up to 255
connections can share a link group and thus also a link and the number
of the work request buffers is decided when the link is allocated,
depending on the workload it can be a bottleneck in a sense that threads
have to wait for work request buffers to become available. Before the
introduction of this control the maximal number of work request buffers
available on the receive path used to be hard coded to 16. With this control
it becomes configurable. The acceptable range is between 2 and 2048.
Please be aware that all the buffers need to be allocated as a physically
continuous array in which each element is a single buffer and has the size
of SMC_WR_BUF_SIZE (48) bytes. If the allocation fails, we keep retrying
with half of the buffer count until it is ether successful or (unlikely)
we dip below the old hard coded value which is 16 where we give up much
like before having this control.
Default: 48

2
include/net/netns/smc.h
View File

@ -24,5 +24,7 @@ struct netns_smc {
int sysctl_rmem;
int sysctl_max_links_per_lgr;
int sysctl_max_conns_per_lgr;
unsigned int sysctl_smcr_max_send_wr;
unsigned int sysctl_smcr_max_recv_wr;
};
#endif

34
net/smc/smc_core.c
View File

@ -810,6 +810,8 @@ int smcr_link_init(struct smc_link_group *lgr, struct smc_link *lnk,
lnk->clearing = 0;
lnk->path_mtu = lnk->smcibdev->pattr[lnk->ibport - 1].active_mtu;
lnk->link_id = smcr_next_link_id(lgr);
lnk->max_send_wr = lgr->max_send_wr;
lnk->max_recv_wr = lgr->max_recv_wr;
lnk->lgr = lgr;
smc_lgr_hold(lgr); /* lgr_put in smcr_link_clear() */
lnk->link_idx = link_idx;
@ -836,27 +838,39 @@ int smcr_link_init(struct smc_link_group *lgr, struct smc_link *lnk,
rc = smc_llc_link_init(lnk);
if (rc)
goto out;
rc = smc_wr_alloc_link_mem(lnk);
if (rc)
goto clear_llc_lnk;
rc = smc_ib_create_protection_domain(lnk);
if (rc)
goto free_link_mem;
rc = smc_ib_create_queue_pair(lnk);
if (rc)
goto dealloc_pd;
goto clear_llc_lnk;
do {
rc = smc_ib_create_queue_pair(lnk);
if (rc)
goto dealloc_pd;
rc = smc_wr_alloc_link_mem(lnk);
if (!rc)
break;
else if (rc != -ENOMEM) /* give up */
goto destroy_qp;
/* retry with smaller ... */
lnk->max_send_wr /= 2;
lnk->max_recv_wr /= 2;
/* ... unless droping below old SMC_WR_BUF_SIZE */
if (lnk->max_send_wr < 16 || lnk->max_recv_wr < 48)
goto destroy_qp;
smc_ib_destroy_queue_pair(lnk);
} while (1);
rc = smc_wr_create_link(lnk);
if (rc)
goto destroy_qp;
goto free_link_mem;
lnk->state = SMC_LNK_ACTIVATING;
return 0;
free_link_mem:
smc_wr_free_link_mem(lnk);
destroy_qp:
smc_ib_destroy_queue_pair(lnk);
dealloc_pd:
smc_ib_dealloc_protection_domain(lnk);
free_link_mem:
smc_wr_free_link_mem(lnk);
clear_llc_lnk:
smc_llc_link_clear(lnk, false);
out:

8
net/smc/smc_core.h
View File

@ -34,6 +34,8 @@
* distributions may modify it to a value between
* 16-255 as needed.
*/
#define SMCR_MAX_SEND_WR_DEF 16 /* Default number of work requests per send queue */
#define SMCR_MAX_RECV_WR_DEF 48 /* Default number of work requests per recv queue */
struct smc_lgr_list { /* list of link group definition */
struct list_head list;
@ -173,6 +175,8 @@ struct smc_link {
struct completion llc_testlink_resp; /* wait for rx of testlink */
int llc_testlink_time; /* testlink interval */
atomic_t conn_cnt; /* connections on this link */
u16 max_send_wr;
u16 max_recv_wr;
};
/* For now we just allow one parallel link per link group. The SMC protocol
@ -366,6 +370,10 @@ struct smc_link_group {
/* max conn can be assigned to lgr */
u8 max_links;
/* max links can be added in lgr */
u16 max_send_wr;
/* number of WR buffers on send */
u16 max_recv_wr;
/* number of WR buffers on recv */
};
struct { /* SMC-D */
struct smcd_gid peer_gid;

10
net/smc/smc_ib.c
View File

@ -669,11 +669,6 @@ int smc_ib_create_queue_pair(struct smc_link *lnk)
.recv_cq = lnk->smcibdev->roce_cq_recv,
.srq = NULL,
.cap = {
/* include unsolicited rdma_writes as well,
* there are max. 2 RDMA_WRITE per 1 WR_SEND
*/
.max_send_wr = SMC_WR_BUF_CNT * 3,
.max_recv_wr = SMC_WR_BUF_CNT * 3,
.max_send_sge = SMC_IB_MAX_SEND_SGE,
.max_recv_sge = lnk->wr_rx_sge_cnt,
.max_inline_data = 0,
@ -683,6 +678,11 @@ int smc_ib_create_queue_pair(struct smc_link *lnk)
};
int rc;
/* include unsolicited rdma_writes as well,
* there are max. 2 RDMA_WRITE per 1 WR_SEND
*/
qp_attr.cap.max_send_wr = 3 * lnk->lgr->max_send_wr;
qp_attr.cap.max_recv_wr = lnk->lgr->max_recv_wr;
lnk->roce_qp = ib_create_qp(lnk->roce_pd, &qp_attr);
rc = PTR_ERR_OR_ZERO(lnk->roce_qp);
if (IS_ERR(lnk->roce_qp))

2
net/smc/smc_llc.c
View File

@ -2157,6 +2157,8 @@ void smc_llc_lgr_init(struct smc_link_group *lgr, struct smc_sock *smc)
init_waitqueue_head(&lgr->llc_msg_waiter);
init_rwsem(&lgr->llc_conf_mutex);
lgr->llc_testlink_time = READ_ONCE(net->smc.sysctl_smcr_testlink_time);
lgr->max_send_wr = (u16)(READ_ONCE(net->smc.sysctl_smcr_max_send_wr));
lgr->max_recv_wr = (u16)(READ_ONCE(net->smc.sysctl_smcr_max_recv_wr));
}
/* called after lgr was removed from lgr_list */

22
net/smc/smc_sysctl.c
View File

@ -29,6 +29,8 @@ static int links_per_lgr_min = SMC_LINKS_ADD_LNK_MIN;
static int links_per_lgr_max = SMC_LINKS_ADD_LNK_MAX;
static int conns_per_lgr_min = SMC_CONN_PER_LGR_MIN;
static int conns_per_lgr_max = SMC_CONN_PER_LGR_MAX;
static unsigned int smcr_max_wr_min = 2;
static unsigned int smcr_max_wr_max = 2048;
static struct ctl_table smc_table[] = {
{
@ -99,6 +101,24 @@ static struct ctl_table smc_table[] = {
.extra1 = SYSCTL_ZERO,
.extra2 = SYSCTL_ONE,
},
{
.procname = "smcr_max_send_wr",
.data = &init_net.smc.sysctl_smcr_max_send_wr,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
.extra1 = &smcr_max_wr_min,
.extra2 = &smcr_max_wr_max,
},
{
.procname = "smcr_max_recv_wr",
.data = &init_net.smc.sysctl_smcr_max_recv_wr,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
.extra1 = &smcr_max_wr_min,
.extra2 = &smcr_max_wr_max,
},
};
int __net_init smc_sysctl_net_init(struct net *net)
@ -130,6 +150,8 @@ int __net_init smc_sysctl_net_init(struct net *net)
WRITE_ONCE(net->smc.sysctl_rmem, net_smc_rmem_init);
net->smc.sysctl_max_links_per_lgr = SMC_LINKS_PER_LGR_MAX_PREFER;
net->smc.sysctl_max_conns_per_lgr = SMC_CONN_PER_LGR_PREFER;
net->smc.sysctl_smcr_max_send_wr = SMCR_MAX_SEND_WR_DEF;
net->smc.sysctl_smcr_max_recv_wr = SMCR_MAX_RECV_WR_DEF;
/* disable handshake limitation by default */
net->smc.limit_smc_hs = 0;

2
net/smc/smc_sysctl.h
View File

@ -25,6 +25,8 @@ static inline int smc_sysctl_net_init(struct net *net)
net->smc.sysctl_autocorking_size = SMC_AUTOCORKING_DEFAULT_SIZE;
net->smc.sysctl_max_links_per_lgr = SMC_LINKS_PER_LGR_MAX_PREFER;
net->smc.sysctl_max_conns_per_lgr = SMC_CONN_PER_LGR_PREFER;
net->smc.sysctl_smcr_max_send_wr = SMCR_MAX_SEND_WR_DEF;
net->smc.sysctl_smcr_max_recv_wr = SMCR_MAX_RECV_WR_DEF;
return 0;
}

31
net/smc/smc_wr.c
View File

@ -547,9 +547,9 @@ void smc_wr_remember_qp_attr(struct smc_link *lnk)
IB_QP_DEST_QPN,
&init_attr);
lnk->wr_tx_cnt = min_t(size_t, SMC_WR_BUF_CNT,
lnk->wr_tx_cnt = min_t(size_t, lnk->max_send_wr,
lnk->qp_attr.cap.max_send_wr);
lnk->wr_rx_cnt = min_t(size_t, SMC_WR_BUF_CNT * 3,
lnk->wr_rx_cnt = min_t(size_t, lnk->max_recv_wr,
lnk->qp_attr.cap.max_recv_wr);
}
@ -741,50 +741,51 @@ int smc_wr_alloc_lgr_mem(struct smc_link_group *lgr)
int smc_wr_alloc_link_mem(struct smc_link *link)
{
/* allocate link related memory */
link->wr_tx_bufs = kcalloc(SMC_WR_BUF_CNT, SMC_WR_BUF_SIZE, GFP_KERNEL);
link->wr_tx_bufs = kcalloc(link->max_send_wr,
SMC_WR_BUF_SIZE, GFP_KERNEL);
if (!link->wr_tx_bufs)
goto no_mem;
link->wr_rx_bufs = kcalloc(SMC_WR_BUF_CNT * 3, link->wr_rx_buflen,
link->wr_rx_bufs = kcalloc(link->max_recv_wr, link->wr_rx_buflen,
GFP_KERNEL);
if (!link->wr_rx_bufs)
goto no_mem_wr_tx_bufs;
link->wr_tx_ibs = kcalloc(SMC_WR_BUF_CNT, sizeof(link->wr_tx_ibs[0]),
GFP_KERNEL);
link->wr_tx_ibs = kcalloc(link->max_send_wr,
sizeof(link->wr_tx_ibs[0]), GFP_KERNEL);
if (!link->wr_tx_ibs)
goto no_mem_wr_rx_bufs;
link->wr_rx_ibs = kcalloc(SMC_WR_BUF_CNT * 3,
link->wr_rx_ibs = kcalloc(link->max_recv_wr,
sizeof(link->wr_rx_ibs[0]),
GFP_KERNEL);
if (!link->wr_rx_ibs)
goto no_mem_wr_tx_ibs;
link->wr_tx_rdmas = kcalloc(SMC_WR_BUF_CNT,
link->wr_tx_rdmas = kcalloc(link->max_send_wr,
sizeof(link->wr_tx_rdmas[0]),
GFP_KERNEL);
if (!link->wr_tx_rdmas)
goto no_mem_wr_rx_ibs;
link->wr_tx_rdma_sges = kcalloc(SMC_WR_BUF_CNT,
link->wr_tx_rdma_sges = kcalloc(link->max_send_wr,
sizeof(link->wr_tx_rdma_sges[0]),
GFP_KERNEL);
if (!link->wr_tx_rdma_sges)
goto no_mem_wr_tx_rdmas;
link->wr_tx_sges = kcalloc(SMC_WR_BUF_CNT, sizeof(link->wr_tx_sges[0]),
link->wr_tx_sges = kcalloc(link->max_send_wr, sizeof(link->wr_tx_sges[0]),
GFP_KERNEL);
if (!link->wr_tx_sges)
goto no_mem_wr_tx_rdma_sges;
link->wr_rx_sges = kcalloc(SMC_WR_BUF_CNT * 3,
link->wr_rx_sges = kcalloc(link->max_recv_wr,
sizeof(link->wr_rx_sges[0]) * link->wr_rx_sge_cnt,
GFP_KERNEL);
if (!link->wr_rx_sges)
goto no_mem_wr_tx_sges;
link->wr_tx_mask = bitmap_zalloc(SMC_WR_BUF_CNT, GFP_KERNEL);
link->wr_tx_mask = bitmap_zalloc(link->max_send_wr, GFP_KERNEL);
if (!link->wr_tx_mask)
goto no_mem_wr_rx_sges;
link->wr_tx_pends = kcalloc(SMC_WR_BUF_CNT,
link->wr_tx_pends = kcalloc(link->max_send_wr,
sizeof(link->wr_tx_pends[0]),
GFP_KERNEL);
if (!link->wr_tx_pends)
goto no_mem_wr_tx_mask;
link->wr_tx_compl = kcalloc(SMC_WR_BUF_CNT,
link->wr_tx_compl = kcalloc(link->max_send_wr,
sizeof(link->wr_tx_compl[0]),
GFP_KERNEL);
if (!link->wr_tx_compl)
@ -905,7 +906,7 @@ int smc_wr_create_link(struct smc_link *lnk)
goto dma_unmap;
}
smc_wr_init_sge(lnk);
bitmap_zero(lnk->wr_tx_mask, SMC_WR_BUF_CNT);
bitmap_zero(lnk->wr_tx_mask, lnk->max_send_wr);
init_waitqueue_head(&lnk->wr_tx_wait);
rc = percpu_ref_init(&lnk->wr_tx_refs, smcr_wr_tx_refs_free, 0, GFP_KERNEL);
if (rc)

2
net/smc/smc_wr.h
View File

@ -19,8 +19,6 @@
#include "smc.h"
#include "smc_core.h"
#define SMC_WR_BUF_CNT 16 /* # of ctrl buffers per link */
#define SMC_WR_TX_WAIT_FREE_SLOT_TIME (10 * HZ)
#define SMC_WR_TX_SIZE 44 /* actual size of wr_send data (<=SMC_WR_BUF_SIZE) */