RDMA/mlx5: Fix undefined shift of user RQ WQE size

set_rq_size() computes the RQ WQE size as "1 << rq_wqe_shift" based on
the user-provided rq_wqe_shift, which is only checked to be greater than
32, so shifts of 32 are still accepted. A shift of 31 also overflows a
signed integer, leading to undefined behavior.

Use check_shl_overflow() to compute the RQ WQE size and reject any
invalid values.

Fixes: e126ba97db ("mlx5: Add driver for Mellanox Connect-IB adapters")
Link: https://patch.msgid.link/r/20260611-maher-sec-fixes-v1-1-cd8eb2542869@nvidia.com
Signed-off-by: Maher Sanalla <msanalla@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
This commit is contained in:
Maher Sanalla 2026-06-11 15:50:42 +03:00 committed by Jason Gunthorpe
parent 666031fed8
commit d881d60223

View File

@ -461,16 +461,13 @@ static int set_rq_size(struct mlx5_ib_dev *dev, struct ib_qp_cap *cap,
if (ucmd) {
qp->rq.wqe_cnt = ucmd->rq_wqe_count;
if (ucmd->rq_wqe_shift > BITS_PER_BYTE * sizeof(ucmd->rq_wqe_shift))
return -EINVAL;
qp->rq.wqe_shift = ucmd->rq_wqe_shift;
if ((1 << qp->rq.wqe_shift) /
sizeof(struct mlx5_wqe_data_seg) <
wq_sig)
if (check_shl_overflow(1, qp->rq.wqe_shift, &wqe_size))
return -EINVAL;
if (wqe_size / sizeof(struct mlx5_wqe_data_seg) < wq_sig)
return -EINVAL;
qp->rq.max_gs =
(1 << qp->rq.wqe_shift) /
sizeof(struct mlx5_wqe_data_seg) -
wqe_size / sizeof(struct mlx5_wqe_data_seg) -
wq_sig;
qp->rq.max_post = qp->rq.wqe_cnt;
} else {