The umdbr field in struct bnxt_re_db_region returns the raw
unmapped PCI BAR address of the doorbell region. Avoid sharing
this field to the userspace. Change this to a reserved field
and stop populating it, keeping the ABI layout and size
unchanged for existing binaries.
Fixes: 1234a9d8ae ("RDMA/bnxt_re: Support doorbell extensions")
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Link: https://patch.msgid.link/20260824172443.33943-1-sriharsha.basavapatna@broadcom.com
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
The current GET_TOGGLE_MEM ioctl requires the caller to supply
a type enum and a raw hardware queue ID (RES_ID). The kernel
looks up the CQ or SRQ by that ID without verifying that the
caller owns the resource.
Add a new, preferred code path that accepts standard uverbs
object handles (BNXT_RE_TOGGLE_MEM_CQ_HANDLE /
BNXT_RE_TOGGLE_MEM_SRQ_HANDLE) instead. The uverbs core validates
that the handle belongs to the calling context as part of resolving
it, so this path no longer needs the driver's own XArray lookup for
ownership checking. As with the legacy path, the toggle_entry's own
mmap-entry refcount (not a CQ/SRQ uobject reference) is what pins
the toggle page for the life of the GET_TOGGLE_MEM handle.
Only newer rdma-core versions support this path, if the
driver reports the supported resp mask
(BNXT_RE_UCNTX_CMASK_TOGGLE_MEM_UOBJ_SUPPORT).
The existing TYPE + RES_ID path is retained for backward
compatibility with older rdma-core.
Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Fix the page lifetime by making the rdma_user_mmap_entry the sole owner
of the toggle page allocation. Creating the rdma_user_mmap_entry and page
during the CQ/SRQ creation time. Freeing the page is handled when the
mmap free is called. Introduce struct bnxt_re_toggle_mem to carry
the mmap_offset for the lifetime of the GET_TOGGLE_MEM uobject handle.
bnxt_re_destroy_cq/srq can erase the entry from the XArray and call
rdma_user_mmap_entry_remove() on the toggle_entry concurrently with
the caller's xa_load() and its subsequent use of that toggle_entry.
Guard against this by taking an extra kref directly on the
toggle_entry's rdma_user_mmap_entry while the GET_TOGGLE_MEM handle
exists, released when the handle is destroyed. This pins exactly the
resource that GET_TOGGLE_MEM hands out (the mmap offset/page),
independent of the CQ/SRQ's own lifetime.
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
The CQ and SRQ hash tables (cq_hash, srq_hash) on struct bnxt_re_dev
were used exclusively to look up a toggle-page pointer from a
user-space-supplied hardware queue ID in the GET_TOGGLE_MEM
ioctl handler. This approach has couple of problems. First,
because the tables are per-device, any user can look up another
user's CQ or SRQ by guessing the hardware queue ID. Second,
concurrent add and remove operations on the hash table are not
protected by any lock, leaving a race window.
The correct fix is to retrieve the CQ and SRQ objects via the uverbs
object handle, which gives built-in ownership verification and reference
pinning for the duration of the ioctl. That is added in a later patch of
this series.
To maintain backward compatibility with older rdma-core versions that
do not send a uverbs object handle, the driver must continue to support
the existing TYPE + RES_ID lookup path. This patch replaces the per-device
hash tables with per-ucontext XArrays (cq_xa and srq_xa on struct
bnxt_re_ucontext), which narrows the lookup scope to the calling context,
eliminating the cross-user visibility. Also adds Xarray locking mechanism
for synchronization.
The GET_TOGGLE_MEM ioctl handler is updated to call xa_load()
in place of the now-removed bnxt_re_search_for_cq()/
bnxt_re_search_for_srq() helpers. No ABI changes are required.
bnxt_re_create_user_cq()/bnxt_re_create_srq() publish the uobject into
cq_xa/srq_xa before returning to the uverbs core, but the core only
sets uobject->object once the create callback has returned success.
Guard the lookup against this so a concurrent GET_TOGGLE_MEM racing an
in-progress create cannot feed a NULL ->object into container_of().
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
If a user calls BNXT_RE_METHOD_GET_TOGGLE_MEM on a device that does not
support the CQ/SRQ toggle feature, uctx_cq_page or uctx_srq_page will
be NULL.
Add an explicit -EOPNOTSUPP return after capturing the address from
uctx_cq_page / uctx_srq_page if the address is zero.
Fixes: e275919d96 ("RDMA/bnxt_re: Share a page to expose per CQ info with userspace")
Fixes: 181028a0d8 ("RDMA/bnxt_re: Share a page to expose per SRQ info with userspace")
Link: https://patch.msgid.link/r/20260615224751.232802-16-selvin.xavier@broadcom.com
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
No need to support the DBR related page allocations if the pacing feature
is disabled. Fail the request if pacing is disabled.
Fixes: ea22248578 ("RDMA/bnxt_re: Update alloc_page uapi for pacing")
Link: https://patch.msgid.link/r/20260615224751.232802-15-selvin.xavier@broadcom.com
Reviewed-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Applications can request multiple WC pages for the same ucontext.
As of now, only 1 WC page per ucontext is supported. Add a lock to
avoid concurrent access and a check to fail repeated requests.
Also, if the mmap entry insert fails for the WC, free the Doorbell
page index mapped for the WC page.
Fixes: eee6268421 ("RDMA/bnxt_re: Move the UAPI methods to a dedicated file")
Fixes: 360da60d6c ("RDMA/bnxt_re: Enable low latency push")
Link: https://patch.msgid.link/r/20260615224751.232802-12-selvin.xavier@broadcom.com
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
dpi is initialized only for BNXT_RE_ALLOC_WC_PAGE, but copied
for all the cases. So initialize the dpi to 0.
Fixes: eee6268421 ("RDMA/bnxt_re: Move the UAPI methods to a dedicated file")
Fixes: 360da60d6c ("RDMA/bnxt_re: Enable low latency push")
Link: https://patch.msgid.link/r/20260615224751.232802-2-selvin.xavier@broadcom.com
Reviewed-by: Anantha Prabhu <anantha.prabhu@broadcom.com>
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
App allocated QPs can use a separate doorbell for each QP.
This doorbell region can be passed through a new driver specific
DBR_HANDLE attribute, during QP creation. When this attribute
is set, associate the QP with the given doorbell region.
While the QP holds a reference to the dbr, the dbr itself
cannot be destroyed and is rejected with EBUSY error.
Link: https://patch.msgid.link/r/20260519150041.7251-9-sriharsha.basavapatna@broadcom.com
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
If the DPI is freed when the dbr object is freed, but if the
process has not unmapped the page yet, then the DPI slot could
get reallocated to another process while the original process
still has it mapped. To prevent this, save the DPI info in the
mmap entry during dbr allocation and free the DPI slot from
bnxt_re_mmap_free(), which enures that there are no references
to it.
This change is needed to support doorbell allocation to QPs
in the next patch.
Link: https://patch.msgid.link/r/20260519150041.7251-8-sriharsha.basavapatna@broadcom.com
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
The current logic in the doorbell cleanup function is not
sufficient for a change in a subsequent patch, that fails
doorbell remove operation in some conditions. The cleanup
should facilitate freeing of the dbr object when the caller
may not retry the teardown operation (implicit teardown:
process-exit/driver-removal).
Extend this counter to use kref mechanism so that the dbr
object gets freed (via kref callback) when there are no more
references to it, rather than directly freeing it in the
cleanup uapi.
Link: https://patch.msgid.link/r/20260519150041.7251-7-sriharsha.basavapatna@broadcom.com
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Some applications may need multiple doorbells to support parallel
processing of threads that each operate on a group of resources.
The following uapi methods have been implemented in this patch.
- BNXT_RE_METHOD_DBR_ALLOC:
This will allow the appliation to create extra doorbell regions
and use the associated doorbell page index in CREATE_QP and
use the associated DB address while ringing the doorbell.
- BNXT_RE_METHOD_DBR_FREE:
Free the allocated doorbell region.
- BNXT_RE_METHOD_GET_DEFAULT_DBR:
Return the default doorbell page index and doorbell page address
associated with the ucontext.
Link: https://patch.msgid.link/r/20260302110036.36387-4-sriharsha.basavapatna@broadcom.com
Co-developed-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
This is in preparation for upcoming patches in the series.
Driver has to support additional UAPIs for some applications.
Moving current UAPI implementation to a new file, uapi.c.
Link: https://patch.msgid.link/r/20260302110036.36387-2-sriharsha.basavapatna@broadcom.com
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
