github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-07 06:17:02 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

scsi: ufs: core: Decouple CQ sweep from request iterator in MCQ

Browse Source 
In MCQ mode, ufshcd_mcq_compl_pending_transfer() uses
blk_mq_tagset_busy_iter() to iterate over busy requests during error
recovery and host reset. However, both iterator callbacks perform
whole-queue operations redundantly for each visited request:

 - force_compl == true: ufshcd_mcq_force_compl_one() calls
   ufshcd_mcq_compl_all_cqes_lock() on every busy request, sweeping the
   entire completion ring (hwq->max_entries slots) once per active request
   under spin_lock_irqsave even though the first sweep already cleared all
   completion entries.

 - force_compl == false: ufshcd_mcq_compl_one() acquires cq_lock and polls
   CQTPy over MMIO via ufshcd_mcq_poll_cqe_lock() for every busy request
   without doing any per-request work.

Sweep or poll each hardware queue (hba->uhq[i]) once at the start of
ufshcd_mcq_compl_pending_transfer(). When force_compl is true, run
blk_mq_tagset_busy_iter() afterward to complete residual in-flight
requests with DID_REQUEUE, and remove the now-unused
ufshcd_mcq_compl_one() callback.

Fixes: ab248643d3 ("scsi: ufs: core: Add error handling for MCQ mode")
Cc: stable@vger.kernel.org
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Stanley Jhu <stanleyjhu@google.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260928035816.1294326-3-stanleyjhu@google.com
Signed-off-by: Martin K. Petersen (Oracle) <mkp@kernel.org>
This commit is contained in:
Stanley Jhu 2026-09-28 11:58:16 +08:00 committed by Martin K. Petersen (Oracle)
parent 375f3a5691
commit 469fa05566
1 changed files with 12 additions and 19 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
drivers/ufs/core/ufshcd.c
View File

@ -6044,8 +6044,6 @@ static bool ufshcd_mcq_force_compl_one(struct request *rq, void *priv)
if (blk_mq_is_reserved_rq(rq) || !hwq)
return true;
ufshcd_mcq_compl_all_cqes_lock(hba, hwq);
/*
* For those cmds of which the cqes are not present in the cq, complete
* them explicitly.
@ -6061,19 +6059,6 @@ static bool ufshcd_mcq_force_compl_one(struct request *rq, void *priv)
return true;
}
static bool ufshcd_mcq_compl_one(struct request *rq, void *priv)
{
struct scsi_device *sdev = rq->q->queuedata;
struct Scsi_Host *shost = sdev->host;
struct ufs_hba *hba = shost_priv(shost);
struct ufs_hw_queue *hwq = ufshcd_mcq_req_to_hwq(hba, rq);
if (!blk_mq_is_reserved_rq(rq) && hwq)
ufshcd_mcq_poll_cqe_lock(hba, hwq);
return true;
}
/**
* ufshcd_mcq_compl_pending_transfer - MCQ mode function. It is
* invoked from the error handler context or ufshcd_host_reset_and_restore()
@ -6088,10 +6073,18 @@ static bool ufshcd_mcq_compl_one(struct request *rq, void *priv)
static void ufshcd_mcq_compl_pending_transfer(struct ufs_hba *hba,
bool force_compl)
{
blk_mq_tagset_busy_iter(&hba->host->tag_set,
force_compl ? ufshcd_mcq_force_compl_one :
ufshcd_mcq_compl_one,
NULL);
int i;
for (i = 0; i < hba->nr_hw_queues; i++) {
if (force_compl)
ufshcd_mcq_compl_all_cqes_lock(hba, &hba->uhq[i]);
else
ufshcd_mcq_poll_cqe_lock(hba, &hba->uhq[i]);
}
if (force_compl)
blk_mq_tagset_busy_iter(&hba->host->tag_set,
ufshcd_mcq_force_compl_one, NULL);
}
/**