From: Alexey Dobriyan <adobriyan@gmail.com>

Date: Thu, 7 May 2020 23:07:04 +0300

Subject: [PATCH] nvme-pci: fix "slimmer CQ head update"

Git-commit: a8de6639169b90e3dc4f27e752a3c5abac5e90da

Patch-mainline: v5.7-rc5

References: git-fixes

Pre-incrementing ->cq_head can't be done in memory because OOB value

can be observed by another context.

This devalues space savings compared to original code :-\

	$ ./scripts/bloat-o-meter ../vmlinux-000 ../obj/vmlinux

	add/remove: 0/0 grow/shrink: 0/4 up/down: 0/-32 (-32)

	Function                                     old     new   delta

	nvme_poll_irqdisable                         464     456      -8

	nvme_poll                                    455     447      -8

	nvme_irq                                     388     380      -8

	nvme_dev_disable                             955     947      -8

But the code is minimal now: one read for head, one read for q_depth,

one increment, one comparison, single instruction phase bit update and

one write for new head.

Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>

Reported-by: John Garry <john.garry@huawei.com>

Tested-by: John Garry <john.garry@huawei.com>

Fixes: e2a366a4b0feaeb ("nvme-pci: slimmer CQ head update")

Signed-off-by: Christoph Hellwig <hch@lst.de>

Signed-off-by: Jens Axboe <axboe@kernel.dk>

Acked-by: Hannes Reinecke <hare@suse.com>

---

 drivers/nvme/host/pci.c | 6 +++++-

 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c

index 4e79e412b276..e13c370de830 100644

--- a/drivers/nvme/host/pci.c

+++ b/drivers/nvme/host/pci.c

@@ -973,9 +973,13 @@ static inline void nvme_handle_cqe(struct nvme_queue *nvmeq, u16 idx)

 static inline void nvme_update_cq_head(struct nvme_queue *nvmeq)

 {

-	if (++nvmeq->cq_head == nvmeq->q_depth) {

+	u16 tmp = nvmeq->cq_head + 1;

+

+	if (tmp == nvmeq->q_depth) {

 		nvmeq->cq_head = 0;

 		nvmeq->cq_phase ^= 1;

+	} else {

+		nvmeq->cq_head = tmp;

 	}

 }

-- 

2.29.2