Daniel Wagner 289f08
From: Zhihao Cheng <chengzhihao1@huawei.com>
Daniel Wagner 289f08
Date: Mon, 5 Jul 2021 21:38:29 +0800
Daniel Wagner 289f08
Subject: nvme-pci: don't WARN_ON in nvme_reset_work if ctrl.state is not
Daniel Wagner 289f08
 RESETTING
Daniel Wagner 289f08
Patch-mainline: v5.14-rc3
Daniel Wagner 289f08
Git-commit: 7764656b108cd308c39e9a8554353b8f9ca232a3
Daniel Wagner 289f08
References: git-fixes
Daniel Wagner 289f08
Daniel Wagner 289f08
Followling process:
Daniel Wagner 289f08
nvme_probe
Daniel Wagner 289f08
  nvme_reset_ctrl
Daniel Wagner 289f08
    nvme_change_ctrl_state(ctrl, NVME_CTRL_RESETTING)
Daniel Wagner 289f08
    queue_work(nvme_reset_wq, &ctrl->reset_work)
Daniel Wagner 289f08
Daniel Wagner 289f08
-------------->	nvme_remove
Daniel Wagner 289f08
		  nvme_change_ctrl_state(&dev->ctrl, NVME_CTRL_DELETING)
Daniel Wagner 289f08
worker_thread
Daniel Wagner 289f08
  process_one_work
Daniel Wagner 289f08
    nvme_reset_work
Daniel Wagner 289f08
    WARN_ON(dev->ctrl.state != NVME_CTRL_RESETTING)
Daniel Wagner 289f08
Daniel Wagner 289f08
, which will trigger WARN_ON in nvme_reset_work():
Daniel Wagner 289f08
[  127.534298] WARNING: CPU: 0 PID: 139 at drivers/nvme/host/pci.c:2594
Daniel Wagner 289f08
[  127.536161] CPU: 0 PID: 139 Comm: kworker/u8:7 Not tainted 5.13.0
Daniel Wagner 289f08
[  127.552518] Call Trace:
Daniel Wagner 289f08
[  127.552840]  ? kvm_sched_clock_read+0x25/0x40
Daniel Wagner 289f08
[  127.553936]  ? native_send_call_func_single_ipi+0x1c/0x30
Daniel Wagner 289f08
[  127.555117]  ? send_call_function_single_ipi+0x9b/0x130
Daniel Wagner 289f08
[  127.556263]  ? __smp_call_single_queue+0x48/0x60
Daniel Wagner 289f08
[  127.557278]  ? ttwu_queue_wakelist+0xfa/0x1c0
Daniel Wagner 289f08
[  127.558231]  ? try_to_wake_up+0x265/0x9d0
Daniel Wagner 289f08
[  127.559120]  ? ext4_end_io_rsv_work+0x160/0x290
Daniel Wagner 289f08
[  127.560118]  process_one_work+0x28c/0x640
Daniel Wagner 289f08
[  127.561002]  worker_thread+0x39a/0x700
Daniel Wagner 289f08
[  127.561833]  ? rescuer_thread+0x580/0x580
Daniel Wagner 289f08
[  127.562714]  kthread+0x18c/0x1e0
Daniel Wagner 289f08
[  127.563444]  ? set_kthread_struct+0x70/0x70
Daniel Wagner 289f08
[  127.564347]  ret_from_fork+0x1f/0x30
Daniel Wagner 289f08
Daniel Wagner 289f08
The preceding problem can be easily reproduced by executing following
Daniel Wagner 289f08
script (based on blktests suite):
Daniel Wagner 289f08
test() {
Daniel Wagner 289f08
  pdev="$(_get_pci_dev_from_blkdev)"
Daniel Wagner 289f08
  sysfs="/sys/bus/pci/devices/${pdev}"
Daniel Wagner 289f08
  for ((i = 0; i < 10; i++)); do
Daniel Wagner 289f08
    echo 1 > "$sysfs/remove"
Daniel Wagner 289f08
    echo 1 > /sys/bus/pci/rescan
Daniel Wagner 289f08
  done
Daniel Wagner 289f08
}
Daniel Wagner 289f08
Daniel Wagner 289f08
Since the device ctrl could be updated as an non-RESETTING state by
Daniel Wagner 289f08
repeating probe/remove in userspace (which is a normal situation), we
Daniel Wagner 289f08
can replace stack dumping WARN_ON with a warnning message.
Daniel Wagner 289f08
Daniel Wagner 289f08
Fixes: 82b057caefaff ("nvme-pci: fix multiple ctrl removal schedulin")
Daniel Wagner 289f08
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Daniel Wagner 289f08
Acked-by: Daniel Wagner <dwagner@suse.de>
Daniel Wagner 289f08
---
Daniel Wagner 289f08
 drivers/nvme/host/pci.c |    5 ++++-
Daniel Wagner 289f08
 1 file changed, 4 insertions(+), 1 deletion(-)
Daniel Wagner 289f08
Daniel Wagner 289f08
--- a/drivers/nvme/host/pci.c
Daniel Wagner 289f08
+++ b/drivers/nvme/host/pci.c
Daniel Wagner 289f08
@@ -2252,8 +2252,11 @@ static void nvme_reset_work(struct work_
Daniel Wagner 289f08
 	int result = -ENODEV;
Daniel Wagner 289f08
 	enum nvme_ctrl_state new_state = NVME_CTRL_LIVE;
Daniel Wagner 289f08
 
Daniel Wagner 289f08
-	if (WARN_ON(dev->ctrl.state != NVME_CTRL_RESETTING))
Daniel Wagner 289f08
+	if (dev->ctrl.state != NVME_CTRL_RESETTING) {
Daniel Wagner 289f08
+		dev_warn(dev->ctrl.device, "ctrl state %d is not RESETTING\n",
Daniel Wagner 289f08
+			 dev->ctrl.state);
Daniel Wagner 289f08
 		goto out;
Daniel Wagner 289f08
+	}
Daniel Wagner 289f08
 
Daniel Wagner 289f08
 	/*
Daniel Wagner 289f08
 	 * If we're called to reset a live controller first shut it down before