From 649bb75d19c93f5459f450191953dff4825fda3e Mon Sep 17 00:00:00 2001
From: Almog Khaikin <almogkh@gmail.com>
Date: Mon, 21 Mar 2022 11:00:59 +0200
Subject: [PATCH] io_uring: fix memory ordering when SQPOLL thread goes to
sleep
Git-commit: 649bb75d19c93f5459f450191953dff4825fda3e
Patch-mainline: v5.18-rc1
References: bsc#1205205
Without a full memory barrier between the store to the flags and the
load of the SQ tail the two operations can be reordered and this can
lead to a situation where the SQPOLL thread goes to sleep while the
application writes to the SQ tail and doesn't see the wakeup flag.
This memory barrier pairs with a full memory barrier in the application
between its store to the SQ tail and its load of the flags.
Signed-off-by: Almog Khaikin <almogkh@gmail.com>
Link: https://lore.kernel.org/r/20220321090059.46313-1-almogkh@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Gabriel Krisman Bertazi <krisman@suse.de>
---
fs/io_uring.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/fs/io_uring.c b/fs/io_uring.c
index 1a65d7880440..48f4540d7dd5 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -8042,6 +8042,13 @@ static int io_sq_thread(void *data)
needs_sched = false;
break;
}
+
+ /*
+ * Ensure the store of the wakeup flag is not
+ * reordered with the load of the SQ tail
+ */
+ smp_mb();
+
if (io_sqring_entries(ctx)) {
needs_sched = false;
break;
--
2.35.3