From: Jens Axboe <axboe@kernel.dk>
Date: Thu, 12 Aug 2021 11:42:53 -0600
Subject: [PATCH] block: provide bio_clear_hipri() helper
Git-commit: 270a1c913ebd745ebee716af5f7215e1c2b30cc0
Patch-mainline: v5.15-rc1
References: jsc#PED-1183
Any case that turns off REQ_HIPRI must also clear BIO_PERCPU_CACHE,
as non-polled IO may complete through hard/soft IRQ and hence isn't
safe for our polled bio alloc cache.
Provide a helper that does just that, and use it in the merging code as
well if we split a bio and turn off polling.
Fixes: be863b9e4348 ("block: clear BIO_PERCPU_CACHE flag if polling isn't supported")
Reported-by: Keith Busch <kbusch@kernel.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Acked-by: Hannes Reinecke <hare@suse.com>
---
block/blk-core.c | 7 ++-----
block/blk-merge.c | 2 +-
block/blk.h | 7 +++++++
3 files changed, 10 insertions(+), 6 deletions(-)
diff --git a/block/blk-core.c b/block/blk-core.c
index 0d4d6b1e5d25..f35d401e65f8 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -832,11 +832,8 @@ static noinline_for_stack bool submit_bio_checks(struct bio *bio)
}
}
- if (!test_bit(QUEUE_FLAG_POLL, &q->queue_flags)) {
- /* can't support alloc cache if we turn off polling */
- bio_clear_flag(bio, BIO_PERCPU_CACHE);
- bio->bi_opf &= ~REQ_HIPRI;
- }
+ if (!test_bit(QUEUE_FLAG_POLL, &q->queue_flags))
+ bio_clear_hipri(bio);
switch (bio_op(bio)) {
case REQ_OP_DISCARD:
diff --git a/block/blk-merge.c b/block/blk-merge.c
index a11b3b53717e..bc25ad409fc1 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -285,7 +285,7 @@ static struct bio *blk_bio_segment_split(struct request_queue *q,
* iopoll in direct IO routine. Given performance gain of iopoll for
* big IO can be trival, disable iopoll when split needed.
*/
- bio->bi_opf &= ~REQ_HIPRI;
+ bio_clear_hipri(bio);
return bio_split(bio, sectors, GFP_NOIO, bs);
}
diff --git a/block/blk.h b/block/blk.h
index cb01429c162c..5a4652a10931 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -364,4 +364,11 @@ extern struct device_attribute dev_attr_events;
extern struct device_attribute dev_attr_events_async;
extern struct device_attribute dev_attr_events_poll_msecs;
+static inline void bio_clear_hipri(struct bio *bio)
+{
+ /* can't support alloc cache if we turn off polling */
+ bio_clear_flag(bio, BIO_PERCPU_CACHE);
+ bio->bi_opf &= ~REQ_HIPRI;
+}
+
#endif /* BLK_INTERNAL_H */
--
2.35.3