Anthony Iliopoulos d228bc
From 22ed903eee23a5b174e240f1cdfa9acf393a5210 Mon Sep 17 00:00:00 2001
Anthony Iliopoulos d228bc
From: "Darrick J. Wong" <djwong@kernel.org>
Anthony Iliopoulos d228bc
Date: Wed, 12 Apr 2023 15:49:23 +1000
Anthony Iliopoulos d228bc
Subject: [PATCH] xfs: verify buffer contents when we skip log replay
Anthony Iliopoulos d228bc
Git-commit: 22ed903eee23a5b174e240f1cdfa9acf393a5210
Anthony Iliopoulos d228bc
Patch-mainline: v6.4-rc1
Anthony Iliopoulos d228bc
References: bsc#1210498 CVE-2023-2124
Anthony Iliopoulos d228bc
Anthony Iliopoulos d228bc
syzbot detected a crash during log recovery:
Anthony Iliopoulos d228bc
Anthony Iliopoulos d228bc
XFS (loop0): Mounting V5 Filesystem bfdc47fc-10d8-4eed-a562-11a831b3f791
Anthony Iliopoulos d228bc
XFS (loop0): Torn write (CRC failure) detected at log block 0x180. Truncating head block from 0x200.
Anthony Iliopoulos d228bc
XFS (loop0): Starting recovery (logdev: internal)
Anthony Iliopoulos d228bc
==================================================================
Anthony Iliopoulos d228bc
Bug: KASAN: slab-out-of-bounds in xfs_btree_lookup_get_block+0x15c/0x6d0 fs/xfs/libxfs/xfs_btree.c:1813
Anthony Iliopoulos d228bc
Read of size 8 at addr ffff88807e89f258 by task syz-executor132/5074
Anthony Iliopoulos d228bc
Anthony Iliopoulos d228bc
Cpu: 0 PID: 5074 Comm: syz-executor132 Not tainted 6.2.0-rc1-syzkaller #0
Anthony Iliopoulos d228bc
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022
Anthony Iliopoulos d228bc
Call Trace:
Anthony Iliopoulos d228bc
 <TASK>
Anthony Iliopoulos d228bc
 __dump_stack lib/dump_stack.c:88 [inline]
Anthony Iliopoulos d228bc
 dump_stack_lvl+0x1b1/0x290 lib/dump_stack.c:106
Anthony Iliopoulos d228bc
 print_address_description+0x74/0x340 mm/kasan/report.c:306
Anthony Iliopoulos d228bc
 print_report+0x107/0x1f0 mm/kasan/report.c:417
Anthony Iliopoulos d228bc
 kasan_report+0xcd/0x100 mm/kasan/report.c:517
Anthony Iliopoulos d228bc
 xfs_btree_lookup_get_block+0x15c/0x6d0 fs/xfs/libxfs/xfs_btree.c:1813
Anthony Iliopoulos d228bc
 xfs_btree_lookup+0x346/0x12c0 fs/xfs/libxfs/xfs_btree.c:1913
Anthony Iliopoulos d228bc
 xfs_btree_simple_query_range+0xde/0x6a0 fs/xfs/libxfs/xfs_btree.c:4713
Anthony Iliopoulos d228bc
 xfs_btree_query_range+0x2db/0x380 fs/xfs/libxfs/xfs_btree.c:4953
Anthony Iliopoulos d228bc
 xfs_refcount_recover_cow_leftovers+0x2d1/0xa60 fs/xfs/libxfs/xfs_refcount.c:1946
Anthony Iliopoulos d228bc
 xfs_reflink_recover_cow+0xab/0x1b0 fs/xfs/xfs_reflink.c:930
Anthony Iliopoulos d228bc
 xlog_recover_finish+0x824/0x920 fs/xfs/xfs_log_recover.c:3493
Anthony Iliopoulos d228bc
 xfs_log_mount_finish+0x1ec/0x3d0 fs/xfs/xfs_log.c:829
Anthony Iliopoulos d228bc
 xfs_mountfs+0x146a/0x1ef0 fs/xfs/xfs_mount.c:933
Anthony Iliopoulos d228bc
 xfs_fs_fill_super+0xf95/0x11f0 fs/xfs/xfs_super.c:1666
Anthony Iliopoulos d228bc
 get_tree_bdev+0x400/0x620 fs/super.c:1282
Anthony Iliopoulos d228bc
 vfs_get_tree+0x88/0x270 fs/super.c:1489
Anthony Iliopoulos d228bc
 do_new_mount+0x289/0xad0 fs/namespace.c:3145
Anthony Iliopoulos d228bc
 do_mount fs/namespace.c:3488 [inline]
Anthony Iliopoulos d228bc
 __do_sys_mount fs/namespace.c:3697 [inline]
Anthony Iliopoulos d228bc
 __se_sys_mount+0x2d3/0x3c0 fs/namespace.c:3674
Anthony Iliopoulos d228bc
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
Anthony Iliopoulos d228bc
 do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
Anthony Iliopoulos d228bc
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
Anthony Iliopoulos d228bc
Rip: 0033:0x7f89fa3f4aca
Anthony Iliopoulos d228bc
Code: 83 c4 08 5b 5d c3 66 2e 0f 1f 84 00 00 00 00 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48
Anthony Iliopoulos d228bc
Rsp: 002b:00007fffd5fb5ef8 EFLAGS: 00000206 ORIG_RAX: 00000000000000a5
Anthony Iliopoulos d228bc
Rax: ffffffffffffffda RBX: 00646975756f6e2c RCX: 00007f89fa3f4aca
Anthony Iliopoulos d228bc
Rdx: 0000000020000100 RSI: 0000000020009640 RDI: 00007fffd5fb5f10
Anthony Iliopoulos d228bc
Rbp: 00007fffd5fb5f10 R08: 00007fffd5fb5f50 R09: 000000000000970d
Anthony Iliopoulos d228bc
R10: 0000000000200800 R11: 0000000000000206 R12: 0000000000000004
Anthony Iliopoulos d228bc
R13: 0000555556c6b2c0 R14: 0000000000200800 R15: 00007fffd5fb5f50
Anthony Iliopoulos d228bc
 </TASK>
Anthony Iliopoulos d228bc
Anthony Iliopoulos d228bc
The fuzzed image contains an AGF with an obviously garbage
Anthony Iliopoulos d228bc
agf_refcount_level value of 32, and a dirty log with a buffer log item
Anthony Iliopoulos d228bc
for that AGF.  The ondisk AGF has a higher LSN than the recovered log
Anthony Iliopoulos d228bc
item.  xlog_recover_buf_commit_pass2 reads the buffer, compares the
Anthony Iliopoulos d228bc
LSNs, and decides to skip replay because the ondisk buffer appears to be
Anthony Iliopoulos d228bc
newer.
Anthony Iliopoulos d228bc
Anthony Iliopoulos d228bc
Unfortunately, the ondisk buffer is corrupt, but recovery just read the
Anthony Iliopoulos d228bc
buffer with no buffer ops specified:
Anthony Iliopoulos d228bc
Anthony Iliopoulos d228bc
	error = xfs_buf_read(mp->m_ddev_targp, buf_f->blf_blkno,
Anthony Iliopoulos d228bc
			buf_f->blf_len, buf_flags, &bp, NULL);
Anthony Iliopoulos d228bc
Anthony Iliopoulos d228bc
Skipping the buffer leaves its contents in memory unverified.  This sets
Anthony Iliopoulos d228bc
us up for a kernel crash because xfs_refcount_recover_cow_leftovers
Anthony Iliopoulos d228bc
reads the buffer (which is still around in XBF_DONE state, so no read
Anthony Iliopoulos d228bc
verification) and creates a refcountbt cursor of height 32.  This is
Anthony Iliopoulos d228bc
impossible so we run off the end of the cursor object and crash.
Anthony Iliopoulos d228bc
Anthony Iliopoulos d228bc
Fix this by invoking the verifier on all skipped buffers and aborting
Anthony Iliopoulos d228bc
log recovery if the ondisk buffer is corrupt.  It might be smarter to
Anthony Iliopoulos d228bc
force replay the log item atop the buffer and then see if it'll pass the
Anthony Iliopoulos d228bc
write verifier (like ext4 does) but for now let's go with the
Anthony Iliopoulos d228bc
conservative option where we stop immediately.
Anthony Iliopoulos d228bc
Anthony Iliopoulos d228bc
Link: https://syzkaller.appspot.com/bug?extid=7e9494b8b399902e994e
Anthony Iliopoulos d228bc
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Anthony Iliopoulos d228bc
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Anthony Iliopoulos d228bc
Signed-off-by: Dave Chinner <david@fromorbit.com>
Anthony Iliopoulos d228bc
Acked-by: Anthony Iliopoulos <ailiop@suse.com>
Anthony Iliopoulos d228bc
Anthony Iliopoulos d228bc
---
Anthony Iliopoulos d228bc
 fs/xfs/xfs_log_recover.c | 9 +++++++++
Anthony Iliopoulos d228bc
 1 file changed, 9 insertions(+)
Anthony Iliopoulos d228bc
Anthony Iliopoulos d228bc
diff --git a/fs/xfs/xfs_log_recover.c b/fs/xfs/xfs_log_recover.c
Anthony Iliopoulos d228bc
index de6aeb6f5dd4..ee6c71ea4194 100644
Anthony Iliopoulos d228bc
--- a/fs/xfs/xfs_log_recover.c
Anthony Iliopoulos d228bc
+++ b/fs/xfs/xfs_log_recover.c
Anthony Iliopoulos d228bc
@@ -2786,6 +2786,15 @@ xlog_recover_buffer_pass2(
Anthony Iliopoulos d228bc
 	if (lsn && lsn != -1 && XFS_LSN_CMP(lsn, current_lsn) >= 0) {
Anthony Iliopoulos d228bc
 		trace_xfs_log_recover_buf_skip(log, buf_f);
Anthony Iliopoulos d228bc
 		xlog_recover_validate_buf_type(mp, bp, buf_f, NULLCOMMITLSN);
Anthony Iliopoulos d228bc
+		/*
Anthony Iliopoulos d228bc
+		 * We're skipping replay of this buffer log item due to the log
Anthony Iliopoulos d228bc
+		 * item LSN being behind the ondisk buffer.  Verify the buffer
Anthony Iliopoulos d228bc
+		 * contents since we aren't going to run the write verifier.
Anthony Iliopoulos d228bc
+		 */
Anthony Iliopoulos d228bc
+		if (bp->b_ops) {
Anthony Iliopoulos d228bc
+			bp->b_ops->verify_read(bp);
Anthony Iliopoulos d228bc
+			error = bp->b_error;
Anthony Iliopoulos d228bc
+		}
Anthony Iliopoulos d228bc
 		goto out_release;
Anthony Iliopoulos d228bc
 	}
Anthony Iliopoulos d228bc
 
Anthony Iliopoulos d228bc
-- 
Anthony Iliopoulos d228bc
2.35.3
Anthony Iliopoulos d228bc