From b64194778a68d1d60b4c5de58e428235ebd48595 Mon Sep 17 00:00:00 2001
From: Peter Zijlstra <peterz@infradead.org>
Date: Thu, 18 Jan 2018 14:38:11 +0100
Subject: [PATCH] jump_label: Add branch hints to static_branch_{un,}likely()
References: bnc#1101669 optimise numa balancing for fast migrate
Patch-mainline: v4.16-rc1
Git-commit: 81dcf89f03204d7e0118018666cf362254e734b6
For some reason these were missing, I've not observed this patch
making a difference in the few code locations I checked, but this
makes sense.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Borislav Petkov <bp@suse.de>
Cc: Jason Baron <jbaron@akamai.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Mel Gorman <mgorman@suse.de>
---
include/linux/jump_label.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h
index 2afd74b9d844..1c4a0d182996 100644
--- a/include/linux/jump_label.h
+++ b/include/linux/jump_label.h
@@ -380,7 +380,7 @@ extern bool ____wrong_branch_error(void);
branch = !arch_static_branch_jump(&(x)->key, true); \
else \
branch = ____wrong_branch_error(); \
- branch; \
+ likely(branch); \
})
#define static_branch_unlikely(x) \
@@ -392,7 +392,7 @@ extern bool ____wrong_branch_error(void);
branch = arch_static_branch(&(x)->key, false); \
else \
branch = ____wrong_branch_error(); \
- branch; \
+ unlikely(branch); \
})
#else /* !HAVE_JUMP_LABEL */