From: Kim Phillips <kim.phillips@amd.com>
Date: Tue, 24 Jan 2023 10:33:15 -0600
Subject: x86/cpu, kvm: Move X86_FEATURE_LFENCE_RDTSC to its native leaf
Git-commit: 84168ae786f8a15a7eb0f79d34f20b8d261ce2f5
Patch-mainline: v6.3-rc1
References: git-fixes
The LFENCE always serializing feature bit was defined as scattered
LFENCE_RDTSC and its native leaf bit position open-coded for KVM. Add
it to its newly added CPUID leaf 0x80000021 EAX proper. With
LFENCE_RDTSC in its proper place, the kernel's set_cpu_cap() will
effectively synthesize the feature for KVM going forward.
Also, DE_CFG[1] doesn't need to be set on such CPUs anymore.
[ bp: Massage and merge diff from Sean. ]
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Acked-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/20230124163319.2277355-5-kim.phillips@amd.com
Acked-by: Nikolay Borisov <nik.borisov@suse.com>
---
arch/x86/include/asm/cpufeatures.h | 5 ++++-
arch/x86/kernel/cpu/amd.c | 2 +-
arch/x86/kvm/cpuid.c | 14 ++++++++++++--
3 files changed, 17 insertions(+), 4 deletions(-)
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -98,7 +98,7 @@
#define X86_FEATURE_SYSENTER32 ( 3*32+15) /* "" sysenter in ia32 userspace */
#define X86_FEATURE_REP_GOOD ( 3*32+16) /* rep microcode works well */
#define X86_FEATURE_MFENCE_RDTSC ( 3*32+17) /* "" Mfence synchronizes RDTSC */
-#define X86_FEATURE_LFENCE_RDTSC ( 3*32+18) /* "" Lfence synchronizes RDTSC */
+/* FREE was #define X86_FEATURE_LFENCE_RDTSC ( 3*32+18) "" Lfence synchronizes RDTSC */
#define X86_FEATURE_ACC_POWER ( 3*32+19) /* AMD Accumulated Power Mechanism */
#define X86_FEATURE_NOPL ( 3*32+20) /* The NOPL (0F 1F) instructions */
#define X86_FEATURE_ALWAYS ( 3*32+21) /* "" Always-present feature */
@@ -374,6 +374,9 @@
#define X86_FEATURE_ARCH_CAPABILITIES (18*32+29) /* IA32_ARCH_CAPABILITIES MSR (Intel) */
#define X86_FEATURE_SPEC_CTRL_SSBD (18*32+31) /* "" Speculative Store Bypass Disable */
+ /* AMD-defined Extended Feature 2 EAX, CPUID level 0x80000021 (EAX), word 20 */
+#define X86_FEATURE_LFENCE_RDTSC (20*32+ 2) /* "" LFENCE always serializing / synchronizes RDTSC */
+
/*
* BUG word(s)
*/
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -966,7 +966,7 @@ static void init_amd(struct cpuinfo_x86
if (c->x86 >= 0xf)
set_cpu_cap(c, X86_FEATURE_K8);
- if (cpu_has(c, X86_FEATURE_XMM2)) {
+ if (!cpu_has(c, X86_FEATURE_LFENCE_RDTSC) && cpu_has(c, X86_FEATURE_XMM2)) {
unsigned long long val;
int ret;
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -397,7 +397,7 @@ static inline int __do_cpuid_ent(struct
const u32 kvm_cpuid_8000_0021_eax_x86_features =
BIT(0) /* NO_NESTED_DATA_BP */ |
- BIT(2) /* LFENCE Always serializing */ | 0 /* SmmPgCfgLock */ |
+ F(LFENCE_RDTSC) /* LFENCE Always serializing */ | 0 /* SmmPgCfgLock */ |
BIT(6) /* NULL_SEL_CLR_BASE */ | 0 /* PrefetchCtlMsr */;
/* cpuid 0xC0000001.edx */
@@ -750,8 +750,18 @@ static inline int __do_cpuid_ent(struct
*/
entry->eax &= kvm_cpuid_8000_0021_eax_x86_features;
+ /*
+ * Synthesize "LFENCE is serializing" into the AMD-defined entry in
+ * KVM's supported CPUID if the feature is reported as supported by the
+ * kernel. LFENCE_RDTSC was a Linux-defined synthetic feature long
+ * before AMD joined the bandwagon, e.g. LFENCE is serializing on most
+ * CPUs that support SSE2. On CPUs that don't support AMD's leaf,
+ * kvm_cpu_cap_mask() will unfortunately drop the flag due to ANDing
+ * the mask with the raw host CPUID, and reporting support in AMD's
+ * leaf can make it easier for userspace to detect the feature.
+ */
if (cpu_feature_enabled(X86_FEATURE_LFENCE_RDTSC))
- entry->eax |= BIT(2);
+ entry->eax |= F(LFENCE_RDTSC);
if (!static_cpu_has_bug(X86_BUG_NULL_SEG))
entry->eax |= BIT(6);
break;