superrob1500
/
mirror-linux
mirror of https://github.com/torvalds/linux
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
mirror-linux/arch/x86/include/asm
History
Sebastian Andrzej Siewior 1c1e7e3c23 x86/percpu: Remove volatile from arch_raw_cpu_ptr(). 
The volatile attribute in the inline assembly of arch_raw_cpu_ptr()
forces the compiler to always generate the code, even if the compiler
can decide upfront that its result is not needed.

For instance invoking __intel_pmu_disable_all(false) (like
intel_pmu_snapshot_arch_branch_stack() does) leads to loading the
address of &cpu_hw_events into the register while compiler knows that it
has no need for it. This ends up with code like:

|	movq	$cpu_hw_events, %rax			#, tcp_ptr__
|	add	%gs:this_cpu_off(%rip), %rax		# this_cpu_off, tcp_ptr__
|	xorl	%eax, %eax				# tmp93

It also creates additional code within local_lock() with !RT &&
!LOCKDEP which is not desired.

By removing the volatile attribute the compiler can place the
function freely and avoid it if it is not needed in the end.
By using the function twice the compiler properly caches only the
variable offset and always loads the CPU-offset.

this_cpu_ptr() also remains properly placed within a preempt_disable()
sections because
- arch_raw_cpu_ptr() assembly has a memory input ("m" (this_cpu_off))
- prempt_{dis,en}able() fundamentally has a 'barrier()' in it

Therefore this_cpu_ptr() is already properly serialized and does not
rely on the 'volatile' attribute.

Remove volatile from arch_raw_cpu_ptr().

[ bigeasy: Added Linus' explanation why this_cpu_ptr() is not moved out
  of a preempt_disable() section without the 'volatile' attribute. ]

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/r/20220328145810.86783-2-bigeasy@linutronix.de
 		2022-04-05 09:59:38 +02:00
..
e820
…
fpu RISCV: 2022-01-16 16:15:14 +02:00
numachip
…
trace x86/fpu: Convert tracing to fpstate 2021-10-20 22:35:04 +02:00
uv
…
vdso clocksource/drivers/hyper-v: Re-enable VDSO_CLOCKMODE_HVCLOCK on X86 2021-05-14 14:55:13 +02:00
xen xen/x86: detect support for extended destination ID 2022-02-10 11:10:17 +01:00
GEN-for-each-reg.h x86/asm: Fix register order 2021-10-28 23:25:26 +02:00
Kbuild x86/syscalls: Switch to generic syscalltbl.sh 2021-05-20 15:03:58 +02:00
acenv.h
…
acpi.h
…
acrn.h
…
agp.h
…
alternative.h x86/alternative: Use .ibt_endbr_seal to seal indirect calls 2022-03-15 10:32:47 +01:00
amd-ibs.h perf/x86/amd/ibs: Add bitfield definitions in new <asm/amd-ibs.h> header 2021-08-26 09:14:36 +02:00
amd_hsmp.h platform/x86: Add AMD system management interface 2022-03-02 11:42:36 +01:00
amd_nb.h x86/amd_nb, EDAC/amd64: Move DF Indirect Read to AMD64 EDAC 2021-11-15 12:44:47 +01:00
apic.h x86/apic: Mark _all_ legacy interrupts when IO/APIC is missing 2021-05-29 11:41:14 +02:00
apicdef.h
…
apm.h
…
arch_hweight.h
…
archrandom.h
…
asm-offsets.h
…
asm-prototypes.h x86/retpoline: Move the retpoline thunk declarations to nospec-branch.h 2021-10-28 23:25:27 +02:00
asm.h x86/extable: Extend extable functionality 2021-12-11 09:09:46 +01:00
atomic.h locking/atomic: make ARCH_ATOMIC a Kconfig symbol 2021-05-26 13:20:49 +02:00
atomic64_32.h
…
atomic64_64.h
…
audit.h
…
barrier.h x86/barriers, kcsan: Use generic instrumentation for non-smp barriers 2021-12-09 16:42:28 -08:00
bios_ebda.h
…
bitops.h include: move find.h from asm_generic to linux 2022-01-15 08:47:31 -08:00
boot.h
…
bootparam_utils.h
…
bug.h x86,objtool: Move the ASM_REACHABLE annotation to objtool.h 2022-03-15 10:32:45 +01:00
bugs.h
…
cache.h
…
cacheflush.h
…
cacheinfo.h
…
ce4100.h
…
checksum.h
…
checksum_32.h
…
checksum_64.h
…
clocksource.h
…
cmdline.h
…
cmpxchg.h
…
cmpxchg_32.h
…
cmpxchg_64.h
…
coco.h x86/coco: Add API to handle encryption mask 2022-02-23 19:14:29 +01:00
compat.h arch: remove compat_alloc_user_space 2021-09-08 15:32:35 -07:00
cpu.h x86/ibt,kexec: Disable CET on kexec 2022-03-15 10:32:39 +01:00
cpu_device_id.h
…
cpu_entry_area.h x86/sev: Make the #VC exception stacks part of the default stacks storage 2021-10-06 21:48:27 +02:00
cpufeature.h x86/asm: Avoid adding register pressure for the init case in static_cpu_has() 2021-09-13 19:48:21 +02:00
cpufeatures.h Add support for Intel CET-IBT, available since Tigerlake (11th gen), which is a 2022-03-27 10:17:23 -07:00
cpuidle_haltpoll.h
…
cpumask.h cpumask: Add a x86-specific cpumask_clear_cpu() helper 2022-02-12 18:20:05 +01:00
crash.h x86/crash: Remove crash_reserve_low_1M() 2021-06-07 12:14:45 +02:00
current.h
…
debugreg.h
…
delay.h
…
desc.h Merge branch 'akpm' (patches from Andrew) 2021-07-02 12:08:10 -07:00
desc_defs.h
…
device.h
…
disabled-features.h x86/cpufeatures: Re-enable ENQCMD 2022-02-15 11:31:43 +01:00
div64.h
…
dma-mapping.h
…
dma.h
…
dmi.h
…
doublefault.h
…
dwarf2.h
…
edac.h
…
efi.h x86/ibt: Disable IBT around firmware 2022-03-15 10:32:40 +01:00
elf.h x86/elf: Support a new ELF aux vector AT_MINSIGSTKSZ 2021-05-19 12:18:45 +02:00
elfcore-compat.h
…
emergency-restart.h
…
emulate_prefix.h
…
enclu.h
…
entry-common.h x86/entry: Correct reference to intended CONFIG_64_BIT 2021-10-06 18:46:02 +02:00
espfix.h
…
exec.h
…
extable.h x86/extable: Extend extable functionality 2021-12-11 09:09:46 +01:00
extable_fixup_types.h x86/entry_32: Fix segment exceptions 2022-01-12 16:38:25 +01:00
fb.h
…
fixmap.h
…
floppy.h floppy: remove redundant assignment to variable st 2021-04-20 08:59:03 -06:00
frame.h
…
fsgsbase.h
…
ftrace.h x86/ftrace: Make function graph use ftrace directly 2021-10-20 23:44:43 -04:00
futex.h x86/futex: Remove .fixup usage 2021-12-11 09:09:47 +01:00
gart.h
…
genapic.h
…
geode.h
…
hardirq.h
…
highmem.h
…
hpet.h
…
hugetlb.h
…
hw_breakpoint.h
…
hw_irq.h
…
hyperv-tlfs.h PCI: hv: Make the code arch neutral by adding arch specific interfaces 2022-01-12 08:21:54 -06:00
hypervisor.h
…
i8259.h x86: Avoid magic number with ELCR register accesses 2021-08-10 23:31:43 +02:00
ia32.h x86: Fix misspelled Kconfig symbols 2021-10-05 21:48:30 +02:00
ia32_unistd.h
…
ibt.h x86/alternative: Use .ibt_endbr_seal to seal indirect calls 2022-03-15 10:32:47 +01:00
idtentry.h x86/ibt: Add IBT feature, MSR and #CP handling 2022-03-15 10:32:39 +01:00
imr.h
…
inat.h
…
inat_types.h
…
init.h
…
insn-eval.h - Get rid of all the .fixup sections because this generates 2022-01-12 16:31:19 -08:00
insn.h x86/insn: Add AVX512-FP16 instructions to the x86 instruction decoder 2022-01-23 20:38:01 +01:00
inst.h
…
intel-family.h x86/cpu: Drop spurious underscore from RAPTOR_LAKE #define 2021-11-30 14:05:48 -08:00
intel-mid.h
…
intel_ds.h perf/x86/intel: Enable PEBS format 5 2022-02-02 13:11:43 +01:00
intel_pconfig.h
…
intel_pt.h perf/x86/intel/pt: Add a capability and config bit for disabling TNTs 2022-02-15 17:47:11 +01:00
intel_punit_ipc.h
…
intel_scu_ipc.h
…
intel_telemetry.h
…
invpcid.h
…
io.h x86/sev: Use CC_ATTR attribute to generalize string I/O unroll 2021-12-08 16:49:42 +01:00
io_apic.h
…
io_bitmap.h
…
iomap.h
…
iommu.h
…
iommu_table.h
…
iosf_mbi.h
…
irq.h
…
irq_remapping.h
…
irq_stack.h x86,objtool: Move the ASM_REACHABLE annotation to objtool.h 2022-03-15 10:32:45 +01:00
irq_vectors.h x86/irq: Add and use NR_EXTERNAL_VECTORS and NR_SYSTEM_VECTORS 2021-05-21 12:36:44 +02:00
irq_work.h
…
irqdomain.h
…
irqflags.h x86/entry,xen: Early rewrite of restore_regs_and_return_to_kernel() 2022-03-15 10:32:34 +01:00
ist.h
…
jailhouse_para.h
…
jump_label.h jump_label, x86: Allow short NOPs 2021-05-12 14:54:56 +02:00
kasan.h
…
kaslr.h
…
kbdleds.h
…
kdebug.h
…
kexec-bzimage64.h
…
kexec.h x86/sme: Replace occurrences of sme_active() with cc_platform_has() 2021-10-04 11:46:46 +02:00
kfence.h kfence, x86: only define helpers if !MODULE 2021-07-30 17:09:01 +02:00
kgdb.h
…
kprobes.h kprobes: treewide: Remove trampoline_address from kretprobe_trampoline_handler() 2021-09-30 21:24:06 -04:00
kvm-x86-ops.h KVM: x86: do not use KVM_X86_OP_OPTIONAL_RET0 for get_mt_mask 2022-03-21 09:28:25 -04:00
kvm_host.h * Only do MSR filtering for MSRs accessed by rdmsr/wrmsr 2022-04-02 12:09:02 -07:00
kvm_page_track.h KVM: x86/mmu: Propagate memslot const qualifier 2021-12-08 04:24:43 -05:00
kvm_para.h x86/kvm: Add AMD SEV specific Hypercall3 2021-11-11 07:37:10 -05:00
kvm_types.h
…
kvm_vcpu_regs.h
…
kvmclock.h x86/kvmclock: Move this_cpu_pvti into kvmclock.h 2021-09-30 04:08:01 -04:00
linkage.h x86/linkage: Add ENDBR to SYM_FUNC_START*() 2022-03-15 10:32:36 +01:00
livepatch.h
…
local.h
…
mach_timer.h
…
mach_traps.h
…
math_emu.h
…
mc146818rtc.h
…
mce.h x86/MCE/AMD, EDAC/mce_amd: Support non-uniform MCA bank type enumeration 2021-12-22 17:22:09 +01:00
mem_encrypt.h Merge branch 'kvm-guest-sev-migration' into kvm-master 2021-11-11 07:40:26 -05:00
memtype.h
…
microcode.h x86/microcode: Use the firmware_loader built-in API 2021-10-22 14:13:50 +02:00
microcode_amd.h treewide: Replace zero-length arrays with flexible-array members 2022-02-17 07:00:39 -06:00
microcode_intel.h treewide: Replace zero-length arrays with flexible-array members 2022-02-17 07:00:39 -06:00
misc.h
…
mmconfig.h
…
mmu.h
…
mmu_context.h
…
mmx.h x86/mmx_32: Remove X86_USE_3DNOW 2021-12-11 09:09:45 +01:00
mmzone.h
…
mmzone_32.h
…
mmzone_64.h
…
module.h
…
mpspec.h
…
mpspec_def.h
…
mshyperv.h hyperv-next for 5.17 2022-01-16 15:53:00 +02:00
msi.h
…
msr-index.h Add support for Intel CET-IBT, available since Tigerlake (11th gen), which is a 2022-03-27 10:17:23 -07:00
msr-trace.h
…
msr.h x86/msr: Remove .fixup usage 2021-12-11 09:09:47 +01:00
mtrr.h x86/mtrr: Remove the mtrr_bp_init() stub 2021-12-22 19:50:26 +01:00
mwait.h
…
nmi.h
…
nops.h x86/asm: Use _ASM_BYTES() in <asm/nops.h> 2021-05-10 12:33:28 +02:00
nospec-branch.h x86/speculation: Add eIBRS + Retpoline options 2022-02-21 10:21:35 +01:00
numa.h
…
numa_32.h
…
olpc.h
…
olpc_ofw.h
…
orc_lookup.h
…
orc_types.h
…
page.h x86: Share definition of __is_canonical_address() 2022-02-02 13:11:42 +01:00
page_32.h x86/mmx_32: Remove X86_USE_3DNOW 2021-12-11 09:09:45 +01:00
page_32_types.h
…
page_64.h x86/mm: Add missing <asm/cpufeatures.h> dependency to <asm/page_64.h> 2021-12-03 09:30:45 -08:00
page_64_types.h x86: Increase exception stack sizes 2021-09-21 13:57:43 +02:00
page_types.h
…
paravirt.h x86/ibt,paravirt: Sprinkle ENDBR 2022-03-15 10:32:36 +01:00
paravirt_api_clock.h sched/headers: Add initial new headers as identity mappings 2022-02-23 10:58:28 +01:00
paravirt_types.h Add support for Intel CET-IBT, available since Tigerlake (11th gen), which is a 2022-03-27 10:17:23 -07:00
parport.h
…
pc-conf-reg.h x86: Add support for 0x22/0x23 port I/O configuration space 2021-08-10 23:31:43 +02:00
pci-direct.h
…
pci-functions.h
…
pci.h treewide: Replace zero-length arrays with flexible-array members 2022-02-17 07:00:39 -06:00
pci_x86.h pci-v5.18-changes 2022-03-25 13:02:05 -07:00
percpu.h x86/percpu: Remove volatile from arch_raw_cpu_ptr(). 2022-04-05 09:59:38 +02:00
perf_event.h perf/x86/intel: Increase max number of the fixed counters 2022-02-02 13:11:44 +01:00
perf_event_p4.h
…
pgalloc.h mm/thp: define default pmd_pgtable() 2021-07-01 11:06:03 -07:00
pgtable-2level.h
…
pgtable-2level_types.h
…
pgtable-3level.h
…
pgtable-3level_types.h
…
pgtable-invert.h
…
pgtable.h x86/coco: Add API to handle encryption mask 2022-02-23 19:14:29 +01:00
pgtable_32.h
…
pgtable_32_areas.h
…
pgtable_32_types.h
…
pgtable_64.h
…
pgtable_64_types.h
…
pgtable_areas.h
…
pgtable_types.h mm: define default value for FIRST_USER_ADDRESS 2021-07-01 11:06:02 -07:00
pkeys.h x86/fault: Fix wrong signal when vsyscall fails with pkey 2021-09-20 22:28:47 +02:00
pkru.h x86/pkey: Fix undefined behaviour with PKRU_WD_BIT 2021-12-19 22:44:34 +01:00
platform_sst_audio.h
…
pm-trace.h
…
posix_types.h
…
preempt.h sched/preempt: Refactor sched_dynamic_update() 2022-02-19 11:11:07 +01:00
probe_roms.h
…
processor-cyrix.h x86: Add support for 0x22/0x23 port I/O configuration space 2021-08-10 23:31:43 +02:00
processor-flags.h
…
processor.h Merge branch 'x86/cpu' into x86/core, to resolve conflicts 2022-03-15 12:52:51 +01:00
prom.h
…
proto.h x86/arch_prctl: Add controls for dynamic XSTATE components 2021-10-26 10:18:09 +02:00
pti.h
…
ptrace.h x86/ptrace: Always inline v8086_mode() for instrumentation 2022-02-12 22:07:13 +01:00
purgatory.h
…
pvclock-abi.h
…
pvclock.h
…
qrwlock.h
…
qspinlock.h x86/qspinlock, kcsan: Instrument barrier of pv_queued_spin_unlock() 2021-12-09 16:42:28 -08:00
qspinlock_paravirt.h x86/ibt,paravirt: Sprinkle ENDBR 2022-03-15 10:32:36 +01:00
realmode.h x86/mm: Flush global TLB when switching to trampoline page-table 2021-12-06 09:54:10 +01:00
reboot.h
…
reboot_fixups.h
…
required-features.h x86/mmx_32: Remove X86_USE_3DNOW 2021-12-11 09:09:45 +01:00
resctrl.h
…
rmwcc.h
…
seccomp.h
…
sections.h
…
segment.h x86/ibt,entry: Sprinkle ENDBR dust 2022-03-15 10:32:35 +01:00
serial.h
…
set_memory.h - Remove a misleading message and an unused function 2022-03-21 11:49:16 -07:00
setup.h x86/ibt: Dont generate ENDBR in .discard.text 2022-03-15 10:32:42 +01:00
setup_arch.h
…
sev-common.h - Share the SEV string unrolling logic with TDX as TDX guests need it too 2022-01-10 09:33:40 -08:00
sev.h x86/sev: Expose sev_es_ghcb_hv_call() for use by HyperV 2021-10-25 18:11:42 +02:00
sgx.h x86/sgx: Remove .fixup usage 2021-12-11 09:09:49 +01:00
shmparam.h
…
sigcontext.h
…
sigframe.h x86/signal: Introduce helpers to get the maximum signal frame size 2021-05-19 11:46:27 +02:00
sighandling.h
…
signal.h compat: make linux/compat.h available everywhere 2021-07-23 14:20:24 +01:00
simd.h
…
smap.h First big cleanup to the paravirt infra to use alternatives and thus 2021-04-26 09:01:29 -07:00
smp.h x86/smp: Factor out parts of native_smp_prepare_cpus() 2021-11-11 13:09:32 +01:00
softirq_stack.h
…
sparsemem.h
…
spec-ctrl.h
…
special_insns.h x86/asm: Fix SETZ size enqcmds() build failure 2021-09-22 19:45:48 +02:00
spinlock.h
…
spinlock_types.h
…
sta2x11.h
…
stackprotector.h x86: Fix leftover comment typos 2021-05-12 20:00:51 +02:00
stacktrace.h x86/mm/64: Improve stack overflow warnings 2021-09-21 13:57:43 +02:00
static_call.h static_call: Properly initialise DEFINE_STATIC_CALL_RET0() 2022-04-05 09:59:38 +02:00
string.h
…
string_32.h x86/mmx_32: Remove X86_USE_3DNOW 2021-12-11 09:09:45 +01:00
string_64.h
…
suspend.h
…
suspend_32.h
…
suspend_64.h
…
svm.h KVM: x86: SVM: move tsc ratio definitions to svm.h 2022-04-02 05:37:25 -04:00
swiotlb.h
…
switch_to.h
…
sync_bitops.h
…
sync_core.h
…
syscall.h arch: remove unused function syscall_set_arguments() 2021-09-14 16:06:20 +02:00
syscall_wrapper.h x86: Remove toolchain check for X32 ABI capability 2022-03-15 10:32:48 +01:00
syscalls.h
…
text-patching.h Add support for Intel CET-IBT, available since Tigerlake (11th gen), which is a 2022-03-27 10:17:23 -07:00
thermal.h x86/thermal: Fix LVT thermal setup for SMI delivery mode 2021-05-31 22:32:26 +02:00
thread_info.h x86: add CPU field to struct thread_info 2021-09-30 14:40:30 +02:00
time.h
…
timer.h
…
timex.h
…
tlb.h
…
tlbbatch.h
…
tlbflush.h x86/mm/64: Flush global TLB on boot and AP bringup 2021-12-06 09:38:48 +01:00
topology.h ACPI updates for 5.18-rc1 2022-03-21 14:17:20 -07:00
trace_clock.h
…
trap_pf.h
…
trapnr.h
…
traps.h x86/ibt: Add IBT feature, MSR and #CP handling 2022-03-15 10:32:39 +01:00
tsc.h
…
uaccess.h uaccess: generalize access_ok() 2022-02-25 09:36:05 +01:00
uaccess_32.h
…
uaccess_64.h arch: remove compat_alloc_user_space 2021-09-08 15:32:35 -07:00
umip.h
…
unistd.h x86/syscalls: Use __NR_syscalls instead of __NR_syscall_max 2021-05-20 15:03:59 +02:00
unwind.h x86,rethook,kprobes: Replace kretprobe with rethook on x86 2022-03-28 19:38:51 -07:00
unwind_hints.h x86/kprobes: Add UNWIND_HINT_FUNC on kretprobe_trampoline() 2021-09-30 21:24:07 -04:00
uprobes.h
…
user.h
…
user32.h
…
user_32.h asm/user.h: killed unused macros 2022-01-30 21:17:00 -05:00
user_64.h asm/user.h: killed unused macros 2022-01-30 21:17:00 -05:00
vdso.h x86: Remove toolchain check for X32 ABI capability 2022-03-15 10:32:48 +01:00
vermagic.h
…
vga.h
…
vgtod.h
…
virtext.h
…
vm86.h
…
vmalloc.h mm/vmalloc: provide fallback arch huge vmap support functions 2021-04-30 11:20:40 -07:00
vmware.h
…
vmx.h KVM: VMX: Add basic handling of VM-Exit from SGX enclave 2021-04-20 04:18:54 -04:00
vmxfeatures.h
…
vsyscall.h
…
vvar.h
…
word-at-a-time.h x86/word-at-a-time: Remove .fixup usage 2021-12-11 09:09:50 +01:00
x86_init.h x86/mm/cpa: Generalize __set_memory_enc_pgtable() 2022-02-23 19:14:29 +01:00
xor.h lib/xor: make xor prototypes more friendly to compiler vectorization 2022-02-11 20:39:39 +11:00
xor_32.h lib/xor: make xor prototypes more friendly to compiler vectorization 2022-02-11 20:39:39 +11:00
xor_64.h
…
xor_avx.h lib/xor: make xor prototypes more friendly to compiler vectorization 2022-02-11 20:39:39 +11:00