Johannes Weiner 3840cbe24c sched: psi: fix bogus pressure spikes from aggregation race ... Brandon reports sporadic, non-sensical spikes in cumulative pressure time (total=) when reading cpu.pressure at a high rate. This is due to a race condition between reader aggregation and tasks changing states. While it affects all states and all resources captured by PSI, in practice it most likely triggers with CPU pressure, since scheduling events are so frequent compared to other resource events. The race context is the live snooping of ongoing stalls during a pressure read. The read aggregates per-cpu records for stalls that have concluded, but will also incorporate ad-hoc the duration of any active state that hasn't been recorded yet. This is important to get timely measurements of ongoing stalls. Those ad-hoc samples are calculated on-the-fly up to the current time on that CPU; since the stall hasn't concluded, it's expected that this is the minimum amount of stall time that will enter the per-cpu records once it does. The problem is that the path that concludes the state uses a CPU clock read that is not synchronized against aggregators; the clock is read outside of the seqlock protection. This allows aggregators to race and snoop a stall with a longer duration than will actually be recorded. With the recorded stall time being less than the last snapshot remembered by the aggregator, a subsequent sample will underflow and observe a bogus delta value, resulting in an erratic jump in pressure. Fix this by moving the clock read of the state change into the seqlock protection. This ensures no aggregation can snoop live stalls past the time that's recorded when the state concludes. Reported-by: Brandon Duffany <brandon@buildbuddy.io> Link: https://bugzilla.kernel.org/show_bug.cgi?id=219194 Link: https://lore.kernel.org/lkml/20240827121851.GB438928@cmpxchg.org/ Fixes: df77430639 ("psi: Reduce calls to sched_clock() in psi") Cc: stable@vger.kernel.org Signed-off-by: Johannes Weiner <hannes@cmpxchg.org> Reviewed-by: Chengming Zhou <chengming.zhou@linux.dev> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>		2024-10-03 16:03:16 -07:00
..
bpf	move asm/unaligned.h to linux/unaligned.h	2024-10-02 17:23:23 -04:00
cgroup	struct fd layout change (and conversion to accessor helpers)	2024-09-23 09:35:36 -07:00
configs	tinyconfig: remove unnecessary 'is not set' for choice blocks	2024-09-01 20:34:38 +09:00
debug	move asm/unaligned.h to linux/unaligned.h	2024-10-02 17:23:23 -04:00
dma	dma-mapping: report unlimited DMA addressing in IOMMU DMA path	2024-09-23 08:38:56 +02:00
entry	treewide: context_tracking: Rename CONTEXT_* into CT_STATE_*	2024-07-29 07:33:10 +05:30
events	uprobes: fix kernel info leak via "[uprobes]" vma	2024-09-30 08:19:11 +09:00
futex	fault-inject: improve build for CONFIG_FAULT_INJECTION=n	2024-09-01 20:43:33 -07:00
gcov	gcov: add support for GCC 14	2024-06-15 10:43:06 -07:00
irq	pci-v6.12-changes	2024-09-23 12:47:06 -07:00
kcsan	kcsan: Use min() to fix Coccinelle warning	2024-08-01 16:40:44 -07:00
livepatch	livepatch: Replace snprintf() with sysfs_emit()	2024-07-02 16:56:18 +02:00
locking	Locking changes for v6.12:	2024-09-29 08:51:30 -07:00
module	Modules changes for v6.12-rc1	2024-09-28 09:06:15 -07:00
power	[tree-wide] finally take no_llseek out	2024-09-27 08:18:43 -07:00
printk	drm next for 6.12-rc1	2024-09-19 10:18:15 +02:00
rcu	slab updates for 6.12	2024-09-18 08:53:53 +02:00
sched	sched: psi: fix bogus pressure spikes from aggregation race	2024-10-03 16:03:16 -07:00
time	[tree-wide] finally take no_llseek out	2024-09-27 08:18:43 -07:00
trace	[tree-wide] finally take no_llseek out	2024-09-27 08:18:43 -07:00
.gitignore	…
Kconfig.freezer	…
Kconfig.hz	…
Kconfig.kexec	…
Kconfig.locks	…
Kconfig.preempt	sched_ext: Build fix on !CONFIG_STACKTRACE[_SUPPORT]	2024-08-01 07:08:01 -10:00
Makefile	mm: move kernel/numa.c to mm/	2024-09-03 21:15:26 -07:00
acct.c	kernel misc: Remove the now superfluous sentinel elements from ctl_table array	2024-04-24 09:43:53 +02:00
async.c	…
audit.c	audit: Make use of str_enabled_disabled() helper	2024-09-03 16:35:16 -04:00
audit.h	…
audit_fsnotify.c	…
audit_tree.c	fsnotify: create a wrapper fsnotify_find_inode_mark()	2024-04-04 16:24:16 +02:00
audit_watch.c	fsnotify: create a wrapper fsnotify_find_inode_mark()	2024-04-04 16:24:16 +02:00
auditfilter.c	audit: use task_tgid_nr() instead of task_pid_nr()	2024-08-28 16:48:28 -04:00
auditsc.c	audit: use task_tgid_nr() instead of task_pid_nr()	2024-08-28 16:48:28 -04:00
backtracetest.c	backtracetest: add MODULE_DESCRIPTION()	2024-06-24 22:24:55 -07:00
bounds.c	bounds: Use the right number of bits for power-of-two CONFIG_NR_CPUS	2024-04-29 08:29:29 -07:00
capability.c	…
cfi.c	…
compat.c	…
configs.c	…
context_tracking.c	context_tracking, rcu: Rename rcu_dyntick trace event into rcu_watching	2024-08-15 21:30:43 +05:30
cpu.c	Updates for timers and timekeeping:	2024-09-17 07:25:37 +02:00
cpu_pm.c	…
crash_core.c	Document/kexec: generalize crash hotplug description	2024-09-01 20:43:37 -07:00
crash_reserve.c	crash: fix crash memory reserve exceed system memory bug	2024-09-01 20:43:30 -07:00
cred.c	…
delayacct.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
dma.c	…
elfcorehdr.c	…
exec_domain.c	…
exit.c	ALong with the usual shower of singleton patches, notable patch series in	2024-09-21 07:29:05 -07:00
exit.h	…
extable.c	…
fail_function.c	…
fork.c	memblock: updates for 6.12-rc1	2024-09-25 11:35:19 -07:00
freezer.c	sched,freezer: Mark TASK_FROZEN special	2024-08-17 11:06:44 +02:00
gen_kheaders.sh	kheaders: use `command -v` to test for existence of `cpio`	2024-05-30 01:13:20 +09:00
groups.c	…
hung_task.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
iomem.c	…
irq_work.c	…
jump_label.c	jump_label: Fix static_key_slow_dec() yet again	2024-09-10 11:57:27 +02:00
kallsyms.c	kallsyms: Match symbols exactly with CONFIG_LTO_CLANG	2024-08-15 09:33:35 -07:00
kallsyms_internal.h	kallsyms: get rid of code for absolute kallsyms	2024-07-20 16:33:21 +09:00
kallsyms_selftest.c	kallsyms: Match symbols exactly with CONFIG_LTO_CLANG	2024-08-15 09:33:35 -07:00
kallsyms_selftest.h	…
kcmp.c	…
kcov.c	Updates for KCOV instrumentation on x86:	2024-09-17 12:40:34 +02:00
kexec.c	crash: add a new kexec flag for hotplug support	2024-04-23 14:59:01 +10:00
kexec_core.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
kexec_elf.c	…
kexec_file.c	kexec_file: fix elfcorehdr digest exclusion when CONFIG_CRASH_HOTPLUG=y	2024-09-01 17:59:01 -07:00
kexec_internal.h	kexec: use atomic_try_cmpxchg_acquire() in kexec_trylock()	2024-09-01 20:43:23 -07:00
kheaders.c	…
kprobes.c	kprobes: Fix to check symbol prefixes correctly	2024-08-05 14:04:03 +09:00
ksyms_common.c	…
ksysfs.c	profiling: remove prof_cpu_mask	2024-07-29 10:45:54 -07:00
kthread.c	kthread: Fix task state in kthread worker if being frozen	2024-09-10 09:51:14 +02:00
latencytop.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
module_signature.c	…
notifier.c	…
nsproxy.c	introduce fd_file(), convert all accessors to it.	2024-08-12 22:00:43 -04:00
padata.c	This update includes the following changes:	2024-09-16 06:28:28 +02:00
panic.c	drm next for 6.12-rc1	2024-09-19 10:18:15 +02:00
params.c	…
pid.c	introduce fd_file(), convert all accessors to it.	2024-08-12 22:00:43 -04:00
pid_namespace.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
pid_sysctl.h	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
profile.c	profiling: remove profile=sleep support	2024-08-04 13:36:28 -07:00
ptrace.c	…
range.c	…
reboot.c	kernel misc: Remove the now superfluous sentinel elements from ctl_table array	2024-04-24 09:43:53 +02:00
regset.c	regset: use kvzalloc() for regset_get_alloc()	2024-04-25 21:07:03 -07:00
relay.c	[tree-wide] finally take no_llseek out	2024-09-27 08:18:43 -07:00
resource.c	ALong with the usual shower of singleton patches, notable patch series in	2024-09-21 07:29:05 -07:00
resource_kunit.c	resource, kunit: add test case for region_intersects()	2024-09-17 01:07:00 -07:00
rseq.c	…
scftorture.c	scftorture: Make torture_type static	2024-05-30 15:31:51 -07:00
scs.c	…
seccomp.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
signal.c	Revert "binfmt_elf, coredump: Log the reason of the failed core dumps"	2024-09-26 11:39:02 -07:00
smp.c	smp: print only local CPU info when sched_clock goes backward	2024-08-15 00:06:48 +05:30
smpboot.c	…
smpboot.h	…
softirq.c	softirq: Remove unused 'action' parameter from action callback	2024-08-20 17:13:40 +02:00
stackleak.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
stacktrace.c	…
static_call.c	…
static_call_inline.c	static_call: Replace pointless WARN_ON() in static_call_module_notify()	2024-09-06 16:29:22 +02:00
stop_machine.c	rcu: Rename rcu_momentary_dyntick_idle() into rcu_momentary_eqs()	2024-08-15 21:30:42 +05:30
sys.c	struct fd layout change (and conversion to accessor helpers)	2024-09-23 09:35:36 -07:00
sys_ni.c	Probes updates for v6.11:	2024-07-18 12:19:20 -07:00
sysctl-test.c	sysctl: Add module description to sysctl-testing	2024-06-03 15:20:37 +02:00
sysctl.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
task_work.c	task_work: make TWA_NMI_CURRENT handling conditional on IRQ_WORK	2024-07-29 12:05:06 -07:00
taskstats.c	introduce fd_file(), convert all accessors to it.	2024-08-12 22:00:43 -04:00
torture.c	torture: Add MODULE_DESCRIPTION()	2024-05-30 15:31:38 -07:00
tracepoint.c	tracepoint: Support iterating tracepoints in a loading module	2024-09-25 23:23:44 +09:00
tsacct.c	tsacct: replace strncpy() with strscpy()	2024-07-12 16:39:53 -07:00
ucount.c	sysctl changes for v6.10-rc1	2024-05-17 17:31:24 -07:00
uid16.c	…
uid16.h	…
umh.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
up.c	…
user-return-notifier.c	…
user.c	uidgid: make sure we fit into one cacheline	2024-09-12 12:16:09 +02:00
user_namespace.c	user_namespace: use kmemdup_array() instead of kmemdup() for multiple allocation	2024-09-09 16:47:42 -07:00
usermode_driver.c	…
utsname.c	…
utsname_sysctl.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
vhost_task.c	vhost_task: Handle SIGKILL by flushing work and exiting	2024-05-22 08:31:15 -04:00
vmcore_info.c	mm: support only one page_type per page	2024-09-03 21:15:43 -07:00
watch_queue.c	introduce fd_file(), convert all accessors to it.	2024-08-12 22:00:43 -04:00
watchdog.c	watchdog: handle the ENODEV failure case of lockup_detector_delay_init() separately	2024-09-01 20:43:32 -07:00
watchdog_buddy.c	…
watchdog_perf.c	watchdog/perf: properly initialize the turbo mode timestamp and rearm counter	2024-07-17 21:11:34 -07:00
workqueue.c	workqueue: Changes for v6.12	2024-09-18 06:59:44 +02:00
workqueue_internal.h	…