We allocate space for added counters ahead of time,
and we arbitrarily chose 24 added counters of each type.
If a future code flow hits this limit and continues,
and then removes counters, we do a phantom increment
and lose track of the number of added counters.
Increment the count only if there is no error.
Fixes: 388e9c8134 ("tools/power turbostat: Make extensible via the --add parameter")
Assisted-by: Claude: Claude Sonnet 4.6
Signed-off-by: Len Brown <len.brown@intel.com>
It is possible to leak a file descriptor if a diretory
open succeeds, but the scan fails. Close the fd on error.
Also, on cleanup, check for NULL fd.
Assisted-by: Claude: Claude Sonnet 4.6
Fixes: 4265a86582 ("tools/power turbostat: Add PMT directory iterator helper")
Signed-off-by: Len Brown <len.brown@intel.com>
Fix off-by-one error causing last CPU on CWF
to never get a pmt counter allocated.
Fixes: 5ce1e9bbb2 ("tools/power turbostat: Add CPU%c1e BIC for CWF")
Assisted-by: GitHub Copilot (claude-sonnet-4.6)
Signed-off-by: Len Brown <len.brown@intel.com>
When turbostat notices a topology change,
it frees perf file descriptors for each CPU
to prepare to re-initialize.
Fix an off-by-one error in that per-cpu loop that
resulted in a file descriptor leak of the file for
the highest numbered CPU in the system.
Fixes: 67bab430f4 ("tools/power turbostat: Group SMI counter with APERF and MPERF")
Assisted-by: GitHub Copilot (claude-sonnet-4.6)
Signed-off-by: Len Brown <len.brown@intel.com>
When measuring over a suspend cycle, GRX%rc6 and SAM%mc6
are observed to print as numbers much greater than 100%.
Those numbers are not useful, and they mess up the columns.
Detect when they are out of range, and print them as NaN.
Signed-off-by: Len Brown <len.brown@intel.com>
Three calloc() calls in turbostat lack NULL return checks, leading to
potential NULL pointer dereferences on allocation failure:
1. rapl_perf_init(): the allocated `domain_visited` is used later via
memset(domain_visited, 0, ...) without checking for NULL.
2. added_perf_counters_init_(): the allocated `domain_visited` is used
later via memset() and domain_visited[next_domain] without checking
for NULL.
3. pmt_add_counter(): the allocated `pcounter` is dereferenced
immediately via strncpy(pcounter->name, ...) without checking for
NULL.
Add NULL checks after each calloc(), using the same error handling
style already present in each function: err(-1, ...) for rapl_perf_init(),
errx(1, ...) for added_perf_counters_init_(), and return 1 for
pmt_add_counter().
Signed-off-by: longlong yan <yanlonglong@kylinos.cn>
Signed-off-by: Len Brown <len.brown@intel.com>
Some systems in the field have perf enabled,
but their kernel perf support lacks the latest L2 stats.
Warn, but don't Err on those systems, and suggest how to avoid the Warn.
Fixes: dd23bfe4c3 ("tools/power turbostat: Add L2 cache statistics")
Reported-by: David Arcari <darcari@redhat.com>
Signed-off-by: Len Brown <len.brown@intel.com>
CPU_SUBSET_MAXCPUS accepts CPU IDs through 8191, but the sysfs and
cgroup CPU-list readers stop at 1024 bytes. A fragmented cpuset or
hybrid PMU CPU list can exceed that size and be truncated.
initialize_cpu_set_from_sysfs() can also pass a full, non-NUL-terminated
buffer to parse_cpu_str(), causing an out-of-bounds read.
Additionally, initialize_cpu_set_from_sysfs() returns with the file
still open on success. It is now called once per CPU for
thread_siblings_list, so large systems can leak thousands of descriptors
during topology discovery.
Use getline() so both readers receive a complete, NUL-terminated line.
Close and free the sysfs CPU-list resources on every path, and report
malformed input from the correct buffer.
Fixes: eb187540d1 ("tools/power turbostat: Increase CPU_SUBSET_MAXCPUS to 8192")
Fixes: 58839fdbd4 ("tools/power turbostat: Process HT siblings in CPU order")
Cc: stable@vger.kernel.org
Signed-off-by: Ali Ahmet Memis <ali@iusegentoo.com>
Signed-off-by: Len Brown <len.brown@intel.com>
Since v2026.02.14
Display HT siblings in cpu# order.
Add Module-ID column.
Print Core-ID and APIC-ID in hex.
Fix misc bugs.
Signed-off-by: Len Brown <len.brown@intel.com>
On large systems with HT sibling cpu#'s more than 32 apart,
HT siblings were processed and displayed in reverse order.
This was due to how set_thread_siblings() parsed the
sibling-bit-mask.
Update set_thread_siblings to instead parse the sibling-list,
like other cpu lists, and to thus order HT siblings
by ascending CPU number, no matter the size of the system.
Signed-off-by: Len Brown <len.brown@intel.com>
Get the "module_id" from the Linux topology "cluster_id".
If the there is more than one id, show it by default.
Module joins Die etc. in the "topology" group.
Display in hex, as it is usually based mask of the APIC-id
Signed-off-by: Len Brown <len.brown@intel.com>
The core_id is based on a mask of the apic_id.
Print them both in hex, rather than decimal,
to make this relationship visibly clear.
Signed-off-by: Len Brown <len.brown@intel.com>
Make printer helper functions more readable by factoring
out a local 'sep' variable.
Remove the redundant parentheses around sprintf() calls.
Remove an unnecessary cast to "unsigned int" by using the '%08llx' instead
of '%08x'.
No functional changes.
[lenb: fix typos, simplify]
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>
When the "--cpu-set" option limits turbostat to run on
a higher numbered HT sibling, it exits upon dividing by zero.
This is because the HT support handles higher numbered siblings
at the same time as lower numbered siblings. But when that lower
number sibling is dis-allowed, the higher numbered sibling is
never processed. The result is a time delta of 0, which results
in a divide by 0 for any of the "per-second" metrics.
Enhance the HT enumeration code to record all siblings (up to SMT4).
Consult this complete HT sibling list to determine when
to process an HT sibling, and when to skip it.
Fixes: a2b4d0f8bf ("tools/power turbostat: Favor cpu# over core#")
Signed-off-by: Len Brown <len.brown@intel.com>
"turbostat --cpu-set 0" appears to hang if cpu0 has an HT sibling.
This is because the initialization code recognizes that it does not
have to open perf files for the HT sibling, but the HT support
in the collection code sees the HT sibling and tries to read
from an uninitialized file descriptor, 0 (standard input).
Access HT siblings only when they are in the allowed set.
Fixes: a2b4d0f8bf ("tools/power turbostat: Favor cpu# over core#")
Signed-off-by: Len Brown <len.brown@intel.com>
Reported-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
The '-P' short option (shorthand for --no-perf) is not present in the
optstring of the second call to getopt_long_only(). This results in
the "unrecognized option" error when the tool reaches the main parsing
loop.
Add 'P' to the second getopt_long_only() call to ensure it is
consistently recognized.
Fixes: a0e86c90b8 ("tools/power turbostat: Add --no-perf option")
Signed-off-by: David Arcari <darcari@redhat.com>
Signed-off-by: Len Brown <len.brown@intel.com>
turbostat.c:8688: rapl_perf_init: Assertion `next_domain < num_domains' failed.
The initial fix for this regression was incomplete, as it did not
handle multi-package systems with sparse core ids.
Fixes: ef0e60083f ("tools/power turbostat: Fix AMD RAPL regression")
Signed-off-by: Len Brown <len.brown@intel.com>
Currently, if perf_l2_init() fails turbostat exits after issuing the
following error (which was encountered on AlderLake):
turbostat: perf_l2_init(cpu0, 0x0, 0xff24) REFS: Invalid argument
This occurs because perf_l2_init() calls err(). However, the code has been
written in such a manner that it is able to perform cleanup and continue.
Therefore, this issue can be addressed by changing the appropriate calls
to err() to warnx().
Additionally, correct the PMU type arguments passed to the warning strings
in the ecore and lcore blocks so the logs accurately reflect the failing
counter type.
Signed-off-by: David Arcari <darcari@redhat.com>
Signed-off-by: Len Brown <len.brown@intel.com>
Commands that add counters, such as 'turbostat --show C1,C1+'
display merged columns without a delimiter.
This is caused by the bad syntax: '(*printed++ ? delim : "")', shared by
print_name()/print_hex_value()/print_decimal_value()/print_float_value()
Use '((*printed)++ ? delim : "")' to correctly increment the value at *printed.
[lenb: fix code and commit message typo, re-word]
Fixes: 56dbb87850 ("tools/power turbostat: Refactor added column header printing")
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>
Problem: individual swidle counter names (C1, C1+, C1-, etc.) cannot be
selected via --show/--hide due to two bugs in probe_cpuidle_counts():
1. The function returns immediately when BIC_cpuidle is not enabled,
without checking deferred_add_index.
2. The deferred name check runs against name_buf before the trailing
newline is stripped, so is_deferred_add("C1\n") never matches "C1".
Fix:
1. Relax the early return to pass through when deferred names are
queued.
2. Strip the trailing newline from name_buf before performing deferred
name checks.
3. Check each suffixed variant (C1+, C1, C1-) individually so that
e.g. "--show C1+" enables only the requested metric.
In addition, introduce a helper function to avoid repeating the
condition (readability cleanup).
Fixes: ec4acd3166 ("tools/power turbostat: disable "cpuidle" invocation counters, by default")
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>
In the perf thread, core, and package counter loops, an incorrect
'mp->format' variable is used instead of 'pp->format'.
[lenb: edit commit message]
Fixes: 696d15cbd8 ("tools/power turbostat: Refactor floating point printout code")
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>
Fix the PMT thread code to use print_float_value(),
to be consistent with the PMT core and package code.
[lenb: commit message]
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>
turbostat always used the same logic to read the microcode patch level,
which is correct for Intel but not for AMD/Hygon.
While Intel stores the patch level in the upper 32 bits of MSR, AMD
stores it in the lower 32 bits, which causes turbostat to report the
microcode version as 0x0 on AMD/Hygon.
Fix by shifting right by 32 for non-AMD/Hygon, preserving the existing
behavior for Intel and unknown vendors.
Fixes: 3e4048466c ("tools/power turbostat: Add --no-msr option")
Signed-off-by: Serhii Pievniev <spevnev16@gmail.com>
Signed-off-by: Len Brown <len.brown@intel.com>
Linux core_id's are a per-package namespace, not a per-node namespace.
Rename topo.cores_per_node to topo.cores_per_pkg to reflect this.
Eliminate topo.nodes_per_pkg from the sizing for core data structures,
since it has no role except to unnecessarily bloat the allocation.
Validated on multiple Intel platforms (ICX/SPR/SRF/EMR/GNR/CWF) with
various CPU online/offline configurations and SMT enabled/disabled
scenarios.
No functional changes.
[lenb: commit message]
Signed-off-by: Zhang Rui <rui.zhang@intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>
I changed my mind about displaying swidle statistics,
which are "added counters". Recently I reverted the
column headers to 8-columns, but kept print_decimal_value()
padding out to 16-columns for all 64-bit counters.
Simplify by keeping print_decimial_value() at %lld -- which
will often fit into 8-columns, and live with the fact
that it can overflow and shift the other columns,
which continue to tab-delimited.
This is a better compromise than inserting a bunch
of space characters that most users don't like.
Fixes: 1a23ba6a1b ("tools/power turbostat: Print wide names only for RAW 64-bit columns")
Reported-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>
When SMT is present and disabled, turbostat may under-size
the thread_data array. This can corrupt results or
cause turbostat to exit with a segmentation fault.
[lenb: commit message]
Fixes: a2b4d0f8bf ("tools/power turbostat: Favor cpu# over core#")
Signed-off-by: Zhang Rui <rui.zhang@intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>
turbostat.c:8688: rapl_perf_init: Assertion `next_domain < num_domains' failed.
Two recent cleanup patches that were not supposed to change anything
broke the core_id code needed for AMD RAPL initialization:
commit 070e92361e ("tools/power turbostat: Enhance HT enumeration")
commit ddf60e38ca ("tools/power turbostat: Simplify global core_id calculation")
Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
Since release 2025.12.02:
Add L2 statistics columns for recent Intel processors:
L2MRPS = L2 Cache M-References Per Second
L2%hit = L2 Cache Hit %
Sort work and output by cpu# rather than core#
This commit:
Version number and white space (indent -l160)
No functional change.
Signed-off-by: Len Brown <len.brown@intel.com>
The "header_iterations" option is commonly used to de-clutter
the screen of redundant header label rows in an interactive session:
Eg. every 10 rows:
$ sudo turbostat --header_iterations 10 -S -q -i 1
But --header_iterations was missing from turbostat.8
Also turbostat help advertised the "-N" short option
that did not actually work:
$ turbostat --help
-N, --header_iterations num
print header every num iterations
Repair "-N"
Document "--header_iterations" on turbostat.8
Signed-off-by: Len Brown <len.brown@intel.com>
Replace strtod() with strtoul() and check errno for -n/-N options, since
num_iterations and header_iterations are unsigned long counters. Reject
zero and conversion errors; negative inputs wrap to large positive values
per standard unsigned semantics.
Signed-off-by: Kaushlendra Kumar <kaushlendra.kumar@intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>
Turbostat collects statistics and outputs results in "topology order",
which means it prioritizes the core# over the cpu#.
The strategy is to minimize wakesups to a core -- which is
important when measuring an idle system.
But core order is problematic, because Linux core#'s are physical
(within each package), and thus subject to APIC-id scrambling
that may be done by the hardware or the BIOS.
As a result users may be are faced with rows in a confusing order:
sudo turbostat -q --show topology,Busy%,CPU%c6,UncMHz sleep 1
Core CPU Busy% CPU%c6 UncMHz
- - 1.25 72.18 3400
0 4 7.74 0.00
1 5 1.77 88.59
2 6 0.48 96.73
3 7 0.21 98.34
4 8 0.14 96.85
5 9 0.26 97.55
6 10 0.44 97.24
7 11 0.12 96.18
8 0 5.41 0.31 3400
8 1 0.19
12 2 0.41 0.22
12 3 0.08
32 12 0.04 99.21
33 13 0.25 94.92
Abandon the legacy "core# topology order" in favor of simply
ordering by cpu#, with a special case to handle HT siblings
that may not have adjacent cpu#'s.
sudo ./turbostat -q --show topology,Busy%,CPU%c6,UncMHz sleep 1
1.003001 sec
Core CPU Busy% CPU%c6 UncMHz
- - 1.38 80.55 1600
8 0 10.94 0.00 1600
8 1 0.53
12 2 2.90 0.45
12 3 0.11
0 4 1.96 91.20
1 5 0.97 96.40
2 6 0.24 94.72
3 7 0.31 98.01
4 8 0.20 98.20
5 9 0.62 96.00
6 10 0.06 98.15
7 11 0.12 99.31
32 12 0.04 99.07
33 13 0.27 95.09
The result is that cpu#'s now take precedence over core#'s.
Signed-off-by: Len Brown <len.brown@intel.com>
Record the cpu_id of each CPU HT sibling -- will need this later.
Rename "thread_id" to "ht_id" to disambiguate that the scope
of this id is within a Core -- it is not a global cpu_id.
No functional change.
Signed-off-by: Len Brown <len.brown@intel.com>
Standardize the generation of globally unique core_id's
in a macro, and simplify the related code.
No functional change.
Signed-off-by: Len Brown <len.brown@intel.com>
Update the syntax of accesses to the even and odd counters
to match the average counters.
No functional change.
Signed-off-by: Len Brown <len.brown@intel.com>
The current static definition of average{} is inconsistent with
the dynamically allocated even{} and odd{} counters.
Allocate average{} counters dynamically.
No functional change.
Signed-off-by: Len Brown <len.brown@intel.com>
Delete redundant core_data.core_id.
Use cpus[].core_id as the single copy of the truth.
No functional change.
Signed-off-by: Len Brown <len.brown@intel.com>
The Linux Kernel topology sysfs is flawed.
core_id is not globally unique, but is per-package.
Turbostat works around this when it needs to, with
rapl_core_id = cpus[cpu].core_id;
rapl_core_id += cpus[cpu].package_id * nr_cores_per_package
Otherwise, turbostat handles core_id as subservient to each package.
As there is only one core_id namespace, rename
physical_core_id to simply be core_id.
No functional change.
Signed-off-by: Len Brown <len.brown@intel.com>
The kernel topology sysfs uses the name "physical_package_id"
because it is allowed to be sparse.
Inside Turbostat, that physical package_id namespace is the only
package_id namespace, so re-name it to simply be "package_id"
in cpus[].
Delete the redundant copy of package_id in pkg_data.
Rely instead on the single copy of the truth in cpus[].
No functional change.
Signed-off-by: Len Brown <len.brown@intel.com>
Disambiguate the uses "base_cpu":
master_cpu: lowest permitted cpu#, read global MSRs here
package_data.first_cpu: lowest permitted cpu# in that package
core_data.first_cpu: lowest permitted cpu# in the core
current_cpu: where I'm running now
No functional change.
Signed-off-by: Len Brown <len.brown@intel.com>
version 2026.02.04
Add support for L2 cache statistics: L2MRPS and L2%hit
L2 statistics join the LLC in the "cache" counter group.
While the underlying LLC perf kernel support was architectural,
L2 perf counters are model-specific:
Support Intel Xeon -- Sapphire Rapids and newer.
Support Intel Atom -- Gracemont and newer.
Support Intel Hybrid -- Alder Lake and newer.
Example:
alder-lake-n$ sudo turbostat --quiet --show CPU,Busy%,cache my_workload
CPU Busy% LLCMRPS LLC%hit L2MRPS L2%hit
- 49.82 1210 85.02 2909 31.63
0 99.14 322 88.89 767 32.38
1 0.91 1 32.47 1 18.86
2 0.20 0 40.78 0 23.34
3 99.17 295 81.79 706 31.89
4 0.68 1 58.71 1 15.61
5 99.16 299 85.65 726 31.32
6 0.08 0 45.35 0 31.71
7 99.21 293 83.63 707 30.92
where "my_workload" is a wrapper for a yogini workload
that has 4 fully-busy threads with 2MB working set each.
Note that analogous to the system summary for multiple LLC systems,
the system summary row for the L2 is the aggregate of all CPUS in the
system -- there is no per-cache roll-up.
Signed-off-by: Len Brown <len.brown@intel.com>
The "is_hybrid" is set and used only in !quiet mode.
Make it valid in both quiet and !quiet mode to allow more uses.
Signed-off-by: Len Brown <len.brown@intel.com>
The purpose of the LLC References per Second LLC column
is to qualify the significance of the LLC%hit column.
If RPS is high, then the hit rate matters.
If RPS is low, then the hit rate is not significant.
Remove unnecessary and distracting precision in the RPS column
by dividing my a million rather than by a thousand.
Signed-off-by: Len Brown <len.brown@intel.com>
Starting in turbostat v2025.01.14, turbostat refused to run
on unsupported hardware, pointing to "RUN THE LATEST VERSION"
on turbostat(8).
At that time, turbostat supported and advertised the "--force"
parameter to run anyway (with unsupported results).
Also document "--force" on turbostat.8.
Signed-off-by: Len Brown <len.brown@intel.com>
Divide-by-zero resulted if LLC references == 0
Pull the percentage division into pct() to centralize sanity checks there.
Fixes: 8808292799 ("tools/power turbostat: Print "nan" for out of range percentages")
Signed-off-by: Len Brown <len.brown@intel.com>
Both Intel and AMD use CPUID.1.ECX[31] to distinguish
between hypervisor and real hardware.
Indicate "HV" or "No-HV" accordingly.
Signed-off-by: Len Brown <len.brown@intel.com>
We dumped selected CPUID(1) features using a format that showed '-'
for a missing feature. Not so friendly to parse a bunch of dashes
when features are missing...
For CPUID(1) adopt the format we used for CPUID(6): 'No-FEATURE'
means that 'FEATURE' is not present.
Signed-off-by: Len Brown <len.brown@intel.com>
