Commit Graph

1175 Commits

Author SHA1 Message Date
Puranjay Mohan
24b620729e perf/core: Fill branch entries with a single assignment
perf_clear_branch_entry_bitfields() clears the bitfields of struct
perf_branch_entry one by one and leaves from/to alone, since callers
overwrite those straight away. The list has to be kept in sync with the
struct by hand and has already fallen behind: new_type and priv were
added to perf_branch_entry and never added here.

Only BRBE writes those two, and neither for every record.
brbe_set_perf_entry_type() leaves new_type alone for a branch type it
does not recognise, and priv is not set for source-only records.
arm_pmuv3.c allocates the per-CPU branch stack with kmalloc(), so such a
record reaches userspace with whatever the slot held: uninitialised
kmalloc() data on the first pass over the buffer, the previous record's
values after that. Nothing under arch/x86/events/ writes either field,
so only arm64 is affected.

Assign the whole entry at each site instead. Everything not named is
then zero, and there is no list to keep in sync. The bitfields add up to
exactly 64 bits, so the struct has no padding to leave undefined.

perf_clear_branch_entry_bitfields() has no callers left, so remove it.
perf_entry_from_brbe_regset() assigns an empty literal instead, since it
fills from/to conditionally. PERF_BR_SPEC_NA is 0, so dropping the
explicit spec assignment changes nothing.

Fixes: b190bc4ac9 ("perf: Extend branch type classification")
Fixes: 5402d25aa5 ("perf: Capture branch privilege information")
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Yifan Wu <wuyifan50@huawei.com>
Link: https://patch.msgid.link/20260810133540.1947118-4-puranjay@kernel.org
2026-09-23 11:48:36 +02:00
Dapeng Mi
d4d9ccbad5 perf/x86/intel: Rename NVL offcore_rsp attribute to offmodule_rsp
NVL introduces Offmodule Response events in place of the legacy
Offcore Response events, but it still exposes the inherited
offcore_rsp PMU attribute for programming the corresponding MSR data.

Rename the NVL PMU attribute to offmodule_rsp so the sysfs interface
matches the underlying event name and avoids user & tooling confusion.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260917015234.981153-13-dapeng1.mi@linux.intel.com
2026-09-22 11:38:17 +02:00
Dapeng Mi
0102c8c7fd perf/x86/intel: Rename DMR offcore_rsp attribute to offmodule_rsp
DMR introduces Offmodule Response events in place of the legacy
Offcore Response events, but it still exposes the inherited
offcore_rsp PMU attribute for programming the corresponding MSR data.

Rename the DMR PMU attribute to offmodule_rsp so the sysfs interface
matches the underlying event name and avoids user & tooling confusion.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260917015234.981153-12-dapeng1.mi@linux.intel.com
2026-09-22 11:37:46 +02:00
Dapeng Mi
ff1621adfd perf/x86/intel: Fix precise OMR event scheduling for DMR/NVL
The latest perfmon event database introduces below precise OMR event
support for DMR/NVL:

- MEM_LOAD_L2_MISS_RETIRED.* (event 0xd6)
- MEM_STORE_L2_MISS_RETIRED.* (event 0x4f)

These events use the same OMR MSRs as the existing OMR events, but
they are not listed in intel_pnc_extra_regs[]. As a result, perf
cannot assign the required OMR extra registers when scheduling them.

Add the new precise OMR events to intel_pnc_extra_regs[] so they can
be scheduled with the correct OMR MSRs. MEM_LOAD_L2_MISS_RETIRED.*
remains limited to GP counters 0-3, while
MEM_STORE_L2_MISS_RETIRED.* is available on all GP counters.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260917015234.981153-11-dapeng1.mi@linux.intel.com
2026-09-22 11:37:00 +02:00
Dapeng Mi
04a7ef3b7a perf/x86/intel: Constrain Panther Cove UOPS_DISPATCHED events to PMCs 0-3
Per the latest Panther Cove event definitions, the following events are
only supported on PMCs 0-3:

- UOPS_DISPATCHED.INT_EU_ALL (0x1b2)
- UOPS_DISPATCHED.ALU (0x2b2)

Add explicit event constraints for these two events so scheduling does
not place them on unsupported counters.

Fixes: d345b6bb88 ("perf/x86/intel: Add core PMU support for DMR")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: <stable@vger.kernel.org> # v7.0+
Link: https://patch.msgid.link/20260917015234.981153-10-dapeng1.mi@linux.intel.com
2026-09-22 11:35:40 +02:00
Dapeng Mi
858b37ca19 perf/x86/intel: Delete dead NVL PEBS data-source initcall
Nova Lake now uses the OMR data-source table for PEBS data-source
decoding and no longer depends on the legacy static pebs_data_source[]
mapping.

Remove the dead intel_pmu_pebs_data_source_lnl() initialization call
for NVL.

Fixes: c847a208f4 ("perf/x86/intel: Add core PMU support for Novalake")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260917015234.981153-9-dapeng1.mi@linux.intel.com
2026-09-22 11:33:49 +02:00
Dapeng Mi
0ac5d6ca2c perf/x86/intel: Fix Panther Cove PEBS data-source snoop states
For Panther Cove, the snoop states for the data source encodings
"Prefetch Promotion" and "Cross Core Prefetch Promotion" should be
SNOOP_NONE instead of SNOOP_MISS.

Fix the incorrect snooping states for Panther Cove.

Fixes: d2bdcde962 ("perf/x86/intel: Add support for PEBS memory auxiliary info field in DMR")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: <stable@vger.kernel.org> # v7.0+
Link: https://patch.msgid.link/20260917015234.981153-8-dapeng1.mi@linux.intel.com
2026-09-22 11:33:06 +02:00
Dapeng Mi
9f93d33ad6 perf/x86/intel: Remove incorrect Panther Cove PEBS data-source constraints
Same issue exists on Panther Cove, PEBS data source is valid only for
these events:

- MEM_TRANS_RETIRED.LOAD_LATENCY (0x1cd)
- MEM_TRANS_RETIRED.STORE_SAMPLE (0x2cd)

The perfmon database (https://github.com/intel/perfmon) previously
tagged additional memory events such as MEM_INST_RETIRED.STLB_MISS_LOADS
with L1_Hit_Indication, implying PEBS data-source support, which is
incorrect. The database has since been fixed, but
intel_pnc_pebs_event_constraints[] still follows the old definition and
marks those events as data-source capable.

As a result, get_data_src() may decode data-source information for
events that do not provide valid PEBS data-source data and mislead
users.

Remove those non-data-source memory events from the Pather Cove PEBS
constraint table so matching falls back to the regular non-PEBS
constraints, which already provide the same counter constraints.

Also update pnc_latency_data() to decode LOAD/STORE flags explicitly
when setting memory operation direction, for consistency with other
*_latency_data() helpers.

Fixes: d345b6bb88 ("perf/x86/intel: Add core PMU support for DMR")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: <stable@vger.kernel.org> # v7.0+
Link: https://patch.msgid.link/20260917015234.981153-7-dapeng1.mi@linux.intel.com
2026-09-22 11:31:40 +02:00
Dapeng Mi
7c944595cc perf/x86/intel: Remove incorrect LionCove PEBS data-source constraints
On Lion Cove, PEBS data source is valid only for these events:

- MEM_TRANS_RETIRED.LOAD_LATENCY (0x1cd)
- MEM_TRANS_RETIRED.STORE_SAMPLE (0x2cd)

The perfmon database (https://github.com/intel/perfmon) previously
tagged additional memory events such as MEM_INST_RETIRED.STLB_MISS_LOADS
with L1_Hit_Indication, implying PEBS data-source support, which is
incorrect. The database has since been fixed, but
intel_lnc_pebs_event_constraints[] still follows the old definition and
marks those events as data-source capable.

As a result, get_data_src() may decode data-source information for
events that do not provide valid PEBS data-source data and mislead
users.

Remove those non-data-source memory events from the Lion Cove PEBS
constraint table so matching falls back to the regular non-PEBS
constraints, which already provide the same counter constraints.

Also update lnc_latency_data() to decode LOAD/STORE flags explicitly
when setting memory operation direction, for consistency with other
*_latency_data() helpers.

Fixes: a932aa0e86 ("perf/x86: Add Lunar Lake and Arrow Lake support")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: <stable@vger.kernel.org>
Link: https://patch.msgid.link/20260917015234.981153-6-dapeng1.mi@linux.intel.com
2026-09-22 11:26:27 +02:00
Dapeng Mi
335b064281 perf/x86/intel: Update arw_latency_data() mem-op direction handling
Align arw_latency_data() with other *_latency_data() helpers by
explicitly decoding LOAD/STORE event flags when setting the sampled
memory operation direction.

This keeps the latency data path behavior consistent across platforms
and avoids relying on implicit direction inference.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260917015234.981153-5-dapeng1.mi@linux.intel.com
2026-09-22 11:19:35 +02:00
Dapeng Mi
8302c5f475 perf/x86/intel: Fix DKT PEBS load/store direction for latency events, to fix sample classification
Same bug exists on Darkmont as on Gracemont:
intel_dkt_pebs_event_constraints[] applies LAT_CONSTRAINT constraints to
MEM_UOPS_RETIRED.{LOAD,STORE}_LATENCY, but does not set explicit
LOAD/STORE flags for those events.

The PEBS latency path (pebs_latency_data(), via cmt_latency_data) uses
the event flags to determine memory operation direction. Without an
explicit STORE flag, samples from MEM_UOPS_RETIRED.STORE_LATENCY can be
misclassified as LOADs.

Set explicit LOAD/STORE flags in intel_dkt_pebs_event_constraints[] for:

- MEM_UOPS_RETIRED.LOAD_LATENCY
- MEM_UOPS_RETIRED.STORE_LATENCY

This fixes incorrect STORE sample classification. Additionally remove
INTEL_HYBRID_LAT_CONSTRAINT() since no one uses it anymore.

Fixes: 65fd435095 ("perf/x86/intel: Update event constraints for PTL")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: <stable@vger.kernel.org> # v7.2+
Link: https://patch.msgid.link/20260917015234.981153-4-dapeng1.mi@linux.intel.com
2026-09-22 11:17:59 +02:00
Dapeng Mi
e961d6db42 perf/x86/intel: Fix CMT PEBS load/store direction for latency events, to fix sample classification
The same bug exists on Crestmont as on Gracemont:
intel_cmt_pebs_event_constraints[] applies LAT_CONSTRAINT constraints to
MEM_UOPS_RETIRED.{LOAD,STORE}_LATENCY, but does not set explicit
LOAD/STORE flags for those events.

The PEBS latency path (pebs_latency_data(), via cmt_latency_data) uses
the event flags to determine memory operation direction. Without an
explicit STORE flag, samples from MEM_UOPS_RETIRED.STORE_LATENCY can be
misclassified as LOADs.

Set explicit LOAD/STORE flags in intel_cmt_pebs_event_constraints[] for:

- MEM_UOPS_RETIRED.LOAD_LATENCY
- MEM_UOPS_RETIRED.STORE_LATENCY

This fixes incorrect STORE sample classification.

Fixes: e99fb45436 ("perf/x86/intel: Update event constraints and cache_extra_regsfor MTL")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: <stable@vger.kernel.org> # v7.2+
Link: https://patch.msgid.link/20260917015234.981153-3-dapeng1.mi@linux.intel.com
2026-09-22 11:17:15 +02:00
Dapeng Mi
89dc568e8c perf/x86/intel: Fix GRT PEBS load/store direction for latency events, to fix sample classification
On Gracemont, intel_grt_pebs_event_constraints[] applies LAT_CONSTRAINT
constraints to MEM_UOPS_RETIRED.{LOAD,STORE}_LATENCY, but does not set
explicit LOAD/STORE flags for those events.

The PEBS latency path (pebs_latency_data(), via __grt_latency_data())
uses the event flags to determine memory operation direction. Without an
explicit STORE flag, samples from MEM_UOPS_RETIRED.STORE_LATENCY can be
misclassified as LOADs.

Set explicit LOAD/STORE flags in intel_grt_pebs_event_constraints[] for:

- MEM_UOPS_RETIRED.LOAD_LATENCY
- MEM_UOPS_RETIRED.STORE_LATENCY

Also update __grt_latency_data() to explicitly interpret these flags when
assigning the sampled memory operation direction.

This fixes incorrect STORE sample classification.

Fixes: 39a41278f0 ("perf/x86/intel: Fix PEBS memory access info encoding for ADL")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: <stable@vger.kernel.org> # v7.2+
Link: https://patch.msgid.link/20260917015234.981153-2-dapeng1.mi@linux.intel.com
2026-09-22 11:17:00 +02:00
Sean Christopherson
a391618e1d perf/x86/intel: Make @data a mandatory param for intel_guest_get_msrs()
Drop "support" for passing a NULL @data/@kvm_pmu param when getting guest
MSRs.  KVM, the only in-tree user, unconditionally passes a non-NULL
pointer, and carrying code that suggests @data may be NULL is confusing,
e.g. incorrectly implies that there are scenarios where KVM doesn't pass
a PMU context.

Fixes: 8183a538cd ("KVM: x86/pmu: Add IA32_DS_AREA MSR emulation to support guest DS")
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Jim Mattson <jmattson@google.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260921191418.950933-5-seanjc@google.com
2026-09-22 11:09:45 +02:00
Sean Christopherson
d06260e99e perf/x86/intel: Don't pointlessly context switch DS_AREA (and PEBS config) if PEBS is unused
When filling the list of MSRs to be loaded by KVM on VM-Enter and VM-Exit,
load the guest values for DS_AREA and (conditionally) MSR_PEBS_DATA_CFG if
and only if PEBS will be active in the guest, i.e. only if a PEBS record
may be generated while running the guest.  As shown by the !pebs_ept path,
it's perfectly safe to run with the host's DS_AREA, so long as PEBS-enabled
counters are disabled via PERF_GLOBAL_CTRL.

Omitting DS_AREA and MSR_PEBS_DATA_CFG when PEBS is unused saves two MSR
writes per MSR on each VMX transition, i.e. eliminates two/four pointless
MSR writes on each VMX roundtrip when PEBS isn't being used by the guest.

Fixes: c59a1f106f ("KVM: x86/pmu: Add IA32_PEBS_ENABLE MSR emulation for extended PEBS")
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Jim Mattson <jmattson@google.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260921191418.950933-4-seanjc@google.com
2026-09-22 11:09:14 +02:00
Sean Christopherson
4b64dbdc58 perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU has PEBS isolation, to fix stuck PEBS_ENABLED
When filling the list of MSRs to be loaded by KVM on VM-Enter and VM-Exit,
*never* insert an entry for PEBS_ENABLED if the CPU properly isolates PEBS
events, in which case disabling counters via PERF_GLOBAL_CTRL is sufficient
to prevent unwanted PEBS events in the guest (or host).  Because perf loads
PEBS_ENABLE with the unfiltered cpu_hw_events.pebs_enabled, i.e. with both
host and guest masks, there is no need to load different values for the
guest versus host, perf+KVM can and should simply control which counters
are enabled/disabled via PERF_GLOBAL_CTRL.

Avoiding touching PEBS_ENABLED "fixes" a bug where PEBS_ENABLED can end up
with "stuck" bits if a PEBS event is throttled between generating the list
and actually entering the guest (Intel CPUs can't arbtitrarily block NMIs).
Fixes in quotes because leaving PEBS_ENABLED as-is doesn't fix the
underlying problem of perf (via PMIs) being able to modify state after the
perf<=>KVM handoff.

But not writing PEBS_ENABLED is desirable no matter what, as stating the
obvious, leaving PEBS_ENABLED as-is avoids three MSR writes on every VMX
transition: one each on entry/exit, and one more explicit WRMSR to zero
PEBS_ENABLED before VM-Entry (KVM assumes the only reason PEBS_ENABLED is
in the load list is if the CPU lacks PEBS isolation and thus needs a
quiescent period).

Opportunistically add comments to (better) explain the rules for generating
the set of PEBS counters that will be active while the guest is running,
along with a FIXME for the suspected hack-a-fix where perf disables guest
PEBS if _any_ PEBS event is configured to count in the host (commit
854250329c ("KVM: x86/pmu: Disable guest PEBS temporarily in two rare
situations") doesn't explain the motivation, at all).

Fixes: c59a1f106f ("KVM: x86/pmu: Add IA32_PEBS_ENABLE MSR emulation for extended PEBS")
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260921191418.950933-3-seanjc@google.com
2026-09-22 11:07:00 +02:00
Sean Christopherson
cec38d5c09 perf/x86/intel: Ensure KVM guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits
When reinstating PEBS counters into PERF_GLOBAL_CTRL for a KVM guest, mask
the value with perf's desired/original PERF_GLOBAL_CTRL value to ensure
KVM doesn't unintentionally set reserved bits in PERF_GLOBAL_CTRL.  E.g.
if the guest's PEBS_ENABLE value had bit 63, "Enable Precise Store", set,
then using the raw guest PEBS value would propagate bit 63 to the guest's
PERF_GLOBAL_CTRL value (which thankfully would be a failed VM-Entry, not
a VMX Abort).

The only reason this bug isn't reachable is because KVM doesn't support
"Enable Precise Store" (which is probably a KVM bug?), i.e. bit 63 can't
be set in kvm_pmu->pebs_enable and thus not in arr[pebs_enable].guest.  In
other words, this _should_ be a glorified NOP in the current code base.

Fixes: c59a1f106f ("KVM: x86/pmu: Add IA32_PEBS_ENABLE MSR emulation for extended PEBS")
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260921191418.950933-2-seanjc@google.com
2026-09-22 11:02:37 +02:00
Dapeng Mi
a56c03a397 perf/x86/intel: Prevent drain_pebs() reentry
The PEBS buffer is shared by all events on a CPU, so drain_pebs() must
not be reentered. If so, one instance may observe stale buffer state and
potentially access out-of-bound memory.

Most invocations happen in NMI context, which naturally prevents reentry.
However, drain_pebs() is also reachable from process context via
intel_pmu_drain_pebs_buffer().

In those paths, the PMU is often already disabled, but not guaranteed.
For example, __intel_pmu_pebs_disable() only disables the target counter,
so other active counters can still raise a PMI and interrupt an in-flight
drain_pebs(). Here is an example,

__perf_addr_filters_adjust()
  perf_event_stop()
    __perf_event_stop()
      x86_pmu_stop() (event->pmu->stop)
        intel_pmu_disable_event()
          intel_pmu_pebs_disable()
            __intel_pmu_pebs_disable()
              intel_pmu_drain_large_pebs()
                intel_pmu_drain_pebs_buffer()

Introduce __intel_pmu_quiesce() and __intel_pmu_resume() helpers and
use them in intel_pmu_drain_large_pebs() to disable the full PMU
around the intel_pmu_drain_pebs_buffer() call, preventing reentry.

Also add a warning in intel_pmu_drain_pebs_buffer() when the full PMU is
not disabled.

Fixes: b752ea0c28 ("perf/x86/intel/ds: Flush PEBS DS when changing PEBS_DATA_CFG")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260813064346.335458-1-dapeng1.mi@linux.intel.com
2026-09-10 10:22:50 +02:00
Dapeng Mi
88aa1223bf perf/x86/intel: Correct pt_regs->flags update for PEBS path
pt_regs->flags holds the saved CPU FLAGS register. In the PEBS path,
it was incorrectly set to PERF_EFLAGS_EXACT instead of being populated
from the PEBS flags snapshot.

Update pt_regs->flags from PEBS GPR flags if GPRs group is present.

Fixes: c22497f583 ("perf/x86/intel: Support adaptive PEBS v4")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260908075102.540715-1-dapeng1.mi@linux.intel.com
2026-09-10 10:22:50 +02:00
Linus Torvalds
ed3b875bea mm.git review status for mm-hotfixes-stable..mm-stable
Everything:
 
 Total patches:       501
 Reviews/patch:       1.66
 Reviewed rate:       70%
 
 Excluding DAMON:
 
 Total patches:       356
 Reviews/patch:       2.26
 Reviewed rate:       90%
 
 Excluding DAMON and selftests:
 
 Total patches:       329
 Reviews/patch:       2.31
 Reviewed rate:       92%
 
 Excluding DAMON, selftests and maple_tree:
 
 Total patches:       328
 Reviews/patch:       2.31
 Reviewed rate:       92%
 
 Summary of patch series in this merge:
 
 - The 2 patch series "mm: drop "sub" prefix from various places" from
   Dev Jain implements some page->folio conversion and a naming cleanup.
 
 - The 2 patch series "mm/kasan: remove redundant initialization for
   kasan_flag_write_only" from Igor Putko provides some KASAN cleanup work.
 
 - The 2 patch series "mm/filemap: reduce unnecessary xarray lookups"
   from Chi Zhiling provides a small speedup in the pagecaache read code.
 
 - The 4 patch series "mm/percpu: Fix possible NOFS/NOIO reclaim
   recursion" from Kaitao Cheng improves a few things in the vmalloc code -
   mainly the avoidance of GFP_KERNEL allocations when the caller asked for
   GFP_NOFS or GFP_NOIO.
 
 - The 3 patch series "mm/kmemleak: avoid soft lockup when scanning task
   stacks" from Breno Leitao avoids a soft lockup watchdog trigger from the
   kmemleak scanning code in extreme situations.
 
 - The 6 patch series "mm/page_owner: misc cleanups" from Ye Liu is a
   collection of unrelated cleanups to the page_owner code.  For some
   reason lots of people have been working on the page_owner code this
   cycle.
 
 - The 4 patch series "mm: convert to walk_page_range_vma() to eliminate
   find_vma()" from Kefeng Wang simplifies and accelerates the page walking
   library function.
 
 - The 3 patch series "mm/migrate: preparatory cleanups for batch copy
   and offload" from Shivank Garg implements cleanups in the migration
   code.
 
 - The 4 patch series "mm/page_owner: add per-fd filter infrastructure
   for print_mode and NUMA filtering" from Zhen Ni provides per-fd
   filtering to page_owner in order to reduce the sometimes vast amount of
   output it can produce.
 
 - The 19 patch series "mm: Refactor bootmem gigantic hugepage
   allocation" from Muchun Song is a "set of fixes and preparatory cleanups
   around bootmem HugeTLB handling, sparse initialization ordering, and
   related vmemmap setup".
 
 - The 4 patch series "mm/zsmalloc: reduce lock contention in zs_free()"
   from Wenchao Hao reduces lock contention in zs_free(), which dominates
   the unmap path under memory pressure on Android (LMK kills) and on x86
   servers running zswap-heavy workloads.  Up to 1.83x improvement in
   microbenchmarking.
 
 - The 2 patch series "move alloc_tag.c file under mm/" from Suren
   Baghdasaryan does that.
 
 - The 6 patch series "samples/damon: handle damon_{start,stop}()
   failures" from SJ Park fixes improper handling of damon_start(),
   damon_stop(), and damon_call() failures across DAMON sample modules to
   prevent potential memory leaks, operation disruptions and use-after-free
   bugs.
 
 - The 11 patch series "mm/damon/sysfs: kobject_del() directories that
   users can create/remove" from SJ Park resolves an issue where delayed
   sysfs directory removal under CONFIG_DEBUG_KOBJECT_RELEASE causes
   creation failures due to duplicate directory names by adding missing
   kobject_del() calls before creating new directories.
 
 - The 3 patch series "mm: cleanup clear_not_present_full_ptes()" from
   David Hildenbrand cleans up the core pte handling code.
 
 - The 3 patch series "selftests/damon: misc fixes for test bugs" from
   Kunwu Chan fixes several bugs in the DAMON selftests.
 
 - The 2 patch series "selftests/damon: fix memcg_path staging handling"
   from Cheng Nie fixes a bug in _damon_sysfs.py for damos_filter
   memcg_path setup, and adds a test case for it in sysfs.py.
 
 - The 2 patch series "selftests/damon: test kdamond refresh_ms" from
   Ruslan Valiyev introduces selftest coverage for DAMON's refresh_ms sysfs
   feature by updating the test control module and verifying that scheme
   stats update automatically without manual intervention.
 
 - The 5 patch series "mm/damon: five misc fixups" from Akinobu Mita
   contains miscellaneous DAMON fixups.
 
 - The 2 patch series "mm/damon/core: detect internal variation above
   max_nr_regions/2" from Jiayuan Chen fixes DAMON's region splitting
   behavior when region counts exceed half the maximum budget by
   dynamically scaling down the split fraction as the limit approaches,
   preventing large regions from staying un-split, and adds corresponding
   KUnit test coverage.
 
 - The 6 patch series "mm: preparatory patches for PMD level swap
   entries" from Usama Arif refactors and cleans up PMD softleaf helpers,
   call sites, and architecture flags to lay the groundwork for a follow-up
   series that introduces PMD page table swap entries.
 
 - The 11 patch series "mm/damon: update, optimize, and clean up doc,
   tests, and code" from SJ Park updates DAMON design and ABI
   documentation, expands unit and selftest coverage, optimizes
   damon_commit_target_regions(), and cleans up recently added sysfs
   interface code for better readability.
 
 - The 2 patch series "mm/vmpressure: reduce CPU, memory and code
   overhead on cgroup v2" from Usama Arif optimizes vmpressure() by
   skipping unnecessary work on cgroup v2 for userspace event notifications
   and refactors v1-only eventfd handling into mm/memcontrol-v1.c to reduce
   memory overhead and code complexity.
 
 - The 10 patch series "selftests/mm: refactor pkey helpers and fix mmap
   error handling" from Hongfu Li refactors pkeys shared tracing and
   assertion helpers into a common file, unifies protection key selftests
   to use consistent diagnostic logging and assertions, and enforces
   standardized MAP_FAILED return checks for mmap() calls across the tests.
 
 - The 18 patch series "mm/damon: optimize out nr_accesses_bp" from SJ
   Park replaces the error-prone, continuously updated nr_accesses_bp field
   in damon_region with an on-demand moving sum function
   (damon_nr_accesses_mvsum()), reducing structure memory overhead and
   avoiding state corruption bugs.
 
 - The 6 patch series "Open HugeTLB allocation routine for more generic
   use" from Ackerley Tng decouples HugeTLB folio allocation from VMA
   dependencies by introducing hugetlb_alloc_folio(), enabling subsystems
   like guest_memfd to allocate HugeTLB folios without standard VMA
   reservations or pseudo-VMAs.
 
 - The 3 patch series "mm/damon: provide pseudo moving sum probe_hits"
   from SJ Park integrates DAMON's probe_hits attribute counter into the
   pseudo moving sum infrastructure, enabling real-time, online monitoring
   without waiting for full aggregation intervals.
 
 - The 18 patch series "mm: Some cleanups for page allocator APIs" from
   Brendan Jackman simplifies and refactors the page allocator entry points
   and flags by unifying allocation paths, adding internal alloc_flags
   arguments, and eliminating redundant __ prefixed alloc_pages variants.
 
 - The 5 patch series "Fix incorrect access of hugetlb pte entries" from
   Dev Jain enforces the consistent use of huge_ptep_get() instead of
   ptep_get() for HugeTLB entries and fixes an unaligned address issue in
   arm64's huge_ptep_get() implementation.
 
 - The 8 patch series "mm/damon: validate all parameters in the core"
   from SJ Park consolidates parameter validation into the DAMON core
   specifically within damon_start() and damon_commit_ctx() to centralize
   error checking, eliminate caller-side redundant checks and to improve
   maintenance efficiency.
 
 - The 3 patch series "tools/mm/page_owner_sort: fix filtering and
   cleanup issues" from Yichong Chen renames is_need() to filter_record()
   for clearer return semantics, fixes per-record allocation memory leaks
   and bounds output copies in search_pattern() to address an existing
   buffer issue.
 
 - The 4 patch series "memcg: bail out reclaim when memcg is dying" from
   Jiayuan Chen mitigates a system-wide stall which occurs when a cgroup is
   removed while one of its memory control files is doing synchronous
   reclaim.
 
 - The 5 patch series "mm/memory-failure: add panic option for
   unrecoverable pages" from Breno Leitao introduces an opt-in
   vm.panic_on_unrecoverable_memory_failure sysctl that immediately panics
   the kernel on unrecoverable memory errors in kernel-owned pages to
   preserve error context and prevent delayed, silent data corruption.
 
 - The 11 patch series "mm/damon: refactor damon_{start,stop,commit}()
   for simple error handling" from SJ Park refactors the DAMON core API
   functions to guarantee that all contexts are fully stopped when
   damon_start(), damon_stop(), or damon_commit() fail, eliminating the
   need for complex and error-prone caller-side cleanup code.
 
 - The 5 patch series "Keep tail page private zero at free and folio
   split" from Zi Yan adds checks to ensure tail_page->private is zero when
   freeing compound or high-order pages and when promoting tail pages
   during large folio splits.  By validating these fields at free and split
   time, it allows the removal of redundant private field clearing inside
   prep_compound_tail().
 
 - The 4 patch series "mm: drop redundant lru_add_drain in anon folio
   reuse paths" from Barry Song eliminates redundant lru_add_drain() calls
   in wp_can_reuse_anon_folio() and do_swap_page() to reduce LRU lock
   contention and system overhead.
 
   By validating folio refcounts against the LRU cache before draining
   and removing unnecessary drains in the swap path, it achieves up to a
   30.5% reduction in drain calls during heavy swap workloads.
 
 - The 3 patch series "mm: clean up folio LRU and swap declarations" from
   Jianyue Wu reorganizes folio LRU and swap code by relocating
   page-cluster state to mm/swap_state.c, renaming mm/swap.c to mm/folio.c,
   and moving MM-internal reclaim declarations into mm/internal.h.
 
 - The 15 patch series "userfaultfd: working set tracking for VM guest
   memory" from Kiryl Shutsemau adds userfaultfd support for tracking the
   working set of VM guest memory, so a VMM can identify hot pages and
   reclaim cold ones to tiered or remote storage.
 
 - The 10 patch series "mm: remove CONFIG_HAVE_BOOTMEM_INFO_NODE (Part
   2)" from David Hildenbrand removes the remaining pieces of
   CONFIG_HAVE_BOOTMEM_INFO_NODE, performing some smaller cleanups around
   freeing of reserved vmemmap pages on the way.
 
 - The 7 patch series "mm/damon: update probe hits for runtime parameter
   commits" from SJ Park ensures that DAMON's probe_hits attribute counter
   is properly updated when monitoring intervals are changed at runtime,
   matching the behavior of nr_accesses.  To achieve this, it refactors and
   renames existing helper functions for shared use, applies the updates to
   probe_hits, and handles edge cases in damon_probe_hits_mvsum() to
   maintain measurement accuracy.
 
 - The 3 patch series "KSM: performance optimizations for rmap_walk_ksm"
   from xu xin resolves a severe KSM reverse-mapping performance bottleneck
   where thousands of split VMAs sharing a single anon_vma cause extended
   lock contention.  By adding an interval-filtering check during the rmap
   walk, it reduces worst-case anon_vma lock hold times from over 500ms
   down to under 2ms, preventing application freezes and latency spikes
   under memory pressure.
 
 - The 3 patch series "mm: split a couple of headers from internal.h"
   from Mike Rapoport splits declarations related to mm_init, memblock,
   vmalloc and sparse into new headers.
 
 - The 2 patch series "KSM: use linear_page_index in collect_procs_ksm()"
   from xu xin applies the interval tree optimization from rmap_walk_ksm()
   to collect_procs_ksm() to avoid iterating over non-matching VMAs during
   KSM memory error handling.  It hoists loop-invariant address
   initialization and restricts the anon_vma_interval_tree_foreach walk to
   a targeted page offset range, reducing redundant checks and improving
   lookup efficiency.
 
 - The 3 patch series "selftests/mm: avoid false failures in hugetlb and
   KSM tests" from Sayali Patil fixes issues in the hugetlb and KSM MM
   selftest categories that can report failures when the prerequisites for
   the tests are not satisfied.
 
 - The 19 patch series "mm/damon: introduce data attributes only
   monitoring" from SJ Park introduces attribute-weighted region management
   in DAMON, allowing users to prioritize specific data attributes (such as
   page sizes or cgroups) over or instead of access monitoring.
 
   By assigning weights to attribute probes, DAMON can completely disable
   access tracking and adjust monitoring regions based on weighted
   probe-hit counters to optimize monitoring quality for attribute-focused
   workloads.
 
 - The 8 patch series "mm/hmm: Add mmap lock-drop support for
   userfaultfd-backed mappings" from Stanislav Kinsburskii extends
   hmm_range_fault() to support userfaultfd-backed regions by allowing the
   mmap lock to be dropped during fault handling via a new
   hmm_range_fault_locked() helper.
 
   By accepting a locked pointer and signaling retry status when lock
   release occurs, it enables page fault resolution in userfaultfd regions
   while preserving backward compatibility for existing callers.
 
 - The 33 patch series "mm: make VMA page offset handling more
   consistent" from Lorenzo Stoakes cleans up and standardizes how
   vma->vm_pgoff is accessed and manipulated across file-backed and
   anonymous mappings in the kernel.
 
   It introduces dedicated helper functions such as vma_start_pgoff(),
   vma_end_pgoff(), vma_set_pgoff() and linear_page_delta() while renaming
   rmap interval tree helpers to better reflect their functionality.
 
   These changes establish a cleaner foundation for future work that will
   unify virtual page offset indexing for all anonymous and CoW'd folios.
 
 - The 3 patch series "mm: handle device-private PMDs in walk callbacks"
   from Usama Arif addresses kernel panics and state corruption caused by
   MM walk callbacks reaching non-present device-private PMD swap entries
   created during HMM migrations.
 
   It ensures that functions which acquire pmd_trans_huge_lock() properly
   recognize device-private PMDs instead of assuming a present THP or a
   standard migration entry.
 
 - The 5 patch series "mm/rmap: Refactor try_to_unmap_one" from Dev Jain
   refactors try_to_unmap_one by modularizing Hugetlb, anonymous-lazyfree,
   and anonymous-swapbacked logic into dedicated functions, laying the
   structural groundwork for batched anonymous large folio unmapping.
 
 - The 4 patch series "Docs/ABI/damon: sysfs ABI document fixes and
   additions" from Song Hu fixes typos and fills in missing entries in the
   DAMON sysfs ABI document.
 
 - The 10 patch series "dax/kmem: atomic whole-device hotplug via sysfs"
   from Gregory Price introduces an atomic sysfs state attribute and
   supporting DAX/MM infrastructure to prevent userland races when
   offlining and removing entire memory regions.
 
   By adding an unplugged state alongside standard online modes, it
   enables whole-device atomic hotplug control while preserving backward
   compatibility.
 
 - The 13 patch series "mm: convert more vm_flags_t users to vma_flags_t"
   from Lorenzo Stoakes continues transitioning the kernel from the
   deprecated vm_flags_t type to vma_flags_t across core memory management
   infrastructure.
 
   It replaces legacy type usage in core functions such as do_mmap(),
   unmapped area allocation, mm->def_vma_flags, and VMA operations like
   mlock, mprotect, and mremap.
 
 - The 2 patch series "Two small patches to clean up mm/mm_slot.h" from
   xu xin refactors mm_slot.h by introducing mm_slot_remove() to unify
   duplicate slot deletion sequences in khugepaged and KSM.  It also adds
   code documentation explaining why mm_slot_lookup and mm_slot_insert must
   remain as preprocessor macros rather than static inline functions.
 
 - The 10 patch series "mm/damon/core: hide core-private struct fields"
   from SJ Park cleans up DAMON core structures by consistently marking
   internal-only fields with private: comment tags to prevent improper
   direct access from outer layers.
 
   It enforces encapsulation across core structures including
   damon_region, damon_target, and damon_ctx and updates DAMON_SYSFS to
   interact through approved access APIs instead of exposing raw struct
   members.
 
 - The 6 patch series "mm/damon: unurgent fixes for infinite loop, NULL
   de-ref and races" from SJ Park addresses potential infinite loops, NULL
   dereferences, and race conditions identified in DAMON.
 
   It fixes an infinite loop triggered by extreme user configurations, a
   NULL pointer dereference within unit tests and minor monitoring
   accuracy degradation caused by subtle runtime races.
 
 - The 2 patch series "mm/page_alloc: fixes for free_pages_nolock() on
   RT/UP" from Brendan Jackman fixes an NMI safety flaw in
   __free_frozen_pages() where freeing pages on non-SMP or PREEMPT_RT
   kernels can bypass can_spin_trylock() checks via non-PCP or isolated
   migration paths.
 
   It also resolves potential kernel crashes and privilege escalation
   risks triggered when BPF tracing runs in NMI context alongside memory
   hotplug or large allocation frees.
 
 - The 4 patch series "mm/page_alloc: couple of followups for recent
   cleanups" from Brendan Jackman cleans up and updates page allocator
   nomenclature, documentation, and debug assertions.
 
   It aligns internal FPI_ flags with the public "nolock" naming
   convention, removes outdated internal implementation details from
   high-level page allocator comments, and eliminates obsolete VM_BUG_ON()
   assertions in allocation paths.
 
 - The 3 patch series "mm/mseal: further cleanups" from Lorenzo Stoakes
   refactors and simplifies the mseal implementation by clarifying API
   boundaries and removing unnecessary code complexity.
 
   It replaces generic do_mseal() usage outside the syscall with a
   dedicated mseal_mmap_page_zero() helper for MMAP_PAGE_ZERO, eliminates
   mm_struct parameters to enforce that sealing applies only to
   current->mm, and streamlines overall logic and comments with no
   functional changes intended.
 
 - The 4 patch series "mm/vmscan: fix swappiness=max and clean up
   per-node proactive reclaim" from Ridong Chen resolves reclaim behavior
   bugs and cleans up function parameters across memory reclaim paths.
 
   It fixes swappiness=max in both standard reclaim and MGLRU so
   unswappable anonymous memory no longer falls back to evicting page
   cache, ensures reclaim_store() returns accurate error codes instead of
   collapsing all failures into -EAGAIN, and removes the obsolete gfp_mask
   parameter from __node_reclaim().
 
 - The 6 patch series "mm: mincore: misc cleanups" from Kefeng Wang
   cleans up and simplifies the mincore code.  Most importantly, it removes
   the historical special behavior that always reports VM_PFNMAP pages as
   non-resident.
 
 - The 2 patch series "mm/huge_memory: drop dead split helper variants"
   from Kiryl Shutsemau implements two trivial cleanups in the folio split
   API.
 
 - The 7 patch series "mm/damon: fix uninitialized DAMOS field and kunit
   exec expectation bugs" from SJ Park resolves minor operational and
   testing bugs in DAMON identified by Sashiko.  It initializes the
   damos->last_applied field to prevent occasional efficiency degradation
   and fixes invalid memory accesses in DAMON KUnit tests during test
   failure handling.
 
 - The 3 patch series "cleanup for stable_page_flags()" from Jinjiang Tu
   cleans up and refactors stable_page_flags() used by /proc/kpageflags
   without altering functionality.
 
   It uses BIT_ULL() to prevent shift-overflow warnings on 64-bit flag
   bits, converts folio-specific flag checks to standard folio_test_*()
   helpers, and removes redundant CONFIG_PAGE_IDLE_FLAG handling.
 
 - The 3 patch series "Batch unmap of uffd-wp file folios" from Dev Jain
   extends batched folio unmapping support to file folios within
   userfaultfd write-protect (uffd-wp) VMAs by adding batching capabilities
   to pte_install_uffd_wp_if_needed().
 
   This removes special-case restrictions on uffd-wp VMAs in
   try_to_unmap_one(), significantly simplifying the function's control
   flow and complexity.
 
 - The 3 patch series "mm/early_ioremap: clarify and clean up
   early_ioremap_reset()" from Sang-Heon Jeon clarifies and cleans up the
   architecture-specific usage of __late_set_fixmap() and
   __late_clear_fixmap() after early_ioremap_reset().
 
   It adds explicit documentation regarding when early_ioremap_reset()
   must be called and removes redundant macro definitions and reset calls
   in the RISC-V and ARM64 architectures.
 
 - The 4 patch series "mm: fix reclaim storms in defrag_mode" from
   Johannes Weiner addresses severe performance regressions, swap storms,
   and spurious OOMs caused by vm.defrag_mode=1 under high memory pressure
   in Meta production.
 
   It updates the page allocator slowpath so non-movable allocation
   requests actively trigger direct reclaim and direct compaction at
   pageblock_order scale, allowing them to claim whole pageblocks rather
   than spinning unproductively.
 
 - The 2 patch series "zram: lockmap tweaks" from Sebastian Siewior
   optimizes and fixes lockdep tracking for zram devices by consolidating
   per-entry lockmaps and isolating lock classes across multiple instances.
 
   It reduces memory overhead by replacing per-entry lockdep_map instances
   with a single map per struct zram, and assigns a dynamic lock_class_key
   to each instance to prevent false deadlock reports when different zram
   devices are backed by distinct filesystems.
 -----BEGIN PGP SIGNATURE-----
 
 iHUEABYKAB0WIQTTMBEPP41GrTpTJgfdBJ7gKXxAjgUCaoUJbQAKCRDdBJ7gKXxA
 jqrzAP9WoPU0hiK4qS/kSjhtoZxhjpS5eLSUCy/utKuEvZbfGgEAu1zA+LH+X9Tm
 THK5ex4iUZxiFbXpWfLMxE/Q9PmQYQ8=
 =QTyb
 -----END PGP SIGNATURE-----

Merge tag 'mm-stable-2026-08-18-18-39' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

Pull MM updates from Andrew Morton:

 - "mm: drop "sub" prefix from various places" (Dev Jain)

   page->folio conversion and a naming cleanup

 - "mm/kasan: remove redundant initialization for kasan_flag_write_only"
   (Igor Putko)

   KASAN cleanup work

 - "mm/filemap: reduce unnecessary xarray lookups" (Chi Zhiling)

   Small speedup in the pagecaache read code

 - "mm/percpu: Fix possible NOFS/NOIO reclaim recursion" (Kaitao Cheng)

   Improve the vmalloc code - mainly the avoidance of GFP_KERNEL
   allocations when the caller asked for GFP_NOFS or GFP_NOIO

 - "mm/kmemleak: avoid soft lockup when scanning task stacks" (Breno
   Leitao)

   Avoid a soft lockup watchdog trigger from the kmemleak scanning code
   in extreme situations

 - "mm/page_owner: misc cleanups" (Ye Liu)

   Cleanups to the page_owner code. For some reason lots of people have
   been working on the page_owner code this cycle.

 - "mm: convert to walk_page_range_vma() to eliminate find_vma()"
   (Kefeng Wang)

   Simplify and accelerate the page walking library function

 - "mm/migrate: preparatory cleanups for batch copy and offload"
   (Shivank Garg)

   Cleanups in the migration code

 - "mm/page_owner: add per-fd filter infrastructure for print_mode and
   NUMA filtering" (Zhen Ni)

   Per-fd filtering to page_owner in order to reduce the sometimes vast
   amount of output it can produce

 - "mm: Refactor bootmem gigantic hugepage allocation" (Muchun Song)

   Fixes and preparatory cleanups around bootmem HugeTLB handling,
   sparse initialization ordering, and related vmemmap setup

 - "mm/zsmalloc: reduce lock contention in zs_free()" (Wenchao Hao)

   Reduce lock contention in zs_free(), which dominates the unmap path
   under memory pressure on Android (LMK kills) and on x86 servers
   running zswap-heavy workloads.

   Up to 1.83x improvement in microbenchmarking.

 - "move alloc_tag.c file under mm/" (Suren Baghdasaryan)

 - "samples/damon: handle damon_{start,stop}() failures" (SJ Park)

   Fix improper handling of damon_start(), damon_stop(), and
   damon_call() failures across DAMON sample modules to prevent
   potential memory leaks, operation disruptions and use-after-free
   bugs

 - "mm/damon/sysfs: kobject_del() directories that users can
   create/remove" (SJ Park)

   Fix delayed sysfs directory removal under DEBUG_KOBJECT_RELEASE
   causeing creation failures due to duplicate directory names by adding
   missing kobject_del() calls before creating new directories

 - "mm: cleanup clear_not_present_full_ptes()" (David Hildenbrand)

   Clean up the core pte handling code

 - "selftests/damon: misc fixes for test bugs" (Kunwu Chan)

   Fix several bugs in the DAMON selftests

 - "selftests/damon: fix memcg_path staging handling" (Cheng Nie)

   Fix a bug in _damon_sysfs.py for damos_filter memcg_path setup, and
   add a test case for it in sysfs.py.

 - "selftests/damon: test kdamond refresh_ms" (Ruslan Valiyev)

   Selftest coverage for DAMON's refresh_ms sysfs feature by updating
   the test control module and verifying that scheme stats update
   automatically without manual intervention

 - "mm/damon: five misc fixups" (Akinobu Mita)

   Miscellaneous DAMON fixups.

 - "mm/damon/core: detect internal variation above max_nr_regions/2"
   (Jiayuan Chen)

   Fix DAMON's region splitting behavior when region counts exceed half
   the maximum budget by dynamically scaling down the split fraction as
   the limit approaches, preventing large regions from staying un-split,
   and add corresponding KUnit test coverage

 - "mm: preparatory patches for PMD level swap entries" (Usama Arif)

   Refactor and clean up PMD softleaf helpers, call sites, and
   architecture flags to lay the groundwork for a follow-up series that
   introduces PMD page table swap entries

 - "mm/damon: update, optimize, and clean up doc, tests, and code" (SJ
   Park)

   Update DAMON design and ABI documentation, expands unit and selftest
   coverage, optimize damon_commit_target_regions(), and clean up
   recently added sysfs interface code for better readability

 - "mm/vmpressure: reduce CPU, memory and code overhead on cgroup v2"
   (Usama Arif)

   Optimize vmpressure() by skipping unnecessary work on cgroup v2 for
   userspace event notifications and refactor v1-only eventfd handling
   into mm/memcontrol-v1.c to reduce memory overhead and code complexity

 - "selftests/mm: refactor pkey helpers and fix mmap error handling"
   (Hongfu Li)

   Refactor pkeys shared tracing and assertion helpers into a common
   file, unify protection key selftests to use consistent diagnostic
   logging and assertions, and enforce standardized MAP_FAILED return
   checks for mmap() calls across the tests

 - "mm/damon: optimize out nr_accesses_bp" (SJ Park)

   Replace the error-prone, continuously updated nr_accesses_bp field in
   damon_region with an on-demand moving sum function, reducing
   structure memory overhead and avoiding state corruption bugs

 - "Open HugeTLB allocation routine for more generic use" (Ackerley Tng)

   Decouple HugeTLB folio allocation from VMA dependencies by
   introducing hugetlb_alloc_folio(), enabling subsystems like
   guest_memfd to allocate HugeTLB folios without standard VMA
   reservations or pseudo-VMAs

 - "mm/damon: provide pseudo moving sum probe_hits" (SJ Park)

   Integrate DAMON's probe_hits attribute counter into the pseudo moving
   sum infrastructure, enabling real-time, online monitoring without
   waiting for full aggregation intervals

 - "mm: Some cleanups for page allocator APIs" (Brendan Jackman)

   Simplify and refactor the page allocator entry points and flags by
   unifying allocation paths, adding internal alloc_flags arguments, and
   eliminating redundant __ prefixed alloc_pages variants.

 - "Fix incorrect access of hugetlb pte entries" (Dev Jain)

   Enforce the consistent use of huge_ptep_get() instead of ptep_get()
   for HugeTLB entries and fixes an unaligned address issue in arm64's
   huge_ptep_get() implementation

 - "mm/damon: validate all parameters in the core" (SJ Park)

   Consolidate parameter validation into the DAMON core specifically
   within damon_start() and damon_commit_ctx() to centralize error
   checking, eliminate caller-side redundant checks and to improve
   maintenance efficiency

 - "tools/mm/page_owner_sort: fix filtering and cleanup issues" (Yichong
   Chen)

   Rename is_need() to filter_record() for clearer return semantics, fix
   per-record allocation memory leaks and bound output copies in
   search_pattern() to address an existing buffer issue

 - "memcg: bail out reclaim when memcg is dying" (Jiayuan Chen)

   Mitigate a system-wide stall which occurs when a cgroup is removed
   while one of its memory control files is doing synchronous reclaim

 - "mm/memory-failure: add panic option for unrecoverable pages" (Breno
   Leitao)

   Introduce an opt-in vm.panic_on_unrecoverable_memory_failure sysctl
   that immediately panics the kernel on unrecoverable memory errors in
   kernel-owned pages to preserve error context and prevent delayed,
   silent data corruption

 - "mm/damon: refactor damon_{start,stop,commit}() for simple error
   handling" (SJ Park)

   Refactor the DAMON core API functions to guarantee that all contexts
   are fully stopped when damon_start(), damon_stop(), or damon_commit()
   fail, eliminating the need for complex and error-prone caller-side
   cleanup code

 - "Keep tail page private zero at free and folio split" (Zi Yan)

   Add checks to ensure tail_page->private is zero when freeing compound
   or high-order pages and when promoting tail pages during large folio
   splits. By validating these fields at free and split time, it allows
   the removal of redundant private field clearing inside
   prep_compound_tail()

 - "mm: drop redundant lru_add_drain in anon folio reuse paths" (Barry
   Song)

   Eliminate redundant lru_add_drain() calls in
   wp_can_reuse_anon_folio() and do_swap_page() to reduce LRU lock
   contention and system overhead

   By validating folio refcounts against the LRU cache before draining
   and removing unnecessary drains in the swap path, it achieves up to a
   30.5% reduction in drain calls during heavy swap workloads

 - "mm: clean up folio LRU and swap declarations" (Jianyue Wu)

   Reorganize folio LRU and swap code by relocating page-cluster state
   to mm/swap_state.c, renaming mm/swap.c to mm/folio.c, and moving
   MM-internal reclaim declarations into mm/internal.h.

 - "userfaultfd: working set tracking for VM guest memory" (Kiryl
   Shutsemau)

   Add userfaultfd support for tracking the working set of VM guest
   memory, so a VMM can identify hot pages and reclaim cold ones to
   tiered or remote storage

 - "mm: remove CONFIG_HAVE_BOOTMEM_INFO_NODE (Part 2)" (David
   Hildenbrand)

   Remove the remaining pieces of CONFIG_HAVE_BOOTMEM_INFO_NODE,
   performing some smaller cleanups around freeing of reserved vmemmap
   pages on the way.

 - "mm/damon: update probe hits for runtime parameter commits" (SJ Park)

   Ensure that DAMON's probe_hits attribute counter is properly updated
   when monitoring intervals are changed at runtime, matching the
   behavior of nr_accesses. To achieve this, it refactors and renames
   existing helper functions for shared use, applies the updates to
   probe_hits, and handles edge cases in damon_probe_hits_mvsum() to
   maintain measurement accuracy.

 - "KSM: performance optimizations for rmap_walk_ksm" (xu xin)

   Resolve a severe KSM reverse-mapping performance bottleneck where
   thousands of split VMAs sharing a single anon_vma cause extended lock
   contention.

   By adding an interval-filtering check during the rmap walk, it
   reduces worst-case anon_vma lock hold times from over 500ms down to
   under 2ms, preventing application freezes and latency spikes under
   memory pressure.

 - "mm: split a couple of headers from internal.h" (Mike Rapoport)

   Split declarations related to mm_init, memblock, vmalloc and sparse
   into new headers

 - "KSM: use linear_page_index in collect_procs_ksm()" (xu xin)

   Apply the interval tree optimization from rmap_walk_ksm() to
   collect_procs_ksm() to avoid iterating over non-matching VMAs during
   KSM memory error handling.

   It hoists loop-invariant address initialization and restricts the
   anon_vma_interval_tree_foreach walk to a targeted page offset range,
   reducing redundant checks and improving lookup efficiency.

 - "selftests/mm: avoid false failures in hugetlb and KSM tests" (Sayali
   Patil)

   Fix issues in the hugetlb and KSM MM selftest categories that can
   report failures when the prerequisites for the tests are not
   satisfied

 - "mm/damon: introduce data attributes only monitoring" (SJ Park)

   Introduce attribute-weighted region management in DAMON, allowing
   users to prioritize specific data attributes (such as page sizes or
   cgroups) over or instead of access monitoring.

   By assigning weights to attribute probes, DAMON can completely
   disable access tracking and adjust monitoring regions based on
   weighted probe-hit counters to optimize monitoring quality for
   attribute-focused workloads.

 - "mm/hmm: Add mmap lock-drop support for userfaultfd-backed mappings"
   (Stanislav Kinsburskii)

   Extend hmm_range_fault() to support userfaultfd-backed regions by
   allowing the mmap lock to be dropped during fault handling via a new
   hmm_range_fault_locked() helper.

   By accepting a locked pointer and signaling retry status when lock
   release occurs, it enables page fault resolution in userfaultfd
   regions while preserving backward compatibility for existing callers.

 - "mm: make VMA page offset handling more consistent" (Lorenzo Stoakes)

   Clean up and standardize how vma->vm_pgoff is accessed and
   manipulated across file-backed and anonymous mappings in the kernel

   It introduces dedicated helper functions such as vma_start_pgoff(),
   vma_end_pgoff(), vma_set_pgoff() and linear_page_delta() while
   renaming rmap interval tree helpers to better reflect their
   functionality.

   These changes establish a cleaner foundation for future work that
   will unify virtual page offset indexing for all anonymous and CoW'd
   folios.

 - "mm: handle device-private PMDs in walk callbacks" (Usama Arif)

   Address kernel panics and state corruption caused by MM walk
   callbacks reaching non-present device-private PMD swap entries
   created during HMM migrations

   It ensures that functions which acquire pmd_trans_huge_lock()
   properly recognize device-private PMDs instead of assuming a present
   THP or a standard migration entry.

 - "mm/rmap: Refactor try_to_unmap_one" (Dev Jain)

   Refactor try_to_unmap_one by modularizing Hugetlb,
   anonymous-lazyfree, and anonymous-swapbacked logic into dedicated
   functions, laying the structural groundwork for batched anonymous
   large folio unmapping.

 - "Docs/ABI/damon: sysfs ABI document fixes and additions" (Song Hu)

   Fix typos and fills in missing entries in the DAMON sysfs ABI
   document

 - "dax/kmem: atomic whole-device hotplug via sysfs" (Gregory Price)

   Introduce an atomic sysfs state attribute and supporting DAX/MM
   infrastructure to prevent userland races when offlining and removing
   entire memory regions

   By adding an unplugged state alongside standard online modes, it
   enables whole-device atomic hotplug control while preserving backward
   compatibility.

 - "mm: convert more vm_flags_t users to vma_flags_t" (Lorenzo Stoakes)

   Continue transitioning the kernel from the deprecated vm_flags_t type
   to vma_flags_t across core memory management infrastructure.

   It replaces legacy type usage in core functions such as do_mmap(),
   unmapped area allocation, mm->def_vma_flags, and VMA operations like
   mlock, mprotect, and mremap.

 - "Two small patches to clean up mm/mm_slot.h" (xu xin)

   Refactor mm_slot.h by introducing mm_slot_remove() to unify duplicate
   slot deletion sequences in khugepaged and KSM. It also adds code
   documentation explaining why mm_slot_lookup and mm_slot_insert must
   remain as preprocessor macros rather than static inline functions.

 - "mm/damon/core: hide core-private struct fields" (SJ Park)

   Clean up DAMON core structures by consistently marking internal-only
   fields with private: comment tags to prevent improper direct access
   from outer layers.

   It enforces encapsulation across core structures including
   damon_region, damon_target, and damon_ctx and updates DAMON_SYSFS to
   interact through approved access APIs instead of exposing raw struct
   members.

 - "mm/damon: unurgent fixes for infinite loop, NULL de-ref and races"
   (SJ Park)

   Address potential infinite loops, NULL dereferences, and race
   conditions identified in DAMON

   It fixes an infinite loop triggered by extreme user configurations, a
   NULL pointer dereference within unit tests and minor monitoring
   accuracy degradation caused by subtle runtime races.

 - "mm/page_alloc: fixes for free_pages_nolock() on RT/UP" (Brendan
   Jackman)

   Fix an NMI safety flaw in __free_frozen_pages() where freeing pages
   on non-SMP or PREEMPT_RT kernels can bypass can_spin_trylock() checks
   via non-PCP or isolated migration paths.

   It also resolves potential kernel crashes and privilege escalation
   risks triggered when BPF tracing runs in NMI context alongside memory
   hotplug or large allocation frees.

 - "mm/page_alloc: couple of followups for recent cleanups" (Brendan
   Jackman)

   Clean up and update page allocator nomenclature, documentation, and
   debug assertions.

   It aligns internal FPI_ flags with the public "nolock" naming
   convention, removes outdated internal implementation details from
   high-level page allocator comments, and eliminates obsolete
   VM_BUG_ON() assertions in allocation paths.

 - "mm/mseal: further cleanups" (Lorenzo Stoakes)

   Refactor and simplify the mseal implementation by clarifying API
   boundaries and removing unnecessary code complexity.

   It replaces generic do_mseal() usage outside the syscall with a
   dedicated mseal_mmap_page_zero() helper for MMAP_PAGE_ZERO,
   eliminates mm_struct parameters to enforce that sealing applies only
   to current->mm, and streamlines overall logic and comments with no
   functional changes intended.

 - "mm/vmscan: fix swappiness=max and clean up per-node proactive
   reclaim" (Ridong Chen)

   Resolve reclaim behavior bugs and clean up function parameters across
   memory reclaim paths

   It fixes swappiness=max in both standard reclaim and MGLRU so
   unswappable anonymous memory no longer falls back to evicting page
   cache, ensures reclaim_store() returns accurate error codes instead
   of collapsing all failures into -EAGAIN, and removes the obsolete
   gfp_mask parameter from __node_reclaim().

 - "mm: mincore: misc cleanups" (Kefeng Wang)

   Clean up and simplifies the mincore code. Most importantly, it
   removes the historical special behavior that always reports VM_PFNMAP
   pages as non-resident.

 - "mm/huge_memory: drop dead split helper variants" (Kiryl Shutsemau)

   Two trivial cleanups in the folio split API

 - "mm/damon: fix uninitialized DAMOS field and kunit exec expectation
   bugs" (SJ Park)

   Resolve minor operational and testing bugs in DAMON identified by
   Sashiko. It initializes the damos->last_applied field to prevent
   occasional efficiency degradation and fixes invalid memory accesses
   in DAMON KUnit tests during test failure handling.

 - "cleanup for stable_page_flags()" (Jinjiang Tu)

   Clean up and refactor stable_page_flags() used by /proc/kpageflags
   without altering functionality.

   It uses BIT_ULL() to prevent shift-overflow warnings on 64-bit flag
   bits, converts folio-specific flag checks to standard folio_test_*()
   helpers, and removes redundant CONFIG_PAGE_IDLE_FLAG handling.

 - "Batch unmap of uffd-wp file folios" (Dev Jain)

   Extend batched folio unmapping support to file folios within
   userfaultfd write-protect (uffd-wp) VMAs by adding batching
   capabilities to pte_install_uffd_wp_if_needed().

   This removes special-case restrictions on uffd-wp VMAs in
   try_to_unmap_one(), significantly simplifying the function's control
   flow and complexity.

 - "mm/early_ioremap: clarify and clean up early_ioremap_reset()"
   (Sang-Heon Jeon)

   Clarify and clean up the architecture-specific usage of
   __late_set_fixmap() and __late_clear_fixmap() after
   early_ioremap_reset()

   It adds explicit documentation regarding when early_ioremap_reset()
   must be called and removes redundant macro definitions and reset
   calls in the RISC-V and ARM64 architectures.

 - "mm: fix reclaim storms in defrag_mode" (Johannes Weiner)

   Address severe performance regressions, swap storms, and spurious
   OOMs caused by vm.defrag_mode=1 under high memory pressure in Meta
   production

   It updates the page allocator slowpath so non-movable allocation
   requests actively trigger direct reclaim and direct compaction at
   pageblock_order scale, allowing them to claim whole pageblocks rather
   than spinning unproductively.

 - "zram: lockmap tweaks" (Sebastian Siewior)

   Optimize and fix lockdep tracking for zram devices by consolidating
   per-entry lockmaps and isolate lock classes across multiple instances

   This reduces memory overhead by replacing per-entry lockdep_map
   instances with a single map per struct zram, and assigns a dynamic
   lock_class_key to each instance to prevent false deadlock reports
   when different zram devices are backed by distinct filesystems.

* tag 'mm-stable-2026-08-18-18-39' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (501 commits)
  selftests/mm: thuge-gen: fix test_shmget() for PAGE_SIZE check
  selftests/mm: unpoison pages in memory-failure teardown
  mm/shmem: downgrade final i_blocks check in shmem_evict_inode() to pr_warn()
  mm/khugepaged: replace mutex_lock/mutex_unlock usage with guard macro
  mm/zsmalloc: fix release order of locks in zs_page_migrate()
  Documentation: zram: remove sections numbering
  ksm: stop iterating VMAs when ksm_test_exit returns true
  mm: fold userfaultfd_rwp() to false without CONFIG_ARCH_HAS_PTE_PROTNONE
  mm/migrate: report RCU-tasks quiescent states in migrate_pages_batch()
  zram: use a custom key for each zram object
  zram: move lockmap to be per-zram instead per table
  selftests/mm: fix gup_longterm EINVAL error message
  mm: page_alloc: fix non-movable reclaim storm in defrag_mode
  mm: page_alloc: move capture_control to the page allocator
  mm: compaction: support non-movable compaction for pageblock requests
  mm: page_alloc: __GFP_FS lockdep annotation for direct compaction
  hugetlb: evaluate subpool free state while locked
  mm/damon: remove trailing semicolons after function definitions
  mm/damon/ops-common: prevent migration fallback to non-target nodes
  mm/damon: update outdated comment about DAMOS filter handling
  ...
2026-08-20 18:17:08 -07:00
Linus Torvalds
ae814200e8 bitmap updates for v7.3
- bitmap_find_next_zero_area_off() optimization (Sunyi);
  - bitmap_find_next_zero_area_off(): return size when no zero area is
    found (Yury);
  - bitmap vs IDA vs Maple Tree performance test (Yury);
  - get rid of cpumap_print_to_pagebuf() (Yury);
  - use nr_node_ids in __nodemask_pr_numnodes() (Li RongQing);
  - bitops: make the *_bit_le functions use unsigned long (Benjamin);
  - bitmap scatter & gather test fix (Christophe);
  - use __ASSEMBLER__ in bitmap header files (Thomas);
 -----BEGIN PGP SIGNATURE-----
 
 iQGzBAABCgAdFiEEi8GdvG6xMhdgpu/4sUSA/TofvsgFAmqDX1YACgkQsUSA/Tof
 vsjcpwwAw7130xQPNFREr+ttdoUn3gN/aVUOl7K2Mi3kUOdlg8i9FEaQ/a07nE5L
 ozXRAWgSM7VxW1I7nkj5McsGRiRX0SRoXmTbjDfPugMiXbD1UBPqCz2n0rYHYR4P
 GfQ0RJK8LUN3Sq1eIu27HNirLG1HU1jCm3fsaTft2naaY6TnSPjklZdVE6Phz87E
 eblgm+AO8//EoKsggoMCnGVCVWKE3N1RrWzsaT4ltErGyHvIzvWEbCCk2cI764Qj
 orLLeefPf2xVDlbzywU7uUklNWYRe4miPxrbnT5TH2KmTipb3M62MkGjPa5dzYIq
 ssOvSnKoQEV9od9cNv1hPFGs5b+RO+kOolcrWKTqWeHFgVTv7dZZL0QvuGHeOfOa
 Fc5GPxE2SiXLPVuUEgMhlrg9DsXGf0d9u7JHTnJF+ZL/QhIE+4wOVnIYWUyBZiqx
 zF7Ic62xmeJOKz9kfYg9JQYPwUblhhyYtiWOmzLIisf8FXi5M3L9eH6rO6CEWcrk
 viEkTMbW
 =q8Lf
 -----END PGP SIGNATURE-----

Merge tag 'bitmap-for-7.3' of https://github.com/norov/linux

Pull bitmap updates from Yury Norov:
 "The usual set of fixes, cleanups and performance improvements together
  with a couple of new tests:

   - bitmap_find_next_zero_area_off() optimization (Sunyi)

   - bitmap_find_next_zero_area_off(): return size when no zero area is
     found (Yury)

   - bitmap vs IDA vs Maple Tree performance test (Yury)

   - get rid of cpumap_print_to_pagebuf() (Yury)

   - use nr_node_ids in __nodemask_pr_numnodes() (Li RongQing)

   - bitops: make the *_bit_le functions use unsigned long (Benjamin)

   - bitmap scatter & gather test fix (Christophe)

   - use __ASSEMBLER__ in bitmap header files (Thomas)"

* tag 'bitmap-for-7.3' of https://github.com/norov/linux: (25 commits)
  lib: test bitmap vs IDA vs Maple Tree performance for region allocations
  bitmap: Return size when no zero area is found
  media: s5p-mfc: Treat bitmap size as allocation failure
  crypto: ccp: Treat bitmap size as allocation failure
  powerpc/msi: Treat bitmap size as allocation failure
  ARM: dma-mapping: Treat bitmap size as allocation failure
  bitmap: drop bitmap_next_set_region()
  nodemask: reduce bitmap width to nr_node_ids in __nodemask_pr_numnodes()
  bitmap: Properly initialise destination bitmap for scatter & gather test
  lib/bitmap-str: get rid of cpumap_print_to_pagebuf()
  perf: Use sysfs_emit() for cpumask show callbacks
  PCI/sysfs: Use sysfs_emit() for cpumask show callbacks
  RDMA/hfi1: Use sysfs_emit() for cpumask show helper
  hwtracing: hisi_ptt: Use sysfs_emit() for cpumask show
  fpga: dfl-fme-perf: Use sysfs_emit() for cpumask show
  devfreq: Use sysfs_emit() for cpumask show callbacks
  cpu: Use sysfs_emit() for cpumask show callback
  x86/events: Use sysfs_emit() for cpumask show callbacks
  powerpc: Use sysfs_emit() for cpumask show callbacks
  arm: Use sysfs_emit() for cpumask show callbacks
  ...
2026-08-20 11:41:46 -07:00
Linus Torvalds
5808ac1889 - Get rid of static_cpu_has() - one less API to care about testing CPU
features
 
 - Unify the handling of CPU core types (performance, efficient, etc) by
   mapping the vendor-specific types to Linux ones
 
 - Continuation of the work of Ahmed Darwish to centralize CPUID leafs
   representation
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEzv7L6UO9uDPlPSfHEsHwGGHeVUoFAmqEomYACgkQEsHwGGHe
 VUqrVhAAmN1E0HRaRm2zUcJqDS2TVz9a//LTWI/i9pNtlhPrdGrPtaHfps89TGj3
 1TBiFPE6I4xbuTlumFBToqfPbJ8mw41s3ZnlAUk/QVkf0J7Qkra/tZ9dRsWpjwqJ
 y/PqzgW7qTtpye881VX0fNaOlV4CRAQiGv6StALRo9fXn1lWRfwCfFBjYeuMHS0H
 e8r0shF0petS7JdHyTWmzS3E50ETrGI6XNbkKjWFyMjolaeP3r4YP1ucj/B5KY1E
 CFjdBkgXM3PQ0K4Nax96PA15VoMorIuGhpjiv0phiA9itBay3x6EQfOhXoMWtgiE
 9gbgUfWUCx/n+U5tXwOaZnFBd8gnyT0QOENmojEf8GCIBoGgeff0g6djNAm36QlF
 Zyfq7EOMBCfkooddWEz6qhZdeP7SwKz0i0dHyJ6OroQKXVjnCXAUveyA3/1venYb
 HT/xP/BNDQc4+GpO1ZPCuottpC6CVPclWsVOrcnU4jBtZJRhvNTWQMbDpZUFUefO
 FIJi40u1cnHw1hk7t5dVACJAME5p4hUz9hlfZFMETbkxjFH7cJEKaGFO2JXhSsui
 NaLkCJQWgnJynnhNhdycRsdbC6+pAiayoajGe80hiwYrr3j0YhG5UKSrgUwD6uiq
 CxuFt7C8jLEsAcWBILVB5+S6vh+3peHRAm9m9LtgiDpRAM4vGGI=
 =IKxX
 -----END PGP SIGNATURE-----

Merge tag 'x86_cpu_for_v7.3_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull x86 cpuid updates from Borislav Petkov:

 - Get rid of static_cpu_has() - one less API to care about testing CPU
   features

 - Unify the handling of CPU core types (performance, efficient, etc) by
   mapping the vendor-specific types to Linux ones

 - Continuation of the work of Ahmed Darwish to centralize CPUID leaf
   representation

* tag 'x86_cpu_for_v7.3_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86/CPU: Rename struct cpuid_read_output to struct cpuid_output
  x86/cpu/scattered: Sort it properly
  x86/cpu: Use parsed CPUID(0x1)
  x86/lib: Add CPUID(0x1) family and model calculation
  x86/cpu: Use parsed CPUID(0x0)
  x86/cpu/transmeta: Rescan CPUID(0x1) after modifying capabilities
  x86/topology: Add TOPO_CPU_TYPE_LOW_POWER
  x86/topology: Name the AMD core-type values
  x86/topo: Map vendor CPU types to generic Linux such types
  x86/bugs: Don't use cpu-type matching in cpu_vuln_blacklist
  x86/cpu: Hide and rename static_cpu_has()
2026-08-18 19:09:42 -07:00
Dapeng Mi
dd384d661f perf/x86/intel: Fix intel_cap handling on hybrid PMUs
intel_cap (IA32_PERF_CAPABILITIES) updates are currently tied to
X86_FEATURE_ARCH_PERFMON_EXT, but these are independent feature paths.
As a result, hybrid PMU capability state can be updated under the wrong
condition.

Also, intel_pmu_broken_perf_cap() is too narrow. Per RPL018, the missing
PERF_METRICS_AVAILABLE bit affects both Raptor Lake and Meteor Lake
parts, not only the currently covered subset.

Move intel_cap updates out of the ARCH_PERFMON_EXT-gated path, extend
intel_pmu_broken_perf_cap() coverage to both RPL and MTL families, and
introduce intel_update_pmu_caps() to centralize PMU capability updates.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Thomas Falcon <thomas.falcon@intel.com>
Reviewed-by: Zide Chen <zide.chen@intel.com>
Link: https://patch.msgid.link/20260717080342.1879573-7-dapeng1.mi@linux.intel.com
2026-08-10 15:05:48 +02:00
Dapeng Mi
3ba87c5bf1 perf/x86: Remove stale fixed counter helper and fix hybrid PMU access
On hybrid systems, init_hw_perf_events() can call check_hw_exists() with
the global PMU pointer after perf_is_hybrid is set. In that case,
fixed_counter_disabled() uses hybrid() on a non-hybrid PMU object, so the
intel_ctrl access is taken from the wrong layout and can read out of
bounds.

fixed_counter_disabled() was added in commit 32451614da
("perf/x86/intel: Support CPUID 10.ECX to disable fixed counters"), when
fixed counters were tracked via num_fixed_counters. Today fixed counters
are represented by fixed_cntr_mask, so this helper is obsolete.

Remove fixed_counter_disabled() and its callers, and rely directly on the
fixed-counter bitmask. With the helper gone, check_hw_exists() no longer
needs a PMU argument, so drop that parameter as well. This removes the
invalid hybrid access and closes the out-of-bounds read risk.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Thomas Falcon <thomas.falcon@intel.com>
Reviewed-by: Zide Chen <zide.chen@intel.com>
Link: https://patch.msgid.link/20260717080342.1879573-6-dapeng1.mi@linux.intel.com
2026-08-10 15:05:48 +02:00
Dapeng Mi
85182f902a perf/x86/intel: Unwind cpuc state if PEBS buffer setup fails
intel_pmu_cpu_prepare() allocates per-CPU perf state first and then sets
up the arch PEBS buffer. If alloc_arch_pebs_buf_on_cpu() fails,
the previously allocated cpuc resources are left behind.

Make the failure path call intel_cpuc_finish(cpuc) to release the per-CPU
state allocated by intel_cpuc_prepare().

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Thomas Falcon <thomas.falcon@intel.com>
Reviewed-by: Zide Chen <zide.chen@intel.com>
Link: https://patch.msgid.link/20260717080342.1879573-5-dapeng1.mi@linux.intel.com
2026-08-10 15:05:47 +02:00
Dapeng Mi
278a3731c9 perf/x86: Guard intel_pmu_cpu_dead() against invalid hybrid PMU casts
In failure paths, cpuc->pmu can still point to the global static pmu
instead of an embedded x86_hybrid_pmu::pmu. Calling hybrid_pmu() on
that pointer causes an invalid container conversion and may lead to
out-of-bounds access.

This can happen in at least two cases:
- init_hybrid_pmu() fails check_hw_exists() and leaves cpuc->pmu as-is.
- CPU hotplug fails between CPUHP_PERF_X86_PREPARE and
CPUHP_AP_PERF_X86_STARTING, and rollback invokes intel_pmu_cpu_dead().

Fix both paths by:
- Clear cpuc->pmu to NULL when check_hw_exists() fails.
- Validat that cpuc->pmu is not the global static pmu before calling
  hybrid_pmu() in intel_pmu_cpu_dead().

A new helper x86_get_static_pmu() is added to get the global static pmu.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Thomas Falcon <thomas.falcon@intel.com>
Reviewed-by: Zide Chen <zide.chen@intel.com>
Link: https://patch.msgid.link/20260717080342.1879573-4-dapeng1.mi@linux.intel.com
2026-08-10 15:05:47 +02:00
Adrian Hunter
2e17bf3a46 perf/x86/intel/pt: Fix stop/start with no update
If pt_event_stop() is called without PERF_EF_UPDATE flag, then
perf_aux_output_end() is not called.  A subsequent call to pt_event_start()
will call perf_aux_output_begin() again which violates the rule against
nesting and triggers a WARNING in perf_aux_output_begin().

Originally, pt_event_stop() was never called without PERF_EF_UPDATE,
because the only code paths to do so are from event overflow, and Intel PT
does not do that.

However the introduction of group throttling by commit 9734e25fbf
("perf: Fix the throttle logic for a group") meant that an Intel PT event
could be throttled if it was part of a group.  Throttling calls PMU
 ->stop() / ->start() callbacks without flags.

An example is when AUX area sampling is used. The following commands
hit the issue:

  echo 10000 > /proc/sys/kernel/perf_event_max_sample_rate

  perf record -F32000 --aux-sample -e '{intel_pt//u,cycles:u}' \
  -- bash -c 'for i in `seq 1 100000` ; do true ; done'

Use PERF_HES_UPTODATE to track whether perf_aux_output_begin() and
perf_aux_output_end() are balanced.  A cleared PERF_HES_UPTODATE bit
indicates that an AUX output context is still open.

Amend pt_event_start() / pt_event_stop() accordingly so that begin/end
stay balanced:

 - In non-snapshot mode, stop() always closes the buffer (the buffer may
   have run out of space, and that accounting is done by the update), so
   a following start() opens a fresh one as before.

 - In snapshot/overwrite mode, stop() without PERF_EF_UPDATE leaves the
   buffer open so that pt_event_snapshot_aux() can still copy from it,
   and start() then only re-enables tracing instead of calling
   perf_aux_output_begin() again.

Note that pt_event_del() calls pt_event_stop() with PERF_EF_UPDATE flag set
(as is required by the documentation), so a final call to
perf_aux_output_end() is assured.

Fixes: 52ca9ced3f ("perf/x86/intel/pt: Add Intel PT PMU driver")
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Yi Lai <yi1.lai@intel.com>
Link: https://patch.msgid.link/20260721070254.13557-4-adrian.hunter@intel.com
2026-08-07 18:27:06 +02:00
Adrian Hunter
265bb4ef75 perf/x86/intel/pt: Use bitwise access for PERF_HES_STOPPED
The Intel PT driver reads and writes event->hw.state as a whole value,
assuming it is either 0 or PERF_HES_STOPPED.  That is true today, but a
subsequent fix needs to also track an open AUX output buffer using the
PERF_HES_UPTODATE bit of the same field.

When more than one bit can be set, whole-value assignments would
overwrite the other bits and whole-value comparisons would fail to match.

Convert all accesses to set, clear and test the PERF_HES_STOPPED bit
individually, in preparation for that change.

No functional change intended: event->hw.state currently only ever
holds 0 or PERF_HES_STOPPED, so the bitwise forms are equivalent.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Yi Lai <yi1.lai@intel.com>
Link: https://patch.msgid.link/20260721070254.13557-3-adrian.hunter@intel.com
2026-08-07 18:27:06 +02:00
Adrian Hunter
c6df517796 perf/x86/intel/pt: Factor out pt_config_enable()
pt_config() enables tracing by allowing NMIs and pause/resume, issuing
the necessary barriers, and calling pt_config_start().  A later change
needs to re-enable tracing on a (re-)start path without repeating the
full pt_config() setup (filters, RTIT_CTL, buffer configuration).

Factor that enabling sequence out into a new helper, pt_config_enable(),
so it can be called on its own.

No functional change intended.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Yi Lai <yi1.lai@intel.com>
Link: https://patch.msgid.link/20260721070254.13557-2-adrian.hunter@intel.com
2026-08-07 18:27:06 +02:00
Randy Dunlap
cfd364903a perf/x86/intel/pt: Drop kernel-doc for deleted struct members
@lost and @topa_index are no longer part of struct pt_buffer so
delete their comment lines.

  Warning: ../arch/x86/events/intel/pt.h:86 Excess struct member 'lost' description in 'pt_buffer'
  Warning: ../arch/x86/events/intel/pt.h:86 Excess struct member 'topa_index' description in 'pt_buffer'

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://patch.msgid.link/20260730233429.285788-2-rdunlap@infradead.org
2026-07-31 11:44:23 +02:00
Brendan Jackman
ca74a82522 perf/x86/intel: use higher-level allocator API
The difference between __alloc_pages_node() and alloc_pages_node() is that
the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

No functional change intended.

Link: https://lore.kernel.org/20260703-alloc-trylock-v5-8-c87b714e19d3@google.com
Signed-off-by: Brendan Jackman <jackmanb@google.com>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Ian Rogers <irogers@google.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: James Clark <james.clark@linaro.org>
Assisted-by: Gemini:unknown-version
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2026-07-30 19:40:42 -07:00
Yury Norov
163d9808b9 x86/events: Use sysfs_emit() for cpumask show callbacks
These callbacks are sysfs show paths.

Use sysfs_emit() and cpumask_pr_args() to emit the masks.

This prepares for removing cpumap_print_to_pagebuf().

Signed-off-by: Yury Norov <ynorov@nvidia.com>
2026-07-22 15:38:37 -04:00
Borislav Petkov (AMD)
3eaa50e1e2 x86/cpu: Hide and rename static_cpu_has()
cpu_feature_enabled() is the one to use to test feature flags so hide
the static thing which doesn't pay attention to disabled mask bits
anyway.

Use the following command to do the replacement:

  $ git grep --files-with-matches -w static_cpu_has -- ':(exclude)*cpufeature.h' \
  | xargs sed -i 's/static_cpu_has(/cpu_feature_enabled\(/g'

There should be no functional changes resulting from this.

Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Nikolay Borisov <nik.borisov@suse.com>
Link: https://patch.msgid.link/20260620015041.336288-1-bp@kernel.org
2026-07-06 21:34:26 -07:00
Dapeng Mi
a6b5fbc331 perf/x86/intel: Drop fixed-counter PEBS constraints for baseline PEBS
On Sapphire Rapids (SPR) guests where pebs_baseline is not advertised,
running command

$ ./perf record -e cpu/event=0x00,umask=0x01,i\
	 name=INST_RETIRED.PREC_DIST/p -c 10000 sleep 1

can trigger:

unchecked MSR access error: WRMSR to 0x3f1 ... in\
	 intel_pmu_pebs_enable_all()

This occurs because SPR-specific PEBS constraints allow fixed-counter
scheduling (for example, INST_RETIRED.PREC_DIST on fixed counter 0).
In guests lacking pebs_baseline, KVM does not support PEBS sampling on
fixed counters, so enabling such events reaches an invalid MSR
programming path.

Starting with Icelake, regardless of whether Extended PEBS or
Architectural PEBS is in use, all counters (including fixed counters)
support PEBS sampling, and the PMU_FL_PEBS_ALL flag is set by default.
As long as PMU_FL_PEBS_ALL is set, constraint lookup automatically falls
back to the corresponding non-PEBS constraints if no matching entry is
found in the PEBS constraints table. Since non-PEBS event constraints
already contain the same fixed-counter constraints, it is safe to remove
these fixed-counter entries from the PEBS constraints table. The fallback
mechanism will ensure that fixed PEBS events are scheduled onto the
correct fixed counters.

So, directly drop the fixed-counter entries from the PEBS constraint
table. Without pebs_baseline, these fixed-counter PEBS events will now
resolve to empty constraints and will not be scheduled or enabled,
thereby avoiding the warning and bypassing the broken guest PEBS path.

Reported-by: Yi Lai <yi1.lai@intel.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260616044654.3468742-7-dapeng1.mi@linux.intel.com
2026-06-30 11:46:07 +02:00
Dapeng Mi
01c153956b perf/x86/intel: Validate the return value of intel_pmu_init_hybrid()
The intel_pmu_init_hybrid() function allocates memory for the
x86_pmu.hybrid_pmu[] array. If this allocation fails under memory
pressure, hybrid PMU initialization will fail.

Currently, the caller does not check the return value of
intel_pmu_init_hybrid(). This can lead to a null-pointer dereference or
invalid memory access when attempting to use the uninitialized array,
potentially triggering a system panic.

Fix this by validating the return value of intel_pmu_init_hybrid().
Additionally, reset x86_pmu.num_hybrid_pmus to 0 on failure, and defer
intel_pmu_arch_lbr_init() until after hybrid PMU initialization
succeeds. This reordering avoids the need to explicitly destroy the
kmem cache if the memory allocation fails.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260616044654.3468742-6-dapeng1.mi@linux.intel.com
2026-06-30 11:46:07 +02:00
Dapeng Mi
e2b0575900 perf/x86/intel: Fix kernel address leakages in LBR stack
Before Arch LBR gained CPL filtering support, a user-only branch stack
could still contain kernel addresses. As a result, kernel branch records
may be exposed to user space even when PERF_SAMPLE_BRANCH_USER is
requested.

For example, on Intel Tiger Lake, the following command can still report
SYSRET/ERET entries with kernel-space from addresses:

$ ./perf record -e cycles:p -o - --branch-filter any,save_type,u -- \
 	./perf bench syscall basic --loop 1000 | \
	./perf script -i - --fields brstack|tr ' ' '\n'| \
	grep -E '0x[89a-f][0-9a-f]{15}'

    Total time: 0.000 [sec]

      0.219000 usecs/op
     4,566,210 ops/sec
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.551 MB - ]
0xffffffff93c001c8/0x7f12a2b1d647/P/-/-/16959/SYSRET/-
0xffffffff93c001c8/0x7f12a2b1d5c2/P/-/-/17535/SYSRET/-
0xffffffff93c01928/0x7f12a2861000/P/-/-/6719/ERET/-
0xffffffff93c01928/0x7f12a297a000/P/-/-/8575/ERET/-

The problem is that intel_pmu_lbr_filter() does not fully validate the
privilege level of sampled entries. It filters some mismatches based on
the branch type and the to address, but it does not reject entries whose
from address violates the requested branch privilege filter.

Fix this by extending software filtering to validate both from and to
addresses against br_sel. Any LBR entry contains kernel address does not
match the requested user filter is dropped. This prevents kernel
addresses from appearing in user-only branch stacks.

Fixes: 47125db27e ("perf/x86/intel/lbr: Support Architectural LBR")
Reported-by: Ian Rogers <irogers@google.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260616044654.3468742-5-dapeng1.mi@linux.intel.com
2026-06-30 11:46:06 +02:00
Dapeng Mi
170cc6b02e perf/x86/intel: Fallback to sw branch type decoding if no hw decoding
intel_pmu_lbr_filter() currently assumes arch-LBR provides hardware
branch-type decoding and skips software decoding on that path.

However, arch-LBR may not always expose branch-type information. In
that case, treating entries as hardware-decoded can misclassify sampled
branches (for example, defaulting to JCC), which breaks branch-type
filtering results.

Fix this by using software branch-type decoding when hardware
branch-type decoding is unavailable (that is, when x86_lbr_type is not
enabled). This keeps branch classification and filtering behavior
correct across arch-LBR configurations.

Since x86_lbr_type is only set to true when arch-LBR is supported,
it's unnecessary to check if the CPU has the arch-LBR feature.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260616044654.3468742-4-dapeng1.mi@linux.intel.com
2026-06-30 11:46:06 +02:00
Dapeng Mi
3c4ec9b2a5 perf/x86/intel: Keep cap_user_rdpmc in sync with RDPMC user-disable state
After introducing the RDPMC user disable feature, user-space RDPMC may
return 0 instead of the actual event count. This creates an inconsistency
with cap_user_rdpmc, where cap_user_rdpmc is set, but user-space RDPMC
only returns 0.

To accurately represent the user-space RDPMC capability, update
cap_user_rdpmc (depending on PERF_EVENT_FLAG_USER_READ_CNT) according to
the RDPMC user disable state. If RDPMC user disable is enabled,
cap_user_rdpmc is updated to false eventually, allowing user-space
programs to fall back to the read() syscall to obtain the real event
count.

Because PERF_EVENT_FLAG_USER_READ_CNT is evaluated in
x86_pmu_event_init(), move intel_pmu_update_rdpmc_user_disable()
earlier into intel_pmu_hw_config(). This ensures that the user-disable
state is updated before updating PERF_EVENT_FLAG_USER_READ_CNT. Note that
since event->ctx is not yet assigned at this stage, use the
PERF_ATTACH_TASK flag to detect whether the event is task-attached.

While at it, fix the indentation of x86_pmu_has_rdpmc_user_disable()
to adhere to the kernel coding style.

Fixes: 59af95e028 ("perf/x86/intel: Add support for rdpmc user disable feature")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260616044654.3468742-3-dapeng1.mi@linux.intel.com
2026-06-30 11:46:06 +02:00
Dapeng Mi
8767b4d730 perf/x86/intel: Remove anythread_deprecated bit from perf_capabilities
AnyThread mode deprecation is enumerated by CPUID.0AH:EDX[15] instead of
PERF_CAPABILITIES MSR. It's not a good practice to define a bit to
represent "anythread deprecation" in perf_capabilities. It leads to the
anythread_deprecated bit could be overwritten by the real value of
PERF_CAPABILITIES MSR, just like the below code in update_pmu_cap() does.

if (!intel_pmu_broken_perf_cap()) {
	/* Perf Metric (Bit 15) and PEBS via PT (Bit 16) are hybrid enumeration */
	rdmsrq(MSR_IA32_PERF_CAPABILITIES, hybrid(pmu, intel_cap).capabilities);
}

It leads to the anythread_deprecated bit is cleared to 0 and the "any"
attribute is incorrectly shown in the /sys/devices/cpu/format/ folder on
these support Perfmon v6 platforms, like Clearwater Forest.

$ grep . /sys/devices/cpu/format/*
/sys/devices/cpu/format/acr_mask:config2:0-63
/sys/devices/cpu/format/any:config:21
/sys/devices/cpu/format/cmask:config:24-31

So remove the anythread_deprecated bit from perf_capabilities structure
and directly depends on CPUID.0AH:EDX[15] to judge if anythread is
deprecated.

Fixes: cadbaa039b ("perf/x86/intel: Make anythread filter support conditional")
Reported-by: Namhyung Kim <namhyung@kernel.org>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Zide Chen <zide.chen@intel.com>
Reviewed-by: Thomas Falcon <thomas.falcon@intel.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260616044654.3468742-2-dapeng1.mi@linux.intel.com
2026-06-30 11:46:06 +02:00
Zide Chen
b25813b179 perf/x86/intel/uncore: Implement lazy setup for MSR/MMIO PMUs
MSR and MMIO uncore PMUs are currently registered at module init time
and appear in sysfs even when no PMU boxes are functional.

Apply the same lazy registration model used by PCI uncore PMUs: the
PMU is registered when the first box is successfully initialized, and
unregistered when the last box exits.  If a box fails to initialize on
a subsequent die, the PMU is marked broken but remains registered to
avoid disrupting any in-flight perf events.

Box allocation and freeing remain at module init/exit time to avoid
repeated kfree/alloc cycles across CPU offline/online events.

Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-9-zide.chen@intel.com
2026-06-30 10:57:05 +02:00
Zide Chen
174f0582e3 perf/x86/intel/uncore: Fix uncore_box ref/unref ordering
In uncore_event_cpu_online(), uncore_box_ref() was called before
uncore_change_context().  uncore_box_ref() gates on box->cpu >= 0,
but box->cpu is still -1 at that point because uncore_change_context()
has not run yet.  As a result, the box is never initialized on the
first CPU to come online in a die, leaving it permanently
uninitialized in the single-CPU-per-die case.

Thus, box->refcnt is one count below the true value, and in the CPU
offline path, the box will be torn down on the second-to-last CPU.

In uncore_event_cpu_offline(), uncore_box_unref() was called after
uncore_change_context(), so box->cpu is already -1 when the collector
CPU goes offline, which prevents it from tearing down the box.

Fix by swapping the call order in both paths so that
uncore_box_{ref,unref}() runs at the point where box->cpu reflects
the correct context.

Move allocate_boxes() out of uncore_box_ref() to enable this
reordering.

Fixes: c74443d92f ("perf/x86/uncore: Support per PMU cpumask")
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-8-zide.chen@intel.com
2026-06-30 10:57:04 +02:00
Zide Chen
30c0a10956 perf/x86/intel/uncore: Introduce PMU flags and broken state
Replace the boolean 'registered' field in intel_uncore_pmu with an
unsigned long 'flags' field, and add a PMU_BROKEN flag to track box
setup failures. The broken flag is sticky, meaning it is cleared only
by a module reload or system reboot.

Broken PMUs are skipped in the CPU hotplug and box allocation paths.

When any box fails to initialize, the PMU is marked broken.  Broken
PMUs reject new event assignments and skip future box setup attempts.
If the PMU was already registered, it remains so to avoid disrupting
in-flight events on other boxes.

Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-7-zide.chen@intel.com
2026-06-30 10:57:04 +02:00
Zide Chen
ae7ca8796d perf/x86/intel/uncore: Factor out box setup code
The PCI uncore PMU path already implements a lazy registration model:
the PMU is registered when the first active box appears and
unregistered when the last active box is removed.

Factor this registration management into a shared helper, so the same
code can be reused by the MSR and MMIO paths in later changes.

No functional change intended.

Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-6-zide.chen@intel.com
2026-06-30 10:57:04 +02:00
Zide Chen
3012af7df3 perf/x86/intel/uncore: Keep PCI PMUs working when MMIO/MSR setup fails
uncore_event_cpu_online() returns -ENOMEM early when both the MSR and
MMIO box allocations fail.  This also aborts PCI uncore setup, even
though PCI PMUs are independent of the MSR/MMIO paths.

Remove the early return so PCI uncore setup always runs regardless
of whether MSR or MMIO box allocation succeeds.

Fixes: 3da04b8a00 ("perf/x86/intel/uncore: Support MMIO type uncore blocks")
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-5-zide.chen@intel.com
2026-06-30 10:57:04 +02:00
Zide Chen
cbbc25209c perf/x86/intel/uncore: Let init_box() callback report failures
The init_box() callback currently returns void, so initialization
failures are silently ignored and the box is still marked initialized.

Change the callback to return int so platform code can report errors
back to the common uncore layer.

Update uncore_box_init() to set the initialized flag only when
init_box() succeeds.  Because box->refcnt guarantees that at most
one CPU calls uncore_box_init() for a given box at a time, plain
__set_bit() is safe for the initialized flag without atomic overhead.

Convert all init_box() implementations to return 0 on success or a
negative error code on failure.  This is a prerequisite for propagating
initialization errors to the caller so they can be handled properly.

Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-4-zide.chen@intel.com
2026-06-30 10:57:03 +02:00
Zide Chen
7d3a9ff988 perf/x86/intel/uncore: Fix refcnt and other cleanups
Fix typo UNCORE_BOX_FLAG_INITIATED to UNCORE_BOX_FLAG_INITIALIZED.

Rename the 'id' parameter in uncore_box_{ref,unref}() to 'die' to
reflect its actual meaning and be consistent with other functions.

box->refcnt is incremented in the PCI PMU register path but has never
been checked or decremented. Although for PCI PMUs box->refcnt
effectively tracks only a single user, add atomic_dec_return() in the
PCI PMU unregister path to make the reference counting complete and
consistent.

Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-3-zide.chen@intel.com
2026-06-30 10:57:03 +02:00
Zide Chen
003267cb94 perf/x86/intel/uncore: Fix PCI PMU cleanup on setup failure
When uncore_pci_pmu_register() fails, pmu->boxes[die] is set to NULL
before returning.  In the uncore_pci_remove() path, this causes
uncore_pci_pmu_unregister() to be skipped entirely, leaking
pmu->activeboxes.  In the uncore_bus_notify() path,
uncore_pci_pmu_unregister() may still be called and must exit early
when pmu->boxes[die] is NULL to avoid a NULL pointer dereference, and
to ensure activeboxes is only decremented for a previously active box.

Additionally, since pci_get_drvdata() returns NULL on registration
failure, uncore_pci_remove() can no longer treat NULL drvdata as an
indicator of an auxiliary PCI device.  Remove the associated
WARN_ON_ONCE().

Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://sashiko.dev/#/patchset/20260512233048.9577-1-zide.chen@intel.com?part=1
Link: https://patch.msgid.link/20260611160033.66760-2-zide.chen@intel.com
2026-06-30 10:57:03 +02:00
Linus Torvalds
ff5ccdb8d5 x86/cpu updates for v7.2:
- CPUID API updates (Ahmed S. Darwish):
 
    - Introduce a centralized CPUID parser
    - Introduce a centralized CPUID data model
    - Introduce <asm/cpuid/leaf_types.h>
    - Rename cpuid_leaf()/cpuid_subleaf() APIs
    - treewide: Explicitly include the x86 CPUID headers
 
  - Update to x86-cpuid-db v3.1 (Maciej Wieczor-Retman)
 
  - Continued removal of pre-i586 support and related
    simplifications (Ingo Molnar)
 
  - Add Intel CPU model number for rugged Panther Lake
    (Tony Luck)
 
  - Misc fixes, updates and cleanups by Arnd Bergmann, Chao Gao,
    Lukas Bulwahn, Sohil Mehta, Maciej Wieczor-Retman.
 
 Signed-off-by: Ingo Molnar <mingo@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmou1LcRHG1pbmdvQGtl
 cm5lbC5vcmcACgkQEnMQ0APhK1h16Q/9GvxrQDX13qd7/XrjOKNYtoLIGh1GhvHM
 2ZdbYh80LX36bDOg7Mhoy0bvElwmzaz2x1Zrb5SAIqpQXYqjfuRCe7a1SPQxYtCK
 A6j1YeD/CJMm36jaQITYkuCYxwaw5LQ62u0ShpEvyzZzarEJt8c8COWJjbE57FN2
 qusr+6K1sBwpEUl5mLabbJWXqhpPRYCz+nl2GF9BCHe6z7Nw7Q8VZf4w1weONJOy
 lMpC2X08w5TE2f4OwvnRbZLelyROz6pr1c3osUoQSVtBkprO9TsYHgd9yBva1E2E
 KxdWm+uSSUXM7cFBzY3RLBzKpG6iLtqircoDFBVdxOlT8I8KFggzbYy4JPlDEHQS
 FqUwWL+2duoRciOncnZn9hydxlg5So4w7OscvTVNp94/Pb2pMdpOB+bsN05WTpXc
 VMQVxrfsmwT1rV1oppzRizTMnb6yGg8BQvZPE+wrMDdH+/wwhzok4GdZzKVCzUoD
 nDFhJL7t8ikRwwuq5RAl1EcT7qLFJx3ba77B5REf2Uos2liTExO/XzcmQnzSV1M1
 RXpajQ1ikEBtJPAD3Zy0ASm/QPjB27o+6Kj6hW3NLlPoi5l0Lu6xQzH3wNrpQpst
 02NCWEn2X00YUQt2ukNcDFJiMHJzXAnBQDBSrWwwd97wol3sKnT9RR2NZVFzn7ls
 GOF8yJ0Wuec=
 =tpai
 -----END PGP SIGNATURE-----

Merge tag 'x86-cpu-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip

Pull x86 cpuid updates from Ingo Molnar:

 - CPUID API updates (Ahmed S. Darwish):
    - Introduce a centralized CPUID parser
    - Introduce a centralized CPUID data model
    - Introduce <asm/cpuid/leaf_types.h>
    - Rename cpuid_leaf()/cpuid_subleaf() APIs
    - treewide: Explicitly include the x86 CPUID headers

 - Update to x86-cpuid-db v3.1 (Maciej Wieczor-Retman)

 - Continued removal of pre-i586 support and related simplifications
   (Ingo Molnar)

 - Add Intel CPU model number for rugged Panther Lake (Tony Luck)

 - Misc fixes, updates and cleanups by Arnd Bergmann, Chao Gao, Lukas
   Bulwahn, Sohil Mehta, Maciej Wieczor-Retman.

* tag 'x86-cpu-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (25 commits)
  x86/cpu: Make CONFIG_X86_CX8 unconditional
  x86/cpu: Remove unused !CONFIG_X86_TSC code
  x86/cpuid: Update bitfields to x86-cpuid-db v3.1
  tools/x86/kcpuid: Update bitfields to x86-cpuid-db v3.1
  x86/cpu: Make CONFIG_X86_TSC unconditional
  MAINTAINERS: Drop obsolete FPU EMULATOR section
  x86/cpu: Fix a F00F bug warning and clean up surrounding code
  x86/cpu: Add Intel CPU model number for rugged Panther Lake
  x86/cpuid: Introduce a centralized CPUID parser
  x86/cpu: Introduce a centralized CPUID data model
  x86/cpuid: Introduce <asm/cpuid/leaf_types.h>
  x86/cpuid: Rename cpuid_leaf()/cpuid_subleaf() APIs
  x86/cpu: Do not include the CPUID API header in asm/processor.h
  Documentation: core-api/cpu_hotplug: Remove stale cpu0_hotplug docs
  x86/cpu, cpufreq: Remove AMD ELAN support
  x86/fpu: Remove the math-emu/ FPU emulation library
  x86/fpu: Remove the 'no387' boot option
  x86/fpu: Remove MATH_EMULATION and related glue code
  treewide: Explicitly include the x86 CPUID headers
  x86/cpu: Remove the CONFIG_X86_INVD_BUG quirk
  ...
2026-06-15 15:25:17 +05:30
Linus Torvalds
7561361d76 x86/msr updates for v7.2:
- Large series to reorganize the rdmsr/wrmsr APIs to remove
    32-bit variants and convert to 64-bit variants (Juergen Gross)
 
  - Fix W=1 warning (HyeongJun An)
 
 Signed-off-by: Ingo Molnar <mingo@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmou1ucRHG1pbmdvQGtl
 cm5lbC5vcmcACgkQEnMQ0APhK1gqYRAAstZaQnaca7PjkVbmcESzqkWSF30V3YmV
 P/Fe4eozPttfcRootPGxW/srm+c0c2u07Dk7zYrS5BF8OuX4960HRLwC4kQs4lvq
 F7ngDAtiGjZm5kNtwnITc2CxJLjDlewW9JqRNNK960IgD0H7X3wdkZWk8kox27Up
 2/SwYSZnzUoYVXLzLJjEpKJ6Lx0sHcql+K3tQOK+uXoNLf/cYHoyjXwWRw7X2H6h
 PnxmnaVf9YmM4ghCZlS0hgxsc32cQUFE93ZnBCMQHe5sF5amGBt7Xy/mVwmKxq6h
 0qKx7Y9SuKjT3dOg+h749C5xzGH9iXhFSfl5Tq3tM7B1aGCfn62X1Uz6KMyeqmgT
 av+iS0oCfCJ3vPdHBRy/Q++jovsIPP3Ty+wWYgIoiRLMP9NtWoFuSsSTGjevkC3U
 NiOecBX5SMYJuqSmwVFEDUjIbKcXRgWAqnIYVzjGO1cT48mE4589GflAPPZHWqua
 BZ7GE7COgAXpxa4tktQFv8/9dfpxfc83CXgFR3BLlv777C/abFDSmJmUyLj61FKe
 Lr4AGpcrjMhhbp9ECZdzUb4YO6xUTnFYRAsquDy9/Cg13KiHuxLeSJ4v1q96ly5c
 gW2cX6KyVLCWMms47i1Hknjn+Vf0dfke1esEwGMd8i2Ji4eqD7Dr4ZvV/Ju9N6UB
 QtBs5sxZVq0=
 =YynE
 -----END PGP SIGNATURE-----

Merge tag 'x86-msr-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip

Pull x86/msr updates from Ingo Molnar:

 - Large series to reorganize the rdmsr/wrmsr APIs to remove
   32-bit variants and convert to 64-bit variants (Juergen Gross)

 - Fix W=1 warning (HyeongJun An)

* tag 'x86-msr-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
  x86/msr: Remove wrmsrl()
  x86/msr: Switch wrmsrl() users to wrmsrq()
  x86/msr: Remove rdmsrl()
  x86/msr: Switch rdmsrl() users to rdmsrq()
  x86/msr: Remove wrmsr_safe_on_cpu()
  x86/msr: Switch wrmsr_safe_on_cpu() users to wrmsrq_safe_on_cpu()
  x86/msr: Remove rdmsr_safe_on_cpu()
  x86/msr: Switch rdmsr_safe_on_cpu() users to rdmsrq_safe_on_cpu()
  x86/msr: Don't use rdmsr_safe_on_cpu() in rdmsrq_safe_on_cpu()
  x86/msr: Remove wrmsr_on_cpu()
  x86/msr: Switch wrmsr_on_cpu() users to wrmsrq_on_cpu()
  x86/msr: Remove rdmsr_on_cpu()
  x86/msr: Switch rdmsr_on_cpu() users to rdmsrq_on_cpu()
  x86/msr: Remove rdmsrl_on_cpu()
  x86/msr: Switch rdmsrl_on_cpu() user to rdmsrq_on_cpu()
  x86/process: Convert rdmsr() to rdmsrq() in arch_post_acpi_subsys_init() to address W=1 warning
2026-06-15 15:08:14 +05:30
Linus Torvalds
0bcc2dc22f Performance events updates for v7.2:
Core perf code updates:
 
  - Reveal PMU type in fdinfo (Chun-Tse Shao)
 
 Intel CPU PMU driver updates:
 
  - Fix various inaccurate hard-coded event configurations
    (Dapeng Mi)
 
 Intel uncore PMU driver updates (Zide Chen):
 
  - Fix discovery unit lookup bug for multi-die systems
  - Guard against invalid box control address
  - Fix PCI device refcount leak in UPI discovery
  - Defer ADL global PMON enable to enable_box() to save power
  - Fix uncore_die_to_cpu() for offline dies
  - Implement global init callback for GNR uncore
 
 AMD CPU PMU driver updates:
 
  - Always use the NMI latency mitigation (Sandipan Das)
 
 AMD uncore PMU driver updates:
 
  - Use Node ID to identify DF and UMC domains (Sandipan Das)
 
 Signed-off-by: Ingo Molnar <mingo@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmouvyARHG1pbmdvQGtl
 cm5lbC5vcmcACgkQEnMQ0APhK1hVSA//RmBeZ9R7QiCJn770b+9ZPvk90PKN8TYQ
 TXcPMTB1gZRtw7q4CF8/CFxUvxGySD+/RaekXkQzfATPurwgHkwKv3xpOZuSGcfW
 dTsuAV3k+Azj3on3dGjmGZoZeWsdG12c0uF7YNylW/hfPQ/KAW1THVKv7ufX6Mcx
 VrEh7oh/5N9paAqtZz5TmesB2wfC05vToj27jE0HL6tnDovF4Q8IsZkfC3P139Kd
 iCwvOnc/jpfke26UP5csPnQIF5zTIy2TZrvDQVlqryPZ4NrmMnA6XC117u8VAIm2
 29JBP72wbMvRT4hXSs8thfvPj1zF/y3YMCN0D+wkxpYD6gqlsjlHhoY9fJ+Ann7t
 gMi0ccSGNC6COVMq1RhzrGdf6JscGcIxXzS1abNJw6cLmfv45UmlbF1ePq9nPV3n
 8ZbotrBovS9TonLPmmXNpR9dKKLSH2URCF/xD92KS5Wbvod7sum8N/rpV3Ncmxxf
 6g8qTsOxpJjEq9pnTndycqvtzZnyf0/68db5Y+5ag1zKNVPhbsV7AU/tVytZt1Hk
 1bTNcc7lCp2GRBPAmDCk5KI235C0UpLS/llG2jEEwXmTzLxk7pzwoJMcl5sABHNA
 VL6AhLP8rmHY+r0e8otdI10xg8Pt/WWvKNuOY4i4PiIf34YG1ARoY0EwzmlwcSmF
 WsKdagUmIqQ=
 =Di4Y
 -----END PGP SIGNATURE-----

Merge tag 'perf-core-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip

Pull performance events updates from Ingo Molnar:
 "Core perf code updates:

   - Reveal PMU type in fdinfo (Chun-Tse Shao)

  Intel CPU PMU driver updates:

   - Fix various inaccurate hard-coded event configurations (Dapeng Mi)

  Intel uncore PMU driver updates (Zide Chen):

   - Fix discovery unit lookup bug for multi-die systems

   - Guard against invalid box control address

   - Fix PCI device refcount leak in UPI discovery

   - Defer ADL global PMON enable to enable_box() to save power

   - Fix uncore_die_to_cpu() for offline dies

   - Implement global init callback for GNR uncore

  AMD CPU PMU driver updates:

   - Always use the NMI latency mitigation (Sandipan Das)

  AMD uncore PMU driver updates:

   - Use Node ID to identify DF and UMC domains (Sandipan Das)"

* tag 'perf-core-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (22 commits)
  perf/x86/amd/uncore: Use Node ID to identify DF and UMC domains
  perf: Reveal PMU type in fdinfo
  perf/x86/intel/uncore: Implement global init callback for GNR uncore
  perf/x86/intel/uncore: Fix uncore_die_to_cpu() for offline dies
  perf/x86/intel/uncore: Move die_to_cpu() to uncore.c
  perf/x86/intel/uncore: Defer ADL global PMON enable to enable_box()
  perf/x86/intel/uncore: Fix PCI device refcount leak in UPI discovery
  perf/x86/intel/uncore: Guard against invalid box control address
  perf/x86/intel/uncore: Fix discovery unit lookup for multi-die systems
  perf/x86/amd/core: Always use the NMI latency mitigation
  perf/x86/intel: Update event constraints and cache_extra_regsfor CWF
  perf/x86/intel: Update event constraints and cache_extra_regsfor SRF
  perf/x86/intel: Update event constraints and cache_extra_regsfor NVL
  perf/x86/intel: Update event constraints for PTL
  perf/x86/intel: Update event constraints and cache_extra_regsfor ARL
  perf/x86/intel: Update event constraints and cache_extra_regsfor LNL
  perf/x86/intel: Update event constraints and cache_extra_regsfor MTL
  perf/x86/intel: Update event constraints and cache_extra_regsfor ADL
  perf/x86/intel: Update event constraints for DMR
  perf/x86/intel: Update event constraints and cache_extra_regsfor SPR
  ...
2026-06-15 14:45:41 +05:30