github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-09-23 22:14:03 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

x86/mm/pat: Don't gate cpa_lock on debug_pagealloc_enabled()

Browse Source 
The splitting and merging of kernel page table mappings between small and
large is protected by cpa_lock. The merging is relatively new but the
splitting is ancient.

The splitting has a locking optimization: since DEBUG_PAGEALLOC forces all
mappings to 4k, there are no large pages to split. So the code that *might*
cause a split can just skip the locking (and a few other things).

This is entertaining, but it adds complexity and makes for weird locking
rules. Plus it's all for a debugging feature which makes the kernel super
slow in the first place. Optimizing something which is already super slow
and not used in production is not the best way to spend our complexity
budget.

Stop gating cpa_lock on debug_pagealloc_enabled() to simplify the code
and the locking rules.

[ dhansen: flesh out changelog ]

Suggested-by: Dave Hansen <dave.hansen@intel.com>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260715144519.934289-1-rppt@kernel.org
Link: https://lore.kernel.org/all/aab44f08-89f8-47fe-bee4-0ab6b25968c6@intel.com/
This commit is contained in:
Mike Rapoport (Microsoft) 2026-07-15 17:45:19 +03:00 committed by Dave Hansen
parent defbd61c2a
commit 5fce67641a
1 changed files with 7 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
arch/x86/mm/pat/set_memory.c
View File

@ -62,10 +62,9 @@ enum cpa_warn {
static const int cpa_warn_level = CPA_PROTECT; static const int cpa_warn_level = CPA_PROTECT;
/* /*
* Serialize cpa() (for !DEBUG_PAGEALLOC which uses large identity mappings) * Serialize cpa() using cpa_lock so that we don't allow any other cpu, with
* using cpa_lock. So that we don't allow any other cpu, with stale large tlb * stale large tlb entries, to change the page attribute in parallel to some
* entries change the page attribute in parallel to some other cpu * other cpu splitting a large page entry along with changing the attribute.
* splitting a large page entry along with changing the attribute.
*/ */
static DEFINE_SPINLOCK(cpa_lock); static DEFINE_SPINLOCK(cpa_lock);
@ -1234,11 +1233,9 @@ static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
{ {
struct ptdesc *ptdesc; struct ptdesc *ptdesc;
if (!debug_pagealloc_enabled()) spin_unlock(&cpa_lock);
spin_unlock(&cpa_lock);
ptdesc = pagetable_alloc(GFP_KERNEL, 0); ptdesc = pagetable_alloc(GFP_KERNEL, 0);
if (!debug_pagealloc_enabled()) spin_lock(&cpa_lock);
spin_lock(&cpa_lock);
if (!ptdesc) if (!ptdesc)
return -ENOMEM; return -ENOMEM;
@ -2022,11 +2019,9 @@ static int __change_page_attr_set_clr(struct cpa_data *cpa, int primary)
if (cpa->flags & (CPA_ARRAY | CPA_PAGES_ARRAY)) if (cpa->flags & (CPA_ARRAY | CPA_PAGES_ARRAY))
cpa->numpages = 1; cpa->numpages = 1;
if (!debug_pagealloc_enabled()) spin_lock(&cpa_lock);
spin_lock(&cpa_lock);
ret = __change_page_attr(cpa, primary); ret = __change_page_attr(cpa, primary);
if (!debug_pagealloc_enabled()) spin_unlock(&cpa_lock);
spin_unlock(&cpa_lock);
if (ret) if (ret)
goto out; goto out;