kpriv/linux-hardened
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

mm/mapping_dirty_helpers: update huge page-table entry callbacks

Browse source 
Following the update of pagewalk code commit a07984d48146 ("mm: pagewalk:
add p4d_entry() and pgd_entry()") we can modify the mapping_dirty_helpers'
huge page-table entry callbacks to avoid splitting when a huge pud or -pmd
is encountered.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Steven Price <steven.price@arm.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Link: http://lkml.kernel.org/r/20200203154305.15045-1-thomas_os@shipmail.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This commit is contained in:
Thomas Hellstrom 2020-04-01 21:07:42 -07:00 committed by Linus Torvalds
parent 48fe267c50
commit b2a403fdd1
1 changed files with 38 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

42
mm/mapping_dirty_helpers.c
View file

@ -111,26 +111,60 @@ static int clean_record_pte(pte_t *pte, unsigned long addr,
return 0;
}
/* wp_clean_pmd_entry - The pagewalk pmd callback. */
/*
* wp_clean_pmd_entry - The pagewalk pmd callback.
*
* Dirty-tracking should take place on the PTE level, so
* WARN() if encountering a dirty huge pmd.
* Furthermore, never split huge pmds, since that currently
* causes dirty info loss. The pagefault handler should do
* that if needed.
*/
static int wp_clean_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
/* Dirty-tracking should be handled on the pte level */
pmd_t pmdval = pmd_read_atomic(pmd);
if (!pmd_trans_unstable(&pmdval))
return 0;
if (pmd_none(pmdval)) {
walk->action = ACTION_AGAIN;
return 0;
}
/* Huge pmd, present or migrated */
walk->action = ACTION_CONTINUE;
if (pmd_trans_huge(pmdval) || pmd_devmap(pmdval))
WARN_ON(pmd_write(pmdval) || pmd_dirty(pmdval));
return 0;
}
/* wp_clean_pud_entry - The pagewalk pud callback. */
/*
* wp_clean_pud_entry - The pagewalk pud callback.
*
* Dirty-tracking should take place on the PTE level, so
* WARN() if encountering a dirty huge puds.
* Furthermore, never split huge puds, since that currently
* causes dirty info loss. The pagefault handler should do
* that if needed.
*/
static int wp_clean_pud_entry(pud_t *pud, unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
/* Dirty-tracking should be handled on the pte level */
pud_t pudval = READ_ONCE(*pud);
if (!pud_trans_unstable(&pudval))
return 0;
if (pud_none(pudval)) {
walk->action = ACTION_AGAIN;
return 0;
}
/* Huge pud */
walk->action = ACTION_CONTINUE;
if (pud_trans_huge(pudval) || pud_devmap(pudval))
WARN_ON(pud_write(pudval) || pud_dirty(pudval));