36e66c554b
In order to pave the way for free page reporting in virtualized environments we will need a way to get pages out of the free lists and identify those pages after they have been returned. To accomplish this, this patch adds the concept of a Reported Buddy, which is essentially meant to just be the Uptodate flag used in conjunction with the Buddy page type. To prevent the reported pages from leaking outside of the buddy lists I added a check to clear the PageReported bit in the del_page_from_free_list function. As a result any reported page that is split, merged, or allocated will have the flag cleared prior to the PageBuddy value being cleared. The process for reporting pages is fairly simple. Once we free a page that meets the minimum order for page reporting we will schedule a worker thread to start 2s or more in the future. That worker thread will begin working from the lowest supported page reporting order up to MAX_ORDER - 1 pulling unreported pages from the free list and storing them in the scatterlist. When processing each individual free list it is necessary for the worker thread to release the zone lock when it needs to stop and report the full scatterlist of pages. To reduce the work of the next iteration the worker thread will rotate the free list so that the first unreported page in the free list becomes the first entry in the list. It will then call a reporting function providing information on how many entries are in the scatterlist. Once the function completes it will return the pages to the free area from which they were allocated and start over pulling more pages from the free areas until there are no longer enough pages to report on to keep the worker busy, or we have processed as many pages as were contained in the free area when we started processing the list. The worker thread will work in a round-robin fashion making its way though each zone requesting reporting, and through each reportable free list within that zone. Once all free areas within the zone have been processed it will check to see if there have been any requests for reporting while it was processing. If so it will reschedule the worker thread to start up again in roughly 2s and exit. Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: Mel Gorman <mgorman@techsingularity.net> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: David Hildenbrand <david@redhat.com> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Cc: Luiz Capitulino <lcapitulino@redhat.com> Cc: Matthew Wilcox <willy@infradead.org> Cc: Michael S. Tsirkin <mst@redhat.com> Cc: Michal Hocko <mhocko@kernel.org> Cc: Nitesh Narayan Lal <nitesh@redhat.com> Cc: Oscar Salvador <osalvador@suse.de> Cc: Pankaj Gupta <pagupta@redhat.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Rik van Riel <riel@surriel.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Wei Wang <wei.w.wang@intel.com> Cc: Yang Zhang <yang.zhang.wz@gmail.com> Cc: wei qi <weiqi4@huawei.com> Link: http://lkml.kernel.org/r/20200211224635.29318.19750.stgit@localhost.localdomain Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
54 lines
1.6 KiB
C
54 lines
1.6 KiB
C
/* SPDX-License-Identifier: GPL-2.0 */
|
|
#ifndef _MM_PAGE_REPORTING_H
|
|
#define _MM_PAGE_REPORTING_H
|
|
|
|
#include <linux/mmzone.h>
|
|
#include <linux/pageblock-flags.h>
|
|
#include <linux/page-isolation.h>
|
|
#include <linux/jump_label.h>
|
|
#include <linux/slab.h>
|
|
#include <asm/pgtable.h>
|
|
#include <linux/scatterlist.h>
|
|
|
|
#define PAGE_REPORTING_MIN_ORDER pageblock_order
|
|
|
|
#ifdef CONFIG_PAGE_REPORTING
|
|
DECLARE_STATIC_KEY_FALSE(page_reporting_enabled);
|
|
void __page_reporting_notify(void);
|
|
|
|
static inline bool page_reported(struct page *page)
|
|
{
|
|
return static_branch_unlikely(&page_reporting_enabled) &&
|
|
PageReported(page);
|
|
}
|
|
|
|
/**
|
|
* page_reporting_notify_free - Free page notification to start page processing
|
|
*
|
|
* This function is meant to act as a screener for __page_reporting_notify
|
|
* which will determine if a give zone has crossed over the high-water mark
|
|
* that will justify us beginning page treatment. If we have crossed that
|
|
* threshold then it will start the process of pulling some pages and
|
|
* placing them in the batch list for treatment.
|
|
*/
|
|
static inline void page_reporting_notify_free(unsigned int order)
|
|
{
|
|
/* Called from hot path in __free_one_page() */
|
|
if (!static_branch_unlikely(&page_reporting_enabled))
|
|
return;
|
|
|
|
/* Determine if we have crossed reporting threshold */
|
|
if (order < PAGE_REPORTING_MIN_ORDER)
|
|
return;
|
|
|
|
/* This will add a few cycles, but should be called infrequently */
|
|
__page_reporting_notify();
|
|
}
|
|
#else /* CONFIG_PAGE_REPORTING */
|
|
#define page_reported(_page) false
|
|
|
|
static inline void page_reporting_notify_free(unsigned int order)
|
|
{
|
|
}
|
|
#endif /* CONFIG_PAGE_REPORTING */
|
|
#endif /*_MM_PAGE_REPORTING_H */
|