• Andrea Arcangeli's avatar
    mm: tail page refcounting optimization for slab and hugetlbfs · 44518d2b
    Andrea Arcangeli authored
    This skips the _mapcount mangling for slab and hugetlbfs pages.
    The main trouble in doing this is to guarantee that PageSlab and
    PageHeadHuge remains constant for all get_page/put_page run on the tail
    of slab or hugetlbfs compound pages.  Otherwise if they're set during
    get_page but not set during put_page, the _mapcount of the tail page
    would underflow.
    PageHeadHuge will remain true until the compound page is released and
    enters the buddy allocator so it won't risk to change even if the tail
    page is the last reference left on the page.
    PG_slab instead is cleared before the slab frees the head page with
    put_page, so if the tail pin is released after the slab freed the page,
    we would have a problem.  But in the slab case the tail pin cannot be
    the last reference left on the page.  This is because the slab code is
    free to reuse the compound page after a kfree/kmem_cache_free without
    having to check if there's any tail pin left.  In turn all tail pins
    must be always released while the head is still pinned by the slab code
    and so we know PG_slab will be still set too.
    Signed-off-by: default avatarAndrea Arcangeli <aarcange@redhat.com>
    Reviewed-by: default avatarKhalid Aziz <khalid.aziz@oracle.com>
    Cc: Pravin Shelar <pshelar@nicira.com>
    Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
    Cc: Ben Hutchings <bhutchings@solarflare.com>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Johannes Weiner <jweiner@redhat.com>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: Andi Kleen <andi@firstfloor.org>
    Cc: Minchan Kim <minchan@kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>