Skip to content
  • Mel Gorman's avatar
    mm, page_alloc: split buffered_rmqueue() · 066b2393
    Mel Gorman authored
    Patch series "Use per-cpu allocator for !irq requests and prepare for a
    bulk allocator", v5.
    
    This series is motivated by a conversation led by Jesper Dangaard Brouer
    at the last LSF/MM proposing a generic page pool for DMA-coherent pages.
    Part of his motivation was due to the overhead of allocating multiple
    order-0 that led some drivers to use high-order allocations and
    splitting them.  This is very slow in some cases.
    
    The first two patches in this series restructure the page allocator such
    that it is relatively easy to introduce an order-0 bulk page allocator.
    A patch exists to do that and has been handed over to Jesper until an
    in-kernel users is created.  The third patch prevents the per-cpu
    allocator being drained from IPI context as that can potentially corrupt
    the list after patch four is merged.  The final patch alters the per-cpu
    alloctor to make it exclusive to !irq requests.  This cuts
    allocation/free overhead by roughly 30%.
    
    Performance tests from both Jesper and me are included in the patch.
    
    This patch (of 4):
    
    buffered_rmqueue removes a page from a given zone and uses the per-cpu
    list for order-0.  This is fine but a hypothetical caller that wanted
    multiple order-0 pages has to disable/reenable interrupts multiple
    times.  This patch structures buffere_rmqueue such that it's relatively
    easy to build a bulk order-0 page allocator.  There is no functional
    change.
    
    [mgorman@techsingularity.net: failed per-cpu refill may blow up]
      Link: http://lkml.kernel.org/r/20170124112723.mshmgwq2ihxku2um@techsingularity.net
    Link: http://lkml.kernel.org/r/20170123153906.3122-2-mgorman@techsingularity.net
    
    
    Signed-off-by: default avatarMel Gorman <mgorman@techsingularity.net>
    Acked-by: default avatarHillf Danton <hillf.zj@alibaba-inc.com>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Cc: Jesper Dangaard Brouer <brouer@redhat.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    066b2393