Commit df3a0a21 authored by Pingfan Liu's avatar Pingfan Liu Committed by Linus Torvalds
Browse files

mm/gup: fix omission of check on FOLL_LONGTERM in gup fast path

FOLL_LONGTERM is a special case of FOLL_PIN.  It suggests a pin which is
going to be given to hardware and can't move.  It would truncate CMA
permanently and should be excluded.

In gup slow path, where
__gup_longterm_locked->check_and_migrate_cma_pages() handles
FOLL_LONGTERM, but in fast path, there lacks such a check, which means a
possible leak of CMA page to longterm pinned.

Place a check in try_grab_compound_head() in the fast path to fix the
leak, and if FOLL_LONGTERM happens on CMA, it will fall back to slow path
to migrate the page.

Some note about the check: Huge page's subpages have the same migrate type
due to either allocation from a free_list[] or alloc_contig_range() with
param MIGRATE_MOVABLE.  So it is enough to check on a single subpage by
Signed-off-by: default avatarPingfan Liu <>
Signed-off-by: default avatarAndrew Morton <>
Reviewed-by: default avatarChristoph Hellwig <>
Reviewed-by: default avatarJason Gunthorpe <>
Cc: Ira Weiny <>
Cc: Mike Rapoport <>
Cc: Dan Williams <>
Cc: Matthew Wilcox <>
Cc: John Hubbard <>
Cc: "Aneesh Kumar K.V" <>
Cc: Christoph Hellwig <>
Cc: Shuah Khan <>
Cc: Jason Gunthorpe <>

Signed-off-by: default avatarLinus Torvalds <>
parent 4628b063
......@@ -88,6 +88,14 @@ static __maybe_unused struct page *try_grab_compound_head(struct page *page,
else if (flags & FOLL_PIN) {
int orig_refs = refs;
* Can't do FOLL_LONGTERM + FOLL_PIN with CMA in the gup fast
* path, so fail and let the caller fall back to the slow path.
if (unlikely(flags & FOLL_LONGTERM) &&
return NULL;
* When pinning a compound page of order > 1 (which is what
* hpage_pincount_available() checks for), use an exact count to
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment