Skip to content
  • Derek's avatar
    mremap should return -ENOMEM when __vm_enough_memory fail · 12215182
    Derek authored
    Recently I straced bash behavior in this dd zero pipe to read test, in
    part of testing under vm.overcommit_memory=2 (OVERCOMMIT_NEVER mode):
    
        # dd if=/dev/zero | read x
    
    The bash sub shell is calling mremap to reallocate more and more memory
    untill it finally failed -ENOMEM (I expect), or to be killed by system OOM
    killer (which should not happen under OVERCOMMIT_NEVER mode); But the
    mremap system call actually failed of -EFAULT, which is a surprise to me,
    I think it's supposed to be -ENOMEM?  then I wrote this piece of C code
    testing confirmed it: https://gist.github.com/crquan/326bde37e1ddda8effe5
    
        $ ./remap
        allocated one page @0x7f686bf71000, (PAGE_SIZE: 4096)
        grabbed 7680512000 bytes of memory (1875125 pages) @ 00007f6690993000.
        mremap failed Bad address (14).
    
    The -EFAULT comes from the branch of security_vm_enough_memory_mm failure,
    underlyingly it calls __vm_enough_memory which returns only 0 for success
    or -ENOMEM; So why vma_to_resize needs to return -EFAULT in this case?
    this sounds like a mistake to me.
    
    Some more digging into git history:
    
    1) Before commit 119f657c ("RLIMIT_AS checking fix") in May 1 2005
       (pre 2.6.12 days) it was returning -ENOMEM for this failure;
    
    2) but commit 119f657c ("untangling do_mremap(), part 1") changed it
       accidentally, to what ever is preserved in local ret, which happened to
       be -EFAULT, in a previous assignment;
    
    3) then in commit 54f5de70
    
     code refactoring, it's explicitly returning
       -EFAULT, should be wrong.
    
    Signed-off-by: default avatarDerek Che <crquan@ymail.com>
    Acked-by: default avatarDavid Rientjes <rientjes@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    12215182