1. 14 Feb, 2017 4 commits
  2. 13 Feb, 2017 2 commits
  3. 28 Jan, 2017 2 commits
  4. 08 Jan, 2017 1 commit
    • Johannes Weiner's avatar
      mm: workingset: fix use-after-free in shadow node shrinker · ea07b862
      Johannes Weiner authored
      Several people report seeing warnings about inconsistent radix tree
      nodes followed by crashes in the workingset code, which all looked like
      use-after-free access from the shadow node shrinker.
      
      Dave Jones managed to reproduce the issue with a debug patch applied,
      which confirmed that the radix tree shrinking indeed frees shadow nodes
      while they are still linked to the shadow LRU:
      
        WARNING: CPU: 2 PID: 53 at lib/radix-tree.c:643 delete_node+0x1e4/0x200
        CPU: 2 PID: 53 Comm: kswapd0 Not tainted 4.10.0-rc2-think+ #3
        Call Trace:
           delete_node+0x1e4/0x200
           __radix_tree_delete_node+0xd/0x10
           shadow_lru_isolate+0xe6/0x220
           __list_lru_walk_one.isra.4+0x9b/0x190
           list_lru_walk_one+0x23/0x30
           scan_shadow_nodes+0x2e/0x40
           shrink_slab.part.44+0x23d/0x5d0
           shrink_node+0x22c/0x330
           kswapd+0x392/0x8f0
      
      This is the WARN_ON_ONCE(!list_empty(&node->private_list)) placed in the
      inlined radix_tree_shrink().
      
      The problem is with 14b46879 ("mm: workingset: move shadow entry
      tracking to radix tree exceptional tracking"), which passes an update
      callback into the radix tree to link and unlink shadow leaf nodes when
      tree entries change, but forgot to pass the callback when reclaiming a
      shadow node.
      
      While the reclaimed shadow node itself is unlinked by the shrinker, its
      deletion from the tree can cause the left-most leaf node in the tree to
      be shrunk.  If that happens to be a shadow node as well, we don't unlink
      it from the LRU as we should.
      
      Consider this tree, where the s are shadow entries:
      
             root->rnode
                  |
             [0       n]
              |       |
           [s    ] [sssss]
      
      Now the shadow node shrinker reclaims the rightmost leaf node through
      the shadow node LRU:
      
             root->rnode
                  |
             [0        ]
              |
          [s     ]
      
      Because the parent of the deleted node is the first level below the
      root and has only one child in the left-most slot, the intermediate
      level is shrunk and the node containing the single shadow is put in
      its place:
      
             root->rnode
                  |
             [s        ]
      
      The shrinker again sees a single left-most slot in a first level node
      and thus decides to store the shadow in root->rnode directly and free
      the node - which is a leaf node on the shadow node LRU.
      
        root->rnode
             |
             s
      
      Without the update callback, the freed node remains on the shadow LRU,
      where it causes later shrinker runs to crash.
      
      Pass the node updater callback into __radix_tree_delete_node() in case
      the deletion causes the left-most branch in the tree to collapse too.
      
      Also add warnings when linked nodes are freed right away, rather than
      wait for the use-after-free when the list is scanned much later.
      
      Fixes: 14b46879
      
       ("mm: workingset: move shadow entry tracking to radix tree exceptional tracking")
      Reported-by: default avatarDave Chinner <david@fromorbit.com>
      Reported-by: default avatarHugh Dickins <hughd@google.com>
      Reported-by: default avatarAndrea Arcangeli <aarcange@redhat.com>
      Reported-and-tested-by: default avatarDave Jones <davej@codemonkey.org.uk>
      Signed-off-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
      Cc: Christoph Hellwig <hch@lst.de>
      Cc: Chris Leech <cleech@redhat.com>
      Cc: Lee Duncan <lduncan@suse.com>
      Cc: Jan Kara <jack@suse.cz>
      Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
      Cc: Matthew Wilcox <mawilcox@linuxonhyperv.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      ea07b862
  5. 15 Dec, 2016 7 commits
  6. 13 Dec, 2016 4 commits
  7. 11 Oct, 2016 1 commit
    • Ross Zwisler's avatar
      radix-tree: 'slot' can be NULL in radix_tree_next_slot() · 915045fe
      Ross Zwisler authored
      There are four cases I can see where we could end up with a NULL 'slot' in
      radix_tree_next_slot().  Yet radix_tree_next_slot() never actually checks
      whether 'slot' is NULL.  It just happens that for the cases where 'slot'
      is NULL, some other combination of factors prevents us from dereferencing
      it.
      
      It would be very easy for someone to unwittingly change one of these
      factors without realizing that we are implicitly depending on it to save
      us from a NULL pointer dereference.
      
      Add a comment documenting the things that allow 'slot' to be safely passed
      as NULL to radix_tree_next_slot().
      
      Here are details on the four cases:
      
      1) radix_tree_iter_retry() via a non-tagged iteration like
      radix_tree_for_each_slot().  In this case we currently aren't seeing a bug
      because radix_tree_iter_retry() sets
      
      	iter->next_index = iter->index;
      
      which means that in in the else case in radix_tree_next_slot(), 'count' is
      zero, so we skip over the while() loop and effectively just return NULL
      without ever dereferencing 'slot'.
      
      2) radix_tree_iter_retry() via tagged iteration like
      radix_tree_for_each_tagged().  This case was giving us NULL pointer
      dereferences in testing, and was fixed with this commit:
      
      commit 3cb9185c ("radix-tree: fix radix_tree_iter_retry() for tagged
      iterators.")
      
      This fix doesn't explicitly check for 'slot' being NULL, though, it works
      around the NULL pointer dereference by instead zeroing iter->tags in
      radix_tree_iter_retry(), which makes us bail out of the if() case in
      radix_tree_next_slot() before we dereference 'slot'.
      
      3) radix_tree_iter_next() via via a non-tagged iteration like
      radix_tree_for_each_slot().  This currently happens in shmem_tag_pins()
      and shmem_partial_swap_usage().
      
      As with non-tagged iteration, 'count' in the else case of
      radix_tree_next_slot() is zero, so we skip over the while() loop and
      effectively just return NULL without ever dereferencing 'slot'.
      
      4) radix_tree_iter_next() via tagged iteration like
      radix_tree_for_each_tagged().  This happens in shmem_wait_for_pins().
      
      radix_tree_iter_next() zeros out iter->tags, so we end up exiting
      radix_tree_next_slot() here:
      
      	if (flags & RADIX_TREE_ITER_TAGGED) {
      		void *canon = slot;
      
      		iter->tags >>= 1;
      		if (unlikely(!iter->tags))
      			return NULL;
      
      Link: http://lkml.kernel.org/r/20160815194237.25967-2-ross.zwisler@linux.intel.com
      
      Signed-off-by: default avatarRoss Zwisler <ross.zwisler@linux.intel.com>
      Cc: Konstantin Khlebnikov <koct9i@gmail.com>
      Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
      Cc: Dmitry Vyukov <dvyukov@google.com>
      Cc: Shuah Khan <shuahkh@osg.samsung.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      915045fe
  8. 05 Oct, 2016 1 commit
    • Johannes Weiner's avatar
      mm: filemap: don't plant shadow entries without radix tree node · d3798ae8
      Johannes Weiner authored
      When the underflow checks were added to workingset_node_shadow_dec(),
      they triggered immediately:
      
        kernel BUG at ./include/linux/swap.h:276!
        invalid opcode: 0000 [#1] SMP
        Modules linked in: isofs usb_storage fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_REJECT nf_reject_ipv6
         soundcore wmi acpi_als pinctrl_sunrisepoint kfifo_buf tpm_tis industrialio acpi_pad pinctrl_intel tpm_tis_core tpm nfsd auth_rpcgss nfs_acl lockd grace sunrpc dm_crypt
        CPU: 0 PID: 20929 Comm: blkid Not tainted 4.8.0-rc8-00087-gbe67d60b #1
        Hardware name: System manufacturer System Product Name/Z170-K, BIOS 1803 05/06/2016
        task: ffff8faa93ecd940 task.stack: ffff8faa7f478000
        RIP: page_cache_tree_insert+0xf1/0x100
        Call Trace:
          __add_to_page_cache_locked+0x12e/0x270
          add_to_page_cache_lru+0x4e/0xe0
          mpage_readpages+0x112/0x1d0
          blkdev_readpages+0x1d/0x20
          __do_page_cache_readahead+0x1ad/0x290
          force_page_cache_readahead+0xaa/0x100
          page_cache_sync_readahead+0x3f/0x50
          generic_file_read_iter+0x5af/0x740
          blkdev_read_iter+0x35/0x40
          __vfs_read+0xe1/0x130
          vfs_read+0x96/0x130
          SyS_read+0x55/0xc0
          entry_SYSCALL_64_fastpath+0x13/0x8f
        Code: 03 00 48 8b 5d d8 65 48 33 1c 25 28 00 00 00 44 89 e8 75 19 48 83 c4 18 5b 41 5c 41 5d 41 5e 5d c3 0f 0b 41 bd ef ff ff ff eb d7 <0f> 0b e8 88 68 ef ff 0f 1f 84 00
        RIP  page_cache_tree_insert+0xf1/0x100
      
      This is a long-standing bug in the way shadow entries are accounted in
      the radix tree nodes. The shrinker needs to know when radix tree nodes
      contain only shadow entries, no pages, so node->count is split in half
      to count shadows in the upper bits and pages in the lower bits.
      
      Unfortunately, the radix tree implementation doesn't know of this and
      assumes all entries are in node->count. When there is a shadow entry
      directly in root->rnode and the tree is later extended, the radix tree
      implementation will copy that entry into the new node and and bump its
      node->count, i.e. increases the page count bits. Once the shadow gets
      removed and we subtract from the upper counter, node->count underflows
      and triggers the warning. Afterwards, without node->count reaching 0
      again, the radix tree node is leaked.
      
      Limit shadow entries to when we have actual radix tree nodes and can
      count them properly. That means we lose the ability to detect refaults
      from files that had only the first page faulted in at eviction time.
      
      Fixes: 449dd698
      
       ("mm: keep page cache radix tree nodes in check")
      Signed-off-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
      Reported-and-tested-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      Reviewed-by: default avatarJan Kara <jack@suse.cz>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: stable@vger.kernel.org
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      d3798ae8
  9. 02 Aug, 2016 1 commit
  10. 26 Jul, 2016 1 commit
  11. 23 Jul, 2016 1 commit
  12. 21 May, 2016 13 commits
  13. 17 May, 2016 1 commit
  14. 17 Mar, 2016 1 commit