1. 23 Aug, 2013 1 commit
  2. 14 Aug, 2013 1 commit
    • Andrey Vagin's avatar
      memcg: don't initialize kmem-cache destroying work for root caches · 3e6b11df
      Andrey Vagin authored
      struct memcg_cache_params has a union.  Different parts of this union
      are used for root and non-root caches.  A part with destroying work is
      used only for non-root caches.
      
      I fixed the same problem in another place v3.9-rc1-16204-gf101a946
      
      , but
      didn't notice this one.
      
      This patch fixes the kernel panic:
      
      [   46.848187] BUG: unable to handle kernel paging request at 000000fffffffeb8
      [   46.849026] IP: [<ffffffff811a484c>] kmem_cache_destroy_memcg_children+0x6c/0xc0
      [   46.849092] PGD 0
      [   46.849092] Oops: 0000 [#1] SMP
      ...
      Signed-off-by: default avatarAndrey Vagin <avagin@openvz.org>
      Cc: Glauber Costa <glommer@openvz.org>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Acked-by: default avatarMichal Hocko <mhocko@suse.cz>
      Cc: Balbir Singh <bsingharora@gmail.com>
      Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Konstantin Khlebnikov <khlebnikov@openvz.org>
      Cc: <stable@vger.kernel.org>    [3.9.x]
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      3e6b11df
  3. 31 Jul, 2013 1 commit
    • Michal Hocko's avatar
      vmpressure: make sure there are no events queued after memcg is offlined · 33cb876e
      Michal Hocko authored
      
      
      vmpressure is called synchronously from reclaim where the target_memcg
      is guaranteed to be alive but the eventfd is signaled from the work
      queue context.  This means that memcg (along with vmpressure structure
      which is embedded into it) might go away while the work item is pending
      which would result in use-after-release bug.
      
      We have two possible ways how to fix this.  Either vmpressure pins memcg
      before it schedules vmpr->work and unpin it in vmpressure_work_fn or
      explicitely flush the work item from the css_offline context (as
      suggested by Tejun).
      
      This patch implements the later one and it introduces vmpressure_cleanup
      which flushes the vmpressure work queue item item.  It hooks into
      mem_cgroup_css_offline after the memcg itself is cleaned up.
      
      [akpm@linux-foundation.org: coding-style fixes]
      Signed-off-by: default avatarMichal Hocko <mhocko@suse.cz>
      Reported-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Anton Vorontsov <anton.vorontsov@linaro.org>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Cc: Li Zefan <lizefan@huawei.com>
      Acked-by: default avatarTejun Heo <tj@kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      33cb876e
  4. 14 Jul, 2013 1 commit
    • Paul Gortmaker's avatar
      kernel: delete __cpuinit usage from all core kernel files · 0db0628d
      Paul Gortmaker authored
      The __cpuinit type of throwaway sections might have made sense
      some time ago when RAM was more constrained, but now the savings
      do not offset the cost and complications.  For example, the fix in
      commit 5e427ec2 ("x86: Fix bit corruption at CPU resume time")
      is a good example of the nasty type of bugs that can be created
      with improper use of the various __init prefixes.
      
      After a discussion on LKML[1] it was decided that cpuinit should go
      the way of devinit and be phased out.  Once all the users are gone,
      we can then finally remove the macros themselves from linux/init.h.
      
      This removes all the uses of the __cpuinit macros from C files in
      the core kernel directories (kernel, init, lib, mm, and include)
      that don't really have a specific maintainer.
      
      [1] https://lkml.org/lkml/2013/5/20/589
      
      Signed-off-by: default avatarPaul Gortmaker <paul.gortmaker@windriver.com>
      0db0628d
  5. 09 Jul, 2013 12 commits
  6. 03 Jul, 2013 2 commits
  7. 12 Jun, 2013 2 commits
    • Johannes Weiner's avatar
      mm: memcontrol: fix lockless reclaim hierarchy iterator · 89dc991f
      Johannes Weiner authored
      
      
      The lockless reclaim hierarchy iterator currently has a misplaced
      barrier that can lead to use-after-free crashes.
      
      The reclaim hierarchy iterator consist of a sequence count and a
      position pointer that are read and written locklessly, with memory
      barriers enforcing ordering.
      
      The write side sets the position pointer first, then updates the
      sequence count to "publish" the new position.  Likewise, the read side
      must read the sequence count first, then the position.  If the sequence
      count is up to date, it's guaranteed that the position is up to date as
      well:
      
        writer:                         reader:
        iter->position = position       if iter->sequence == expected:
        smp_wmb()                           smp_rmb()
        iter->sequence = sequence           position = iter->position
      
      However, the read side barrier is currently misplaced, which can lead to
      dereferencing stale position pointers that no longer point to valid
      memory.  Fix this.
      Signed-off-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
      Reported-by: default avatarTejun Heo <tj@kernel.org>
      Reviewed-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarMichal Hocko <mhocko@suse.cz>
      Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Glauber Costa <glommer@parallels.com>
      Cc: <stable@kernel.org>		[3.10+]
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      89dc991f
    • Andrey Vagin's avatar
      memcg: don't initialize kmem-cache destroying work for root caches · f101a946
      Andrey Vagin authored
      
      
      struct memcg_cache_params has a union.  Different parts of this union
      are used for root and non-root caches.  A part with destroying work is
      used only for non-root caches.
      
        BUG: unable to handle kernel paging request at 0000000fffffffe0
        IP: kmem_cache_alloc+0x41/0x1f0
        Modules linked in: netlink_diag af_packet_diag udp_diag tcp_diag inet_diag unix_diag ip6table_filter ip6_tables i2c_piix4 virtio_net virtio_balloon microcode i2c_core pcspkr floppy
        CPU: 0 PID: 1929 Comm: lt-vzctl Tainted: G      D      3.10.0-rc1+ #2
        Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
        RIP: kmem_cache_alloc+0x41/0x1f0
        Call Trace:
         getname_flags.part.34+0x30/0x140
         getname+0x38/0x60
         do_sys_open+0xc5/0x1e0
         SyS_open+0x22/0x30
         system_call_fastpath+0x16/0x1b
        Code: f4 53 48 83 ec 18 8b 05 8e 53 b7 00 4c 8b 4d 08 21 f0 a8 10 74 0d 4c 89 4d c0 e8 1b 76 4a 00 4c 8b 4d c0 e9 92 00 00 00 4d 89 f5 <4d> 8b 45 00 65 4c 03 04 25 48 cd 00 00 49 8b 50 08 4d 8b 38 49
        RIP  [<ffffffff8116b641>] kmem_cache_alloc+0x41/0x1f0
      Signed-off-by: default avatarAndrey Vagin <avagin@openvz.org>
      Cc: Konstantin Khlebnikov <khlebnikov@openvz.org>
      Cc: Glauber Costa <glommer@parallels.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: Balbir Singh <bsingharora@gmail.com>
      Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Reviewed-by: default avatarMichal Hocko <mhocko@suse.cz>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: <stable@vger.kernel.org>	[3.9.x]
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      f101a946
  8. 24 May, 2013 1 commit
  9. 08 May, 2013 1 commit
  10. 29 Apr, 2013 11 commits
    • Li Zefan's avatar
      memcg: take reference before releasing rcu_read_lock · ca0dde97
      Li Zefan authored
      
      
      The memcg is not referenced, so it can be destroyed at anytime right
      after we exit rcu read section, so it's not safe to access it.
      
      To fix this, we call css_tryget() to get a reference while we're still
      in rcu read section.
      
      This also removes a bogus comment above __memcg_create_cache_enqueue().
      Signed-off-by: default avatarLi Zefan <lizefan@huawei.com>
      Acked-by: default avatarGlauber Costa <glommer@parallels.com>
      Acked-by: default avatarMichal Hocko <mhocko@suse.cz>
      Acked-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      ca0dde97
    • David Rientjes's avatar
      mm, memcg: give exiting processes access to memory reserves · 465adcf1
      David Rientjes authored
      
      
      A memcg may livelock when oom if the process that grabs the hierarchy's
      oom lock is never the first process with PF_EXITING set in the memcg's
      task iteration.
      
      The oom killer, both global and memcg, will defer if it finds an
      eligible process that is in the process of exiting and it is not being
      ptraced.  The idea is to allow it to exit without using memory reserves
      before needlessly killing another process.
      
      This normally works fine except in the memcg case with a large number of
      threads attached to the oom memcg.  In this case, the memcg oom killer
      only gets called for the process that grabs the hierarchy's oom lock;
      all others end up blocked on the memcg's oom waitqueue.  Thus, if the
      process that grabs the hierarchy's oom lock is never the first
      PF_EXITING process in the memcg's task iteration, the oom killer is
      constantly deferred without anything making progress.
      
      The fix is to give PF_EXITING processes access to memory reserves so
      that we've marked them as oom killed without any iteration.  This allows
      __mem_cgroup_try_charge() to succeed so that the process may exit.  This
      makes the memcg oom killer exemption for TIF_MEMDIE tasks, now
      immediately granted for processes with pending SIGKILLs and those in the
      exit path, to be equivalent to what is done for the global oom killer.
      Signed-off-by: default avatarDavid Rientjes <rientjes@google.com>
      Acked-by: default avatarMichal Hocko <mhocko@suse.cz>
      Acked-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      465adcf1
    • Li Zefan's avatar
      memcg: avoid accessing memcg after releasing reference · fd0ccaf2
      Li Zefan authored
      
      
      This might cause a use-after-free bug.
      Signed-off-by: default avatarLi Zefan <lizefan@huawei.com>
      Cc: Glauber Costa <glommer@parallels.com>
      Acked-by: default avatarMichal Hocko <mhocko@suse.cz>
      Acked-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      fd0ccaf2
    • Anton Vorontsov's avatar
      memcg: add memory.pressure_level events · 70ddf637
      Anton Vorontsov authored
      With this patch userland applications that want to maintain the
      interactivity/memory allocation cost can use the pressure level
      notifications.  The levels are defined like this:
      
      The "low" level means that the system is reclaiming memory for new
      allocations.  Monitoring this reclaiming activity might be useful for
      maintaining cache level.  Upon notification, the program (typically
      "Activity Manager") might analyze vmstat and act in advance (i.e.
      prematurely shutdown unimportant services).
      
      The "medium" level means that the system is experiencing medium memory
      pressure, the system might be making swap, paging out active file
      caches, etc.  Upon this event applications may decide to further analyze
      vmstat/zoneinfo/memcg or internal memory usage statistics and free any
      resources that can be easily reconstructed or re-read from a disk.
      
      The "critical" level means that the system is actively thrashing, it is
      about to out of memory (OOM) or even the in-kernel OOM killer is on its
      way to trigger.  Applications should do whatever they can to help the
      system.  It might be too late to consult with vmstat or any other
      statistics, so it's advisable to take an immediate action.
      
      The events are propagated upward until the event is handled, i.e.  the
      events are not pass-through.  Here is what this means: for example you
      have three cgroups: A->B->C.  Now you set up an event listener on
      cgroups A, B and C, and suppose group C experiences some pressure.  In
      this situation, only group C will receive the notification, i.e.  groups
      A and B will not receive it.  This is done to avoid excessive
      "broadcasting" of messages, which disturbs the system and which is
      especially bad if we are low on memory or thrashing.  So, organize the
      cgroups wisely, or propagate the events manually (or, ask us to
      implement the pass-through events, explaining why would you need them.)
      
      Performance wise, the memory pressure notifications feature itself is
      lightweight and does not require much of bookkeeping, in contrast to the
      rest of memcg features.  Unfortunately, as of current memcg
      implementation, pages accounting is an inseparable part and cannot be
      turned off.  The good news is that there are some efforts[1] to improve
      the situation; plus, implementing the same, fully API-compatible[2]
      interface for CONFIG_MEMCG=n case (e.g.  embedded) is also a viable
      option, so it will not require any changes on the userland side.
      
      [1] http://permalink.gmane.org/gmane.linux.kernel.cgroups/6291
      [2] http://lkml.org/lkml/2013/2/21/454
      
      
      
      [akpm@linux-foundation.org: coding-style fixes]
      [akpm@linux-foundation.org: fix CONFIG_CGROPUPS=n warnings]
      Signed-off-by: default avatarAnton Vorontsov <anton.vorontsov@linaro.org>
      Acked-by: default avatarKirill A. Shutemov <kirill@shutemov.name>
      Acked-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Tejun Heo <tj@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Pekka Enberg <penberg@kernel.org>
      Cc: Mel Gorman <mgorman@suse.de>
      Cc: Glauber Costa <glommer@parallels.com>
      Cc: Michal Hocko <mhocko@suse.cz>
      Cc: Luiz Capitulino <lcapitulino@redhat.com>
      Cc: Greg Thelen <gthelen@google.com>
      Cc: Leonid Moiseichuk <leonid.moiseichuk@nokia.com>
      Cc: KOSAKI Motohiro <kosaki.motohiro@gmail.com>
      Cc: Minchan Kim <minchan@kernel.org>
      Cc: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
      Cc: John Stultz <john.stultz@linaro.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      70ddf637
    • Michel Lespinasse's avatar
      mm/memcontrol.c: remove unnecessary ; · 573b400d
      Michel Lespinasse authored
      
      
      Just a trivial issue I stumbled on while doing something else...
      Signed-off-by: default avatarMichel Lespinasse <walken@google.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      573b400d
    • Michal Hocko's avatar
      memcg: do not check for do_swap_account in mem_cgroup_{read,write,reset} · acb6d558
      Michal Hocko authored
      Since commit 2d11085e
      
       ("memcg: do not create memsw files if swap
      accounting is disabled") memsw files are created only if memcg swap
      accounting is enabled so it doesn't make any sense to check for it
      explicitly in mem_cgroup_read(), mem_cgroup_write() and
      mem_cgroup_reset().
      Signed-off-by: default avatarMichal Hocko <mhocko@suse.cz>
      Cc: Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Tejun Heo <tj@kernel.org>
      Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      acb6d558
    • Michal Hocko's avatar
      memcg: further simplify mem_cgroup_iter · 16248d8f
      Michal Hocko authored
      
      
      mem_cgroup_iter basically does two things currently.  It takes care of
      the house keeping (reference counting, raclaim cookie) and it iterates
      through a hierarchy tree (by using cgroup generic tree walk).  The code
      would be much more easier to follow if we move the iteration outside of
      the function (to __mem_cgrou_iter_next) so the distinction is more
      clear.  This patch doesn't introduce any functional changes.
      Signed-off-by: default avatarMichal Hocko <mhocko@suse.cz>
      Acked-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Ying Han <yinghan@google.com>
      Cc: Tejun Heo <htejun@gmail.com>
      Cc: Glauber Costa <glommer@parallels.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      16248d8f
    • Michal Hocko's avatar
      memcg: simplify mem_cgroup_iter · 19f39402
      Michal Hocko authored
      
      
      The current implementation of mem_cgroup_iter has to consider both css
      and memcg to find out whether no group has been found (css==NULL - aka
      the loop is completed) and that no memcg is associated with the found
      node (!memcg - aka css_tryget failed because the group is no longer
      alive).  This leads to awkward tweaks like tests for css && !memcg to
      skip the current node.
      
      It will be much easier if we got rid off css variable altogether and
      only rely on memcg.  In order to do that the iteration part has to skip
      dead nodes.  This sounds natural to me and as a nice side effect we will
      get a simple invariant that memcg is always alive when non-NULL and all
      nodes have been visited otherwise.
      
      We could get rid of the surrounding while loop but keep it in for now to
      make review easier.  It will go away in the following patch.
      Signed-off-by: default avatarMichal Hocko <mhocko@suse.cz>
      Acked-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Ying Han <yinghan@google.com>
      Cc: Tejun Heo <htejun@gmail.com>
      Cc: Glauber Costa <glommer@parallels.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      19f39402
    • Michal Hocko's avatar
      memcg: relax memcg iter caching · 5f578161
      Michal Hocko authored
      
      
      Now that the per-node-zone-priority iterator caches memory cgroups
      rather than their css ids we have to be careful and remove them from the
      iterator when they are on the way out otherwise they might live for
      unbounded amount of time even though their group is already gone (until
      the global/targeted reclaim triggers the zone under priority to find out
      the group is dead and let it to find the final rest).
      
      We can fix this issue by relaxing rules for the last_visited memcg.
      Instead of taking a reference to the css before it is stored into
      iter->last_visited we can just store its pointer and track the number of
      removed groups from each memcg's subhierarchy.
      
      This number would be stored into iterator everytime when a memcg is
      cached.  If the iter count doesn't match the curent walker root's one we
      will start from the root again.  The group counter is incremented
      upwards the hierarchy every time a group is removed.
      
      The iter_lock can be dropped because racing iterators cannot leak the
      reference anymore as the reference count is not elevated for
      last_visited when it is cached.
      
      Locking rules got a bit complicated by this change though.  The iterator
      primarily relies on rcu read lock which makes sure that once we see a
      valid last_visited pointer then it will be valid for the whole RCU walk.
      smp_rmb makes sure that dead_count is read before last_visited and
      last_dead_count while smp_wmb makes sure that last_visited is updated
      before last_dead_count so the up-to-date last_dead_count cannot point to
      an outdated last_visited.  css_tryget then makes sure that the
      last_visited is still alive in case the iteration races with the cached
      group removal (css is invalidated before mem_cgroup_css_offline
      increments dead_count).
      
      In short:
      mem_cgroup_iter
       rcu_read_lock()
       dead_count = atomic_read(parent->dead_count)
       smp_rmb()
       if (dead_count != iter->last_dead_count)
       	last_visited POSSIBLY INVALID -> last_visited = NULL
       if (!css_tryget(iter->last_visited))
       	last_visited DEAD -> last_visited = NULL
       next = find_next(last_visited)
       css_tryget(next)
       css_put(last_visited) 	// css would be invalidated and parent->dead_count
       			// incremented if this was the last reference
       iter->last_visited = next
       smp_wmb()
       iter->last_dead_count = dead_count
       rcu_read_unlock()
      
      cgroup_rmdir
       cgroup_destroy_locked
        atomic_add(CSS_DEACT_BIAS, &css->refcnt) // subsequent css_tryget fail
         mem_cgroup_css_offline
          mem_cgroup_invalidate_reclaim_iterators
           while(parent = parent_mem_cgroup)
           	atomic_inc(parent->dead_count)
        css_put(css) // last reference held by cgroup core
      
      Spotted by Ying Han.
      
      Original idea from Johannes Weiner.
      
      [akpm@linux-foundation.org: coding-style fixes]
      Signed-off-by: default avatarMichal Hocko <mhocko@suse.cz>
      Signed-off-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
      Acked-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Ying Han <yinghan@google.com>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Tejun Heo <htejun@gmail.com>
      Cc: Glauber Costa <glommer@parallels.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      5f578161
    • Michal Hocko's avatar
      memcg: rework mem_cgroup_iter to use cgroup iterators · 542f85f9
      Michal Hocko authored
      mem_cgroup_iter curently relies on css->id when walking down a group
      hierarchy tree.  This is really awkward because the tree walk depends on
      the groups creation ordering.  The only guarantee is that a parent node is
      visited before its children.
      
      Example:
      
       1) mkdir -p a a/d a/b/c
       2) mkdir -a a/b/c a/d
      
      Will create the same trees but the tree walks will be different:
      
       1) a, d, b, c
       2) a, b, c, d
      
      Commit 574bd9f7
      
       ("cgroup: implement generic child / descendant walk
      macros") has introduced generic cgroup tree walkers which provide either
      pre-order or post-order tree walk.  This patch converts css->id based
      iteration to pre-order tree walk to keep the semantic with the original
      iterator where parent is always visited before its subtree.
      
      cgroup_for_each_descendant_pre suggests using post_create and
      pre_destroy for proper synchronization with groups addidition resp.
      removal.  This implementation doesn't use those because a new memory
      cgroup is initialized sufficiently for iteration in mem_cgroup_css_alloc
      already and css reference counting enforces that the group is alive for
      both the last seen cgroup and the found one resp.  it signals that the
      group is dead and it should be skipped.
      
      If the reclaim cookie is used we need to store the last visited group
      into the iterator so we have to be careful that it doesn't disappear in
      the mean time.  Elevated reference count on the css keeps it alive even
      though the group have been removed (parked waiting for the last dput so
      that it can be freed).
      
      Per node-zone-prio iter_lock has been introduced to ensure that
      css_tryget and iter->last_visited is set atomically.  Otherwise two
      racing walkers could both take a references and only one release it
      leading to a css leak (which pins cgroup dentry).
      Signed-off-by: default avatarMichal Hocko <mhocko@suse.cz>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Ying Han <yinghan@google.com>
      Cc: Tejun Heo <htejun@gmail.com>
      Cc: Glauber Costa <glommer@parallels.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      542f85f9
    • Michal Hocko's avatar
      memcg: keep prev's css alive for the whole mem_cgroup_iter · c40046f3
      Michal Hocko authored
      The patchset tries to make mem_cgroup_iter saner in the way how it walks
      hierarchies.  css->id based traversal is far from being ideal as it is not
      deterministic because it depends on the creation ordering.  Additional to
      that css_id is considered a burden for cgroup maintainers because it is
      quite some code and memcg is the last user of it.  After this series only
      the swap accounting uses css_id but that one will follow up later.
      
      Diffstat (if we exclude removed/added comments) looks quite
      promising. We got rid of some code:
      
        $ git diff mmotm... | grep -v "^[+-][[:space:]]*[/ ]\*" | diffstat
         b/include/linux/cgroup.h |    3 ---
         kernel/cgroup.c          |   33 ---------------------------------
         mm/memcontrol.c          |    4 +++-
         3 files changed, 3 insertions(+), 37 deletions(-)
      
      The first patch is just preparatory and it changes when we release css of
      the previously returned memcg.  Nothing controlversial.
      
      The second patch is the core of the patchset and it replaces css_get_next
      based on css_id by the generic cgroup pre-order.  This brings some
      chalanges for the last visited group caching during the reclaim
      (mem_cgroup_per_zone::reclaim_iter).  We have to use memcg pointers
      directly now which means that we have to keep a reference to those groups'
      css to keep them alive.
      
      I also folded iter_lock introduced by https://lkml.org/lkml/2013/1/3/295
      in the previous version into this patch.  Johannes felt the race I was
      describing should be mostly harmless and I haven't been able to trigger it
      so the lock doesn't deserve its own patch.  It is still needed
      temporarily, though, because the reference counting on iter->last_visited
      depends on it.  It will go away with the next patch.
      
      The next patch fixups an unbounded cgroup removal holdoff caused by the
      elevated css refcount.  The issue has been observed by Ying Han.  Johannes
      wasn't impressed by the previous version of the fix
      (https://lkml.org/lkml/2013/2/8/379
      
      ) which cleaned up pending references
      during mem_cgroup_css_offline when a group is removed.  He has suggested a
      different way when the iterator checks whether a cached memcg is still
      valid or no.  More on that in the patch but the basic idea is that every
      memcg tracks the number removed subgroups and iterator records this number
      when a group is cached.  These numbers are checked before
      iter->last_visited is about to be used and the iteration is restarted if
      it is invalid.
      
      The fourth and fifth patches are an attempt for simplification of the
      mem_cgroup_iter.  css juggling is removed and the iteration logic is moved
      to a helper so that the reference counting and iteration are separated.
      
      The last patch just removes css_get_next as there is no user for it any
      longer.
      
      My testing looked as follows:
              A (use_hierarchy=1, limit_in_bytes=150M)
             /|\
            1 2 3
      
      Children groups were created so that the number is never higher than 3 and
      their limits were random between 50-100M.  Each group hosts a kernel build
      (starting with tar -xf so the tree is not shared and make -jNUM_CPUs/3)
      and terminated after random time - up to 5 minutes) and then it is
      removed.
      
      This should exercise both leaf and hierarchical reclaim as well as races
      with cgroup removals and debugging messages I added on top proved that.
      100 groups were created during the test.
      
      This patch:
      
      css reference counting keeps the cgroup alive even though it has been
      already removed.  mem_cgroup_iter relies on this fact and takes a
      reference to the returned group.  The reference is then released on the
      next iteration or mem_cgroup_iter_break.  mem_cgroup_iter currently
      releases the reference right after it gets the last css_id.
      
      This is correct because neither prev's memcg nor cgroup are accessed after
      then.  This will change in the next patch so we need to hold the group
      alive a bit longer so let's move the css_put at the end of the function.
      Signed-off-by: default avatarMichal Hocko <mhocko@suse.cz>
      Acked-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Ying Han <yinghan@google.com>
      Cc: Tejun Heo <htejun@gmail.com>
      Cc: Glauber Costa <glommer@parallels.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      c40046f3
  11. 15 Apr, 2013 1 commit
  12. 07 Apr, 2013 1 commit
  13. 08 Mar, 2013 1 commit
    • Konstantin Khlebnikov's avatar
      memcg: initialize kmem-cache destroying work earlier · 15cf17d2
      Konstantin Khlebnikov authored
      
      
      Fix a warning from lockdep caused by calling cancel_work_sync() for
      uninitialized struct work.  This path has been triggered by destructon
      kmem-cache hierarchy via destroying its root kmem-cache.
      
        cache ffff88003c072d80
        obj ffff88003b410000 cache ffff88003c072d80
        obj ffff88003b924000 cache ffff88003c20bd40
        INFO: trying to register non-static key.
        the code is fine but needs lockdep annotation.
        turning off the locking correctness validator.
        Pid: 2825, comm: insmod Tainted: G           O 3.9.0-rc1-next-20130307+ #611
        Call Trace:
          __lock_acquire+0x16a2/0x1cb0
          lock_acquire+0x8a/0x120
          flush_work+0x38/0x2a0
          __cancel_work_timer+0x89/0xf0
          cancel_work_sync+0xb/0x10
          kmem_cache_destroy_memcg_children+0x81/0xb0
          kmem_cache_destroy+0xf/0xe0
          init_module+0xcb/0x1000 [kmem_test]
          do_one_initcall+0x11a/0x170
          load_module+0x19b0/0x2320
          SyS_init_module+0xc6/0xf0
          system_call_fastpath+0x16/0x1b
      
      Example module to demonstrate:
      
        #include <linux/module.h>
        #include <linux/slab.h>
        #include <linux/mm.h>
        #include <linux/workqueue.h>
      
        int __init mod_init(void)
        {
        	int size = 256;
        	struct kmem_cache *cache;
        	void *obj;
        	struct page *page;
      
        	cache = kmem_cache_create("kmem_cache_test", size, size, 0, NULL);
        	if (!cache)
        		return -ENOMEM;
      
        	printk("cache %p\n", cache);
      
        	obj = kmem_cache_alloc(cache, GFP_KERNEL);
        	if (obj) {
        		page = virt_to_head_page(obj);
        		printk("obj %p cache %p\n", obj, page->slab_cache);
        		kmem_cache_free(cache, obj);
        	}
      
        	flush_scheduled_work();
      
        	obj = kmem_cache_alloc(cache, GFP_KERNEL);
        	if (obj) {
        		page = virt_to_head_page(obj);
        		printk("obj %p cache %p\n", obj, page->slab_cache);
        		kmem_cache_free(cache, obj);
        	}
      
        	kmem_cache_destroy(cache);
      
        	return -EBUSY;
        }
      
        module_init(mod_init);
        MODULE_LICENSE("GPL");
      Signed-off-by: default avatarKonstantin Khlebnikov <khlebnikov@openvz.org>
      Cc: Glauber Costa <glommer@parallels.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      15cf17d2
  14. 24 Feb, 2013 4 commits