1. 09 May, 2017 1 commit
  2. 14 Jul, 2016 1 commit
    • Paul Gortmaker's avatar
      x86/mm: Audit and remove any unnecessary uses of module.h · 4b599fed
      Paul Gortmaker authored
      
      
      Historically a lot of these existed because we did not have
      a distinction between what was modular code and what was providing
      support to modules via EXPORT_SYMBOL and friends.  That changed
      when we forked out support for the latter into the export.h file.
      
      This means we should be able to reduce the usage of module.h
      in code that is obj-y Makefile or bool Kconfig.  The advantage
      in doing so is that module.h itself sources about 15 other headers;
      adding significantly to what we feed cpp, and it can obscure what
      headers we are effectively using.
      
      Since module.h was the source for init.h (for __init) and for
      export.h (for EXPORT_SYMBOL) we consider each obj-y/bool instance
      for the presence of either and replace accordingly where needed.
      
      Note that some bool/obj-y instances remain since module.h is
      the header for some exception table entry stuff, and for things
      like __init_or_module (code that is tossed when MODULES=n).
      Signed-off-by: default avatarPaul Gortmaker <paul.gortmaker@windriver.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Link: http://lkml.kernel.org/r/20160714001901.31603-3-paul.gortmaker@windriver.com
      
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      4b599fed
  3. 02 Feb, 2014 1 commit
  4. 03 Jul, 2013 1 commit
  5. 31 Jan, 2013 1 commit
    • Dave Hansen's avatar
      x86-32, mm: Rip out x86_32 NUMA remapping code · f03574f2
      Dave Hansen authored
      This code was an optimization for 32-bit NUMA systems.
      
      It has probably been the cause of a number of subtle bugs over
      the years, although the conditions to excite them would have
      been hard to trigger.  Essentially, we remap part of the kernel
      linear mapping area, and then sometimes part of that area gets
      freed back in to the bootmem allocator.  If those pages get
      used by kernel data structures (say mem_map[] or a dentry),
      there's no big deal.  But, if anyone ever tried to use the
      linear mapping for these pages _and_ cared about their physical
      address, bad things happen.
      
      For instance, say you passed __GFP_ZERO to the page allocator
      and then happened to get handed one of these pages, it zero the
      remapped page, but it would make a pte to the _old_ page.
      There are probably a hundred other ways that it could screw
      with things.
      
      We don't need to hang on to performance optimizations for
      these old boxes any more.  All my 32-bit NUMA systems are long
      dead and buried, and I probably had access to more than most
      people.
      
      This code is causing real things to break today:
      
      	https://lkml.org/lkml/2013/1/9/376
      
      I looked in to actually fixing this, but it requires surgery
      to way too much brittle code, as well as stuff like
      per_cpu_ptr_to_phys().
      
      [ hpa: Cc: this for -stable, since it is a memory corruption issue.
        However, an alternative is to simply mark NUMA as depends BROKEN
        rather than EXPERIMENTAL in the X86_32 subclause... ]
      
      Link: http://lkml.kernel.org/r/20130131005616.1C79F411@kernel.stglabs.ibm.com
      
      Signed-off-by: default avatarH. Peter Anvin <hpa@linux.intel.com>
      Cc: <stable@vger.kernel.org>
      f03574f2
  6. 14 Jul, 2011 1 commit
  7. 13 Jul, 2011 2 commits
  8. 02 May, 2011 10 commits
    • Tejun Heo's avatar
      x86, NUMA: Make 32bit use common NUMA init path · bd6709a9
      Tejun Heo authored
      
      
      With both _numa_init() methods converted and the rest of init code
      adjusted, numa_32.c now can switch from the 32bit only init code to
      the common one in numa.c.
      
      * Shim get_memcfg_*()'s are dropped and initmem_init() calls
        x86_numa_init(), which is updated to handle NUMAQ.
      
      * All boilerplate operations including node range limiting, pgdat
        alloc/init are handled by numa_init().  32bit only implementation is
        removed.
      
      * 32bit numa_add_memblk(), numa_set_distance() and
        memory_add_physaddr_to_nid() removed and common versions in
        numa_32.c enabled for 32bit.
      
      This change causes the following behavior changes.
      
      * NODE_DATA()->node_start_pfn/node_spanned_pages properly initialized
        for 32bit too.
      
      * Much more sanity checks and configuration cleanups.
      
      * Proper handling of node distances.
      
      * The same NUMA init messages as 64bit.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      bd6709a9
    • Tejun Heo's avatar
      x86, NUMA: Initialize and use remap allocator from setup_node_bootmem() · 7888e96b
      Tejun Heo authored
      
      
      setup_node_bootmem() is taken from 64bit and doesn't use remap
      allocator.  It's about to be shared with 32bit so add support for it.
      If NODE_DATA is remapped, it's noted in the debug message and node
      locality check is skipped as the __pa() of the remapped address
      doesn't reflect the actual physical address.
      
      On 64bit, remap allocator becomes noop and doesn't affect the
      behavior.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      7888e96b
    • Tejun Heo's avatar
      x86-32, NUMA: Add @start and @end to init_alloc_remap() · 99cca492
      Tejun Heo authored
      
      
      Instead of dereferencing node_start/end_pfn[] directly, make
      init_alloc_remap() take @start and @end and let the caller be
      responsible for making sure the range is sane.  This is to prepare for
      use from unified NUMA init code.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      99cca492
    • Tejun Heo's avatar
      x86, NUMA: Enable build of generic NUMA init code on 32bit · 744baba0
      Tejun Heo authored
      
      
      Generic NUMA init code was moved to numa.c from numa_64.c but is still
      guaraded by CONFIG_X86_64.  This patch removes the compile guard and
      enables compiling on 32bit.
      
      * numa_add_memblk() and numa_set_distance() clash with the shim
        implementation in numa_32.c and are left out.
      
      * memory_add_physaddr_to_nid() clashes with 32bit implementation and
        is left out.
      
      * MAX_DMA_PFN definition in dma.h moved out of !CONFIG_X86_32.
      
      * node_data definition in numa_32.c removed in favor of the one in
        numa.c.
      
      There are places where ulong is assumed to be 64bit.  The next patch
      will fix them up.  Note that although the code is compiled it isn't
      used yet and this patch doesn't cause any functional change.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      744baba0
    • Tejun Heo's avatar
      x86-32, NUMA: Update numaq to use new NUMA init protocol · 299a180a
      Tejun Heo authored
      
      
      Update numaq such that it calls numa_add_memblk() and sets
      numa_nodes_parsed instead of directly diddling with NUMA states.  The
      original get_memcfg_numaq() is renamed to numaq_numa_init() and new
      get_memcfg_numaq() is created in numa_32.c.
      
      The shim numa_add_memblk() implementation handles node_start/end_pfn[]
      and node_set_online() for nodes with memory.  The new
      get_memcfg_numaq() exactly the same with get_memcfg_from_srat() other
      than calling the numaq init function.  Things get_memcfgs_numaq() do
      are not strictly necessary for numaq but added for consistency and to
      help unifying NUMA init handling.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      299a180a
    • Tejun Heo's avatar
      x86-32, NUMA: Replace srat_32.c with srat.c · 5acd91ab
      Tejun Heo authored
      
      
      SRAT support implementation in srat_32.c and srat.c are generally
      similar; however, there are some differences.
      
      First of all, 64bit implementation supports more types of SRAT
      entries.  64bit supports x2apic, affinity, memory and SLIT.  32bit
      only supports processor and memory.
      
      Most other differences stem from different initialization protocols
      employed by 64bit and 32bit NUMA init paths.
      
      On 64bit,
      
      * Mappings among PXM, node and apicid are directly done in each SRAT
        entry callback.
      
      * Memory affinity information is passed to numa_add_memblk() which
        takes care of all interfacing with NUMA init.
      
      * Doesn't directly initialize NUMA configurations.  All the
        information is recorded in numa_nodes_parsed and memblks.
      
      On 32bit,
      
      * Checks numa_off.
      
      * Things go through one more level of indirection via private tables
        but eventually end up initializing the same mappings.
      
      * node_start/end_pfn[] are initialized and
        memblock_x86_register_active_regions() is called for each memory
        chunk.
      
      * node_set_online() is called for each online node.
      
      * sort_node_map() is called.
      
      There are also other minor differences in sanity checking and messages
      but taking 64bit version should be good enough.
      
      This patch drops the 32bit specific implementation and makes the 64bit
      implementation common for both 32 and 64bit.
      
      The init protocol differences are dealt with in two places - the
      numa_add_memblk() shim added in the previous patch and new temporary
      numa_32.c:get_memcfg_from_srat() which wraps invocation of
      x86_acpi_numa_init().
      
      The shim numa_add_memblk() handles the folowings.
      
      * node_start/end_pfn[] initialization.
      
      * node_set_online() for memory nodes.
      
      * Invocation of memblock_x86_register_active_regions().
      
      The shim get_memcfg_from_srat() handles the followings.
      
      * numa_off check.
      
      * node_set_online() for CPU nodes.
      
      * sort_node_map() invocation.
      
      * Clearing of numa_nodes_parsed and active_ranges on failure.
      
      The shims are temporary and will be removed as the generic NUMA init
      path in 32bit is replaced with 64bit one.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      5acd91ab
    • Tejun Heo's avatar
      x86-32, NUMA: implement temporary NUMA init shims · b0d31080
      Tejun Heo authored
      
      
      To help transition to common NUMA init, implement temporary 32bit
      shims for numa_add_memblk() and numa_set_distance().
      numa_add_memblk() registers the memblk and adjusts
      node_start/end_pfn[].  numa_set_distance() is noop.
      
      These shims will allow using 64bit NUMA init functions on 32bit and
      gradual transition to common NUMA init path.
      
      For detailed description, please read description of commits which
      make use of the shim functions.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      b0d31080
    • Tejun Heo's avatar
      x86-32, NUMA: Move get_memcfg_numa() into numa_32.c · daf4f480
      Tejun Heo authored
      
      
      There's no reason get_memcfg_numa() to be implemented inline in
      mmzone_32.h.  Move it to numa_32.c and also make
      get_memcfg_numa_flag() static.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      daf4f480
    • Tejun Heo's avatar
      x86-32, NUMA: use sparse_memory_present_with_active_regions() · 797390d8
      Tejun Heo authored
      
      
      Instead of calling memory_present() for each region from NUMA init,
      call sparse_memory_present_with_active_regions() from paging_init()
      similarly to x86-64.
      
      For flat and numaq, this results in exactly the same memory_present()
      calls.  For srat, if there are multiple memory chunks for a node,
      after this change, memory_present() will be called separately for each
      chunk instead of being called once to encompass the whole range, which
      doesn't cause any harm and actually is the better behavior.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      797390d8
    • Tejun Heo's avatar
      x86, NUMA: Unify 32/64bit numa_cpu_node() implementation · 6bd26273
      Tejun Heo authored
      
      
      Currently, the only meaningful user of apic->x86_32_numa_cpu_node() is
      NUMAQ which returns valid mapping only after CPU is initialized during
      SMP bringup; thus, the previous patch to set apicid -> node in
      setup_local_APIC() makes __apicid_to_node[] always contain the correct
      mapping whether custom apic->x86_32_numa_cpu_node() is used or not.
      
      So, there is no reason to keep separate 32bit implementation.  We can
      always consult __apicid_to_node[].  Move 64bit implementation from
      numa_64.c to numa.c and remove 32bit implementation from numa_32.c.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      6bd26273
  9. 07 Apr, 2011 14 commits
  10. 16 Feb, 2011 2 commits
    • Tejun Heo's avatar
      x86, NUMA: Move *_numa_init() invocations into initmem_init() · d8fc3afc
      Tejun Heo authored
      
      
      There's no reason for these to live in setup_arch().  Move them inside
      initmem_init().
      
      - v2: x86-32 initmem_init() weren't updated breaking 32bit builds.
        Fixed.  Found by Ankita.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Ankita Garg <ankita@in.ibm.com>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      d8fc3afc
    • Tejun Heo's avatar
      x86, NUMA: Drop @start/last_pfn from initmem_init() · 86ef4dbf
      Tejun Heo authored
      
      
      initmem_init() extensively accesses and modifies global data
      structures and the parameters aren't even followed depending on which
      path is being used.  Drop @start/last_pfn and let it deal with
      @max_pfn directly.  This is in preparation for further NUMA init
      cleanups.
      
      - v2: x86-32 initmem_init() weren't updated breaking 32bit builds.
        Fixed.  Found by Yinghai.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      86ef4dbf
  11. 28 Jan, 2011 2 commits
    • Tejun Heo's avatar
      x86: Unify NUMA initialization between 32 and 64bit · 8db78cc4
      Tejun Heo authored
      
      
      Now that everything else is unified, NUMA initialization can be
      unified too.
      
      * numa_init_array() and init_cpu_to_node() are moved from
        numa_64 to numa.
      
      * numa_32::initmem_init() is updated to call numa_init_array()
        and setup_arch() to call init_cpu_to_node() on 32bit too.
      
      * x86_cpu_to_node_map is now initialized to NUMA_NO_NODE on
        32bit too. This is safe now as numa_init_array() will initialize
        it early during boot.
      
      This makes NUMA mapping fully initialized before
      setup_per_cpu_areas() on 32bit too and thus makes the first
      percpu chunk which contains all the static variables and some of
      dynamic area allocated with NUMA affinity correctly considered.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: yinghai@kernel.org
      Cc: brgerst@gmail.com
      Cc: gorcunov@gmail.com
      Cc: shaohui.zheng@intel.com
      Cc: rientjes@google.com
      LKML-Reference: <1295789862-25482-17-git-send-email-tj@kernel.org>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      Reported-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Reviewed-by: default avatarPekka Enberg <penberg@kernel.org>
      8db78cc4
    • Tejun Heo's avatar
      x86: Unify cpu/apicid <-> NUMA node mapping between 32 and 64bit · bbc9e2f4
      Tejun Heo authored
      
      
      The mapping between cpu/apicid and node is done via
      apicid_to_node[] on 64bit and apicid_2_node[] +
      apic->x86_32_numa_cpu_node() on 32bit. This difference makes it
      difficult to further unify 32 and 64bit NUMA handling.
      
      This patch unifies it by replacing both apicid_to_node[] and
      apicid_2_node[] with __apicid_to_node[] array, which is accessed
      by two accessors - set_apicid_to_node() and numa_cpu_node().  On
      64bit, numa_cpu_node() always consults __apicid_to_node[]
      directly while 32bit goes through apic->numa_cpu_node() method
      to allow apic implementations to override it.
      
      srat_detect_node() for amd cpus contains workaround for broken
      NUMA configuration which assumes relationship between APIC ID,
      HT node ID and NUMA topology.  Leave it to access
      __apicid_to_node[] directly as mapping through CPU might result
      in undesirable behavior change.  The comment is reformatted and
      updated to note the ugliness.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Reviewed-by: default avatarPekka Enberg <penberg@kernel.org>
      Cc: eric.dumazet@gmail.com
      Cc: yinghai@kernel.org
      Cc: brgerst@gmail.com
      Cc: gorcunov@gmail.com
      Cc: shaohui.zheng@intel.com
      Cc: rientjes@google.com
      LKML-Reference: <1295789862-25482-14-git-send-email-tj@kernel.org>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      Cc: David Rientjes <rientjes@google.com>
      bbc9e2f4
  12. 06 Oct, 2010 1 commit
    • Yinghai Lu's avatar
      x86-32, memblock: Make add_highpages honor early reserved ranges · 1d931264
      Yinghai Lu authored
      
      
      Originally the only early reserved range that is overlapped with high
      pages is "KVA RAM", but we already do remove that from the active ranges.
      
      However, It turns out Xen could have that kind of overlapping to support memory
      ballooning.x
      
      So we need to make add_highpage_with_active_regions() to subtract
      memblock reserved just like low ram; this is the proper design anyway.
      
      In this patch, refactering get_freel_all_memory_range() to make it can
      be used by add_highpage_with_active_regions().  Also we don't need to
      remove "KVA RAM" from active ranges.
      Signed-off-by: default avatarYinghai Lu <yinghai@kernel.org>
      LKML-Reference: <4CABB183.1040607@kernel.org>
      Signed-off-by: default avatarH. Peter Anvin <hpa@zytor.com>
      1d931264
  13. 27 Aug, 2010 2 commits
  14. 12 Feb, 2010 1 commit