Skip to content
  • Christoph Lameter's avatar
    vmstat: User per cpu atomics to avoid interrupt disable / enable · 7c839120
    Christoph Lameter authored
    
    
    Currently the operations to increment vm counters must disable interrupts
    in order to not mess up their housekeeping of counters.
    
    So use this_cpu_cmpxchg() to avoid the overhead. Since we can no longer
    count on preremption being disabled we still have some minor issues.
    The fetching of the counter thresholds is racy.
    A threshold from another cpu may be applied if we happen to be
    rescheduled on another cpu.  However, the following vmstat operation
    will then bring the counter again under the threshold limit.
    
    The operations for __xxx_zone_state are not changed since the caller
    has taken care of the synchronization needs (and therefore the cycle
    count is even less than the optimized version for the irq disable case
    provided here).
    
    The optimization using this_cpu_cmpxchg will only be used if the arch
    supports efficient this_cpu_ops (must have CONFIG_CMPXCHG_LOCAL set!)
    
    The use of this_cpu_cmpxchg reduces the cycle count for the counter
    operations by %80 (inc_zone_page_state goes from 170 cycles to 32).
    
    Signed-off-by: default avatarChristoph Lameter <cl@linux.com>
    7c839120