1. 07 Feb, 2014 2 commits
    • Brian Foster's avatar
      xfs: use tr_qm_dqalloc log reservation for dquot alloc · 410b11a6
      Brian Foster authored
      The dquot allocation path in xfs_qm_dqread() currently uses the
      attribute set log reservation, which appears to be incorrect. We
      have reports of transaction reservation overruns with the current
      code. E.g., a repeated run of xfstests test generic/270 on a 512b
      block size fs occassionally produces the following in dmesg:
      
      	XFS (sdN): xlog_write: reservation summary:
      	  trans type  = QM_DQALLOC (30)
      	  unit res    = 7080 bytes
      	  current res = -632 bytes
      	  total reg   = 0 bytes (o/flow = 0 bytes)
      	  ophdrs      = 0 (ophdr space = 0 bytes)
      	  ophdr + reg = 0 bytes
      	  num regions = 0
      
      	XFS (sdN): xlog_write: reservation ran out. Need to up reservation
      
      The dquot allocation case should consist of a write reservation
      (i.e., we are allocating a range of the internal quota file) plus
      the size of the actual dquots. We already have a log reservation
      definition for this operation (tr_qm_dqalloc). Use it in
      xfs_qm_dqread() and update the log reservation calculation function
      to use the write res. calculation function rather than reading the
      assumed to be pre-calculated value directly.
      Signed-off-by: default avatarBrian Foster <bfoster@redhat.com>
      Reviewed-by: default avatarJie Liu <jeff.liu@oracle.com>
      Reviewed-by: default avatarBen Myers <bpm@sgi.com>
      Signed-off-by: default avatarDave Chinner <david@fromorbit.com>
      
      410b11a6
    • Eric Sandeen's avatar
      xfs: remove unused tr_swrite · c19ec235
      Eric Sandeen authored
      tr_swrite is never used, remove it.
      
      From a very quick look, I think the usage of it (and its ancestor
      XFS_SWRITE_LOG_RES) went away in commit 13e6d5cd "xfs: merge fsync
      and O_SYNC handling" back in 2009.
      Signed-off-by: default avatarEric Sandeen <sandeen@redhat.com>
      Reviewed-by: default avatarJie Liu <jeff.liu@oracle.com>
      Signed-off-by: default avatarDave Chinner <david@fromorbit.com>
      c19ec235
  2. 13 Dec, 2013 2 commits
  3. 18 Nov, 2013 1 commit
    • Dave Chinner's avatar
      xfs: increase inode cluster size for v5 filesystems · 8f80587b
      Dave Chinner authored
      v5 filesystems use 512 byte inodes as a minimum, so read inodes in
      clusters that are effectively half the size of a v4 filesystem with
      256 byte inodes. For v5 fielsystems, scale the inode cluster size
      with the size of the inode so that we keep a constant 32 inodes per
      cluster ratio for all inode IO.
      
      This only works if mkfs.xfs sets the inode alignment appropriately
      for larger inode clusters, so this functionality is made conditional
      on mkfs doing the right thing. xfs_repair needs to know about
      the inode alignment changes, too.
      
      Wall time:
      	create	bulkstat	find+stat	ls -R	unlink
      v4	237s	161s		173s		201s	299s
      v5	235s	163s		205s		 31s	356s
      patched	234s	160s		182s		 29s	317s
      
      System time:
      	create	bulkstat	find+stat	ls -R	unlink
      v4	2601s	2490s		1653s		1656s	2960s
      v5	2637s	2497s		1681s		  20s	3216s
      patched	2613s	2451s		1658s		  20s	3007s
      
      So, wall time same or down across the board, system time same or
      down across the board, and cache hit rates all improve except for
      the ls -R case which is a pure cold cache directory read workload
      on v5 filesystems...
      
      So, this patch removes most of the performance and CPU usage
      differential between v4 and v5 filesystems on traversal related
      workloads.
      
      Note: while this patch is currently for v5 filesystems only, there
      is no reason it can't be ported back to v4 filesystems.  This hasn't
      been done here because bringing the code back to v4 requires
      forwards and backwards kernel compatibility testing.  i.e. to
      deterine if older kernels(*) do the right thing with larger inode
      alignments but still only using 8k inode cluster sizes. None of this
      testing and validation on v4 filesystems has been done, so for the
      moment larger inode clusters is limited to v5 superblocks.
      
      (*) a current default config v4 filesystem should mount just fine on
      2.6.23 (when lazy-count support was introduced), and so if we change
      the alignment emitted by mkfs without a feature bit then we have to
      make sure it works properly on all kernels since 2.6.23. And if we
      allow it to be changed when the lazy-count bit is not set, then it's
      all kernels since v2 logs were introduced that need to be tested for
      compatibility...
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
      Reviewed-by: default avatarEric Sandeen <sandeen@redhat.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      8f80587b
  4. 23 Oct, 2013 4 commits
    • Dave Chinner's avatar
      xfs: decouple inode and bmap btree header files · a4fbe6ab
      Dave Chinner authored
      Currently the xfs_inode.h header has a dependency on the definition
      of the BMAP btree records as the inode fork includes an array of
      xfs_bmbt_rec_host_t objects in it's definition.
      
      Move all the btree format definitions from xfs_btree.h,
      xfs_bmap_btree.h, xfs_alloc_btree.h and xfs_ialloc_btree.h to
      xfs_format.h to continue the process of centralising the on-disk
      format definitions. With this done, the xfs inode definitions are no
      longer dependent on btree header files.
      
      The enables a massive culling of unnecessary includes, with close to
      200 #include directives removed from the XFS kernel code base.
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Reviewed-by: default avatarBen Myers <bpm@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      a4fbe6ab
    • Dave Chinner's avatar
      xfs: decouple log and transaction headers · 239880ef
      Dave Chinner authored
      xfs_trans.h has a dependency on xfs_log.h for a couple of
      structures. Most code that does transactions doesn't need to know
      anything about the log, but this dependency means that they have to
      include xfs_log.h. Decouple the xfs_trans.h and xfs_log.h header
      files and clean up the includes to be in dependency order.
      
      In doing this, remove the direct include of xfs_trans_reserve.h from
      xfs_trans.h so that we remove the dependency between xfs_trans.h and
      xfs_mount.h. Hence the xfs_trans.h include can be moved to the
      indicate the actual dependencies other header files have on it.
      
      Note that these are kernel only header files, so this does not
      translate to any userspace changes at all.
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Reviewed-by: default avatarBen Myers <bpm@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      239880ef
    • Dave Chinner's avatar
      xfs: unify directory/attribute format definitions · 57062787
      Dave Chinner authored
      The on-disk format definitions for the directory and attribute
      structures are spread across 3 header files right now, only one of
      which is dedicated to defining on-disk structures and their
      manipulation (xfs_dir2_format.h). Pull all the format definitions
      into a single header file - xfs_da_format.h - and switch all the
      code over to point at that.
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Reviewed-by: default avatarBen Myers <bpm@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      57062787
    • Dave Chinner's avatar
      xfs: create a shared header file for format-related information · 70a9883c
      Dave Chinner authored
      All of the buffer operations structures are needed to be exported
      for xfs_db, so move them all to a common location rather than
      spreading them all over the place. They are verifying the on-disk
      format, so while xfs_format.h might be a good place, it is not part
      of the on disk format.
      
      Hence we need to create a new header file that we centralise these
      related definitions. Start by moving the bffer operations
      structures, and then also move all the other definitions that have
      crept into xfs_log_format.h and xfs_format.h as there was no other
      shared header file to put them in.
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      70a9883c
  5. 30 Aug, 2013 1 commit
    • Dave Chinner's avatar
      xfs: inode log reservations are too small · 23956703
      Dave Chinner authored
      We've been seeing occasional problems with log space leaks and
      transaction underruns such as this for some time:
      
       XFS (dm-0): xlog_write: reservation summary:
         trans type  = FSYNC_TS (36)
         unit res    = 2740 bytes
         current res = -4 bytes
         total reg   = 0 bytes (o/flow = 0 bytes)
         ophdrs      = 0 (ophdr space = 0 bytes)
         ophdr + reg = 0 bytes
         num regions = 0
      
      Turns out that xfstests generic/311 is reliably reproducing this
      problem with the test it runs at sequence 16 of it execution. It is
      a 100% reliable reproducer with the mkfs configuration of "-b
      size=1024 -m crc=1" on a 10GB scratch device.
      
      The problem? Inode forks in btree format are logged in memory
      format, not disk format (i.e. bmbt format, not bmdr format). That
      means there is a btree block header being logged, when such a
      structure is never written to the inode fork in bmdr format. The
      bmdr header in the inode is only 4 bytes, while the bmbt header is
      24 bytes for v4 filesystems and 72 bytes for v5 filesystems.
      
      We currently reserve the inode size plus the rounded up overhead of
      a logging a buffer, which is 128 bytes. That means the reservation
      for a 512 byte inode is 640 bytes. What we can actually log is:
      
      	inode core, data and attr fork = 512 bytes
      	inode log format + log op header = 56 + 12 = 68 bytes
      	data fork bmbt hdr = 24/72 bytes
      	attr fork bmbt hdr = 24/72 bytes
      
      So, for a v2 inodes we can log at least 628 bytes, but if we split that
      inode over the end of the log across log buffers, we need to also
      another log op header, which takes us to 640 bytes. If there's
      another reservation taken out of this that I haven't taken into
      account (perhaps multiple iclog splits?) or I haven't corectly
      calculated the bmbt format space used (entirely possible), then
      we will overun it.
      
      For v3 inodes the maximum is actually 724 bytes, and even a
      single maximally sized btree format fork can blow it (652 bytes).
      And that's exactly what is happening with the FSYNC_TS transaction
      in the above output - it's consumed 644 bytes of space after the CIL
      context took the space reserved for it (2100 bytes).
      
      This problem has always been present in the XFS code - the btree
      format inode forks have always been logged in this manner. Hence
      there has always been the possibility of an overrun with such a
      transaction. The CRC code has just exposed it frequently enough to
      be able to debug and understand the root cause....
      
      So, let's fix all the inode log space reservations.
      
      [ I'm so glad we spent the effort to clean up the transaction
        reservation code. This is an easy fix now. ]
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Reviewed-by: default avatarMark Tinguely <tinguely@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      23956703
  6. 12 Aug, 2013 5 commits