Skip to content
  • Konrad Rzeszutek Wilk's avatar
    xen/mmu: On early bootup, flush the TLB when changing RO->RW bits Xen provided pagetables. · b2222794
    Konrad Rzeszutek Wilk authored
    
    
    Occassionaly on a DL380 G4 the guest would crash quite early with this:
    
    (XEN) d244:v0: unhandled page fault (ec=0003)
    (XEN) Pagetable walk from ffffffff84dc7000:
    (XEN)  L4[0x1ff] = 00000000c3f18067 0000000000001789
    (XEN)  L3[0x1fe] = 00000000c3f14067 000000000000178d
    (XEN)  L2[0x026] = 00000000dc8b2067 0000000000004def
    (XEN)  L1[0x1c7] = 00100000dc8da067 0000000000004dc7
    (XEN) domain_crash_sync called from entry.S
    (XEN) Domain 244 (vcpu#0) crashed on cpu#3:
    (XEN) ----[ Xen-4.1.3OVM  x86_64  debug=n  Not tainted ]----
    (XEN) CPU:    3
    (XEN) RIP:    e033:[<ffffffff81263f22>]
    (XEN) RFLAGS: 0000000000000216   EM: 1   CONTEXT: pv guest
    (XEN) rax: 0000000000000000   rbx: ffffffff81785f88   rcx: 000000000000003f
    (XEN) rdx: 0000000000000000   rsi: 00000000dc8da063   rdi: ffffffff84dc7000
    
    The offending code shows it to be a loop writting the value zero
    (%rax) in the %rdi (the L4 provided by Xen) register:
    
       0: 44 00 00             add    %r8b,(%rax)
       3: 31 c0                 xor    %eax,%eax
       5: b9 40 00 00 00       mov    $0x40,%ecx
       a: 66 0f 1f 84 00 00 00 nopw   0x0(%rax,%rax,1)
      11: 00 00
      13: ff c9                 dec    %ecx
      15:* 48 89 07             mov    %rax,(%rdi)     <-- trapping instruction
      18: 48 89 47 08           mov    %rax,0x8(%rdi)
      1c: 48 89 47 10           mov    %rax,0x10(%rdi)
    
    which fails. xen_setup_kernel_pagetable recycles some of the Xen's
    page-table entries when it has switched over to its Linux page-tables.
    
    Right before try to clear the page, we  make a hypercall to change
    it from _RO to  _RW and that works (otherwise we would hit an BUG()).
    And the _RW flag is set for that page:
    (XEN)  L1[0x1c7] = 001000004885f067 0000000000004dc7
    
    The error code is 3, so PFEC_page_present and PFEC_write_access, so page is
    present (correct), and we tried to write to the page, but a violation
    occurred. The one theory is that the the page entries in hardware
    (which are cached) are not up to date with what we just set. Especially
    as we have just done an CR3 write and flushed the multicalls.
    
    This patch does solve the problem by flusing out the TLB page
    entry after changing it from _RO to _RW and we don't hit this
    issue anymore.
    
    Fixed-Oracle-Bug: 16243091 [ON OCCASIONS VM START GOES INTO
    'CRASH' STATE: CLEAR_PAGE+0X12 ON HP DL380 G4]
    Reported-and-Tested-by: default avatarSaar Maoz <Saar.Maoz@oracle.com>
    Signed-off-by: default avatarKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
    b2222794