We cannot assert the fence is not yet changed as the next thread may
change it prior to acquiring our lock.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200225082233.274530-1-chris@chris-wilson.co.uk