• Aleksandar Markovic's avatar
    MIPS: math-emu: RINT.<D|S>: Fix several problems by reimplementation · 3ec404d8
    Aleksandar Markovic authored
    Reimplement RINT.<D|S> kernel emulation so that all RINT.<D|S>
    specifications are met.
    For the sake of simplicity, let's analyze RINT.S only. Prior to
    this patch, RINT.S emulation was essentially implemented as (in
    pseudocode) <output> = ieee754sp_flong(ieee754sp_tlong(<input>)),
    where ieee754sp_tlong() and ieee754sp_flong() are functions
    providing conversion from double to integer, and from integer
    to double, respectively. On surface, this implementation looks
    correct, but actually fails in many cases. Following problems
    were detected:
    1. NaN and infinity cases will not be handled properly. The
       function ieee754sp_flong() never returns NaN nor infinity.
    2. For RINT.S, for all inputs larger than LONG_MAX, and smaller
       than FLT_MAX, the result will be wrong, and the overflow
       exception will be erroneously set. A similar problem for
       negative inputs exists as well.
    3. For some rounding modes, for some negative inputs close to zero,
       the return value will be zero, and should be -zero. This is
       because ieee754sp_flong() never returns -zero.
    This patch removes the problems above by implementing dedicated
    functions for RINT.<D|S> emulation.
    The core of the new function functionality is adapted version of
    the core of the function ieee754sp_tlong(). However, there are many
    details that are implemented to match RINT.<D|S> specification. It
    should be said that the functionality of ieee754sp_tlong() actually
    closely corresponds to CVT.L.S instruction, and it is used while
    emulating CVT.L.S. However, RINT.S and CVT.L.S instructions differ
    in many aspects. This patch fulfills missing support for RINT.<D|S>.
    Signed-off-by: default avatarMiodrag Dinic <miodrag.dinic@imgtec.com>
    Signed-off-by: default avatarGoran Ferenc <goran.ferenc@imgtec.com>
    Signed-off-by: default avatarAleksandar Markovic <aleksandar.markovic@imgtec.com>
    Cc: David S. Miller <davem@davemloft.net>
    Cc: Douglas Leung <douglas.leung@imgtec.com>
    Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
    Cc: Hans Verkuil <hans.verkuil@cisco.com>
    Cc: James Hogan <james.hogan@imgtec.com>
    Cc: Maciej W. Rozycki <macro@imgtec.com>
    Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
    Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
    Cc: Paul Burton <paul.burton@imgtec.com>
    Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
    Cc: Raghu Gandham <raghu.gandham@imgtec.com>
    Cc: Randy Dunlap <rdunlap@infradead.org>
    Cc: linux-mips@linux-mips.org
    Cc: linux-kernel@vger.kernel.org
    Patchwork: https://patchwork.linux-mips.org/patch/17141/Signed-off-by: default avatarRalf Baechle <ralf@linux-mips.org>