• Roland Scheidegger's avatar
    util: don't flush overflowing values to infinity in half-float conversion · d86fddc8
    Roland Scheidegger authored
    I am not able to find _any_ rounding behavior specified for OpenGL for
    float to half-float conversions. However, it is specified for fp11/fp10
    which suggests round to next finite value but round-to-zero would also
    be allowed, but finite values must not be flushed to infinity in either
    Hence I believe it makes sense to do the same for half-floats too.
    We could probably also use round-to-zero consistently, which is in fact
    required by d3d10 (but it doesn't seem to matter much).
    Does not match the mesa core function doing the same though (which is
    saying it was built to match intel gpus which I don't believe for a
    second as it would cause failures in d3d10, moreover the PRM (for
    ivy bridge, not listed in older manuals) while not specifying rounding
    behavior clearly states finite numbers are never flushed to infinity).
    Reviewed-by: default avatarJose Fonseca <jfonseca@vmware.com>
u_half.h 3.27 KB