softfloat: fix floatx80 remainder pseudo-denormal check for zero

The floatx80 remainder implementation ignores the high bit of the
significand when checking whether an operand (numerator) with zero
exponent is zero. This means it mishandles a pseudo-denormal
representation of 0x1p-16382L by treating it as zero. Fix this by
checking the whole significand instead.

Backports commit 499a2f7b554a295cfc10f8cd026d9b20a38fe664 from qemu
This commit is contained in:
Joseph Myers 2021-02-25 13:35:16 -05:00 committed by Lioncash
parent b08d204a37
commit 6b63555a00

View file

@ -5703,7 +5703,7 @@ floatx80 floatx80_modrem(floatx80 a, floatx80 b, bool mod,
normalizeFloatx80Subnormal( bSig, &bExp, &bSig );
}
if ( aExp == 0 ) {
if ( (uint64_t) ( aSig0<<1 ) == 0 ) return a;
if ( aSig0 == 0 ) return a;
normalizeFloatx80Subnormal( aSig0, &aExp, &aSig0 );
}
bSig |= UINT64_C(0x8000000000000000);