softfloat: fix floatx80 pseudo-denormal comparisons

The softfloat floatx80 comparisons fail to allow for pseudo-denormals,
which should compare equal to corresponding values with biased
exponent 1 rather than 0. Add an adjustment for that case when
comparing numbers with the same sign.

Backports commit be53fa785ab766d2722628403edee75b3e6ab599 from qemu
This commit is contained in:
Joseph Myers 2020-05-15 23:58:47 -04:00 committed by Lioncash
parent 85964d48d2
commit 3d4a7e34e1

View file

@ -7928,6 +7928,13 @@ static inline int floatx80_compare_internal(floatx80 a, floatx80 b,
return 1 - (2 * aSign);
}
} else {
/* Normalize pseudo-denormals before comparison. */
if ((a.high & 0x7fff) == 0 && a.low & UINT64_C(0x8000000000000000)) {
++a.high;
}
if ((b.high & 0x7fff) == 0 && b.low & UINT64_C(0x8000000000000000)) {
++b.high;
}
if (a.low == b.low && a.high == b.high) {
return float_relation_equal;
} else {