fpu: add mechanism to check for invalid long double formats

All operations that take a floatx80 as an operand need to have their
inputs checked for malformed encodings. In all of these cases, use the
function floatx80_invalid_encoding to perform the check. If an invalid
operand is found, raise an invalid operation exception, and then return
either NaN (for fp-typed results) or the integer indefinite value (the
minimum representable signed integer value, for int-typed results).

For the non-quiet comparison operations, this touches adjacent code in
order to pass style checks.

Backports cast correction portion of commit d1eb8f2acba579830cf3798c3c15ce51be852c56m from qemu
This commit is contained in:
Andrew Dutcher 2018-02-26 02:26:59 -05:00 committed by Lioncash
parent 9e6fec8741
commit 26b36e5ff8
No known key found for this signature in database
GPG key ID: 4E3C3CC1031BA9C7
2 changed files with 3 additions and 3 deletions

View file

@ -4859,7 +4859,7 @@ int64_t floatx80_to_int64(floatx80 a, float_status *status)
if (floatx80_invalid_encoding(a)) {
float_raise(float_flag_invalid, status);
return (uint64_t)1 << 63;
return 1ULL << 63;
}
aSig = extractFloatx80Frac( a );
aExp = extractFloatx80Exp( a );
@ -4904,7 +4904,7 @@ int64_t floatx80_to_int64_round_to_zero(floatx80 a, float_status *status)
if (floatx80_invalid_encoding(a)) {
float_raise(float_flag_invalid, status);
return (uint64_t)1 << 63;
return 1ULL << 63;
}
aSig = extractFloatx80Frac( a );
aExp = extractFloatx80Exp( a );

View file

@ -677,7 +677,7 @@ static inline int floatx80_is_any_nan(floatx80 a)
*----------------------------------------------------------------------------*/
static inline bool floatx80_invalid_encoding(floatx80 a)
{
return (a.low & ((uint64_t)1 << 63)) == 0 && (a.high & 0x7FFF) != 0;
return (a.low & (1ULL << 63)) == 0 && (a.high & 0x7FFF) != 0;
}
#define floatx80_zero make_floatx80(0x0000, 0x0000000000000000LL)