target/i386: Fix decoding of certain BMI instructions

This patch fixes a translation bug for a subset of x86 BMI instructions
such as the following:

c4 e2 f9 f7 c0 shlxq %rax, %rax, %rax

Currently, these incorrectly generate an undefined instruction exception
when SSE is disabled via CR4, while instructions like "shrxq" work fine.

The problem appears to be related to BMI instructions encoded using VEX
and with a mandatory prefix of "0x66" (data). Instructions with this
data prefix (such as shlxq) are currently rejected. Instructions with
other mandatory prefixes (such as shrxq) translate as expected.

This patch removes the incorrect check in "gen_sse" that causes the
exception to be generated. For the non-BMI cases, the check is
redundant: prefixes are already checked at line 3696.

Buglink: https://bugs.launchpad.net/qemu/+bug/1748296

Backports 51909241d26fe6fe18a08def93ccc8273f61a8b3
This commit is contained in:
David Greenaway 2021-03-04 18:08:37 -05:00 committed by Lioncash
parent 56afe9f919
commit 0c1c359b5c

View file

@ -3531,7 +3531,7 @@ static void gen_sse(CPUX86State *env, DisasContext *s, int b,
}
if (is_xmm
&& !(s->flags & HF_OSFXSR_MASK)
&& ((b != 0x38 && b != 0x3a) || (s->prefix & PREFIX_DATA))) {
&& (b != 0x38 && b != 0x3a)) {
goto unknown_op;
}
if (b == 0x0e) {