Commit graph

562 commits

Author SHA1 Message Date
Richard Henderson b7b142ed79
tcg/i386: Implement INDEX_op_extr{lh}_i64_i32 for 32-bit guests
This preserves the invariant that all TCG_TYPE_I32 values are
zero-extended in the 64-bit host register.

Backports commit 75478279a0c1eafc7b69d5382356da138f58f1bd from qemu
2018-12-18 05:38:55 -05:00
Richard Henderson 4e882a95f3
tcg/i386: Propagate is64 to tcg_out_qemu_ld_slow_path
This helps preserve the invariant that all TCG_TYPE_I32 values
are stored zero-extended in the 64-bit host registers.

Backports commit 3dbc8c61de4e0d0a2afe0897cda7ab28cd37a164 from qemu
2018-12-18 05:36:58 -05:00
Richard Henderson bdd6118105
tcg/i386: Propagate is64 to tcg_out_qemu_ld_direct
This helps preserve the invariant that all TCG_TYPE_I32 values
are stored zero-extended in the 64-bit host registers.

Backports commit 1d21d95b6101786d44d3b4a12400eb80a1ecc647 from qemu
2018-12-18 05:35:34 -05:00
Richard Henderson 7927f3cff5
tcg/s390x: Return false on failure from patch_reloc
This does require an extra two checks within the slow paths
to replace the assert that we're moving. Also add two checks
within existing functions that lacked any kind of assert for
out of range branch.

Backports commit 55dfd8fedceb1311d9cdded1a0f94b2da91a387d from qemu
2018-12-18 05:34:00 -05:00
Richard Henderson 51b802223a
tcg/ppc: Return false on failure from patch_reloc
The reloc_pc{14,24}_val routines retain their asserts.
Use these directly within the slow paths.

Backports commit d5132903518fadad579ef2de9e45fce98eefaa63 from qemu
2018-12-18 05:32:12 -05:00
Richard Henderson 8ecb82062f
tcg/arm: Return false on failure from patch_reloc
This does require an extra two checks within the slow paths
to replace the assert that we're moving.

Backports commit 43fabd30e2f411e8d70ff347902a7c8ed308233e from qemu
2018-12-18 05:30:11 -05:00
Richard Henderson a22387f919
tcg/aarch64: Return false on failure from patch_reloc
This does require an extra two checks within the slow paths
to replace the assert that we're moving.

Backports commit 214bfe83d5a5af70bac2b8d0bd649b018c33c03b from qemu
2018-12-18 05:28:45 -05:00
Richard Henderson fc86fd34ff
tcg/i386: Return false on failure from patch_reloc
Backports commit bec3afd5fc6ab0b6e9d8a01575d58db8d1ad82ce from qemu
2018-12-18 05:27:14 -05:00
Richard Henderson 46189d87b3
tcg: Return success from patch_reloc
This will move the assert for success from within (subroutines of)
patch_reloc into the callers. It will also let new code do something
different when a relocation is out of range.

For the moment, all backends are trivially converted to return true.

Backports commit 6ac1778676f4259c10b0629ccd9df319a5d1baeb from qemu
2018-12-18 05:25:45 -05:00
Richard Henderson 294573899f
tcg/mips: Remove retranslation code
There is no longer a need for preserving branch offset operands,
as we no longer re-translate.

Backports commit 8c1b079279fadaee10dc39ca9a58c4c91c7a1854 from qemu
2018-12-18 05:22:25 -05:00
Richard Henderson ad9aec6f35
tcg/sparc: Remove retranslation code
There is no longer a need for preserving branch offset operands,
as we no longer re-translate.

Backports commit 791645f0227c9d52ce5fe1ad6e1cda55a9bfe633 from qemu
2018-12-18 05:21:50 -05:00
Richard Henderson a124110db4
tcg/s390: Remove retranslation code
There is no longer a need for preserving branch offset operands,
as we no longer re-translate.

Backports commit 3661612fc3e4b65be03482bf6bafd116101881e1 from qemu
2018-12-18 05:21:03 -05:00
Richard Henderson 85485dc20e
tcg/ppc: Fold away noaddr branch routines
There is no longer a need for preserving branch offset operands,
as we no longer re-translate.

Backports commit f9c7246faa279237200a2a53beacaa8100ea1900 from qemu
2018-12-18 05:18:59 -05:00
Richard Henderson b49a353adb
tcg/arm: Fold away noaddr branch routines
There are one use apiece for these. There is no longer a need for
preserving branch offset operands, as we no longer re-translate.

Backports commit 37ee93a974c49ab9edfcd1db0aad3838b0395b14 from qemu
2018-12-18 05:17:22 -05:00
Richard Henderson 1167aa481d
tcg/arm: Remove reloc_pc24_atomic
It is unused since 3fb53fb4d12f2e7833bd1659e6013237b130ef20.

Backports commit 2672ccc7eee742e23928f4bf60a13a77d64f540d from qemu
2018-12-18 05:16:29 -05:00
Richard Henderson 0a8bc142d3
tcg/aarch64: Fold away noaddr branch routines
There are one use apiece for these. There is no longer a need for
preserving branch offset operands, as we no longer re-translate.

Backports commit 733589b3382afcb0ae9f43e72e083a5ddd38abd5 from qemu
2018-12-18 05:15:41 -05:00
Richard Henderson cbe1065e83
tcg/aarch64: Remove reloc_pc26_atomic
It is unused since b68686bd4bfeb70040b4099df993dfa0b4f37b03.

Backports commit 90d6cb781130891f96eb54f8315e29fbd4e99a71 from qemu
2018-12-18 05:14:22 -05:00
Richard Henderson 091b4fa1ff
tcg/i386: Move TCG_REG_CALL_STACK from define to enum
Backports commit 66c0285df4270d184afce5ac8b97ac175c89562f from qemu
2018-12-18 05:13:47 -05:00
Richard Henderson f3a8a4a306
tcg/i386: Always use %ebp for TCG_AREG0
For x86_64, this can remove a REX prefix resulting in smaller code
when manipulating globals of type i32, as we move them between backing
store via cpu_env, aka TCG_AREG0.

Backports commit 5740d9f714835964873325d1210b26811252843f from qemu
2018-12-18 05:13:05 -05:00
Richard Henderson 7ab51fc012
target/sparc: Remove the constant pool
Partially reverts ab20bdc1162. The 14-bit displacement that we
allowed to reach the constant pool is not always sufficient.
Retain the tb-relative addressing, as that is how most return
values from the tb are computed.

Backports commit f6823cbe3787aa47db62deede6683077e3da9a2c from qemu
2018-12-18 05:12:11 -05:00
Thomas Huth 3ba2114043
tcg/tcg.h: Remove GCC check for tcg_debug_assert() macro
Both GCC v4.8 and Clang v3.4 (our minimum versions) support
__builtin_unreachable(), so we can remove the version check here now.

Backports commit 6fa2cef205a60b5c5c3b058f53852416b885c455 from qemu
2018-12-18 03:53:56 -05:00
Peter Maydell 78906db067
tcg/tcg-op.h: Add multiple include guard
The tcg-op.h header was missing the usual guard against multiple
inclusion; add it.

(Spotted by lgtm.com's static analyzer.)

Backports commit a7ce790a029bd94eb320d8c69f38900f5233997e from qemu
2018-11-11 08:51:51 -05:00
Craig Janeczek 58dc377890
target/mips: Introduce MXU registers
Define and initialize the 16 MXU registers - 15 general computational
register, and 1 control register). There is also a zero register, but
it does not have any corresponding variable.

Backports commit eb5559f67dc8dc12335dd996877bb6daaea32eb2 from qemu.
2018-11-11 05:50:52 -05:00
Richard Henderson d74e00a30a
tcg: Split CONFIG_ATOMIC128
GCC7+ will no longer advertise support for 16-byte __atomic operations
if only cmpxchg is supported, as for x86_64. Fortunately, x86_64 still
has support for __sync_compare_and_swap_16 and we can make use of that.
AArch64 does not have, nor ever has had such support, so open-code it.

Backports commit e6cd4bb59b8154fa00da611200beef7eb4e8ec56 from qemu
2018-10-23 15:17:39 -04:00
Emilio G. Cota e5b43d2794
tcg: plug holes in struct TCGProfile
This plugs two 4-byte holes in 64-bit.

Backports commit dd1d7da23b0abef87f46d9ab39ba9b0974eaec04 from qemu
2018-10-23 14:38:16 -04:00
Emilio G. Cota 223975ada0
tcg: fix use of uninitialized variable under CONFIG_PROFILER
We forgot to initialize n in commit 15fa08f845 ("tcg: Dynamically
allocate TCGOps", 2017-12-29).

Backports commit c1f543b739086733024e31d74a52d9e41553f316 from qemu
2018-10-23 14:37:37 -04:00
Richard Henderson e01deeb9ba
tcg: Implement CPU_LOG_TB_NOCHAIN during expansion
Rather than test NOCHAIN before linking, do not emit the
goto_tb opcode at all. We already do this for goto_ptr.

Backports commit d7f425fdea991f052241c6479acd9feae834063b from qemu
2018-10-23 14:35:12 -04:00
Lioncash cc3d618e61
tcg: Remove unnecessary MSVC ifdef
All relevant arrays have at least one member in them now, making this
unnecessary.
2018-10-06 05:08:17 -04:00
Lioncash 766c70f608
arm: Move cpu_M0 to DisasContext 2018-10-06 03:32:39 -04:00
Lioncash 787fd448b1
arm: Move cpu_V1 to DisasContext 2018-10-06 03:28:42 -04:00
Lioncash 1aa20da917
arm: Move cpu_V0 to DisasContext 2018-10-06 03:26:52 -04:00
Lioncash 06c21baaa4
arm: Move cpu_F1d to DisasContext 2018-10-06 03:11:54 -04:00
Lioncash 5f3dd68f9c
arm: Move cpu_F0d to DisasContext 2018-10-06 03:07:42 -04:00
Lioncash e457ce8ccc
arm: Move cpu_F1s to DisasContext 2018-10-06 03:02:06 -04:00
Lioncash 97a5955a2a
tcg: Remove leftover unused variable from TCGContext
This was previously used by the i386 target, however all of the locals
were moved to the DisasContext struct, leaving this unused.
2018-10-06 02:46:27 -04:00
Emilio G. Cota b9bb6cead9
target/i386: move x86_64_hregs to DisasContext
And convert it to a bool to use an existing hole
in the struct.

Backports commit 1dbe15ef57abdf7b6a26c8e638abf6413a4b9d0c from qemu
2018-10-04 04:02:50 -04:00
Emilio G. Cota 04530acab2
target/i386: move cpu_tmp3_i32 to DisasContext
Backports commit 4f82446de695f080ed148a0e47fc141e928665af from qemu
2018-10-04 03:56:05 -04:00
Emilio G. Cota 781e6bde41
target/i386: move cpu_tmp2_i32 to DisasContext
Backports commit 6bd48f6f206b6f32a5bbeebc3ae6886d4f587981 from qemu
2018-10-04 03:53:31 -04:00
Emilio G. Cota c13337d1bc
target/i386: move cpu_ptr1 to DisasContext
Backports commit 6387e8303ffb26cfb40b0f93372f1519229b4d2c from qemu
2018-10-04 03:48:09 -04:00
Emilio G. Cota 3e442d4480
target/i386: move cpu_ptr0 to DisasContext
Backports commit 2ee2646491a293a92d1c85e90e12419a8c199ed0 from qemu
2018-10-04 03:46:53 -04:00
Emilio G. Cota cc872aa711
target/i386: move cpu_tmp4 to DisasContext
Backports commit 5022f28f1e4033eb369b744ad61b96d086beca1b from qemu
2018-10-04 03:45:28 -04:00
Emilio G. Cota d2752ebc42
target/i386: move cpu_tmp0 to DisasContext
Backports commit fbd80f02df3fe272ba0f4825df27b8459dafbc14 from qemu
2018-10-04 03:41:13 -04:00
Emilio G. Cota b704b6c205
target/i386: move cpu_T1 to DisasContext
Backports commit b48597b0eda32d4c7ade2ba3f98f06f62289e3e2 from qemu
2018-10-04 03:35:10 -04:00
Emilio G. Cota 70b327dc82
target/i386: move cpu_T0 to DisasContext
Backports commit c66f97273f677d76afaaeb0e688eb08499701b1b from qemu
2018-10-04 03:29:13 -04:00
Emilio G. Cota c1d70758ea
target/i386: move cpu_A0 to DisasContext
Backports commit 6b672b5d6b14422c131969c5725f738751e12847 from qemu
2018-10-04 01:16:35 -04:00
Emilio G. Cota 30c66bcca3
target/i386: move cpu_cc_srcT to DisasContext
Backports commit 93a3e108eb6a9bb781ab7db6e92d91528e482030 from qemu
2018-10-04 00:59:00 -04:00
Roman Kapl 33e69342e3
tcg/i386: fix vector operations on 32-bit hosts
The TCG backend uses LOWREGMASK to get the low 3 bits of register numbers.
This was defined as no-op for 32-bit x86, with the assumption that we have
eight registers anyway. This assumption is not true once we have xmm regs.

Since LOWREGMASK was a no-op, xmm register indidices were wrong in opcodes
and have overflown into other opcode fields, wreaking havoc.

To trigger these problems, you can try running the "movi d8, #0x0" AArch64
instruction on 32-bit x86. "vpxor %xmm0, %xmm0, %xmm0" should be generated,
but instead TCG generated "vpxor %xmm0, %xmm0, %xmm2".

Fixes: 770c2fc7bb ("Add vector operations")

Backports commit 93bf9a42733321fb632bcb9eafd049ef0e3d9417 from qemu
2018-10-02 04:22:35 -04:00
Richard Henderson 9e8c8a617b
tcg/optimize: Do not skip default processing of dup_vec
If we do not opimize away dup_vec, we must mark its output as changed.

Backports commit 1fb57da72ae0886eba1234a2d98ddd10e88a9efc from qemu
2018-08-09 00:53:07 -04:00
Richard Henderson a4c2dbef3e
tcg/i386: Mark xmm registers call-clobbered
When host vector registers and operations were introduced, I failed
to mark the registers call clobbered as required by the ABI.

Fixes: 770c2fc7bb7

Backports commit 672189cd586ea38a2c1d8ab91eb1f9dcff5ceb05 from qemu
2018-07-23 20:00:26 -04:00
Alex Bennée 11948dd1cc
tcg/aarch64: limit mul_vec size
In AdvSIMD we can only do 32x32 integer multiples although SVE is
capable of larger 64 bit multiples. As a result we can end up
generating invalid opcodes. Fix this by only reprting we can emit
mul vector ops if the size is small enough.

Fixes a crash on:

sve-all-short-v8.3+sve@vq3/insn_mul_z_zi___INC.risu.bin

When running on AArch64 hardware.

Backports commit e65a5f227d77a5dbae7a7123c3ee915ee4bd80cf from qemu
2018-07-21 14:15:59 -04:00
Richard Henderson f1aaf5be62
tcg: Restrict check_size_impl to multiples of the line size
Normally this is automatic in the size restrictions that are placed
on vector sizes coming from the implementation. However, for the
legitimate size tuple [oprsz=8, maxsz=32], we need to clear the final
24 bytes of the vector register. Without this check, do_dup selects
TCG_TYPE_V128 and clears only 16 bytes.

Backports commit 499748d7683198a765d17b4fdf6901ab9dca920c from qemu
2018-07-09 16:41:53 -04:00
John Arbuckle 22c3206738
tcg/i386: Use byte form of xgetbv instruction
The assembler in most versions of Mac OS X is pretty old and does not
support the xgetbv instruction. To go around this problem, the raw
encoding of the instruction is used instead.

Backports commit 1019242af11400252f6735ca71a35f81ac23a66d from qemu
2018-06-28 13:23:32 -05:00
Richard Henderson 10e2b13650
tcg: Pass tb and index to tcg_gen_exit_tb separately
Do the cast to uintptr_t within the helper, so that the compiler
can type check the pointer argument. We can also do some more
sanity checking of the index argument.

Backports commit 07ea28b41830f946de3841b0ac61a3413679feb9 from qemu
2018-06-07 11:56:32 -04:00
Emilio G. Cota 7e8902eccc
tcg: fix s/compliment/complement/ typos
Backports commit 1d349821551c2da4dfefe36c6ac17319f33ebbd5 from qemu
2018-05-22 00:29:51 -04:00
Richard Henderson de1708aadc
tcg: Introduce atomic helpers for integer min/max
Given that this atomic operation will be used by both risc-v
and aarch64, let's not duplicate code across the two targets.

Backports commit 5507c2bf35aa6b4705939349184e71afd5e058b2 from qemu
2018-05-14 08:06:42 -04:00
Richard Henderson eef66443b2
tcg: Introduce helpers for integer min/max
These operations are re-invented by several targets so far.
Several supported hosts have insns for these, so place the
expanders out-of-line for a future introduction of tcg opcodes.

Backports commit b87fb8cd9f9a0ba599ff79e7bf03222da02e5724 from qemu
2018-05-14 07:31:50 -04:00
Richard Henderson f417df19b7
tcg: Limit the number of ops in a TB
In 6001f7729e12 we partially attempt to address the branch
displacement overflow caused by 15fa08f845.

However, gcc/testsuite/gcc.target/aarch64/advsimd-intrinsics/vqtbX.c
is a testcase that contains a TB so large as to overflow anyway.
The limit here of 8000 ops produces a maximum output TB size of
24112 bytes on a ppc64le host with that test case. This is still
much less than the maximum forward branch distance of 32764 bytes.

Backports commit abebf92597186be2bc48d487235da28b1127860f from qemu
2018-05-11 11:25:01 -04:00
Richard Henderson 33f7f6f09a
tcg/i386: Fix dup_vec in non-AVX2 codepath
The VPUNPCKLD* instructions are all "non-destructive source",
indicated by "NDS" in the encoding string in the x86 ISA manual.
This means that they take two source operands, one of which is
encoded in the VEX.vvvv field. We were incorrectly treating them
as if they were destructive-source and passing 0 as the 'v'
argument of tcg_out_vex_modrm(). This meant we were always
using %xmm0 as one of the source operands, causing incorrect
results if the register allocator happened to want to use
something else. For instance the input AArch64 insn:
DUP v26.16b, w21
which becomes TCG IR ops:
dup_vec v128,e8,tmp2,x21
st_vec v128,e8,tmp2,env,$0xa40
was assembled to:
0x607c568c: c4 c1 7a 7e 86 e8 00 00 vmovq 0xe8(%r14), %xmm0
0x607c5694: 00
0x607c5695: c5 f9 60 c8 vpunpcklbw %xmm0, %xmm0, %xmm1
0x607c5699: c5 f9 61 c9 vpunpcklwd %xmm1, %xmm0, %xmm1
0x607c569d: c5 f9 70 c9 00 vpshufd $0, %xmm1, %xmm1
0x607c56a2: c4 c1 7a 7f 8e 40 0a 00 vmovdqu %xmm1, 0xa40(%r14)
0x607c56aa: 00

when the vpunpcklwd insn should be "%xmm1, %xmm1, %xmm1".
This resulted in our incorrectly setting the output vector to
q26=0000320000003200:0000320000003200
when given an input of x21 == 0000000002803200
rather than the expected all-zeroes.

Pass the correct source register number to tcg_out_vex_modrm()
for these insns.

Backports commit 7eb30ef0ba2eb59e7430d4848ae8d4bf4e50f768 from qemu
2018-05-11 11:22:38 -04:00
Laurent Vivier ec12091943
tcg: workaround branch instruction overflow in tcg_out_qemu_ld/st
ppc64 uses a BC instruction to call the tcg_out_qemu_ld/st
slow path. BC instruction uses a relative address encoded
on 14 bits.

The slow path functions are added at the end of the generated
instructions buffer, in the reverse order of the callers.
So more we have slow path functions more the distance between
the caller (BC) and the function increases.

This patch changes the behavior to generate the functions in
the same order of the callers.

Backports commit 6001f7729e12dd1d810291e4cbf83cee8e07441d from qemu
2018-05-03 15:09:07 -04:00
Richard Henderson 2150745db4
tcg: Improve TCGv_ptr support
Drop TCGV_PTR_TO_NAT and TCGV_NAT_TO_PTR internal macros.

Add tcg_temp_local_new_ptr, tcg_gen_brcondi_ptr, tcg_gen_ext_i32_ptr,
tcg_gen_trunc_i64_ptr, tcg_gen_extu_ptr_i64, tcg_gen_trunc_ptr_i32.

Use inlines instead of macros where possible.

Backports commit 5bfa803448638a45542441fd6b7cc1241403ea72 from qemu
2018-05-03 15:05:43 -04:00
Richard Henderson 4fa9ea2ae1
tcg: Allow wider vectors for cmp and mul
In db432672, we allow wide inputs for operations such as add.
However, in 212be173 and 3774030a we didn't do the same for
compare and multiply.

Backports commit 9a938d86b04025ac605db0ea9819e5896bf576ec from qemu
2018-05-03 14:42:57 -04:00
Henry Wertz 090e2e9d0e
tcg/arm: Fix memory barrier encoding
I found with qemu 2.11.x or newer that I would get an illegal instruction
error running some Intel binaries on my ARM chromebook. On investigation,
I found it was quitting on memory barriers.

qemu instruction:
mb $0x31
was translating as:
0x604050cc: 5bf07ff5 blpl #0x600250a8

After patch it gives:
0x604050cc: f57ff05b dmb ish

In short, I found INSN_DMB_ISH (memory barrier for ARMv7) appeared to be
correct based on online docs, but due to some endian-related shenanigans it
had to be byte-swapped to suit qemu; it appears INSN_DMB_MCR (memory
barrier for ARMv6) also should be byte swapped (and this patch does so).
I have not checked for correctness of aarch64's barrier instruction.

Backports commit 3f814b803797c007abfe5c4041de754e01723031 from qemu
2018-05-03 14:41:36 -04:00
Richard Henderson 16a55143dc
tcg: Document INDEX_mul[us]h_*
Backports commit d103021269ca9307ed7ca0d845d2b9e6c387509a from qemu
2018-05-03 14:40:49 -04:00
Peter Maydell 778d0c47df
tcg/mips: Handle large offsets from target env to tlb_table
The MIPS TCG target makes the assumption that the offset from the
target env pointer to the tlb_table is less than about 64K. This
used to be true, but gradual addition of features to the Arm
target means that it's no longer true there. This results in
the build-time assertion failing:

In file included from /home/pm215/qemu/include/qemu/osdep.h:36:0,
from /home/pm215/qemu/tcg/tcg.c:28:
/home/pm215/qemu/tcg/mips/tcg-target.inc.c: In function ‘tcg_out_tlb_load’:
/home/pm215/qemu/include/qemu/compiler.h:90:36: error: static assertion failed: "not expecting: offsetof(CPUArchState, tlb_table[NB_MMU_MODES - 1][1]) > 0x7ff0 + 0x7fff"
^
/home/pm215/qemu/include/qemu/compiler.h:98:30: note: in expansion of macro ‘QEMU_BUILD_BUG_MSG’
^
/home/pm215/qemu/tcg/mips/tcg-target.inc.c:1236:9: note: in expansion of macro ‘QEMU_BUILD_BUG_ON’
QEMU_BUILD_BUG_ON(offsetof(CPUArchState,
^
/home/pm215/qemu/rules.mak:66: recipe for target 'tcg/tcg.o' failed

An ideal long term approach would be to rearrange the CPU state
so that the tlb_table was not so far along it, but this is tricky
because it would move it from the "not cleared on CPU reset" part
of the struct to the "cleared on CPU reset" part. As a simple fix
for the 2.12 release, make the MIPS TCG target handle an arbitrary
offset by emitting more add instructions. This will mean an extra
instruction in the fastpath for TCG loads and stores for the
affected guests (currently just aarch64-softmmu)

Backports commit 161dfd1e7fad1203840c0390f235030eba3fd23c from qemu
2018-04-16 13:44:39 -04:00
Richard Henderson 49476ebf5e
tcg: Introduce tcg_set_insn_start_param
The parameters for tcg_gen_insn_start are target_ulong, which may be split
into two TCGArg parameters for storage in the opcode on 32-bit hosts.

Fixes the ARM target and its direct use of tcg_set_insn_param, which would
set the wrong argument in the 64-on-32 case.

Backports commit 9743cd5736263e90d312b2c33bd739ffe1eae70d from qemu
2018-04-11 19:34:18 -04:00
Richard Henderson c2e46f2931
tcg: Mark muluh_i64 and mulsh_i64 as 64-bit ops
Failure to do so results in the tcg optimizer sign-extending
any constant fold from 32-bits. This turns out to be visible
in the RISC-V testsuite using a host that emits these opcodes
(e.g. any non-x86_64).

Backports commit f2f1dde75160cac6ede330f3db50dc817d01a2d6 from qemu
2018-03-29 14:03:00 -04:00
Lioncash 6bdfeb35ec
tcg/i386: Perform comparison pass against qemu
Ensures formatting and code are consistent.
2018-03-20 06:29:06 -04:00
Richard Henderson 0dcb2d20ed
tcg: Add choose_vector_size
This unifies 5 copies of checks for supported vector size,
and in the process fixes a missing check in tcg_gen_gvec_2s.

This lead to an assertion failure for 64-bit vector multiply,
which is not available in the AVX instruction set.

Bakports commit adb196cbd5cff26547bc32a208074f03f4c4a627 from qemu
2018-03-17 20:22:31 -04:00
Richard Henderson 2310bd4887
tcg/i386: Support INDEX_op_dup2_vec for -m32
Unknown why -m32 was passing with gcc but not clang; it should have
failed for both. This would be used for tcg_gen_dup_i64_vec, and
visible with the right TB and an aarch64 guest.

Backports commit 7f34ed4bcdfda55f978f51aadca64aa970c9f4b6 from qemu
2018-03-17 20:22:24 -04:00
Richard Henderson e9eee21efd
tcg: Improve tcg_gen_muli_i32/i64
Convert multiplication by power of two to left shift.

Backports commit b2e3ae9452fa55eb036739ec39c33f0782a97504 from qemu
2018-03-17 20:22:10 -04:00
Richard Henderson 31e93018f3
tcg: Allow 6 arguments to TCG helpers
We already handle this in the backends, and the lifetime datum
for the TCGOp is already large enough.

Backports commit 1df3caa946e08b387511dfba3a37d78910e51796 from qemu
2018-03-17 18:29:04 -04:00
Lioncash 035f1afa7d
tcg: move tcg backend files into accel/tcg/
move tcg-runtime.c, translate-all.(ch) and translate-common.c into
accel/tcg/ subdirectory and updated related trace-events file.

Backports commit 244f144134d0dd182f1af8654e7f9a79fe770368 and applies
relevant changes made in db432672dc50ed86dda17ac821b7eb07411a90af and
d9bb58e51068dfc48746c6af0179926c8dc05bce from qemu
2018-03-13 11:48:15 -04:00
Lioncash 99dbbf1571
tcg/optimize: Perform comparison pass with qemu
Keeps formatting and code synced
2018-03-12 18:06:29 -04:00
Lioncash 21b0afe218
tcg: Perform comparison pass with qemu
Makes formatting and code consistent with qemu
2018-03-12 18:03:06 -04:00
Lioncash b28c64ed34
tcg/i386: Amend bad merge 2018-03-12 10:11:03 -04:00
Richard Henderson a16ee979fc
tcg/i386: Always use TZCNT when available
I think this is cleaner than sometimes using BSF.

Backports commit 39f099ec9d6d420b6fe6f7f4f8ed80ae29c65ff2 from qemu
2018-03-12 05:11:42 -04:00
Richard Henderson 7e327aaf84
util: Introduce include/qemu/cpuid.h
Clang 3.9 passes the CONFIG_AVX2_OPT configure test. However, the
supplied <cpuid.h> does not contain the bit_AVX2 define that we use
when detecting whether the routine can be enabled.

Introduce a qemu-specific header that uses the compiler's definition
of __cpuid et al, but supplies any missing bit_* definitions needed.
This avoids introducing any extra ifdefs to util/bufferiszero.c, and
allows quite a few to be removed from tcg/i386/tcg-target.inc.c.

Backports commit 5dd8990841a9e331d9d4838a116291698208cbb6 from qemu
2018-03-09 12:12:00 -05:00
Richard Henderson d1da0b8f6d
tcg/aarch64: Add vector operations
Backports commit 14e4c1e2355473ccb2939afc69ac8f25de103b92 from qemu
2018-03-07 08:07:58 -05:00
Richard Henderson b3e89e9996
tcg/i386: Add vector operations
The x86 vector instruction set is extremely irregular. With newer
editions, Intel has filled in some of the blanks. However, we don't
get many 64-bit operations until SSE4.2, introduced in 2009.

The subsequent edition was for AVX1, introduced in 2011, which added
three-operand addressing, and adjusts how all instructions should be
encoded.

Given the relatively narrow 2 year window between possible to support
and desirable to support, and to vastly simplify code maintainence,
I am only planning to support AVX1 and later cpus.

Backports commit 770c2fc7bb70804ae9869995fd02dadd6d7656ac from qemu
2018-03-07 08:07:40 -05:00
Richard Henderson 7f55d6ed69
tcg/optimize: Handle vector opcodes during optimize
Trivial move and constant propagation. Some identity and constant
function folding, but nothing that requires knowledge of the size
of the vector element.

Backports commit 170ba88f45bd7b1c5593021ed8e174f663b0bd1a from qemu
2018-03-06 16:10:09 -05:00
Richard Henderson ac4d051b05
tcg: Add generic vector helpers with a scalar operand
Use dup to convert a non-constant scalar to a third vector.

Add addition, multiplication, and logical operations with an immediate.
Add addition, subtraction, multiplication, and logical operations with
a non-constant scalar. Allow for the front-end to build operations in
which the scalar operand comes first.

Backports commit 22fc3527034678489ec554e82fd52f8a7f05418e from qemu
2018-03-06 16:10:09 -05:00
Richard Henderson 57bdf0faa2
tcg: Add generic helpers for saturating arithmetic
No vector ops as yet. SSE only has direct support for 8- and 16-bit
saturation; handling 32- and 64-bit saturation is much more expensive.

Backports commit f49b12c6e6a75a5bd109bcbbda072b24e5fb8dfd from qemu
2018-03-06 16:10:09 -05:00
Richard Henderson ab8579123e
tcg: Add generic vector ops for multiplication
Backports commit 3774030a3e523689df24a7ed22854ce7a06b0116 from qemu
2018-03-06 16:10:08 -05:00
Richard Henderson f9c4930ecd
tcg: Add generic vector ops for comparisons
Backports commit 212be173f01e85e6589fd76676827953a84a732b from qemu
2018-03-06 16:09:38 -05:00
Richard Henderson 577ee114c3
tcg: Add generic vector ops for constant shifts
Opcodes are added for scalar and vector shifts, but considering the
varied semantics of these do not expose them to the front ends. Do
go ahead and provide them in case they are needed for backend expansion.

Backports commit d0ec97967f940bbc11dced83422b39c224127f1e from qemu
2018-03-06 14:03:30 -05:00
Richard Henderson 64365612bf
tcg: Add generic vector expanders
Backports commit db432672dc50ed86dda17ac821b7eb07411a90af from qemu
2018-03-06 13:42:52 -05:00
Richard Henderson 12fb906688
tcg: Standardize integral arguments to expanders
Some functions use intN_t arguments, some use uintN_t, some just
used "unsigned". To aid putting function pointers in tables, we
need consistency.

Backports commit 474b2e8f0f765515515b495e6872b5e18a660baf from qemu
2018-03-06 12:18:28 -05:00
Richard Henderson b9cd924fa5
tcg: Add types and basic operations for host vectors
Nothing uses or enables them yet.

Backports commit d2fd745fe8b9ac574d28b7ac63c39f6529749bd2 from qemu
2018-03-06 12:13:32 -05:00
Richard Henderson 9ef32fc039
tcg: Allow multiple word entries into the constant pool
This will be required for storing vector constants.

Backports commit da73a4abca6acefc4bb55d30bd0242bdaddb6045 from qemu
2018-03-06 11:43:21 -05:00
Lioncash 02eee6d5f7
tcg/ppc: Update to commit 030ffe39dd4128eb90483af82a5b23b23054a466 2018-03-06 09:16:37 -05:00
Richard Henderson 6212981120
tcg/ppc: Support tlb offsets larger than 64k
AArch64 with SVE has an offset of 80k to the 8th TLB.

Backports commit 4a64e0fd6876e45b34cd87b700ee30ef5c10c87a from qemu
2018-03-06 09:14:05 -05:00
Richard Henderson c4f6a7d06d
tcg/arm: Support tlb offsets larger than 64k
AArch64 with SVE has an offset of 80k to the 8th TLB.

Backports commit 71f9cee9d0a36dc4c00dfeeeca1301f265268f62 from qemu
2018-03-06 09:13:17 -05:00
Richard Henderson 9cd6985799
tcg/arm: Fix double-word comparisons
The code sequence we were generating was only good for unsigned
comparisons. For signed comparisions, use the sequence from gcc.

Fixes booting of ppc64 firmware, with a patch changing the code
sequence for ppc comparisons.

Backports commit 7170ac33135e6ecf89752d3949bcecf9b9766d1c from qemu
2018-03-06 09:12:14 -05:00
Richard Henderson bbd87f9d73
tcg: Add tcg_signed_cond
Complimenting the existing tcg_unsigned_cond.

Backports commit 923ed1750186591b04d7d61399f6d68b4e0608f2 from qemu
2018-03-05 16:55:17 -05:00
Richard Henderson 140058221d
tcg: Generalize TCGOp parameters
We had two fields specific to INDEX_op_call. Rename these and
add some macros so that the fields may be reused for other opcodes.

Backports commit cd9090aa9dbba30db8aec9a2fc103aaf1ab0f5a7 from qemu
2018-03-05 16:53:50 -05:00
Richard Henderson 7fe5f620df
tcg: Dynamically allocate TCGOps
With no fixed array allocation, we can't overflow a buffer.
This will be important as optimizations related to host vectors
may expand the number of ops used.

Use QTAILQ to link the ops together.

Backports commit 15fa08f8451babc88d733bd411d4c94976f9d0f8 from qemu
2018-03-05 16:34:40 -05:00
Richard Henderson 5f074f09ab
tcg: Remove TCGV_UNUSED* and TCGV_IS_UNUSED*
These are now trivial sets and tests against NULL. Unwrap.

Backports commit f764718d0cb30af9f1f8e1d6a33622cc05ca4155 from qemu
2018-03-05 15:58:15 -05:00
Richard Henderson 5ef155a68f
tcg/s390x: Use constant pool for prologue
Rather than have separate code only used for guest_base,
rely on a recent change to handle constant pool entries.

Backports commit ba2c747992f8c315c2fbddba196ce9137430d61d from qemu
2018-03-05 11:28:39 -05:00
Richard Henderson ef3f552229
tcg: Allow constant pool entries in the prologue
Both ARMv6 and AArch64 currently may drop complex guest_base values
into the constant pool. But generic code wasn't expecting that, and
the pool is not emitted. Correct that.

Backports commit 5b38ee31616d1532c3c3a6dc644a9160d608ed2f from qemu
2018-03-05 11:25:56 -05:00
Richard Henderson ab9df6244c
tcg: Use offsets not indices for TCGv_*
Using the offset of a temporary, relative to TCGContext, rather than
its index means that we don't use 0. That leaves offset 0 free for
a NULL representation without having to leave index 0 unused.

Backports commit e89b28a63501c0ad6d2501fe851d0c5202055e70 from qemu
2018-03-05 10:12:08 -05:00
Richard Henderson 4d9c8583fa
tcg: Remove TCGV_EQUAL*
When we used structures for TCGv_*, we needed a macro in order to
perform a comparison. Now that we use pointers, this is just clutter

Backports commit 11f4e8f8bfaa2caaab24bef6bbbb8a0205015119 from qemu
2018-03-05 09:16:07 -05:00
Richard Henderson d450156414
tcg: Remove GET_TCGV_* and MAKE_TCGV_*
The GET and MAKE functions weren't really specific enough.
We now have a full complement of functions that convert exactly
between temporaries, arguments, tcgv pointers, and indices.

The target/sparc change is also a bug fix, which would have affected
a host that defines TCG_TARGET_HAS_extr[lh]_i64_i32, i.e. MIPS64.

Backports commit dc41aa7d34989b552efe712ffe184236216f960b from qemu
2018-03-05 09:12:26 -05:00
Richard Henderson 960eb3f4f9
tcg: Introduce temp_tcgv_{i32,i64,ptr}
Backports commit 085272b35e0644fea373c33b5265c1818b7a978c from qemu
2018-03-05 08:55:52 -05:00
Richard Henderson 2bb5011b18
tcg: Introduce tcgv_{i32,i64,ptr}_{arg,temp}
Transform TCGv_* to an "argument" or a temporary.
For now, an argument is simply the temporary index.

Backports commit ae8b75dc6ec808378487064922f25f1e7ea7a9be from qemu
2018-03-05 08:46:12 -05:00
Richard Henderson 9f8c6a456b
tcg: Use per-temp state data in optimize
While we're touching many of the lines anyway, adjust the naming
of the functions to better distinguish when "TCGArg" vs "TCGTemp"
should be used.

Backports commit 6349039d0b06eda59820629b934944246b14a1c1 from qemu
2018-03-05 08:24:06 -05:00
Richard Henderson 387060ccf5
tcg: Remove unused TCG_CALL_DUMMY_TCGV
Backports commit 54534d7cfd3bdff1aa1f6c9472d94243d2303656 from qemu
2018-03-05 07:52:35 -05:00
Richard Henderson d104b792a6
tcg: Change temp_allocate_frame arg to TCGTemp
Backports commit 2272e4a791b7e1a01ffac143616ba4ece9a5762d from qemu
2018-03-05 07:51:40 -05:00
Richard Henderson 35a7a9c9a4
tcg: Avoid loops against variable bounds
Copy s->nb_globals or s->nb_temps to a local variable for the purposes
of iteration. This should allow the compiler to use low-overhead
looping constructs on some hosts.

Backports commit ac3b88911ebc6fc841f28898ee8aed40839debe2 from qemu
2018-03-05 07:50:06 -05:00
Richard Henderson 1f4ac863bf
tcg: Use per-temp state data in liveness
This avoids having to allocate external memory for each temporary.

Backports commit b83eabeac06e38706738bd5e92b1ba117a1b554d from qemu
2018-03-05 07:47:51 -05:00
Richard Henderson 87f2067aac
tcg: Introduce temp_arg, export temp_idx
At the same time, drop the TCGContext argument and use tcg_ctx instead.

Backports commit 1807f4c40098070008eb84b2032e25b7ac42569e from qemu
2018-03-05 07:24:17 -05:00
Richard Henderson a659a03ff5
tcg: Return NULL temp for TCG_CALL_DUMMY_ARG
Backports commit c6c7d84df8889b9d6298466999b88a8a42e5f976 from qemu
2018-03-05 07:22:38 -05:00
Richard Henderson 010ded3088
tcg: Add temp_global bit to TCGTemp
This avoids needing to test the index of a temp against nb_globals.

Backports commit fa477d25470187030614288d35bc734edffa41ee from qemu
2018-03-05 07:21:10 -05:00
Richard Henderson a9c46ad7a0
tcg: Introduce arg_temp
Backports commit 434391390ba99996af1591b427a73b3f5c05065e from qemu
2018-03-05 07:17:44 -05:00
Richard Henderson c8f0f6901e
tcg: Propagate TCGOp down to allocators
Backports commit dd186292017641d5b31fc13225a420677e1d20d3 from qemu
2018-03-05 07:12:48 -05:00
Richard Henderson f1e2ea6847
tcg: Propagate args to op->args in tcg.c
Backports commit efee3746fa471852daba7674b0d34f8c88be7559 from qemu
2018-03-05 07:06:50 -05:00
Richard Henderson 845cfc2ae9
tcg: Propagate args to op->args in optimizer
Backports commit acd937019bdaf933fcf1a7b57679ba07119c89b7 from qemu
2018-03-05 06:56:06 -05:00
Richard Henderson eb488f5bd6
tcg: Merge opcode arguments into TCGOp
Rather than have a separate buffer of 10*max_ops entries,
give each opcode 10 entries. The result is actually a bit
smaller and should have slightly more cache locality.

Backports commit 75e8b9b7aa0b95a761b9add7e2f09248b101a392 from qemu
2018-03-05 04:45:20 -05:00
Jiang Biao 60ef6d016d
tcg/mips: delete commented out extern keyword
Backports commit 8df8d529ed958de4e23dcbf38bd34eff1a4716f2 from qemu
2018-03-05 03:24:25 -05:00
Emilio G. Cota 239e9771df
tcg: define TCG_HIGHWATER
Will come in handy very soon.

Backports commit a505785cd221994dd3713bde860861869a059940 from qemu
2018-03-05 03:22:27 -05:00
Emilio G. Cota 8552d95c52
exec-all: extract tb->tc_* into a separate struct tc_tb
In preparation for adding tc.size to be able to keep track of
TB's using the binary search tree implementation from glib.

Backports commit e7e168f41364c6e83d0f75fc1b3ce7f9c41ccf76 from qemu
2018-03-05 02:57:22 -05:00
Emilio G. Cota 5fae6dd433
tcg: remove addr argument from lookup_tb_ptr
It is unlikely that we will ever want to call this helper passing
an argument other than the current PC. So just remove the argument,
and use the pc we already get from cpu_get_tb_cpu_state.

This change paves the way to having a common "tb_lookup" function.

Backports commit 7f11636dbee89b0e4d03e9e2b96e14649a7db778 from qemu
2018-03-05 02:16:34 -05:00
Emilio G. Cota f1d6630893
tcg/mips: constify tcg_target_callee_save_regs
Backports commit d453ec78251d03cbd4ffc28dbf6070931c8ae469 from qemu
2018-03-05 02:08:36 -05:00
Emilio G. Cota 3cf23eb256
tcg/i386: constify tcg_target_callee_save_regs
Backports commit e268f4c036d2b47a4f8bf293c1371b328e03ca04 from qemu
2018-03-05 02:08:02 -05:00
Richard Henderson 7168f72d4d
tcg/mips: Fully convert tcg_target_op_def
Backports commit 89b2e37e6506d92b00ac478e7953be6ddd7a86a9 from qemu
2018-03-04 23:54:26 -05:00
Richard Henderson 24c5be0472
tcg/sparc: Fully convert tcg_target_op_def
Backports commit 9be44a16c258287aab5a3accda153d3a5144359f from qemu
2018-03-04 23:52:18 -05:00
Richard Henderson d3b1c8d5a4
tcg/ppc: Fully convert tcg_target_op_def
Backports commit 6cb3658a04149b2c1fb92e2ea9d2e2f6cecc0014 from qemu
2018-03-04 23:50:58 -05:00
Richard Henderson 3094e7927e
tcg/arm: Fully convert tcg_target_op_def
Backports commit 7536b82d28876d1ffe0359667b28c93d49386fa0 from qemu
2018-03-04 23:48:55 -05:00
Richard Henderson 47ed20fdd4
tcg/aarch64: Fully convert tcg_target_op_def
Backports commit 1897cc2eb8be2d8be23380b45a2d3c1a2808723f from qemu
2018-03-04 23:46:38 -05:00
Richard Henderson fe632c4df8
tcg: Fix types in tcg_regset_{set,reset}_reg
There was a potential problem here with an ILP32 host
with 64 host registers.

Backports commit 80a8b9a910e14d4a1937f70dce944891990f3441 from qemu
2018-03-04 23:44:13 -05:00
Richard Henderson fc8b4316a9
tcg: Remove tcg_regset_set32
It's not even clear what the interface REG and VAL32 were supposed to mean.
All uses had REG = 0 and VAL32 was the bitset assigned to the destination.

Backports commit f46934df662182097dce07d57ec00f37e4d2abf1 from qemu
2018-03-04 23:42:59 -05:00
Richard Henderson 9a9c2ede4a
tcg: Remove tcg_regset_{or,and,andnot,not}
Backports commit 07ddf036fa66bca279590c09fe1c46bcdcc5bcff from qemu
2018-03-04 23:34:16 -05:00
Richard Henderson 7ba6f6f5e6
tcg: Remove tcg_regset_set
Backports commit d21369f5fb41299d5e7b032ec6da12da7f95f72f from qemu
2018-03-04 23:31:35 -05:00
Richard Henderson 49d09d6888
tcg: Remove tcg_regset_clear
Backports commit ccb1bb66ea2a42e773bfa04178d8b383ff86d4d8 from qemu
2018-03-04 23:24:45 -05:00
Richard Henderson 7b68a8f0ca
tcg: Add tcg_op_supported
Backports commit be0f34b5840312bbe9627c2b9f68a25f32903dae from qemu
2018-03-04 23:20:28 -05:00
Lioncash 3c5f8b2800
tcg/ppc: Update to commit 53c89efd02cef626040165cc8f06b5cf2c15355d 2018-03-04 23:00:03 -05:00
Lioncash c786137691
tcg/arm: Update to commit afe74dbd6a58031741b68e99843c1f1d390996b2 2018-03-04 22:58:36 -05:00
Richard Henderson 504bdad70d
tcg/arm: Tighten tlb indexing offset test
We are not going to use ldrd for loading the comparator
for 32-bit guests, so don't limit cmp_off to 8 bits then.
This eliminates one insn in the tlb load for some guests.

Backports commit 95ede84f4de18747d03d79c148013cff99acd60b from qemu
2018-03-04 22:57:04 -05:00
Richard Henderson e4d05c2567
tcg/arm: Improve tlb load for armv7
Use UBFX to avoid limitation on CPU_TLB_BITS. Since we're dropping
the initial shift, we need to replace the page masking. We can use
MOVW+BIC to do this without shifting. The result is the same size
as the armv6 path with one less conditional instruction.

Backports commit 647ab96aaf5defeb138e48d610f7f633c587b40d from qemu
2018-03-04 22:56:27 -05:00
Richard Henderson b3fd6a8c8c
tcg/sparc: Use constant pool for movi
Backports commit e9823b4c3347370414b63010ec4a2a4754e4abb5 from qemu
2018-03-04 22:53:59 -05:00
Richard Henderson b786e2d27e
tcg/sparc: Introduce TCG_REG_TB
Backports commit ab20bdc11624837bd0c8aea83c603b66f0406e8b from qemu
2018-03-04 22:51:38 -05:00
Richard Henderson 0c3781e7eb
tcg/aarch64: Use constant pool for movi
Backports commit 55129955e92ec164ee2d778f20070dc214109bc6 from qemu
2018-03-04 22:46:50 -05:00
Richard Henderson 5150970625
tcg/s390: Use constant pool for cmpi
Also use CHI/CGHI for 16-bit signed constants.

Backports commit a534bb15f30ff7e420434b3e5746bcad595c5429 from qemu
2018-03-04 22:44:26 -05:00
Richard Henderson c08620b984
tcg/s390: Use constant pool for xori
Backports commit 5bf67a9217a31512f35b036924e1db1baf2f9ebf from qemu
2018-03-04 22:39:14 -05:00
Lioncash 35d3118469
tcg/s390: Use constant pool for ori 2018-03-04 22:35:27 -05:00
Richard Henderson bdadfa7520
tcg/s390: Use constant pool for andi
Backports commit bdcd5d1926a7ae42c060efdcaa15074930a92ebb from qemu
2018-03-04 22:33:08 -05:00
Richard Henderson bc23bab79d
tcg/s390: Use constant pool for movi
Split out maybe_out_small_movi for use with other operations
that want to add to the constant pool.

Backports commit 28eef8aaece5e83df4568d9842ab9611ec130b2c from qemu
2018-03-04 22:32:04 -05:00
Richard Henderson ba1563eb2f
tcg/s390: Fix sign of patch_reloc addend
We were passing in -2 instead of +2, but then ignoring
the actual contents of addend in the calculation.

Backports commit e692a3492d04500355bcf23575eed7cf137b38d5 from qemu
2018-03-04 22:28:24 -05:00
Richard Henderson 2fff7d54cb
tcg/s390: Introduce TCG_REG_TB
Backports commit 829e1376d94009a7ccacc0535bffcc679f7bb507 from qemu
2018-03-04 22:26:52 -05:00
Richard Henderson b96f53e8a3
tcg/i386: Store out-of-range call targets in constant pool
Already it saves 2 bytes per call, but also the constant pool
entry may well be shared across multiple calls.

Backports commit 4e45f23943c0bb91588627de3801826546155ad8 from qemu
2018-03-04 22:22:49 -05:00
Richard Henderson e9d8cef430
tcg: Infrastructure for managing constant pools
A new shared header tcg-pool.inc.c adds new_pool_label,
for registering a tcg_target_ulong to be emitted after
the generated code, plus relocation data to install a
pointer to the data.

A new pointer is added to the TCGContext, so that we
dump the constant pool as data, not code.

Backports commit 57a269469dbf70013dab3a176e1735636010a772 from qemu
2018-03-04 22:17:33 -05:00