Commit graph

153 commits

Author SHA1 Message Date
Richard Henderson 5cce52a04b tcg: Implement gvec support for rotate by immediate
No host backend support yet, but the interfaces for rotli
are in place. Canonicalize immediate rotate to the left,
based on a survey of architectures, but provide both left
and right shift interfaces to the translators.

Backports commit b0f7e7444c03da17e41bf327c8aea590104a28ab from qemu
2020-06-14 21:26:58 -04:00
Tony Nguyen f75368cd0f
tcg: TCGMemOp is now accelerator independent MemOp
Preparation for collapsing the two byte swaps, adjust_endianness and
handle_bswap, along the I/O path.

Target dependant attributes are conditionalized upon NEED_CPU_H.

Backports commit 14776ab5a12972ea439c7fb2203a4c15a09094b4 from qemu
2019-11-28 03:01:12 -05:00
Richard Henderson 2ea6dfbd63
tcg: Add support for vector compare select
Perform a per-element conditional move. This combination operation is
easier to implement on some host vector units than plain cmp+bitsel.
Omit the usual gvec interface, as this is intended to be used by
target-specific gvec expansion call-backs.

Backports commit f75da2988eb2457fa23d006d573220c5c680ec4e from qemu
2019-05-24 18:21:13 -04:00
Richard Henderson ca58be9cb4
tcg: Add support for vector bitwise select
This operation performs d = (b & a) | (c & ~a), and is present
on a majority of host vector units. Include gvec expanders.

Backports commit 38dc12947ec9106237f9cdbd428792c985cd86ae from qemu
2019-05-24 18:15:10 -04:00
Richard Henderson 6d5e7856ff
tcg: Add support for vector absolute value
Backports commit bcefc90208f8a1d6f619d61c2647281d92277015 from qemu
2019-05-16 16:33:43 -04:00
Richard Henderson 66e6bea084
tcg: Add INDEX_op_dupm_vec
Allow the backend to expand dup from memory directly, instead of
forcing the value into a temp first. This is especially important
if integer/vector register moves do not exist.

Note that officially tcg_out_dupm_vec is allowed to fail.
If it did, we could fix this up relatively easily:

VECE == 32/64:
Load the value into a vector register, then dup.
Both of these must work.

VECE == 8/16:
If the value happens to be at an offset such that an aligned
load would place the desired value in the least significant
end of the register, go ahead and load w/garbage in high bits.

Load the value w/INDEX_op_ld{8,16}_i32.
Attempt a move directly to vector reg, which may fail.
Store the value into the backing store for OTS.
Load the value into the vector reg w/TCG_TYPE_I32, which must work.
Duplicate from the vector reg into itself, which must work.

All of which is well and good, except that all supported
hosts can support dupm for all vece, so all of the failure
paths would be dead code and untestable.

Backports commit 37ee55a081b7863ffab2151068dd1b2f11376914 from qemu
2019-05-16 15:38:02 -04:00
Richard Henderson d4e7c6a8c5
tcg: Add tcg_out_dupm_vec to the backend interface
Currently stubbed out in all backends that support vectors.

Backports commit d6ecb4a978b718dbe108a9fa9ecccc8b7f7cb579 from qemu
2019-05-16 15:24:48 -04:00
Richard Henderson cf238d3544
tcg: Manually expand INDEX_op_dup_vec
This case is similar to INDEX_op_mov_* in that we need to do
different things depending on the current location of the source.

Backports commit bab1671f0fa928fd678a22f934739f06fd5fd035 from qemu
2019-05-16 15:22:29 -04:00
Richard Henderson 3d20e1678c
tcg: Promote tcg_out_{dup,dupi}_vec to backend interface
The i386 backend already has these functions, and the aarch64 backend
could easily split out one. Nothing is done with these functions yet,
but this will aid register allocation of INDEX_op_dup_vec in a later patch.

Adjust the aarch64 tcg_out_dupi_vec signature to match the new interface.

Backports commit e7632cfa8b76cdbbc1c76e8737338ef5844e7d60 from qemu
2019-05-16 15:18:48 -04:00
Richard Henderson d58d9ad16e
tcg: Support cross-class moves without instruction support
PowerPC Altivec does not support direct moves between vector registers
and general registers. So when tcg_out_mov fails, we can use the
backing memory for the temporary to perform the move.

Backports commit 240c08d0998f402c325fce489de0d14831048128 from qemu
2019-05-16 15:16:23 -04:00
Richard Henderson f86bd1c5d6
tcg: Return bool success from tcg_out_mov
This patch merely changes the interface, aborting on all failures,
of which there are currently none.

Backports commit 78113e83e0007e869c9f0cb4c0497a77538988e3 from qemu
2019-05-16 15:14:42 -04:00
Richard Henderson fef5700c9c
tcg: Assert fixed_reg is read-only
The only fixed_reg is cpu_env, and it should not be modified
during any TB. Therefore code that tries to special-case moves
into a fixed_reg is dead. Remove it.

Backports commit d63e3b6e694ad6c887be135dddb9cd4893f1a844 from qemu
2019-05-16 15:09:37 -04:00
Richard Henderson 6145e3fdd7
tcg: Restart TB generation after out-of-line ldst overflow
This is part c of relocation overflow handling.

Backports commit aeee05f53a5d67304a521d2644dc0a607e3c8b28 from qemu
2019-04-30 10:06:53 -04:00
Richard Henderson 196631e0a4
tcg: Restart TB generation after constant pool overflow
This is part b of relocation overflow handling.

Backports commit 1768987b73fa7e23e58b7844abe5882490ff8e42 from qemu
2019-04-30 10:00:52 -04:00
Richard Henderson 45315fd8ef
tcg: Restart TB generation after relocation overflow
If the TB generates too much code, such that backend relocations
overflow, try again with a smaller TB. In support of this, move
relocation processing from a random place within tcg_out_op, in
the handling of branch opcodes, to a new function at the end of
tcg_gen_code.

This is not a complete solution, as there are additional relocs
generated for out-of-line ldst handling and constant pools.

Backports commit 7ecd02a06f8f4c0bbf872ecc15e37035b7e1df5f from qemu
2019-04-30 09:58:45 -04:00
Richard Henderson 434b3ab9ec
tcg: Restart after TB code generation overflow
If a TB generates too much code, try again with fewer insns.

Fixes: https://bugs.launchpad.net/bugs/1824853

Backports commit 6e6c4efed995d9eca6ae0cfdb2252df830262f50 from qemu
2019-04-30 09:52:57 -04:00
Richard Henderson 269fa0daba
tcg: Add INDEX_op_extract2_{i32,i64}
This will let backends implement the double-word shift operation.

Backports commit fce1296f135669eca85dc42154a2a352c818ad76 from qemu
2019-04-30 09:29:05 -04:00
Lioncash f0c271ca2f
tcg: Correct special-cased brcond handling 2019-04-26 10:25:46 -04:00
Lioncash 4a64ebf95e
tcg: Synchronize with qemu 2019-04-26 09:32:20 -04:00
Lioncash 3996153514
tcg: Synchronize with qemu 2019-04-26 08:48:32 -04:00
Lioncash 006a13026a
tcg: Remove inconsistent g_strdup usage
The ts's name was allocated with strdup, but ts2's was being done with
g_strdup. Makes them consistent with upstream Qemu.
2019-04-26 08:48:32 -04:00
Lioncash e579832dcb
tcg: Synchronize with qemu 2019-04-18 04:57:19 -04:00
Richard Henderson f7c5f0ccbe
tcg: Diagnose referenced labels that have not been emitted
Currently, a jump to a label that is not defined anywhere will
be emitted not be relocated. This results in a jump to a random
jump target. With tcg debugging, print a diagnostic to the -d op
file and abort.

This could help debug or detect errors like
c2d9644e6d ("target/arm: Fix crash on conditional instruction in an IT block")

Backports commit bef16ab4e641636b4e85c3d863b4257ce0be4e6f from qemu
2019-02-12 11:35:11 -05:00
Richard Henderson fb684825c8
tcg: Add opcodes for vector minmax arithmetic
Backports commit dd0a0fcdd8848c2a18970c44a62bd8f394c2b495 from qemu
2019-01-29 16:24:52 -05:00
Richard Henderson e0266239ea
tcg: Add opcodes for vector saturated arithmetic
Backports commit 8afaf0506606f8003ef696df849c5a98637a7a83 from qemu
2019-01-29 16:14:34 -05:00
Richard Henderson 6ed82f77b4
tcg: Improve call argument loading
Free the argument register only after we have verified that the
temporary is not already in that register. This case is likely
now that we are back propagating the preferred register.

Backports commit 4250da10923347c9ee907f8d72bd93dfa5ee8742 from qemu
2019-01-05 07:24:08 -05:00
Richard Henderson 64843e8c09
tcg: Record register preferences during liveness
With these preferences, we can arrange for function call arguments to
be computed into the proper registers instead of requiring extra moves.

Backports commit 25f49c5f1508ddf081ce89fa6bbfd87a51eea37b from qemu
2019-01-05 07:22:57 -05:00
Richard Henderson 63cf164724
tcg: Split out more subroutines from liveness_pass_1
Backports commit f65a061c39cc4f9d088201031050e42eb23d5b2a from qemu
2019-01-05 07:07:49 -05:00
Richard Henderson c348ceba56
tcg: Rename and adjust liveness_pass_1 helpers
No need for a "tcg_" prefix for a static function; we already
have another "la_" prefix for indicating liveness analysis.
Pass in nb_globals and nb_temps, as we will already have them
in registers for other loops within the parent function.

Backports commit 2616c8082143373e794b62444bf81754f50dbf6b from qemu
2019-01-05 07:05:58 -05:00
Richard Henderson b356212b33
tcg: Dump register preference info with liveness
Backports commit 1894f69a612b35c2a39b44a824da04d74bfe324a from qemu
2019-01-05 07:00:21 -05:00
Richard Henderson 494d802781
tcg: Improve register allocation for matching constraints
Try harder to honor the output_pref. When we're forced to allocate
a second register for the input, it does not need to use the input
constraint; that will be honored by the register we allocate for the
output and a move is already required.

Backports commit d62816f2db439b2dd761c674f0256f21d9dd2ed0 from qemu
2019-01-05 06:57:56 -05:00
Richard Henderson 83a7de2566
tcg: Add output_pref to TCGOp
Allocate storage for, but do not yet fill in, per-opcode
preferences for the output operands. Pass it in to the
register allocation routines for output operands.

Backports commit 69e3706d2b473815e382552e729d12590339e0ac from qemu
2019-01-05 06:54:40 -05:00
Richard Henderson 19bde1a9cf
tcg: Add preferred_reg argument to tcg_reg_alloc_do_movi
Pass this through to temp_sync.

Backports commit ba87719cd267e6f07b17f6cda08246bf483146d4 from qemu
2019-01-05 06:51:55 -05:00
Richard Henderson c3aa567b03
tcg: Add preferred_reg argument to temp_sync
Pass this through to tcg_reg_alloc.

Backports commit 98b4e186c1ccb8f1868c61a33a3be8c2b82654f3 from qemu
2019-01-05 06:50:22 -05:00
Richard Henderson 96b6640f3b
tcg: Add preferred_reg argument to temp_load
Pass this through to tcg_reg_alloc.

Backports commit b722452aefb089e003b16946a4d73bad1fd3b79b from qemu
2019-01-05 06:48:19 -05:00
Richard Henderson 5e73b27607
tcg: Add preferred_reg argument to tcg_reg_alloc
This new argument will aid register allocation by indicating how
the temporary will be used in future. If the preference cannot
be satisfied, fall back to the constraints of the current insn.

Short circuit the preference when it cannot be satisfied or if
it does not further constrain the operation.

With an eye toward optimizing function call sequences, optimize
for the preferred_reg set containing a single register.

For the moment, all users pass 0 for preference.

Backports commit b016486e7baddb43cfc1e51909b05cde9cf82e0c from qemu
2019-01-05 06:45:15 -05:00
Richard Henderson 6aea2880d2
tcg: Add reachable_code_pass
Delete trivially dead code that follows unconditional branches and
noreturn helpers. These can occur either via optimization or via
the structure of a target's translator following an exception.

Backports commit b4fc67c7afd2c338d6e7c73a7f428dfe05ae0603 from qemu
2019-01-05 06:41:16 -05:00
Richard Henderson 26ab4d6560
tcg: Reference count labels
Increment when adding branches, and decrement when removing them.

Backports commit d88a117eaa39b1d0eb1a79fe84c81840a39eb233 from qemu
2019-01-05 06:39:20 -05:00
Emilio G. Cota 0567c69235
tcg: Drop nargs from tcg_op_insert_{before,after}
It's unused since 75e8b9b7aa0b95a761b9add7e2f09248b101a392.

Backports commit ac1043f6d607aaac206c8aac42bc32f634f59395 from qemu
2018-12-18 06:00:13 -05:00
Richard Henderson 46189d87b3
tcg: Return success from patch_reloc
This will move the assert for success from within (subroutines of)
patch_reloc into the callers. It will also let new code do something
different when a relocation is out of range.

For the moment, all backends are trivially converted to return true.

Backports commit 6ac1778676f4259c10b0629ccd9df319a5d1baeb from qemu
2018-12-18 05:25:45 -05:00
Emilio G. Cota 223975ada0
tcg: fix use of uninitialized variable under CONFIG_PROFILER
We forgot to initialize n in commit 15fa08f845 ("tcg: Dynamically
allocate TCGOps", 2017-12-29).

Backports commit c1f543b739086733024e31d74a52d9e41553f316 from qemu
2018-10-23 14:37:37 -04:00
Lioncash cc3d618e61
tcg: Remove unnecessary MSVC ifdef
All relevant arrays have at least one member in them now, making this
unnecessary.
2018-10-06 05:08:17 -04:00
Richard Henderson f417df19b7
tcg: Limit the number of ops in a TB
In 6001f7729e12 we partially attempt to address the branch
displacement overflow caused by 15fa08f845.

However, gcc/testsuite/gcc.target/aarch64/advsimd-intrinsics/vqtbX.c
is a testcase that contains a TB so large as to overflow anyway.
The limit here of 8000 ops produces a maximum output TB size of
24112 bytes on a ppc64le host with that test case. This is still
much less than the maximum forward branch distance of 32764 bytes.

Backports commit abebf92597186be2bc48d487235da28b1127860f from qemu
2018-05-11 11:25:01 -04:00
Laurent Vivier ec12091943
tcg: workaround branch instruction overflow in tcg_out_qemu_ld/st
ppc64 uses a BC instruction to call the tcg_out_qemu_ld/st
slow path. BC instruction uses a relative address encoded
on 14 bits.

The slow path functions are added at the end of the generated
instructions buffer, in the reverse order of the callers.
So more we have slow path functions more the distance between
the caller (BC) and the function increases.

This patch changes the behavior to generate the functions in
the same order of the callers.

Backports commit 6001f7729e12dd1d810291e4cbf83cee8e07441d from qemu
2018-05-03 15:09:07 -04:00
Richard Henderson 2150745db4
tcg: Improve TCGv_ptr support
Drop TCGV_PTR_TO_NAT and TCGV_NAT_TO_PTR internal macros.

Add tcg_temp_local_new_ptr, tcg_gen_brcondi_ptr, tcg_gen_ext_i32_ptr,
tcg_gen_trunc_i64_ptr, tcg_gen_extu_ptr_i64, tcg_gen_trunc_ptr_i32.

Use inlines instead of macros where possible.

Backports commit 5bfa803448638a45542441fd6b7cc1241403ea72 from qemu
2018-05-03 15:05:43 -04:00
Lioncash 21b0afe218
tcg: Perform comparison pass with qemu
Makes formatting and code consistent with qemu
2018-03-12 18:03:06 -04:00
Richard Henderson ab8579123e
tcg: Add generic vector ops for multiplication
Backports commit 3774030a3e523689df24a7ed22854ce7a06b0116 from qemu
2018-03-06 16:10:08 -05:00
Richard Henderson f9c4930ecd
tcg: Add generic vector ops for comparisons
Backports commit 212be173f01e85e6589fd76676827953a84a732b from qemu
2018-03-06 16:09:38 -05:00
Richard Henderson 577ee114c3
tcg: Add generic vector ops for constant shifts
Opcodes are added for scalar and vector shifts, but considering the
varied semantics of these do not expose them to the front ends. Do
go ahead and provide them in case they are needed for backend expansion.

Backports commit d0ec97967f940bbc11dced83422b39c224127f1e from qemu
2018-03-06 14:03:30 -05:00
Richard Henderson 64365612bf
tcg: Add generic vector expanders
Backports commit db432672dc50ed86dda17ac821b7eb07411a90af from qemu
2018-03-06 13:42:52 -05:00