Remove a function of the same name from target/arm/.
Use a branchless implementation of abs gleaned from gcc.
Backports commit ff1f11f7f8710a768f9313f24bd7f509d3db27e5 from qemu
Allow expansion either via shift by scalar or by replicating
the scalar for shift by vector.
Backports commit b4578cd91cda4cef1c413304353ca6dc5b957b60 from qemu
The gvec expanders perform a modulo on the shift count. If the target
requires alternate behaviour, then it cannot use the generic gvec
expanders anyway, and will have to have its own custom code.
Backports commit 5ee5c14cacda27e904cd6b0d9e7ffe1acff42838 from qemu
Allow the backend to expand dup from memory directly, instead of
forcing the value into a temp first. This is especially important
if integer/vector register moves do not exist.
Note that officially tcg_out_dupm_vec is allowed to fail.
If it did, we could fix this up relatively easily:
VECE == 32/64:
Load the value into a vector register, then dup.
Both of these must work.
VECE == 8/16:
If the value happens to be at an offset such that an aligned
load would place the desired value in the least significant
end of the register, go ahead and load w/garbage in high bits.
Load the value w/INDEX_op_ld{8,16}_i32.
Attempt a move directly to vector reg, which may fail.
Store the value into the backing store for OTS.
Load the value into the vector reg w/TCG_TYPE_I32, which must work.
Duplicate from the vector reg into itself, which must work.
All of which is well and good, except that all supported
hosts can support dupm for all vece, so all of the failure
paths would be dead code and untestable.
Backports commit 37ee55a081b7863ffab2151068dd1b2f11376914 from qemu
The LD1R instruction does all the work. Note that the only
useful addressing mode is a base register with no offset.
Backports commit f23e5e15edfd49d5dd72cab2ed2d85ac354b2eeb from qemu
This case is similar to INDEX_op_mov_* in that we need to do
different things depending on the current location of the source.
Backports commit bab1671f0fa928fd678a22f934739f06fd5fd035 from qemu
The i386 backend already has these functions, and the aarch64 backend
could easily split out one. Nothing is done with these functions yet,
but this will aid register allocation of INDEX_op_dup_vec in a later patch.
Adjust the aarch64 tcg_out_dupi_vec signature to match the new interface.
Backports commit e7632cfa8b76cdbbc1c76e8737338ef5844e7d60 from qemu
PowerPC Altivec does not support direct moves between vector registers
and general registers. So when tcg_out_mov fails, we can use the
backing memory for the temporary to perform the move.
Backports commit 240c08d0998f402c325fce489de0d14831048128 from qemu
This patch merely changes the interface, aborting on all failures,
of which there are currently none.
Backports commit 78113e83e0007e869c9f0cb4c0497a77538988e3 from qemu
We have a function that takes an additional condition parameter
over the standard backend interface. It already takes care of
eliding no-op moves.
Backports commit c16f52b2c5d91c36e121795bd3b386cea0b7573c from qemu
The only fixed_reg is cpu_env, and it should not be modified
during any TB. Therefore code that tries to special-case moves
into a fixed_reg is dead. Remove it.
Backports commit d63e3b6e694ad6c887be135dddb9cd4893f1a844 from qemu
Replace the single opcode in .opc with a null-terminated
array in .opt_opc. We still require that all opcodes be
used with the same .vece.
Validate the contents of this list with CONFIG_DEBUG_TCG.
All tcg_gen_*_vec functions will check any list active
during .fniv expansion. Swap the active list in and out
as we expand other opcodes, or take control away from the
front-end function.
Convert all existing vector aware front ends.
Backports commit 53229a7703eeb2bbe101a19a33ef22aaf960c65b from qemu
PowerPC Altivec does not support add and subtract of 64-bit elements.
Prepare for that configuration by not assuming the operation is
universally supported.
Backports commit ce27c5d1a38e93da38653af71fb468c5eded4c7b from qemu
Use tcg_can_emit_vec_op instead of just TCG_TARGET_HAS_neg_vec,
so that we check the type and vece for the actual operation.
Backports commit ac383dde33405106469d04a78de1d76f1a730cb1 from qemu
Let's add tcg_gen_gvec_3i(), similar to tcg_gen_gvec_2i(), however
without introducing "gen_helper_gvec_3i *fnoi", as it isn't needed
for now.
Backports commit e1227bb6e59173117f094a6a13b998587b45c928 from qemu
Thereby decoupling the resulting translated code from the current state
of the system.
The tb->cflags field is not passed to tcg generation functions. So
we add a field to TCGContext, storing there a copy of tb->cflags.
Most architectures have <= 32 registers, which results in a 4-byte hole
in TCGContext. Use this hole for the new field.
Backports commit e82d5a2460b0e176128027651ff9b104e4bdf5cc from qemu
This will not necessarily restrict the size of the TB, since for v7
the majority of constant pool usage is for calls from the out-of-line
ldst code, which is already at the end of the TB. But this does
allow us to save one insn per reference on the off-chance.
Backports commit b4b82d7e9caff7ccca5c621817b5a4b8e95eb9b1 from qemu
There is no point in coding for a 2GB offset when the max TB size
is already limited to 64k. If we further restrict to 32k then we
can eliminate the extra ADDIS instruction.
Backports commit a7cdaf710f2aaaf0be855a338dd67463d4bb99e2 from qemu
If the TB generates too much code, such that backend relocations
overflow, try again with a smaller TB. In support of this, move
relocation processing from a random place within tcg_out_op, in
the handling of branch opcodes, to a new function at the end of
tcg_gen_code.
This is not a complete solution, as there are additional relocs
generated for out-of-line ldst handling and constant pools.
Backports commit 7ecd02a06f8f4c0bbf872ecc15e37035b7e1df5f from qemu
If a TB generates too much code, try again with fewer insns.
Fixes: https://bugs.launchpad.net/bugs/1824853
Backports commit 6e6c4efed995d9eca6ae0cfdb2252df830262f50 from qemu
Will be helpful for s390x. Input 128 bit and output 64 bit only,
which is sufficient for now.
Backports commit 2089fcc9e7b4174d1c351eaa7d277c02188a6dd2 from qemu
This ports over the RISC-V architecture from Qemu. This is currently a
very barebones transition. No code hooking or any fancy stuff.
Currently, you can feed it instructions and query the CPU state itself.
This also allows choosing whether or not RISC-V 32-bit or RISC-V 64-bit
is desirable through Unicorn's interface as well.
Extremely basic examples of executing a single instruction have been
added to the samples directory to help demonstrate how to use the basic
functionality.
The last update to this file was 9 years ago. In the meantime,
4 of the 6 ideas have actually been completed. The lat two do
not actually make sense anymore.
Backports commit 9e564a1dde5abc7ae4cebc115142f685d98938d7 from qemu
Completely rewrite conditional stores handling. Use cmpxchg.
This eliminates need for separate implementations of SC instruction
emulation for user and system emulation.
Backports commit 33a07fa2db66376e6ee780d4a8b064dc5118cf34 from qemu
Due to a cut/paste error in the original implementation, the unsigned
vector saturating arithmetic was erroneously being calculated as signed
vector saturating arithmetic.
Fixes: 8ffafbcec2 ("tcg/i386: Implement vector saturating arithmetic")
Backports commit 3115584d39afe8cf2a84a40549029f53792abca5 from qemu
Currently, a jump to a label that is not defined anywhere will
be emitted not be relocated. This results in a jump to a random
jump target. With tcg debugging, print a diagnostic to the -d op
file and abort.
This could help debug or detect errors like
c2d9644e6d ("target/arm: Fix crash on conditional instruction in an IT block")
Backports commit bef16ab4e641636b4e85c3d863b4257ce0be4e6f from qemu
It's either "GNU *Library* General Public version 2" or "GNU Lesser
General Public version *2.1*", but there was no "version 2.0" of the
"Lesser" library. So assume that version 2.1 is meant here.
Backports commit fb0343d5b4dd4b9b9e96e563d913a3e0c709fe4e from qemu
The avx instruction set does not directly provide MO_64.
We can still implement 64-bit with comparison and vpblendvb.
Backports commit bc37faf4cb2baa77c44298c01558970b88d32808 from qemu
Only MO_8 and MO_16 are implemented, since that's all the
instruction set provides.
Backports commit 8ffafbcec275e61f6a1a17ac1d0bd918d5b23db3 from qemu
We handle many of these during integer expansion, and the
rest of them during integer optimization.
Backports commit 9a9eda78e4e56051485efb65e01748084f99ac3c from qemu
The 32 R5900 128-bit registers are split into two 64-bit halves:
the lower halves are the GPRs and the upper halves are accessible
by the R5900-specific multimedia instructions.
Backports commit a168a796e1c251787fcdf2d9ca1e9e69cb86ffcd from qemu
Free the argument register only after we have verified that the
temporary is not already in that register. This case is likely
now that we are back propagating the preferred register.
Backports commit 4250da10923347c9ee907f8d72bd93dfa5ee8742 from qemu
With these preferences, we can arrange for function call arguments to
be computed into the proper registers instead of requiring extra moves.
Backports commit 25f49c5f1508ddf081ce89fa6bbfd87a51eea37b from qemu
Use this to notice the opcodes that exit the TB, which implies
that local temps are really dead and need not be synced.
Previously we so marked the true end of the TB, but that was
immediately overwritten by the la_bb_end invoked by any
TCG_OPF_BB_END opcode, like exit_tb.
Backports commit ae36a246ed1a0e96c6c4f478f03d047dfa3a8898 from qemu
No need for a "tcg_" prefix for a static function; we already
have another "la_" prefix for indicating liveness analysis.
Pass in nb_globals and nb_temps, as we will already have them
in registers for other loops within the parent function.
Backports commit 2616c8082143373e794b62444bf81754f50dbf6b from qemu
Try harder to honor the output_pref. When we're forced to allocate
a second register for the input, it does not need to use the input
constraint; that will be honored by the register we allocate for the
output and a move is already required.
Backports commit d62816f2db439b2dd761c674f0256f21d9dd2ed0 from qemu
Allocate storage for, but do not yet fill in, per-opcode
preferences for the output operands. Pass it in to the
register allocation routines for output operands.
Backports commit 69e3706d2b473815e382552e729d12590339e0ac from qemu
This new argument will aid register allocation by indicating how
the temporary will be used in future. If the preference cannot
be satisfied, fall back to the constraints of the current insn.
Short circuit the preference when it cannot be satisfied or if
it does not further constrain the operation.
With an eye toward optimizing function call sequences, optimize
for the preferred_reg set containing a single register.
For the moment, all users pass 0 for preference.
Backports commit b016486e7baddb43cfc1e51909b05cde9cf82e0c from qemu
Delete trivially dead code that follows unconditional branches and
noreturn helpers. These can occur either via optimization or via
the structure of a target's translator following an exception.
Backports commit b4fc67c7afd2c338d6e7c73a7f428dfe05ae0603 from qemu
Previously, the low 4 bits were used for TCG_CALL_TYPE_MASK,
which was removed in 6a18ae2d2947532d5c26439548afa0481c4529f9.
Backports commit 3b50352b05eeafeb95cccd770f7aaba00bbdf6fe from qemu
Backporting 6fa2cef205a60b5c5c3b058f53852416b885c455 by Thomas Huth
started invoking assertions on clang. This means Unicorn is doing
something silly. This should be tracked down, but in the meantime,
restore behavior to allow tests to still be run.
For now, defined universally as true, since we previously required
backends to implement swapped memory operations. Future patches
may now remove that support where it is onerous.
Backports commit e1dcf3529d0797b25bb49a20e94b62eb93e7276a from qemu
Somehow we forgot these operations, once upon a time.
This will allow immediate stores to have their bswap
optimized away.
Backports commit 6498594c8eda83c5f5915afc34bd03396f8de6df from qemu
Based on the only current user, Sparc:
New code uses 2 constants that take 2 insns to load from constant pool,
plus 13. Old code used 6 constants that took 1 or 2 insns to create,
plus 21. The result is a new total of 17 vs an old total of 29.
Backports commit 9e821eab0ab708add35fa0446d880086e845ee3e from qemu
Based on the only current user, Sparc:
New code uses 1 constant that takes 2 insns to create, plus 8.
Old code used 2 constants that took 2 insns to create, plus 9.
The result is a new total of 10 vs an old total of 13.
Backports commit a686dc71d89b1d7934becd95c843aa1375cdb7e7 from qemu
We now have an invariant that all TCG_TYPE_I32 values are
zero-extended, which means that we do not need to extend
them again during qemu_ld/st, either explicitly via a separate
tcg_out_ext32u or implicitly via P_ADDR32.
Backports commit 4810d96f03be4d3820563e3c6bf13dfc0627f205 from qemu
This preserves the invariant that all TCG_TYPE_I32 values are
zero-extended in the 64-bit host register.
Backports commit 75478279a0c1eafc7b69d5382356da138f58f1bd from qemu
This helps preserve the invariant that all TCG_TYPE_I32 values
are stored zero-extended in the 64-bit host registers.
Backports commit 3dbc8c61de4e0d0a2afe0897cda7ab28cd37a164 from qemu
This helps preserve the invariant that all TCG_TYPE_I32 values
are stored zero-extended in the 64-bit host registers.
Backports commit 1d21d95b6101786d44d3b4a12400eb80a1ecc647 from qemu
This does require an extra two checks within the slow paths
to replace the assert that we're moving. Also add two checks
within existing functions that lacked any kind of assert for
out of range branch.
Backports commit 55dfd8fedceb1311d9cdded1a0f94b2da91a387d from qemu
The reloc_pc{14,24}_val routines retain their asserts.
Use these directly within the slow paths.
Backports commit d5132903518fadad579ef2de9e45fce98eefaa63 from qemu
This does require an extra two checks within the slow paths
to replace the assert that we're moving.
Backports commit 43fabd30e2f411e8d70ff347902a7c8ed308233e from qemu
This does require an extra two checks within the slow paths
to replace the assert that we're moving.
Backports commit 214bfe83d5a5af70bac2b8d0bd649b018c33c03b from qemu
This will move the assert for success from within (subroutines of)
patch_reloc into the callers. It will also let new code do something
different when a relocation is out of range.
For the moment, all backends are trivially converted to return true.
Backports commit 6ac1778676f4259c10b0629ccd9df319a5d1baeb from qemu
There is no longer a need for preserving branch offset operands,
as we no longer re-translate.
Backports commit 8c1b079279fadaee10dc39ca9a58c4c91c7a1854 from qemu
There is no longer a need for preserving branch offset operands,
as we no longer re-translate.
Backports commit 791645f0227c9d52ce5fe1ad6e1cda55a9bfe633 from qemu
There is no longer a need for preserving branch offset operands,
as we no longer re-translate.
Backports commit 3661612fc3e4b65be03482bf6bafd116101881e1 from qemu
There is no longer a need for preserving branch offset operands,
as we no longer re-translate.
Backports commit f9c7246faa279237200a2a53beacaa8100ea1900 from qemu
There are one use apiece for these. There is no longer a need for
preserving branch offset operands, as we no longer re-translate.
Backports commit 37ee93a974c49ab9edfcd1db0aad3838b0395b14 from qemu
There are one use apiece for these. There is no longer a need for
preserving branch offset operands, as we no longer re-translate.
Backports commit 733589b3382afcb0ae9f43e72e083a5ddd38abd5 from qemu
For x86_64, this can remove a REX prefix resulting in smaller code
when manipulating globals of type i32, as we move them between backing
store via cpu_env, aka TCG_AREG0.
Backports commit 5740d9f714835964873325d1210b26811252843f from qemu
Partially reverts ab20bdc1162. The 14-bit displacement that we
allowed to reach the constant pool is not always sufficient.
Retain the tb-relative addressing, as that is how most return
values from the tb are computed.
Backports commit f6823cbe3787aa47db62deede6683077e3da9a2c from qemu
Both GCC v4.8 and Clang v3.4 (our minimum versions) support
__builtin_unreachable(), so we can remove the version check here now.
Backports commit 6fa2cef205a60b5c5c3b058f53852416b885c455 from qemu
The tcg-op.h header was missing the usual guard against multiple
inclusion; add it.
(Spotted by lgtm.com's static analyzer.)
Backports commit a7ce790a029bd94eb320d8c69f38900f5233997e from qemu
Define and initialize the 16 MXU registers - 15 general computational
register, and 1 control register). There is also a zero register, but
it does not have any corresponding variable.
Backports commit eb5559f67dc8dc12335dd996877bb6daaea32eb2 from qemu.
GCC7+ will no longer advertise support for 16-byte __atomic operations
if only cmpxchg is supported, as for x86_64. Fortunately, x86_64 still
has support for __sync_compare_and_swap_16 and we can make use of that.
AArch64 does not have, nor ever has had such support, so open-code it.
Backports commit e6cd4bb59b8154fa00da611200beef7eb4e8ec56 from qemu
We forgot to initialize n in commit 15fa08f845 ("tcg: Dynamically
allocate TCGOps", 2017-12-29).
Backports commit c1f543b739086733024e31d74a52d9e41553f316 from qemu
Rather than test NOCHAIN before linking, do not emit the
goto_tb opcode at all. We already do this for goto_ptr.
Backports commit d7f425fdea991f052241c6479acd9feae834063b from qemu
The TCG backend uses LOWREGMASK to get the low 3 bits of register numbers.
This was defined as no-op for 32-bit x86, with the assumption that we have
eight registers anyway. This assumption is not true once we have xmm regs.
Since LOWREGMASK was a no-op, xmm register indidices were wrong in opcodes
and have overflown into other opcode fields, wreaking havoc.
To trigger these problems, you can try running the "movi d8, #0x0" AArch64
instruction on 32-bit x86. "vpxor %xmm0, %xmm0, %xmm0" should be generated,
but instead TCG generated "vpxor %xmm0, %xmm0, %xmm2".
Fixes: 770c2fc7bb ("Add vector operations")
Backports commit 93bf9a42733321fb632bcb9eafd049ef0e3d9417 from qemu
When host vector registers and operations were introduced, I failed
to mark the registers call clobbered as required by the ABI.
Fixes: 770c2fc7bb7
Backports commit 672189cd586ea38a2c1d8ab91eb1f9dcff5ceb05 from qemu
In AdvSIMD we can only do 32x32 integer multiples although SVE is
capable of larger 64 bit multiples. As a result we can end up
generating invalid opcodes. Fix this by only reprting we can emit
mul vector ops if the size is small enough.
Fixes a crash on:
sve-all-short-v8.3+sve@vq3/insn_mul_z_zi___INC.risu.bin
When running on AArch64 hardware.
Backports commit e65a5f227d77a5dbae7a7123c3ee915ee4bd80cf from qemu
Normally this is automatic in the size restrictions that are placed
on vector sizes coming from the implementation. However, for the
legitimate size tuple [oprsz=8, maxsz=32], we need to clear the final
24 bytes of the vector register. Without this check, do_dup selects
TCG_TYPE_V128 and clears only 16 bytes.
Backports commit 499748d7683198a765d17b4fdf6901ab9dca920c from qemu
The assembler in most versions of Mac OS X is pretty old and does not
support the xgetbv instruction. To go around this problem, the raw
encoding of the instruction is used instead.
Backports commit 1019242af11400252f6735ca71a35f81ac23a66d from qemu
Do the cast to uintptr_t within the helper, so that the compiler
can type check the pointer argument. We can also do some more
sanity checking of the index argument.
Backports commit 07ea28b41830f946de3841b0ac61a3413679feb9 from qemu
Given that this atomic operation will be used by both risc-v
and aarch64, let's not duplicate code across the two targets.
Backports commit 5507c2bf35aa6b4705939349184e71afd5e058b2 from qemu
These operations are re-invented by several targets so far.
Several supported hosts have insns for these, so place the
expanders out-of-line for a future introduction of tcg opcodes.
Backports commit b87fb8cd9f9a0ba599ff79e7bf03222da02e5724 from qemu
In 6001f7729e12 we partially attempt to address the branch
displacement overflow caused by 15fa08f845.
However, gcc/testsuite/gcc.target/aarch64/advsimd-intrinsics/vqtbX.c
is a testcase that contains a TB so large as to overflow anyway.
The limit here of 8000 ops produces a maximum output TB size of
24112 bytes on a ppc64le host with that test case. This is still
much less than the maximum forward branch distance of 32764 bytes.
Backports commit abebf92597186be2bc48d487235da28b1127860f from qemu
The VPUNPCKLD* instructions are all "non-destructive source",
indicated by "NDS" in the encoding string in the x86 ISA manual.
This means that they take two source operands, one of which is
encoded in the VEX.vvvv field. We were incorrectly treating them
as if they were destructive-source and passing 0 as the 'v'
argument of tcg_out_vex_modrm(). This meant we were always
using %xmm0 as one of the source operands, causing incorrect
results if the register allocator happened to want to use
something else. For instance the input AArch64 insn:
DUP v26.16b, w21
which becomes TCG IR ops:
dup_vec v128,e8,tmp2,x21
st_vec v128,e8,tmp2,env,$0xa40
was assembled to:
0x607c568c: c4 c1 7a 7e 86 e8 00 00 vmovq 0xe8(%r14), %xmm0
0x607c5694: 00
0x607c5695: c5 f9 60 c8 vpunpcklbw %xmm0, %xmm0, %xmm1
0x607c5699: c5 f9 61 c9 vpunpcklwd %xmm1, %xmm0, %xmm1
0x607c569d: c5 f9 70 c9 00 vpshufd $0, %xmm1, %xmm1
0x607c56a2: c4 c1 7a 7f 8e 40 0a 00 vmovdqu %xmm1, 0xa40(%r14)
0x607c56aa: 00
when the vpunpcklwd insn should be "%xmm1, %xmm1, %xmm1".
This resulted in our incorrectly setting the output vector to
q26=0000320000003200:0000320000003200
when given an input of x21 == 0000000002803200
rather than the expected all-zeroes.
Pass the correct source register number to tcg_out_vex_modrm()
for these insns.
Backports commit 7eb30ef0ba2eb59e7430d4848ae8d4bf4e50f768 from qemu
ppc64 uses a BC instruction to call the tcg_out_qemu_ld/st
slow path. BC instruction uses a relative address encoded
on 14 bits.
The slow path functions are added at the end of the generated
instructions buffer, in the reverse order of the callers.
So more we have slow path functions more the distance between
the caller (BC) and the function increases.
This patch changes the behavior to generate the functions in
the same order of the callers.
Backports commit 6001f7729e12dd1d810291e4cbf83cee8e07441d from qemu
Drop TCGV_PTR_TO_NAT and TCGV_NAT_TO_PTR internal macros.
Add tcg_temp_local_new_ptr, tcg_gen_brcondi_ptr, tcg_gen_ext_i32_ptr,
tcg_gen_trunc_i64_ptr, tcg_gen_extu_ptr_i64, tcg_gen_trunc_ptr_i32.
Use inlines instead of macros where possible.
Backports commit 5bfa803448638a45542441fd6b7cc1241403ea72 from qemu
In db432672, we allow wide inputs for operations such as add.
However, in 212be173 and 3774030a we didn't do the same for
compare and multiply.
Backports commit 9a938d86b04025ac605db0ea9819e5896bf576ec from qemu
