Commit graph

603 commits

Author SHA1 Message Date
Richard Henderson e3356f9bad tcg: Drop union from TCGArgConstraint
The union is unused; let "regs" appear in the main structure
without the "u.regs" wrapping.

Backports 9be0d08019465b38e2f1a605960961a491430c21
2021-03-01 19:29:19 -05:00
Richard Henderson 1551f6be9d tcg: Adjust simd_desc size encoding
With larger vector sizes, it turns out oprsz == maxsz, and we only
need to represent mismatch for oprsz <= 32. We do, however, need
to represent larger oprsz and do so without reducing SIMD_DATA_BITS.

Reduce the size of the oprsz field and increase the maxsz field.
Steal the oprsz value of 24 to indicate equality with maxsz.

Backports e2e7168a214b0ed98dc357bba96816486a289762
2021-03-01 19:23:37 -05:00
Richard Henderson cd79d2a915 tcg: Implement 256-bit dup for tcg_gen_gvec_dup_mem
We already support duplication of 128-bit blocks. This extends
that support to 256-bit blocks. This will be needed by SVE2.

Backports commit fe4b0b5bfa96c38ad1cad0689a86cca9f307e353
2021-03-01 18:10:07 -05:00
Richard Henderson b478ce5052 tcg: Eliminate one store for in-place 128-bit dup_mem
Do not store back to the exact memory from which we just loaded.

Backports 6a17646176e011ddc463a2870a64c7aaccfe9c50
2021-03-01 18:06:17 -05:00
Stephen Long c9dc750058 tcg: Fix tcg gen for vectorized absolute value
The fallback inline expansion for vectorized absolute value,
when the host doesn't support such an insn was flawed.

E.g. when a vector of bytes has all elements negative, mask
will be 0xffff_ffff_ffff_ffff. Subtracting mask only adds 1
to the low element instead of all elements becase -mask is 1
and not 0x0101_0101_0101_0101.

Backports commit e7e8f33fb603c3bfa0479d7d924f2ad676a84317
2021-03-01 18:04:46 -05:00
Peter Maydell e0000d1700 target/arm/translate.c: Delete/amend incorrect comments
In arm_tr_init_disas_context() we have a FIXME comment that suggests
"cpu_M0 can probably be the same as cpu_V0". This isn't in fact
possible: cpu_V0 is used as a temporary inside gen_iwmmxt_shift(),
and that function is called in various places where cpu_M0 contains a
live value (i.e. between gen_op_iwmmxt_movq_M0_wRn() and
gen_op_iwmmxt_movq_wRn_M0() calls). Remove the comment.

We also have a comment on the declarations of cpu_V0/V1/M0 which
claims they're "for efficiency". This isn't true with modern TCG, so
replace this comment with one which notes that they're only used with
the iwmmxt decode

Backports 8b4c9a50dc9531a729ae4b5941d287ad0422db48
2021-02-26 11:23:52 -05:00
LIU Zhiwei 0968caa249 target/riscv: add vector extension field in CPURISCVState
The 32 vector registers will be viewed as a continuous memory block.
It avoids the convension between element index and (regno, offset).
Thus elements can be directly accessed by offset from the first vector
base address.

Backports ad9e5aa2ae8032f19a8293b6b8f4661c06167bf0 from qemu
2021-02-26 02:17:49 -05:00
Richard Henderson 55369d710c tcg: Save/restore vecop_list around minmax fallback
Forgetting this asserts when tcg_gen_cmp_vec is called from
within tcg_gen_cmpsel_vec.

Fixes: 72b4c792c7a

Backports commit 69c918d2ef319ac63cd759c527debc2a2bdf3a0c from qemu
2021-02-25 23:33:24 -05:00
Richard Henderson 65d5288563 tcg: Fix do_nonatomic_op_* vs signed operations
The smin/smax/umin/umax operations require the operands to be
properly sign extended. Do not drop the MO_SIGN bit from the
load, and additionally extend the val input.

Backports commit 852f933e482518797f7785a2e017a215b88df815 from qemu
2021-02-25 23:10:40 -05:00
Richard Henderson 0e68fa345e tcg: Improve move ops in liveness_pass_2
If the output of the move is dead, then the last use is in
the store. If we propagate the input to the store, then we
can remove the move opcode entirely.

Backports commit 61f15c487fc2aea14f6b0e52c459ae8b7d252a65 from qemu
2020-06-14 22:13:04 -04:00
Richard Henderson 6b91e9bae1 tcg/i386: Implement INDEX_op_rotl{i,s,v}_vec
For immediates, we must continue the special casing of 8-bit
elements. The other element sizes and shift types are trivially
implemented with shifts.

Backports commit 885b1706df6f0211a22e120fac910fb3abf3e733 from qemu
2020-06-14 22:09:24 -04:00
Richard Henderson cc3187b1e4 tcg: Implement gvec support for rotate by scalar
No host backend support yet, but the interfaces for rotls
are in place. Only implement left-rotate for now, as the
only known use of vector rotate by scalar is s390x, so any
right-rotate would be unused and untestable.

Backports commit 23850a74afb641102325b4b7f74071d929fc4594 from qemu
2020-06-14 22:00:50 -04:00
Richard Henderson 2aa9d13120 tcg: Remove expansion to shift by vector from do_shifts
We do not reflect this expansion in tcg_can_emit_vecop_list,
so it is unused and unusable. However, we actually perform
the same expansion in do_gvec_shifts, so it is also unneeded.

Backports commit 3d5bb2ea5cc9ed54f65a6929a6e6baa01cabd98b from qemu
2020-06-14 21:53:36 -04:00
Richard Henderson be78062fd8 tcg: Implement gvec support for rotate by vector
No host backend support yet, but the interfaces for rotlv
and rotrv are in place.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
v3: Drop the generic expansion from rot to shift; we can do better
for each backend, and then this code becomes unused.

Backports commit 5d0ceda902915e3f0e21c39d142c92c4e97c3ebb from qemu
2020-06-14 21:43:46 -04:00
Richard Henderson 5cce52a04b tcg: Implement gvec support for rotate by immediate
No host backend support yet, but the interfaces for rotli
are in place. Canonicalize immediate rotate to the left,
based on a survey of architectures, but provide both left
and right shift interfaces to the translators.

Backports commit b0f7e7444c03da17e41bf327c8aea590104a28ab from qemu
2020-06-14 21:26:58 -04:00
Richard Henderson 742301a7c1 tcg: Fix integral argument type to tcg_gen_rot[rl]i_i{32,64}
For the benefit of compatibility of function pointer types,
we have standardized on int32_t and int64_t as the integral
argument to tcg expanders.

We converted most of them in 474b2e8f0f7, but missed the rotates.

Backports commit 07dada0336a83002dfa8673a9220a88e13d9a45c from qemu
2020-05-07 10:41:01 -04:00
Richard Henderson 0bcd0ca93d tcg: Add load_dest parameter to GVecGen2
We have this same parameter for GVecGen2i, GVecGen3,
and GVecGen3i. This will make some SVE2 insns easier
to parameterize.

Backports commit ac09ae627e9a2c65c8a452b69c3dac33c29d0719 from qemu
2020-05-07 10:35:47 -04:00
Richard Henderson f02f71f38f tcg: Improve vector tail clearing
Better handling of non-power-of-2 tails as seen with Arm 8-byte
vector operations.

Backports commit f47db80cc073c0a7a22136c8296b5eca20c0e199 from qemu
2020-05-07 10:24:00 -04:00
Richard Henderson 549b0ec3c5 tcg: Add tcg_gen_gvec_dup_tl
For use when a target needs to pass a configure-specific
target_ulong value to duplicate.

Backports commit 0f039e3ad9131966d9fe509c231b756868b015e2 from qemu
2020-05-07 10:12:09 -04:00
Richard Henderson e65806c356 tcg: Remove tcg_gen_gvec_dup{8,16,32,64}i
These interfaces are now unused.

Backports commit 398f21412aeec158338963e3f71c9313bc126a71 form qemu
2020-05-07 10:11:00 -04:00
Richard Henderson 43a72b0540 tcg: Use tcg_gen_gvec_dup_imm in logical simplifications
Replace the outgoing interface.

Backports commit 03ddb6f315ca6d02dfdba0aecc43aa97c728c428 from qemu
2020-05-07 10:09:53 -04:00
Richard Henderson 07f622e57d tcg: Add tcg_gen_gvec_dup_imm
Add a version of tcg_gen_dup_* that takes both immediate and
a vector element size operand. This will replace the set of
tcg_gen_gvec_dup{8,16,32,64}i functions that encode the element
size within the function name.

Backports commit 44c94677febd15488f9190b11eaa4a08e8ac696b from qemu
2020-05-07 09:55:25 -04:00
lixinyu 7c32c5b0a4 tcg/mips: mips sync* encode error
OPC_SYNC_WMB, OPC_SYNC_MB, OPC_SYNC_ACQUIRE, OPC_SYNC_RELEASE and
OPC_SYNC_RMB have wrong encode. According to the mips manual,
their encode should be 'OPC_SYNC | 0x?? << 6' rather than
'OPC_SYNC | 0x?? << 5'. Wrong encode can lead illegal instruction
errors. These instructions often appear with multi-threaded
simulation.

Fixes: 6f0b99104a3 ("tcg/mips: Add support for fence")

Backports commit a4e57084c16d5b0eff3651693fba04f26b30b551 from qemu
2020-04-30 07:24:57 -04:00
Richard Henderson 299ba4e867 tcg/i386: Fix INDEX_op_dup2_vec
We were only constructing the 64-bit element, and not
replicating the 64-bit element across the rest of the vector.

Backports commit e20cb81d9c5a3d0f9c08f3642728a210a1c162c9 from qemu
2020-04-30 07:15:08 -04:00
Richard Henderson b358f771f6 tcg/i386: Bound shift count expanding sari_vec
A given RISU testcase for SVE can produce

tcg-op-vec.c:511: do_shifti: Assertion `i >= 0 && i < (8 << vece)' failed.

because expand_vec_sari gave a shift count of 32 to a MO_32
vector shift.

In 44f1441dbe1, we changed from direct expansion of vector opcodes
to re-use of the tcg expanders. So while the comment correctly notes
that the hw will handle such a shift count, we now have to take our
own sanity checks into account. Which is easy in this particular case.

Fixes: 44f1441dbe1

Backports commit 312b426fea4d6dd322d7472c80010a8ba7a166d2 from qemu
2020-04-30 06:26:42 -04:00
Richard Henderson 12b4e01d9c tcg: Add tcg_gen_gvec_5_ptr
Extend the vector generator infrastructure to handle
5 vector arguments.

Backports commit 2445971604c1cfd3ec484457159f4ac300fb04d2 from qemu
2020-03-21 16:54:01 -04:00
Tony Nguyen f75368cd0f
tcg: TCGMemOp is now accelerator independent MemOp
Preparation for collapsing the two byte swaps, adjust_endianness and
handle_bswap, along the I/O path.

Target dependant attributes are conditionalized upon NEED_CPU_H.

Backports commit 14776ab5a12972ea439c7fb2203a4c15a09094b4 from qemu
2019-11-28 03:01:12 -05:00
Emilio G. Cota ca15f620b0
tcg/README: fix typo s/afterwise/afterwards/
Afterwise is "wise after the fact", as in "hindsight".
Here we meant "afterwards" (as in "subsequently"). Fix it.
2019-11-28 02:37:51 -05:00
tony.nguyen@bt.com b4c2c94602
configure: Define target access alignment in configure
This patch moves the define of target access alignment earlier from
target/foo/cpu.h to configure.

Suggested in Richard Henderson's reply to "[PATCH 1/4] tcg: TCGMemOp is now
accelerator independent MemOp"

Backports commit 52bf9771fdfce98e98cea36a17a18915be6f6b7f from qemu
2019-11-18 21:41:35 -05:00
Richard Henderson d291a311ee
tcg/aarch64: Fix output of extract2 opcodes
This patch fixes two problems:
(1) The inputs to the EXTR insn were reversed,
(2) The input constraints use rZ, which means that we need to use
the REG0 macro in order to supply XZR for a constant 0 input.

Fixes: 464c2969d5d

Backports commit 1789d4274b851fb8fdf4a947ce5474c63e813d0d from qemu
2019-08-08 19:25:37 -04:00
Richard Henderson b2d75f4955
tcg: Fix constant folding of INDEX_op_extract2_i32
On a 64-bit host, discard any replications of the 32-bit
sign bit when performing the shift and merge.

Backports commit 80f4d7c3ae216c191fb403e149bcba88d6aa40bb from qemu
2019-08-08 19:25:01 -04:00
Richard Henderson 955661ad7b
tcg: Fix expansion of INDEX_op_not_vec
This operation can always be emitted, even if we need to
fall back to xor. Adjust the assertions to match.

Backports commit 11978f6f58f1d3d66429f7ff897524f693d823ce from qemu
2019-08-08 19:23:03 -04:00
Richard Henderson d7ea41c3a3
cpu: Move icount_decr to CPUNegativeOffsetState
Amusingly, we had already ignored the comment to keep this value
at the end of CPUState. This restores the minimum negative offset
from TCG_AREG0 for code generation.

For the couple of uses within qom/cpu.c, without NEED_CPU_H, add
a pointer from the CPUState object to the IcountDecr object within
CPUNegativeOffsetState.

Backports commit 5e1401969b25f676fee6b1c564441759cf967a43 from qemu
2019-06-13 15:34:28 -04:00
Richard Henderson ac176ccb38
cpu: Move ENV_OFFSET to exec/gen-icount.h
Now that we have ArchCPU, we can define this generically,
in the one place that needs it.

Backports commit 677c4d69ac21961e76a386f9bfc892a44923acc0 from qemu
2019-06-12 12:20:21 -04:00
Markus Armbruster 5e5197b136
Supply missing header guards
Backports applicable parts of commit
f91005e195e7e1485e60cb121731589960f1a3c9 from qemu
2019-06-12 10:59:10 -04:00
Richard Henderson 67f0af4282
tcg/aarch64: Allow immediates for vector ORR and BIC
The allows immediates to be used for ORR and BIC,
as well as the trivial inversions, ORC and AND.

Backports commit 9e27f58b9902834dffc0d66d9eb62f78d9c2a632 from qemu
2019-05-24 18:47:07 -04:00
Richard Henderson 5ecfba4fe6
tcg/aarch64: Build vector immediates with two insns
Use MOVI+ORR or MVNI+BIC in order to build some vector constants,
as opposed to dropping them to the constant pool. This includes
all 16-bit constants and a similar set of 32-bit constants.

Backports commit 02f3a5b4744885258758d07ebe09cf965de78bcf from qemu
2019-05-24 18:43:54 -04:00
Richard Henderson 06058ef648
tcg/aarch64: Use MVNI in tcg_out_dupi_vec
The compliment of a subset of immediates can be computed
with a single instruction.

Backports commit 7e308e003e5b6ddd3130e09711e1d33693230696 from qemu
2019-05-24 18:42:40 -04:00
Richard Henderson c18ec586dc
tcg/aarch64: Split up is_fimm
There are several sub-classes of vector immediate, and only MOVI
can use them all. This will enable usage of MVNI and ORRI, which
use progressively fewer sub-classes.

This patch adds no new functionality, merely splits the function
and moves part of the logic into tcg_out_dupi_vec.

Backports commit 984fdcee342473dfe797897758929dad654693c8 from qemu
2019-05-24 18:41:37 -04:00
Richard Henderson 0ea4c05dc3
tcg/aarch64: Support vector bitwise select value
The instruction set has 3 insns that perform the same operation,
only varying in which operand must overlap the destination. We
can represent the operation without overlap and choose based on
the operands seen.

Backports commit a9e434a5dc16f71ee156428619fc3c3765b68f26 from qemu
2019-05-24 18:38:37 -04:00
Richard Henderson c79510378f
tcg/i386: Use umin/umax in expanding unsigned compare
Using umin(a, b) == a as an expansion for TCG_COND_LEU is a
better alternative to (a - INT_MIN) <= (b - INT_MIN).

Backports commit ebcfb91abed8c0fb180a968b9004419c208dcc02 from qemu
2019-05-24 18:36:32 -04:00
Richard Henderson ffdbc1a233
tcg/i386: Remove expansion for missing minmax
This is now handled by code within tcg-op-vec.c.

Backports commit 3ec3538a45f2fead475b0cca6945092c87927b4f from qemu
2019-05-24 18:34:44 -04:00
Richard Henderson 68cb096196
tcg/i386: Support vector comparison select value
We already had backend support for this feature. Expand the new
cmpsel opcode using vpblendb. The combination allows us to avoid
an extra NOT for some comparison codes.

Backports commit 904c5e19672778cc3349f4975437cfdf3371abb6 from qemu
2019-05-24 18:33:16 -04:00
Richard Henderson a868533297
tcg: Add TCG_OPF_NOT_PRESENT if TCG_TARGET_HAS_foo is negative
If INDEX_op_foo is always expanded by tcg_expand_vec_op, then
there may be no reasonable set of constraints to return from
tcg_target_op_def for that opcode.

Let TCG_TARGET_HAS_foo be specified as -1 in that case. Thus a
boolean test for TCG_TARGET_HAS_foo is true, but we will not
assert within process_op_defs when no constraints are specified.

Compare this with tcg_can_emit_vec_op, which already uses this
tri-state indication.

Backports commit 25c012b4009256505be3430480954a0233de343e from qemu
2019-05-24 18:28:11 -04:00
Richard Henderson 568da655c6
tcg: Expand vector minmax using cmp+cmpsel
Provide a generic fallback for the min/max operations.

Backports commit 72b4c792c7a576d9246207a8e9a940ed9e191722 from qemu
2019-05-24 18:26:53 -04:00
Richard Henderson 56d35e80aa
tcg: Introduce do_op3_nofail for vector expansion
This makes do_op3 match do_op2 in allowing for failure,
and thus fall back expansions.

Backports commit 17f79944ebeace8bf43047a33b7775ba5ed9070e from qemu
2019-05-24 18:24:44 -04:00
Richard Henderson 2ea6dfbd63
tcg: Add support for vector compare select
Perform a per-element conditional move. This combination operation is
easier to implement on some host vector units than plain cmp+bitsel.
Omit the usual gvec interface, as this is intended to be used by
target-specific gvec expansion call-backs.

Backports commit f75da2988eb2457fa23d006d573220c5c680ec4e from qemu
2019-05-24 18:21:13 -04:00
Richard Henderson ca58be9cb4
tcg: Add support for vector bitwise select
This operation performs d = (b & a) | (c & ~a), and is present
on a majority of host vector units. Include gvec expanders.

Backports commit 38dc12947ec9106237f9cdbd428792c985cd86ae from qemu
2019-05-24 18:15:10 -04:00
Richard Henderson fa363c3d6d
tcg: Fix missing checks and clears in tcg_gen_gvec_dup_mem
The paths through tcg_gen_dup_mem_vec and through MO_128 were
missing the check_size_align. The path through MO_128 was also
missing the expand_clr. This last was not visible because the
only user is ARM SVE, which would set oprsz == maxsz, and not
require the clear.

Fix by adding the check_size_align and using do_dup directly
instead of duplicating the check in tcg_gen_gvec_dup_{i32,i64}.

Backports commit 532ba368a13712724137228b5e7e9435994d25e1 from qemu
2019-05-24 18:07:28 -04:00
Richard Henderson 60cfe541b2
tcg/i386: Fix dupi/dupm for avx1 and 32-bit hosts
The VBROADCASTSD instruction only allows %ymm registers as destination.
Rather than forcing VEX.L and writing to the entire 256-bit register,
revert to using MOVDDUP with an %xmm register. This is sufficient for
an avx1 host since we do not support TCG_TYPE_V256 for that case.

Also fix the 32-bit avx2, which should have used VPBROADCASTW.

Fixes: 1e262b49b533

Backports commit 7b60ef3264e9627ac6efb34e9a6130647e9b55c0 from qemu
2019-05-24 18:04:08 -04:00