Commit graph

107 commits

Author SHA1 Message Date
Richard Henderson 6e38e5004f tcg: Use tcg_constant_{i32,i64,vec} with gvec expanders
Backports 88d4005b098427638d7551aa04ebde4fdd06835b
2021-03-04 13:05:25 -05:00
Richard Henderson 8edc9b76dd tcg: Introduce TYPE_CONST temporaries
These will hold a single constant for the duration of the TB.
They are hashed, so that each value has one temp across the TB.

Not used yet, this is all infrastructure.

Backports c0522136adf550c7a0ef7c0755c1f9d1560d2757
2021-03-03 21:29:40 -05:00
Richard Henderson a90476c897 tcg/i386: Adjust TCG_TARGET_HAS_MEMORY_BSWAP
Always true when movbe is available, otherwise leave
this to generic code.

Backports d2ef1b83a7a2047e0e36d7b62b3a5d151ab958f5
2021-03-03 19:40:00 -05:00
Peter Maydell d350644817 target/arm: AArch32 VCVT fixed-point to float is always round-to-nearest
For AArch32, unlike the VCVT of integer to float, which honours the
rounding mode specified by the FPSCR, VCVT of fixed-point to float is
always round-to-nearest. (AArch64 fixed-point-to-float conversions
always honour the FPCR rounding mode.)

Implement this by providing _round_to_nearest versions of the
relevant helpers which set the rounding mode temporarily when making
the call to the underlying softfloat function.

We only need to change the VFP VCVT instructions, because the
standard- FPSCR value used by the Neon VCVT is always set to
round-to-nearest, so we don't need to do the extra work of saving
and restoring the rounding mode.

Backports commit 61db12d9f9eb36761edba4d9a414cd8dd34c512b
2021-03-01 20:04:31 -05:00
Peter Maydell 8c6affbca4 target/arm/vec_helper: Add gvec fp indexed multiply-and-add operations
Add gvec helpers for doing Neon-style indexed non-fused fp
multiply-and-accumulate operations.

Backports commit c50d8d144098a8261233ca31b47e3bc487e112fe
2021-03-01 17:52:31 -05:00
Peter Maydell 681218b4ab target/arm: Implement fp16 for Neon VRINTX
Convert the Neon VRINTX insn to use gvec, and use this to implement
fp16 support for it.

Backports 23afcdd2511f2a3dc05bed650d27bd25cf9b2a3c
2021-03-01 17:47:25 -05:00
Peter Maydell 53aba9d900 target/arm: Implement fp16 for Neon VRINT-with-specified-rounding-mode
Convert the Neon VRINT-with-specified-rounding-mode insns to gvec,
and use this to implement the fp16 versions.

Backports 18725916b1438b54d6d6533980833d2251a20b7c
2021-03-01 17:44:49 -05:00
Peter Maydell eb4054d04f target/arm: Implement fp16 for Neon VCVT with rounding modes
Convert the Neon VCVT with-specified-rounding-mode instructions
to gvec, and use this to implement fp16 support for them.

Backports ca88a6efdf4ce96b646a896059f9bd324c2cebc4
2021-03-01 17:40:36 -05:00
Peter Maydell 56fe927d40 target/arm: Implement fp16 for Neon VCVT fixed-point
Implement fp16 for the Neon VCVT insns which convert between
float and fixed-point.

Backports 24018cf3990b692b51e50183c5fbd98d17b3fa40
2021-03-01 17:36:43 -05:00
Peter Maydell 948b01ad01 target/arm: Convert Neon VCVT fixed-point to gvec
Convert the Neon VCVT float<->fixed-point insns to a
gvec style, in preparation for adding fp16 support.

Backports 7b959c5890deb9a6d71bc6800006a0eae0a84c60
2021-03-01 17:33:20 -05:00
Peter Maydell c324c6817e target/arm: Implement fp16 for Neon float-integer VCVT
Convert the Neon float-integer VCVT insns to gvec, and use this
to implement fp16 support for them.

Note that unlike the VFP int<->fp16 VCVT insns we converted
earlier and which convert to/from a 32-bit integer, these
Neon insns convert to/from 16-bit integers. So we can use
the existing vfp conversion helpers for the f32<->u32/i32
case but need to provide our own for f16<->u16/i16.

Backports 7782a9afec81d1efe23572135c1ed777691ccde5
2021-03-01 17:29:02 -05:00
Peter Maydell 82f4a7e135 target/arm: Implement fp16 for Neon pairwise fp ops
Convert the Neon pairwise fp ops to use a single gvic-style
helper to do the full operation instead of one helper call
for each 32-bit part. This allows us to use the same
framework to implement the fp16.

Backports 1dc587ee9bfe804406eb3e0bacf47a80644d8abc
2021-03-01 17:25:19 -05:00
Peter Maydell b08ea84374 target/arm: Implement fp16 for Neon VRSQRTS
Convert the Neon VRSQRTS insn to using a gvec helper,
and use this to implement the fp16 case.

As with VRECPS, we adjust the phrasing of the new implementation
slightly so that the fp32 version parallels the fp16 one.

Backports 40fde72dda2da8d55b820fa6c5efd85814be2023
2021-03-01 17:20:22 -05:00
Peter Maydell f4ebbba9fd target/arm: Implement fp16 for Neon VRECPS
Convert the Neon VRECPS insn to using a gvec helper, and
use this to implement the fp16 case.

The phrasing of the new float32_recps_nf() is slightly different from
the old recps_f32() so that it parallels the f16 version; for f16 we
can't assume that flush-to-zero is always enabled.

Backports ac8c62c4e5a3f24e6d47f52ec1bfb20994caefa5
2021-03-01 17:09:16 -05:00
Peter Maydell 5776c594e4 target/arm: Implement fp16 for Neon fp compare-vs-0
Convert the neon floating-point vector compare-vs-0 insns VCEQ0,
VCGT0, VCLE0, VCGE0 and VCLT0 to use a gvec helper, and use this to
implement the fp16 case.

Backport 635187aaa92f21ab001e2868e803b3c5460261ca
2021-03-01 17:05:03 -05:00
Peter Maydell 8de258c3cb target/arm: Implement fp16 for Neon VFMA, VMFS
Convert the neon floating-point vector operations VFMA and VFMS
to use a gvec helper, and use this to implement the fp16 case.

This is the last use of do_3same_fp() so we can now delete
that function.

Backports commit cf722d75b329ef3f86b869e7e68cbfb1607b3bde
2021-03-01 17:00:49 -05:00
Peter Maydell 587c3549b7 target/arm: Implement fp16 for Neon VMLA, VMLS operations
Convert the Neon floating-point VMLA and VMLS insns over to using a
gvec helper, and use this to implement the fp16 case.

Backports e5adc70665ecaf4009c2fb8d66775ea718a85abd
2021-03-01 16:57:20 -05:00
Peter Maydell 0068d12355 target/arm: Implement fp16 for Neon VMAXNM, VMINNM
Convert the Neon floating point VMAXNM and VMINNM insns to
using a gvec helper and use this to implement the fp16 case.

Backports e22705bb941d82d6c2a09e8b2031084326902be3
2021-03-01 16:53:57 -05:00
Peter Maydell 465cfb54c4 target/arm: Implement fp16 for Neon VMAX, VMIN
Convert the Neon float-point VMAX and VMIN insns over to using
a gvec helper, and use this to implement the fp16 case.

Backport e43268c54b6cbcb197d179409df7126e81f8cd52
2021-03-01 16:50:23 -05:00
Peter Maydell 6dd4a8e93f target/arm: Implement fp16 for VACGE, VACGT
Convert the neon floating-point vector absolute comparison ops
VACGE and VACGT over to using a gvec hepler and use this to
implement the fp16 case.

Backports bb2741da186ebaebc7d5189372be4401e1ff9972
2021-03-01 16:47:44 -05:00
Peter Maydell 4eb39f1b2f target/arm: Implement fp16 for VCEQ, VCGE, VCGT comparisons
Convert the Neon floating-point vector comparison ops VCEQ,
VCGE and VCGT over to using a gvec helper and use this to
implement the fp16 case.

(We put the float16_ceq() etc functions above the DO_2OP()
macro definition because later when we convert the
compare-against-zero instructions we'll want their
definitions to be visible at that point in the source file.)

Backports ad505db233b89b7fd4b5a98b6f0e8ac8d05b11db
2021-03-01 16:44:34 -05:00
Peter Maydell 4850377f01 target/arm: Implement FP16 for Neon VADD, VSUB, VABD, VMUL
Implement FP16 support for the Neon insns which use the DO_3S_FP_GVEC
macro: VADD, VSUB, VABD, VMUL.

For VABD this requires us to implement a new gvec_fabd_h helper
using the machinery we have already for the other helpers.

Backport e4a6d4a69e239becfd83bdcd996476e7b8e1138d
2021-03-01 16:31:54 -05:00
Peter Maydell 90aa9647e0 target/arm: Implement VFP fp16 VRINT*
Implement the fp16 version of the VFP VRINT* insns.

Backports 0a6f4b4cb338665b81ad824d9a6868932461b7f7
2021-03-01 16:15:21 -05:00
Peter Maydell 9c5b6f06a2 target/arm: Use macros instead of open-coding fp16 conversion helpers
Now the VFP_CONV_FIX macros can handle fp16's distinction between the
width of the operation and the width of the type used to pass operands,
use the macros rather than the open-coded functions.

This creates an extra six helper functions, all of which we are going
to need for the AArch32 VFP fp16 instructions.

Backports commit 414ba270c4fb758d987adf37ae9bfe531715c604
2021-02-28 05:08:44 -05:00
Peter Maydell 5d98e14545 target/arm: Implement VFP fp16 VCMP
Implement fp16 version of VCMP.

Backports 1b88b054c5b201e8581114d29527c6a5a7e088c9
2021-02-28 04:56:24 -05:00
Peter Maydell 2d9abf7c0b target/arm: Implement VFP fp16 for VABS, VNEG, VSQRT
Implement VFP fp16 for VABS, VNEG and VSQRT. This is all
the fp16 insns that use the DO_VFP_2OP macro, because there
is no fp16 version of VMOV_reg.

Notes:
* the gen_helper_vfp_negh already exists as we needed to create
it for the fp16 multiply-add insns
* as usual we need to use the f16 version of the fp_status;
this is only relevant for VSQRT

Backports ce2d65a5d191380756cdac7a1fd1ba76bd1621cf
2021-02-28 04:48:28 -05:00
Peter Maydell 6ac2c597ab target/arm: Implement VFP fp16 for fused-multiply-add
Implement VFP fp16 support for fused multiply-add insns
VFNMA, VFNMS, VFMA, VFMS.

Backports 9886fe2834b064a3cf0675a4659942ed547aed42
2021-02-28 04:39:21 -05:00
Peter Maydell a42ecfe203 target/arm: Implement VFP fp16 VMLA, VMLS, VNMLS, VNMLA, VNMUL
Implement fp16 versions of the VFP VMLA, VMLS, VNMLS, VNMLA, VNMUL
instructions. (These are all the remaining ones which we implement
via do_vfp_3op_[hsd]p().)

Backports commit e7cb0ded52c6d7b86585b09935fe7caeb9e38b69
2021-02-28 04:29:37 -05:00
Peter Maydell eae621098d target/arm: Implement VFP fp16 for VFP_BINOP operations
Implmeent VFP fp16 support for simple binary-operator VFP insns VADD,
VSUB, VMUL, VDIV, VMINNM and VMAXNM:

* make the VFP_BINOP() macro generate float16 helpers as well as
float32 and float64
* implement a do_vfp_3op_hp() function similar to the existing
do_vfp_3op_sp()
* add decode for the half-precision insn patterns

Note that the VFP_BINOP macro use creates a couple of unused helper
functions vfp_maxh and vfp_minh, but they're small so it's not worth
splitting the BINOP operations into "needs halfprec" and "no
halfprec" groups.

Backports commit 120a0eb3ea23a5b06fae2f3daebd46a4035864cf
2021-02-28 04:24:39 -05:00
LIU Zhiwei d26cd63ad6 softfloat: Define misc operations for bfloat16
Backports 5ebf5f4be66c378fd5f3dee85f54dd4942171d57
2021-02-27 16:41:46 -05:00
LIU Zhiwei d8168a8142 softfloat: Define convert operations for bfloat16
Backports 34f0c0a98a5f3bb6706088c0384f937f7a294d3e
2021-02-27 16:37:11 -05:00
LIU Zhiwei b0be0d28cc softfloat: Define operations for bfloat16
Backports 8282310d8535cc2a8431c516e907da79f92df6eb
2021-02-26 15:20:30 -05:00
Frank Chang d97454eb63 softfloat: Add fp16 and uint8/int8 conversion functions
Backports 0d93d8ec632154dea2627a9e989972ee09721187
2021-02-26 15:11:57 -05:00
Lioncash f5a21abc0b target/arm: Convert sq{, r}dmulh to gvec for aa64 advsimd 2021-02-26 15:01:44 -05:00
LIU Zhiwei 9b7f4b72fc target/riscv: vector single-width integer multiply instructions 2021-02-26 10:46:26 -05:00
LIU Zhiwei ab81642440 target/riscv: vector integer min/max instructions
558fa7797c919c4f21ac10980f3ed28160d6d3cb
2021-02-26 10:43:13 -05:00
LIU Zhiwei 965af9986a target/riscv: vector integer comparison instructions
1366fc79be04fa56a0e3f078ba4f26c27ac67e89
2021-02-26 10:40:33 -05:00
LIU Zhiwei 56c0e253c2 target/riscv: vector bitwise logical instructions
Backports d3842924cf93d104f691c5ea9090d6700ccef281
2021-02-26 10:30:33 -05:00
LIU Zhiwei 05153c6d7c target/riscv: vector integer add-with-carry / subtract-with-borrow instructions
3a6f8f68ad2f4a22d9ae8287f336b5dcc80b6448
2021-02-26 10:19:48 -05:00
LIU Zhiwei b9814de4c3 target/riscv: vector widening integer add and subtract
Backports 8fcdf77630290591a6068c2d82ca2935338c3b0c
2021-02-26 10:05:43 -05:00
LIU Zhiwei f564388e89 target/riscv: vector single-width integer add and subtract
Backports 43740e3a3b3bb66456103684e622ba4e9baae297
2021-02-26 09:58:31 -05:00
LIU Zhiwei 7d0d7338c2 target/riscv: add vector amo operations
Vector AMOs operate as if aq and rl bits were zero on each element
with regard to ordering relative to other instructions in the same hart.
Vector AMOs provide no ordering guarantee between element operations
in the same vector AMO instruction

Backports 268fcca66bde62257960ec8d859de374315a5e3d
2021-02-26 09:47:32 -05:00
LIU Zhiwei 152934bade target/riscv: add fault-only-first unit stride load
The unit-stride fault-only-fault load instructions are used to
vectorize loops with data-dependent exit conditions(while loops).
These instructions execute as a regular load except that they
will only take a trap on element 0.

Backports commit 022b4ecf775ffeff522eaea4f0d94edcfe00a0a9 from qemu
2021-02-26 09:28:19 -05:00
LIU Zhiwei 887c29bc79 target/riscv: add vector index load and store instructions
Vector indexed operations add the contents of each element of the
vector offset operand specified by vs2 to the base effective address
to give the effective address of each element.

Backports f732560e3551c0823cee52efba993fbb8f689a36
2021-02-26 03:00:45 -05:00
LIU Zhiwei c7a17d04a2 target/riscv: add vector stride load and store instructions
Vector strided operations access the first memory element at the base address,
and then access subsequent elements at address increments given by the byte
offset contained in the x register specified by rs2.

Vector unit-stride operations access elements stored contiguously in memory
starting from the base effective address. It can been seen as a special
case of strided operations.

Backports 751538d5da557e5c10e5045c2d27639580ea54a7
2021-02-26 02:55:14 -05:00
LIU Zhiwei 9db3b70869 target/riscv: add vector configure instruction
vsetvl and vsetvli are two configure instructions for vl, vtype. TB flags
should update after configure instructions. The (ill, lmul, sew ) of vtype
and the bit of (VSTART == 0 && VL == VLMAX) will be placed within tb_flags.

Backports 2b7168fc43fb270fb89e1dddc17ef54714712f3a from qemu
2021-02-26 02:37:59 -05:00
Richard Henderson 94b0876f15 target/arm: Add sve infrastructure for page lookup
For contiguous predicated memory operations, we want to
minimize the number of tlb lookups performed. We have
open-coded this for sve_ld1_r, but for correctness with
MTE we will need this for all of the memory operations.

Create a structure that holds the bounds of active elements,
and metadata for two pages. Add routines to find those
active elements, lookup the pages, and run watchpoints
for those pages.

Temporarily mark the functions unused to avoid Werror.

Backports commit b4cd95d2f4c7197b844f51b29871d888063ea3e7 from qemu
2021-02-25 20:28:23 -05:00
Richard Henderson 2e03f74a53 target/arm: Use cpu_*_data_ra for sve_ldst_tlb_fn
Use the "normal" memory access functions, rather than the
softmmu internal helper functions directly.

Since fb901c9, cpu_mem_index is now a simple extract
from env->hflags and not a large computation.  Which means
that it's now more work to pass around this value than it
is to recompute it.

This only adjusts the primitives, and does not clean up
all of the uses within sve_helper.c.
2021-02-25 20:16:38 -05:00
Richard Henderson 5b3ddcf2e2 target/arm: Simplify DC_ZVA
Now that we know that the operation is on a single page,
we need not loop over pages while probing.

Backports commit e26d0d226892f67435cadcce86df0ddfb9943174 from qemu
2021-02-25 15:55:46 -05:00
Joseph Myers b08d204a37 softfloat: merge floatx80_mod and floatx80_rem
The m68k-specific softfloat code includes a function floatx80_mod that
is extremely similar to floatx80_rem, but computing the remainder
based on truncating the quotient toward zero rather than rounding it
to nearest integer. This is also useful for emulating the x87 fprem
and fprem1 instructions. Change the floatx80_rem implementation into
floatx80_modrem that can perform either operation, with both
floatx80_rem and floatx80_mod as thin wrappers available for all
targets.

There does not appear to be any use for the _mod operation for other
floating-point formats in QEMU (the only other architectures using
_rem at all are linux-user/arm/nwfpe, for FPA emulation, and openrisc,
for instructions that have been removed in the latest version of the
architecture), so no change is made to the code for other formats.

Backports commit 6b8b0136ab3018e4b552b485f808bf66bcf19ead from qemu
2021-02-25 13:34:05 -05:00