Commit graph

7315 commits

Author SHA1 Message Date
Richard Henderson f7e831a7e4 target/arm: Ignore HCR_EL2.ATA when {E2H,TGE} != 11
Unlike many other bits in HCR_EL2, the description for this
bit does not contain the phrase "if ... this field behaves
as 0 for all purposes other than", so do not squash the bit
in arm_hcr_el2_eff.

Instead, replicate the E2H+TGE test in the two places that
require it.

Backports 4301acd7d7d455792ea873ced75c0b5d653618b1
2021-03-01 20:12:36 -05:00
Richard Henderson 4f00eacb11 target/arm: Fix reported EL for mte_check_fail
The reporting in AArch64.TagCheckFail only depends on PSTATE.EL,
and not the AccType of the operation. There are two guest
visible problems that affect LDTR and STTR because of this:

(1) Selecting TCF0 vs TCF1 to decide on reporting,
(2) Report "data abort same el" not "data abort lower el".

Backports 50244cc76abcac3296cff3d84826f5ff71808c80
2021-03-01 20:10:44 -05:00
Richard Henderson 511636a3f4 target/arm: Remove redundant mmu_idx lookup
We already have the full ARMMMUIdx as computed from the
function parameter.

For the purpose of regime_has_2_ranges, we can ignore any
difference between AccType_Normal and AccType_Unpriv, which
would be the only difference between the passed mmu_idx
and arm_mmu_idx_el.

Backports 4aedfc0f633fd06dd2a5dc8ffa93f4c56117e37f
2021-03-01 20:09:51 -05:00
Peter Maydell d350644817 target/arm: AArch32 VCVT fixed-point to float is always round-to-nearest
For AArch32, unlike the VCVT of integer to float, which honours the
rounding mode specified by the FPSCR, VCVT of fixed-point to float is
always round-to-nearest. (AArch64 fixed-point-to-float conversions
always honour the FPCR rounding mode.)

Implement this by providing _round_to_nearest versions of the
relevant helpers which set the rounding mode temporarily when making
the call to the underlying softfloat function.

We only need to change the VFP VCVT instructions, because the
standard- FPSCR value used by the Neon VCVT is always set to
round-to-nearest, so we don't need to do the extra work of saving
and restoring the rounding mode.

Backports commit 61db12d9f9eb36761edba4d9a414cd8dd34c512b
2021-03-01 20:04:31 -05:00
Peter Maydell 31013d5a8f target/arm: Fix SMLAD incorrect setting of Q bit
The SMLAD instruction is supposed to:
* signed multiply Rn[15:0] * Rm[15:0]
* signed multiply Rn[31:16] * Rm[31:16]
* perform a signed addition of the products and Ra
* set Rd to the low 32 bits of the theoretical
infinite-precision result
* set the Q flag if the sign-extension of Rd
would differ from the infinite-precision result
(ie on overflow)

Our current implementation doesn't quite do this, though: it performs
an addition of the products setting Q on overflow, and then it adds
Ra, again possibly setting Q. This sometimes incorrectly sets Q when
the architecturally mandated only-check-for-overflow-once algorithm
does not. For instance:
r1 = 0x80008000; r2 = 0x80008000; r3 = 0xffffffff
smlad r0, r1, r2, r3
This is (-32768 * -32768) + (-32768 * -32768) - 1

The products are both 0x4000_0000, so when added together as 32-bit
signed numbers they overflow (and QEMU sets Q), but because the
addition of Ra == -1 brings the total back down to 0x7fff_ffff
there is no overflow for the complete operation and setting Q is
incorrect.

Fix this edge case by resorting to 64-bit arithmetic for the
case where we need to add three values together.

Backports commit 5288145d716338ace0f83e3ff05c4d07715bb4f4
2021-03-01 19:58:39 -05:00
Peter Maydell 6cd06169ee target/arm: Make '-cpu max' have a 48-bit PA
QEMU supports a 48-bit physical address range, but we don't currently
expose it in the '-cpu max' ID registers (you get the same range as
Cortex-A57, which is 44 bits).

Set the ID_AA64MMFR0.PARange field to indicate 48 bits.

Backports d1b6b7017572e8d82f26eb827a1dba0e8cf3cae6
2021-03-01 19:50:28 -05:00
Richard Henderson c648361597 tcg: Remove TCG_TARGET_HAS_cmp_vec
The cmp_vec opcode is mandatory; this symbol is unused.

Backports cae5d53b9e72d7a1e43cebeb36471d77a16c6e43
2021-03-01 19:49:02 -05:00
Richard Henderson 45af31fcb4 tcg/optimize: Fold dup2_vec
When the two arguments are identical, this can be reduced to
dup_vec or to mov_vec from a tcg_constant_vec.

Backports commit 1dc4fe70128db05237a00eda6eb15e2b44deb31f
2021-03-01 19:46:14 -05:00
Richard Henderson 456fb66617 tcg: Fix generation of dupi_vec for 32-bit host
The definition of INDEX_op_dupi_vec is that it operates on
units of tcg_target_ulong -- in this case 32 bits. It does
not work to use this for a uint64_t value that happens to be
small enough to fit in tcg_target_ulong.

Backports a5b30d950c42b14bc9da24d1e68add6538d23336
2021-03-01 19:45:30 -05:00
Richard Henderson 578673be68 tcg/i386: Fix dupi for avx2 32-bit hosts
The previous change wrongly stated that 32-bit avx2 should have
used VPBROADCASTW. But that's a 16-bit broadcast and we want a
32-bit broadcast.

Backports f80d09b599a5e0fd7f44653f23b04104cb703f7a
2021-03-01 19:44:09 -05:00
Richard Henderson 50b3632ab4 tcg: Remove TCGOpDef.used
The last user of this field disappeared in f69d277ece4.
2021-03-01 19:43:37 -05:00
Richard Henderson 7813c57f9e tcg: Move some TCG_CT_* bits to TCGArgConstraint bitfields
These are easier to set and test when they have their own fields.
Reduce the size of alias_index and sort_index to 4 bits, which is
sufficient for TCG_MAX_OP_ARGS. This leaves only the bits indicating
constants within the ct field.

Move all initialization to allocation time, rather than init
individual fields in process_op_defs.

Backports bc2b17e6ea582ef3ade2bdca750de269c674c915
2021-03-01 19:41:34 -05:00
Richard Henderson 71a34d84e5 tcg: Remove TCG_CT_REG
This wasn't actually used for anything, really. All variable
operands must accept registers, and which are indicated by the
set in TCGArgConstraint.regs.

Backports commit 74a117906b87ff9220e4baae5a7431d6f4eadd45
2021-03-01 19:38:00 -05:00
Richard Henderson ae075d324d tcg: Move sorted_args into TCGArgConstraint.sort_index
This uses an existing hole in the TCGArgConstraint structure
and will be convenient for keeping the data in one place.

Backports 66792f90f14fef18b25a168922877a367ecdca05
2021-03-01 19:33:45 -05:00
Richard Henderson e3356f9bad tcg: Drop union from TCGArgConstraint
The union is unused; let "regs" appear in the main structure
without the "u.regs" wrapping.

Backports 9be0d08019465b38e2f1a605960961a491430c21
2021-03-01 19:29:19 -05:00
Richard Henderson 1551f6be9d tcg: Adjust simd_desc size encoding
With larger vector sizes, it turns out oprsz == maxsz, and we only
need to represent mismatch for oprsz <= 32. We do, however, need
to represent larger oprsz and do so without reducing SIMD_DATA_BITS.

Reduce the size of the oprsz field and increase the maxsz field.
Steal the oprsz value of 24 to indicate equality with maxsz.

Backports e2e7168a214b0ed98dc357bba96816486a289762
2021-03-01 19:23:37 -05:00
Richard Henderson 567fa21c65 target/arm: Fix SVE splice
While converting to gen_gvec_ool_zzzp, we lost passing
a->esz as the data argument to the function.

Backports commit dd701fafe55a78e655d4823d29226d92250a6b56
2021-03-01 19:20:44 -05:00
Richard Henderson ccb293911f target/arm: Fix sve ldr/str
The mte update missed a bit when producing clean addresses.

Fixes: b2aa8879b88

Backports d8227b098301935ea8e0e032e7d41e5dc3e97590
2021-03-01 19:20:04 -05:00
Peter Maydell 79feec40df target/arm: Make isar_feature_aa32_fp16_arith() handle M-profile
The M-profile definition of the MVFR1 ID register differs slightly
from the A-profile one, and in particular the check for "does the CPU
support fp16 arithmetic" is not the same.

We don't currently implement any M-profile CPUs with fp16 arithmetic,
so this is not yet a visible bug, but correcting the logic now
disarms this beartrap for when we eventually do.

Backports commit dfc523a84b06b6a4b583ed4c29d24fd980dd37a0
2021-03-01 19:17:23 -05:00
Peter Maydell 09a7d6381e target/arm: Move id_pfr0, id_pfr1 into ARMISARegisters
Move the id_pfr0 and id_pfr1 fields into the ARMISARegisters
sub-struct. We're going to want id_pfr1 for an isar_features
check, and moving both at the same time avoids an odd
inconsistency.

Changes other than the ones to cpu.h and kvm64.c made
automatically with:
perl -p -i -e 's/cpu->id_pfr/cpu->isar.id_pfr/' target/arm/*.c hw/intc/armv7m_nvic.c

Backports commit 8a130a7be6e222965641e1fd9469fd3ee752c7d4
2021-03-01 19:15:10 -05:00
Peter Maydell ed92f3c42b target/arm: Replace ARM_FEATURE_PXN with ID_MMFR0.VMSA check
The ARM_FEATURE_PXN bit indicates whether the CPU supports the PXN
bit in short-descriptor translation table format descriptors. This
is indicated by ID_MMFR0.VMSA being at least 0b0100. Replace the
feature bit with an ID register check, in line with our preference
for ID register checks over feature bits.

Backports commit 0ae0326b984e77a55c224b7863071bd3d8951231
2021-03-01 19:06:15 -05:00
Xiaoyao Li d9d68cc128 i386/cpu: Clear FEAT_XSAVE_COMP_{LO,HI} when XSAVE is not available
Per Intel SDM vol 1, 13.2, if CPUID.1:ECX.XSAVE[bit 26] is 0, the
processor provides no further enumeration through CPUID function 0DH.
QEMU does not do this for "-cpu host,-xsave".

Backports 19ca8285fcd61a8f60f2f44f789a561e0958e8e6
2021-03-01 19:04:03 -05:00
Richard Henderson 5e6196ea6b target/riscv: Set instance_align on RISCVCPU TypeInfo
Fix alignment of CPURISCVState.vreg.

Backports 5de5b99b3101a1648ed583193db8d92eea0c4545
2021-03-01 19:00:27 -05:00
Richard Henderson cdf40f7ff6 target/arm: Set instance_align on CPUARM TypeInfo
Fix alignment of CPUARMState.vfp.zregs.

Backports d03087bda4ba17076b430fd2af083020d7c5112a
2021-03-01 18:58:44 -05:00
Richard Henderson 86dd30850d qom: Allow objects to be allocated with increased alignment
It turns out that some hosts have a default malloc alignment less
than that required for vectors.

We assume that, with compiler annotation on CPUArchState, that we
can properly align the vector portion of the guest state. Fix the
alignment of the allocation by using qemu_memalloc when required.
2021-03-01 18:32:51 -05:00
Eduardo Habkost 6baafeafd4 qom: Correct object_class_dynamic_cast_assert() documentation
object_class_dynamic_cast_assert() is not used by
INTERFACE_CHECK, remove misleading mention of that function in
the documentation.
2021-03-01 18:29:34 -05:00
Aaron Lindsay 97702da7ad target/arm: Count PMU events when MDCR.SPME is set
This check was backwards when introduced in commit
033614c47de78409ad3fb39bb7bd1483b71c6789:

target/arm: Filter cycle counter based on PMCCFILTR_EL0

Backports commit db1f3afb17269cf2bd86c222e1bced748487ef71
2021-03-01 18:25:25 -05:00
Peter Maydell 16ad0d93d9 target/arm: Convert VCMLA, VCADD size field to MO_* in decode
The VCMLA and VCADD insns have a size field which is 0 for fp16
and 1 for fp32 (note that this is the reverse of the Neon 3-same
encoding!). Convert it to MO_* values in decode for consistency.

Backports d186a4854c04e9832907b0b4240a47731da20993
2021-03-01 18:23:34 -05:00
Peter Maydell 61abec1908 target/arm: Convert Neon VCVT fp size field to MO_* in decode
Convert the insns using the 2reg_vcvt and 2reg_vcvt_f16 formats
to pass the size through to the trans function as a MO_* value
rather than the '0==f32, 1==f16' used in the fp 3-same encodings.

Backports commit 0ae715c658a02af1834b63563c56112a6d8842cb
2021-03-01 18:20:11 -05:00
Peter Maydell 524b54bc7b target/arm: Convert Neon 3-same-fp size field to MO_* in decode
In the Neon instructions, some instruction formats have a 2-bit size
field which corresponds exactly to QEMU's MO_8/16/32/64. However the
floating-point insns in the 3-same group have a 1-bit size field
which is "0 for 32-bit float and 1 for 16-bit float". Currently we
pass these values directly through to trans_ functions, which means
that when reading a particular trans_ function you need to know if
that insn uses a 2-bit size or a 1-bit size.

Move the handling of the 1-bit size to the decodetree file, so that
all these insns consistently pass a size to the trans_ function which
is an MO_8/16/32/64 value.

In this commit we switch over the insns using the 3same_fp and
3same_fp_q0 formats.

Backports commit 6cf0f240e0b980a877abed12d2995f740eae6515
2021-03-01 18:15:18 -05:00
Richard Henderson cd79d2a915 tcg: Implement 256-bit dup for tcg_gen_gvec_dup_mem
We already support duplication of 128-bit blocks. This extends
that support to 256-bit blocks. This will be needed by SVE2.

Backports commit fe4b0b5bfa96c38ad1cad0689a86cca9f307e353
2021-03-01 18:10:07 -05:00
Richard Henderson b478ce5052 tcg: Eliminate one store for in-place 128-bit dup_mem
Do not store back to the exact memory from which we just loaded.

Backports 6a17646176e011ddc463a2870a64c7aaccfe9c50
2021-03-01 18:06:17 -05:00
Stephen Long c9dc750058 tcg: Fix tcg gen for vectorized absolute value
The fallback inline expansion for vectorized absolute value,
when the host doesn't support such an insn was flawed.

E.g. when a vector of bytes has all elements negative, mask
will be 0xffff_ffff_ffff_ffff. Subtracting mask only adds 1
to the low element instead of all elements becase -mask is 1
and not 0x0101_0101_0101_0101.

Backports commit e7e8f33fb603c3bfa0479d7d924f2ad676a84317
2021-03-01 18:04:46 -05:00
Eduardo Habkost cefb1666c0 arm: Fix typo in AARCH64_CPU_GET_CLASS definition
There's a typo in the type name of AARCH64_CPU_GET_CLASS. This
was never detected because the macro is not used by any code.

Backports 37e3d65043229bb20bd07af74dc0866e12071415
2021-03-01 18:03:29 -05:00
Peter Maydell ff74ede2fd target/arm: Enable FP16 in '-cpu max'
Set the MVFR1 ID register FPHP and SIMDHP fields to indicate
that our "-cpu max" has v8.2-FP16.

Backports commit 5f07817eb94542e39a419baafa3026b15e8d33f7
2021-03-01 18:00:13 -05:00
Peter Maydell b948636c4a target/arm: Implement fp16 for Neon VMUL, VMLA, VMLS
Convert the Neon floating-point VMUL, VMLA and VMLS to use gvec,
and use this to implement fp16 support.

Backports fc8ae790311882afa3c7816df004daf978c40e9a
2021-03-01 17:57:36 -05:00
Peter Maydell 8c6affbca4 target/arm/vec_helper: Add gvec fp indexed multiply-and-add operations
Add gvec helpers for doing Neon-style indexed non-fused fp
multiply-and-accumulate operations.

Backports commit c50d8d144098a8261233ca31b47e3bc487e112fe
2021-03-01 17:52:31 -05:00
Peter Maydell 3cc3099e36 target/arm/vec_helper: Handle oprsz less than 16 bytes in indexed operations
In the gvec helper functions for indexed operations, for AArch32
Neon the oprsz (total size of the vector) can be less than 16 bytes
if the operation is on a D reg. Since the inner loop in these
helpers always goes from 0 to segment, we must clamp it based
on oprsz to avoid processing a full 16 byte segment when asked to
handle an 8 byte wide vector.

Backports commit d7ce81e553e6789bf27657105b32575668d60b1c
2021-03-01 17:48:42 -05:00
Peter Maydell 681218b4ab target/arm: Implement fp16 for Neon VRINTX
Convert the Neon VRINTX insn to use gvec, and use this to implement
fp16 support for it.

Backports 23afcdd2511f2a3dc05bed650d27bd25cf9b2a3c
2021-03-01 17:47:25 -05:00
Peter Maydell 53aba9d900 target/arm: Implement fp16 for Neon VRINT-with-specified-rounding-mode
Convert the Neon VRINT-with-specified-rounding-mode insns to gvec,
and use this to implement the fp16 versions.

Backports 18725916b1438b54d6d6533980833d2251a20b7c
2021-03-01 17:44:49 -05:00
Peter Maydell eb4054d04f target/arm: Implement fp16 for Neon VCVT with rounding modes
Convert the Neon VCVT with-specified-rounding-mode instructions
to gvec, and use this to implement fp16 support for them.

Backports ca88a6efdf4ce96b646a896059f9bd324c2cebc4
2021-03-01 17:40:36 -05:00
Peter Maydell 56fe927d40 target/arm: Implement fp16 for Neon VCVT fixed-point
Implement fp16 for the Neon VCVT insns which convert between
float and fixed-point.

Backports 24018cf3990b692b51e50183c5fbd98d17b3fa40
2021-03-01 17:36:43 -05:00
Peter Maydell 948b01ad01 target/arm: Convert Neon VCVT fixed-point to gvec
Convert the Neon VCVT float<->fixed-point insns to a
gvec style, in preparation for adding fp16 support.

Backports 7b959c5890deb9a6d71bc6800006a0eae0a84c60
2021-03-01 17:33:20 -05:00
Peter Maydell c324c6817e target/arm: Implement fp16 for Neon float-integer VCVT
Convert the Neon float-integer VCVT insns to gvec, and use this
to implement fp16 support for them.

Note that unlike the VFP int<->fp16 VCVT insns we converted
earlier and which convert to/from a 32-bit integer, these
Neon insns convert to/from 16-bit integers. So we can use
the existing vfp conversion helpers for the f32<->u32/i32
case but need to provide our own for f16<->u16/i16.

Backports 7782a9afec81d1efe23572135c1ed777691ccde5
2021-03-01 17:29:02 -05:00
Peter Maydell 82f4a7e135 target/arm: Implement fp16 for Neon pairwise fp ops
Convert the Neon pairwise fp ops to use a single gvic-style
helper to do the full operation instead of one helper call
for each 32-bit part. This allows us to use the same
framework to implement the fp16.

Backports 1dc587ee9bfe804406eb3e0bacf47a80644d8abc
2021-03-01 17:25:19 -05:00
Peter Maydell b08ea84374 target/arm: Implement fp16 for Neon VRSQRTS
Convert the Neon VRSQRTS insn to using a gvec helper,
and use this to implement the fp16 case.

As with VRECPS, we adjust the phrasing of the new implementation
slightly so that the fp32 version parallels the fp16 one.

Backports 40fde72dda2da8d55b820fa6c5efd85814be2023
2021-03-01 17:20:22 -05:00
Peter Maydell f4ebbba9fd target/arm: Implement fp16 for Neon VRECPS
Convert the Neon VRECPS insn to using a gvec helper, and
use this to implement the fp16 case.

The phrasing of the new float32_recps_nf() is slightly different from
the old recps_f32() so that it parallels the f16 version; for f16 we
can't assume that flush-to-zero is always enabled.

Backports ac8c62c4e5a3f24e6d47f52ec1bfb20994caefa5
2021-03-01 17:09:16 -05:00
Peter Maydell 5776c594e4 target/arm: Implement fp16 for Neon fp compare-vs-0
Convert the neon floating-point vector compare-vs-0 insns VCEQ0,
VCGT0, VCLE0, VCGE0 and VCLT0 to use a gvec helper, and use this to
implement the fp16 case.

Backport 635187aaa92f21ab001e2868e803b3c5460261ca
2021-03-01 17:05:03 -05:00
Peter Maydell 8de258c3cb target/arm: Implement fp16 for Neon VFMA, VMFS
Convert the neon floating-point vector operations VFMA and VFMS
to use a gvec helper, and use this to implement the fp16 case.

This is the last use of do_3same_fp() so we can now delete
that function.

Backports commit cf722d75b329ef3f86b869e7e68cbfb1607b3bde
2021-03-01 17:00:49 -05:00
Peter Maydell 587c3549b7 target/arm: Implement fp16 for Neon VMLA, VMLS operations
Convert the Neon floating-point VMLA and VMLS insns over to using a
gvec helper, and use this to implement the fp16 case.

Backports e5adc70665ecaf4009c2fb8d66775ea718a85abd
2021-03-01 16:57:20 -05:00