Commit graph

1388 commits

Author SHA1 Message Date
Peter Maydell 1cc6451cb6 target/arm: Convert Neon VPMAX/VPMIN 3-reg-same insns to decodetree
Convert the Neon integer VPMAX and VPMIN 3-reg-same insns to
decodetree. These are 'pairwise' operations.

Backports commit 059c2398a2b1ae86c6722c45e79fb0d0f4d95b1d from qemu
2020-05-15 22:59:10 -04:00
Peter Maydell f35ae14ab4 target/arm: Convert Neon VQSHL, VRSHL, VQRSHL 3-reg-same insns to decodetree
Convert the VQSHL, VRSHL and VQRSHL insns in the 3-reg-same
group to decodetree. We have already implemented the size==0b11
case of these insns; this commit handles the remaining sizes

Backports commit 6812dfdc6b0286730d6f903ebfbdc4f81b80c29b from qemu
2020-05-15 22:53:27 -04:00
Peter Maydell 5308fb324e target/arm: Convert Neon VRHADD, VHSUB 3-reg-same insns to decodetree
Convert the Neon VRHADD and VHSUB 3-reg-same insns to decodetree.
(These are all the other insns in 3-reg-same which were using
GEN_NEON_INTEGER_OP() and which are not pairwise or
reversed-operands.)

Backports commit 8e44d03f4b5590e19a4f7910ca1c327609933dd7 from qemu
2020-05-15 22:50:02 -04:00
Peter Maydell ec327c7fc8 target/arm: Convert Neon VABA/VABD 3-reg-same to decodetree
Convert the Neon VABA and VABD insns in the 3-reg-same group to
decodetree.

Backports commit 7715098f93ff5205334edf161e5fe156346122b0 from qemu
2020-05-15 22:46:02 -04:00
Peter Maydell f1028fe4a7 target/arm: Convert Neon VHADD 3-reg-same insns
Convert the Neon VHADD insns in the 3-reg-same group to decodetree.

Backports commit cb294bca866f1cd776e44e03e5e432942bc676e8 from qemu
2020-05-15 22:43:01 -04:00
Peter Maydell 4098e0b80a target/arm: Convert Neon 64-bit element 3-reg-same insns
Convert the 64-bit element insns in the 3-reg-same group
to decodetree. This covers VQSHL, VRSHL and VQRSHL where
size==0b11.

Backports commit 35d4352fa9e94b35bf17f58181cb16c184b98d56 from qemu
2020-05-15 22:40:48 -04:00
Peter Maydell e2b703a82c target/arm: Convert Neon 3-reg-same SHA to decodetree
Convert the Neon SHA instructions in the 3-reg-same group
to decodetree

Backports commit 21290edfc29d8929741c0ed043733c23c69bc3b9 from qemu
2020-05-15 22:34:40 -04:00
Richard Henderson 1740e018f4 target/arm: Convert Neon 3-reg-same VQRDMLAH/VQRDMLSH to decodetree
Convert the Neon VQRDMLAH and VQRDMLSH insns in the 3-reg-same group
to decodetree. These don't use do_3same() because they want to
operate on VFP double registers, whose offsets are different from the
neon_reg_offset() calculations do_3same does.

Backports commit a063569508af8295cf6271e06700e5b956bb402d from qemu
2020-05-15 22:20:23 -04:00
Richard Henderson 451683ee79 target/arm: Vectorize SABA/UABA
Include 64-bit element size in preparation for SVE2.

Backports commit cfdb2c0c95ae9205b0dd7f0f5e970cdec50fef20 from qemu
2020-05-15 22:15:14 -04:00
Richard Henderson 98c79f9afc target/arm: Vectorize SABD/UABD
Include 64-bit element size in preparation for SVE2.

Backports commit 50c160d44eb059c7fc7f348ae2c3b0cb41437044 from qemu
2020-05-15 22:01:29 -04:00
Richard Henderson 765dbb57f0 target/arm: Clear tail in gvec_fmul_idx_*, gvec_fmla_idx_*
Must clear the tail for AdvSIMD when SVE is enabled.

Fixes: ca40a6e6e39

Backports commit 525d9b6d42844e187211d25b69be8b378785bc24 from qemu
2020-05-15 21:50:30 -04:00
Richard Henderson 73d08253a2 target/arm: Pass pointer to qc to qrdmla/qrdmls
Pass a pointer directly to env->vfp.qc[0], rather than env.
This will allow SVE2, which does not modify QC, to pass a
pointer to dummy storage.

Change the return type of inl_qrdml.h_s16 to match the
sense of the operation: signed.

Backports commit e286bf4a72fe3a60490b8d6e3f28d6335677e08c from qemu
2020-05-15 21:48:35 -04:00
Richard Henderson 3c4f226e00 target/arm: Create gen_gvec_{qrdmla,qrdmls}
Provide a functional interface for the vector expansion.
This fits better with the existing set of helpers that
we provide for other operations.

Backports commit 146aa66ce58b686b8037d0eb3921c1125942dbde from qemu
2020-05-15 21:43:22 -04:00
Richard Henderson efdcad70b1 target/arm: Remove fp_status from helper_{recpe, rsqrte}_u32
These operations do not touch fp_status.

Backports commit fe6fb4beb2f9bb0afc813e565504b66a92bbf04b from qemu
2020-05-15 21:32:03 -04:00
Richard Henderson 9dfc0479ff target/arm: Create gen_gvec_{uqadd, sqadd, uqsub, sqsub}
Provide a functional interface for the vector expansion.
This fits better with the existing set of helpers that
we provide for other operations.

Backports commit c7715b6b51a6f7a5412c5fcb40a4c8586105e597 from qemu
2020-05-15 21:25:06 -04:00
Richard Henderson 4abfe5156d target/arm: Create gen_gvec_{cmtst,ushl,sshl}
Provide a functional interface for the vector expansion.
This fits better with the existing set of helpers that
we provide for other operations.

Backports commit 8161b75357095fef54c76b1a6ed1e54d0e8655e0 from qemu
2020-05-15 21:15:49 -04:00
Richard Henderson 15b2850f4d target/arm: Swap argument order for VSHL during decode
Rather than perform the argument swap during code generation,
perform it during decode. This means it doesn't have to be
special cased later, and we can share code with aarch64 code
generation. Hopefully the decode comment addresses any confusion
that might arise in between.

Backports commit e9eee5316ffec5f37643de806b2e5577c5c189cf from qemu
2020-05-15 21:07:59 -04:00
Richard Henderson 546db9089c target/arm: Create gen_gvec_{mla,mls}
Provide a functional interface for the vector expansion.
This fits better with the existing set of helpers that
we provide for other operations.

Backports commit 271063206a46062a45fc6bab8dabe45f0b88159d from qemu
2020-05-15 21:06:06 -04:00
Richard Henderson 340f97bf4c target/arm: Create gen_gvec_{ceq,clt,cle,cgt,cge}0
Provide a functional interface for the vector expansion.
This fits better with the existing set of helpers that
we provide for other operations.

Macro-ize the 5 nearly identical comparisons.

Backports commit 69d5e2bf8c3cefedbfa1c1670137e636dbd7faa5 from qemu
2020-05-15 20:57:33 -04:00
Richard Henderson e08c2b8ece target/arm: Tidy handle_vec_simd_shri
Now that we've converted all cases to gvec, there is quite a bit
of dead code at the end of the function. Remove it.

Sink the call to gen_gvec_fn2i to the end, loading a function
pointer within the switch statement.

Backports commit 3f08f0bce841e7857ec98ce7909629d0c335005e from qemu
2020-05-15 20:47:47 -04:00
Richard Henderson 7a1750d691 target/arm: Remove unnecessary range check for VSHL
In 1dc8425e551, while converting to gvec, I added an extra range check
against the shift count. This was unnecessary because the encoding of
the shift count produces 0 to the element size - 1.

Backports commit 2f27c5244db300387f15d9ffa5067a204ffd625d from qemu
2020-05-15 20:42:12 -04:00
Richard Henderson 6190be3191 target/arm: Create gen_gvec_{sri,sli}
The functions eliminate duplication of the special cases for
this operation. They match up with the GVecGen2iFn typedef.

Add out-of-line helpers. We got away with only having inline
expanders because the neon vector size is only 16 bytes, and
we know that the inline expansion will always succeed.
When we reuse this for SVE, tcg-gvec-op may decide to use an
out-of-line helper due to longer vector lengths.

Backports commit 893ab0542aa385a287cbe46d5535c8b9e95ce699 from qemu
2020-05-15 20:39:28 -04:00
Richard Henderson 2609e6f319 target/arm: Create gen_gvec_{u,s}{rshr,rsra}
Create vectorized versions of handle_shri_with_rndacc
for shift+round and shift+round+accumulate. Add out-of-line
helpers in preparation for longer vector lengths from SVE.

Backports commit 6ccd48d4ea244c1c46a24dfa50bfb547f11422dd from qemu
2020-05-15 20:28:44 -04:00
Richard Henderson 5d7c46204d target/arm: Create gen_gvec_[us]sra
The functions eliminate duplication of the special cases for
this operation. They match up with the GVecGen2iFn typedef.

Add out-of-line helpers. We got away with only having inline
expanders because the neon vector size is only 16 bytes, and
we know that the inline expansion will always succeed.
When we reuse this for SVE, tcg-gvec-op may decide to use an
out-of-line helper due to longer vector lengths.

Backports commit 631e565450c483e0622eec3d8b61d7fa41d16bca from qemu
2020-05-15 20:10:32 -04:00
Richard Henderson 4be4ca57b1 target/arm: Fix tcg_gen_gvec_dup_imm vs DUP (indexed)
DUP (indexed) can duplicate 128-bit elements, so using esz
unconditionally can assert in tcg_gen_gvec_dup_imm.

Fixes: 8711e71f9cbb

Backports commit 7e17d50ebd359ee5fa3d65d7fdc0fe0336d60694 from qemu
2020-05-11 17:22:52 -04:00
Lioncash 5c03efd5d6 arm/helper: Amend sign conversion warning 2020-05-11 17:21:25 -04:00
Lioncash 08cc2c6dcc arm/cpu64: Remove unused variable 2020-05-11 17:18:13 -04:00
Richard Henderson f93deb0786 target/arm: Use tcg_gen_gvec_5_ptr for sve FMLA/FCMLA
Now that we can pass 7 parameters, do not encode register
operands within simd_data.

Backports commit 08975da9f0bfcfa654628cae71201a351ba5449a from qemu
2020-05-11 17:17:17 -04:00
Thomas Huth dfe548117e target/arm: Make set_feature() available for other files
Move the common set_feature() and unset_feature() functions
from cpu.c and cpu64.c to cpu.h.

Backports commit 5fda95041d7237ab35733ceb66e0cb89f6107169 from qemu
2020-05-11 17:02:21 -04:00
Philippe Mathieu-Daudé cfe94f63f3 target/arm/cpu: Use ARRAY_SIZE() to iterate over ARMCPUInfo[]
Since on the aarch64-linux-user build, arm_cpus[] is empty, add
the cpu_count variable and only iterate when it is non-zero.

Backports commit 92b6a659388ab3735e5fbb17ac486923b681f57f from qemu
2020-05-11 16:59:54 -04:00
Edgar E. Iglesias 91dbd53f77 target/arm: Drop access_el3_aa32ns_aa64any()
Calling access_el3_aa32ns() works for AArch32 only cores
but it does not handle 32-bit EL2 on top of 64-bit EL3
for mixed 32/64-bit cores.

Merge access_el3_aa32ns_aa64any() into access_el3_aa32ns()
and only use the latter.

Fixes: 68e9c2fe65 ("target-arm: Add VTCR_EL2")

Backports commit 93dd1e6140e2652347cfe7208591d4cd32762d08 from qemu
2020-05-11 16:39:40 -04:00
MerryMage 9255fbce96 target/arm: Introduce add_reg_for_lit (fixup)
Backports commit 16e0d8234ef9291747332d2c431e46808a060472 from qemu

Missed from original backporting commit a2e60445de
2020-05-10 12:30:52 +01:00
Richard Henderson b0f6374149 target/arm: Use tcg_gen_gvec_dup_imm
In a few cases, we're able to remove some manual replication.

Backports commit 8711e71f9cbb692d614e6ecf5d51222372f7b77e from qemu
2020-05-07 10:05:49 -04:00
Peter Maydell d350125eab target/arm: Move gen_ function typedefs to translate.h
We're going to want at least some of the NeonGen* typedefs
for the refactored 32-bit Neon decoder, so move them all
to translate.h since it makes more sense to keep them in
one group.

Backports commit 9aefc6cf9b73f66062d2f914a0136756e7a28211 from qemu
2020-05-07 09:51:52 -04:00
Peter Maydell 652165d671 target/arm: Convert Neon 3-reg-same VMUL, VMLA, VMLS, VSHL to decodetree
Convert the Neon VMUL, VMLA, VMLS and VSHL insns in the
3-reg-same grouping to decodetree.

Backports commit 0de34fd48ad4e44bf5caa2330657ebefa93cea7d from qemu
2020-05-07 09:50:44 -04:00
Peter Maydell 17bd8930fc target/arm: Convert Neon 3-reg-same VQADD/VQSUB to decodetree
Convert the Neon VQADD/VQSUB insns in the 3-reg-same grouping
to decodetree.

Backports commit 7a9497f1cf73667a4744d09673b808c20e067915 from qemu
2020-05-07 09:47:18 -04:00
Peter Maydell d52b830ce3 target/arm: Convert Neon 3-reg-same comparisons to decodetree
Convert the Neon comparison ops in the 3-reg-same grouping
to decodetree.

Backports commit 02bd0cdb64b3e79419ba3a8746cb86430883b3ae from qemu
2020-05-07 09:45:03 -04:00
Peter Maydell c6f9fb54fd target/arm: Convert Neon 3-reg-same VMAX/VMIN to decodetree
Convert the Neon 3-reg-same VMAX and VMIN insns to decodetree.

Backports commit 36b59310c38d45213bf860affa90618aa5eeca93 from qemu
2020-05-07 09:42:04 -04:00
Peter Maydell d30f99ca79 target/arm: Convert Neon 3-reg-same logic ops to decodetree
Convert the Neon logic ops in the 3-reg-same grouping to decodetree.
Note that for the logic ops the 'size' field forms part of their
decode and the actual operations are always bitwise.

Backports commit 35a548edb6f5043386183b9f6b4139d99d1f130a from qemu
2020-05-07 09:40:10 -04:00
Peter Maydell eae3ce9899 target/arm: Convert Neon 3-reg-same VADD/VSUB to decodetree
Convert the Neon 3-reg-same VADD and VSUB insns to decodetree.

Note that we don't need the neon_3r_sizes[op] check here because all
size values are OK for VADD and VSUB; we'll add this when we convert
the first insn that has size restrictions.

For this we need one of the GVecGen*Fn typedefs currently in
translate-a64.h; move them all to translate.h as a block so they
are visible to the 32-bit decoder.

Backports commit a4e143ac5b9185f670d2f17ee9cc1a430047cb65 from qemu
2020-05-07 09:36:28 -04:00
Peter Maydell c7a31355fc target/arm: Convert Neon 'load/store single structure' to decodetree
Convert the Neon "load/store single structure to one lane" insns to
decodetree.

As this is the last set of insns in the neon load/store group,
we can remove the whole disas_neon_ls_insn() function.

Backports commit 123ce4e3daba26b760b472687e1fb1ad82cf1993 from qemu
2020-05-07 09:32:17 -04:00
Peter Maydell 302506f2f6 target/arm: Convert Neon 'load single structure to all lanes' to decodetree
Convert the Neon "load single structure to all lanes" insns to
decodetree.

Backports commit 3698747c48db871d876a398592c5a23d7580ed4a from qemu
2020-05-07 09:29:03 -04:00
Peter Maydell 7aad825fa6 target/arm: Convert Neon load/store multiple structures to decodetree
Convert the Neon "load/store multiple structures" insns to decodetree.

Backports commit a27b46304352a0eced45e560e96515dbe3cc174f from qemu
2020-05-07 09:25:51 -04:00
Peter Maydell 9814c1722f target/arm: Convert VFM[AS]L (scalar) to decodetree
Convert the VFM[AS]L (scalar) insns in the 2reg-scalar-ext group
to decodetree. These are the last ones in the group so we can remove
all the legacy decode for the group.

Note that in disas_thumb2_insn() the parts of this encoding space
where the decodetree decoder returns false will correctly be directed
to illegal_op by the "(insn & (1 << 28))" check so they won't fall
into disas_coproc_insn() by mistake.

Backports commit d27e82f7d02f35e5919bd9cbbcb157f3537069a0 from qemu
2020-05-07 09:20:35 -04:00
Peter Maydell 49cdb7e2db target/arm: Convert V[US]DOT (scalar) to decodetree
Convert the V[US]DOT (scalar) insns in the 2reg-scalar-ext group
to decodetree.

Backports commit 35f5d4d1747558c6af2d914bcd848dcc30c3b531 from qemu
2020-05-07 09:17:32 -04:00
Peter Maydell 73dbfbe4d7 target/arm: Convert VCMLA (scalar) to decodetree
Convert VCMLA (scalar) in the 2reg-scalar-ext group to decodetree.

Backports commit 7e1b5d615361bb0038cda0e08af41e350e42d081 from qemu
2020-05-07 09:15:30 -04:00
Peter Maydell 1ab06d3eb5 target/arm: Convert VFM[AS]L (vector) to decodetree
Convert the VFM[AS]L (vector) insns to decodetree. This is the last
insn in the legacy decoder for the 3same_ext group, so we can
delete the legacy decoder function for the group entirely.

Note that in disas_thumb2_insn() the parts of this encoding space
where the decodetree decoder returns false will correctly be directed
to illegal_op by the "(insn & (1 << 28))" check so they won't fall
into disas_coproc_insn() by mistake.

Backports commit 9a107e7b8a3c87ab63ec830d3d60f319fc577ff7 from qemu
2020-05-07 09:13:36 -04:00
Peter Maydell c06bdf4cc2 target/arm: Convert V[US]DOT (vector) to decodetree
Convert the V[US]DOT (vector) insns to decodetree.

Backports commit 32da0e330d3e5218b669079826496751fb52c1ca from qemu
2020-05-07 09:09:24 -04:00
Peter Maydell 1d4dba1e5a target/arm: Convert VCADD (vector) to decodetree
Convert the VCADD (vector) insns to decodetree.

Backports commit 94d5eb7b3f72fbbdee55d7908e9cb6de95949f4b from qemu
2020-05-07 09:05:55 -04:00
Peter Maydell d8287755b2 target/arm: Convert VCMLA (vector) to decodetree
Convert the VCMLA (vector) insns in the 3same extension group to
decodetree.

Backports commit afff8de0d4d55b4ce7c36eb9cdfafe477a35dd75 from qemu
2020-05-07 09:02:52 -04:00
Peter Maydell c2c628eb71 target/arm: Add stubs for AArch32 Neon decodetree
Add the infrastructure for building and invoking a decodetree decoder
for the AArch32 Neon encodings. At the moment the new decoder covers
nothing, so we always fall back to the existing hand-written decode.

We follow the same pattern we did for the VFP decodetree conversion
(commit 78e138bc1f672c145ef6ace74617d and following): code that deals
with Neon will be moving gradually out to translate-neon.vfp.inc,
which we #include into translate.c.

In order to share the decode files between A32 and T32, we
split Neon into 3 parts:
* data-processing
* load-store
* 'shared' encodings

The first two groups of instructions have similar but not identical
A32 and T32 encodings, so we need to manually transform the T32
encoding into the A32 one before calling the decoder; the third group
covers the Neon instructions which are identical in A32 and T32.

Backports commit 625e3dd44a15dfbe9532daa6454df3f86cf04d3e from qemu
2020-05-07 08:59:42 -04:00
Peter Maydell 518d18062f target/arm: Don't allow Thumb Neon insns without FEATURE_NEON
We were accidentally permitting decode of Thumb Neon insns even if
the CPU didn't have the FEATURE_NEON bit set, because the feature
check was being done before the call to disas_neon_data_insn() and
disas_neon_ls_insn() in the Arm decoder but was omitted from the
Thumb decoder. Push the feature bit check down into the called
functions so it is done for both Arm and Thumb encodings.

Backports commit d1a6d3b594157425232a1ae5ea7f51b7a1c1aa2e from qemu
2020-05-07 08:55:02 -04:00
Peter Maydell 1964e4b9c9 target/arm/translate-vfp.inc.c: Remove duplicate simd_r32 check
Somewhere along theline we accidentally added a duplicate
"using D16-D31 when they don't exist" check to do_vfm_dp()
(probably an artifact of a patchseries rebase). Remove it.

Backports commit 0d787cf1f3c88fa29477e054f8523f6d82d91c98 from qemu
2020-05-07 08:52:42 -04:00
Philippe Mathieu-Daudé 8f90b77a6d target/arm: Use uint64_t for midr field in CPU state struct
MIDR_EL1 is a 64-bit system register with the top 32-bit being RES0.
Represent it in QEMU's ARMCPU struct with a uint64_t, not a
uint32_t.

This fixes an error when compiling with -Werror=conversion
because we were manipulating the register value using a
local uint64_t variable:

target/arm/cpu64.c: In function ‘aarch64_max_initfn’:
target/arm/cpu64.c:628:21: error: conversion from ‘uint64_t’ {aka ‘long unsigned int’} to ‘uint32_t’ {aka ‘unsigned int’} may change value [-Werror=conversion]
628 | cpu->midr = t;
| ^

and future-proofs us against a possible future architecture
change using some of the top 32 bits.

Backports commit e544f80030121040c8932ff1bd4006f390266c0f from qemu
2020-05-07 08:51:28 -04:00
Peter Maydell d28059f4ea target/arm: Use correct variable for setting 'max' cpu's ID_AA64DFR0
In aarch64_max_initfn() we update both 32-bit and 64-bit ID
registers. The intended pattern is that for 64-bit ID registers we
use FIELD_DP64 and the uint64_t 't' register, while 32-bit ID
registers use FIELD_DP32 and the uint32_t 'u' register. For
ID_AA64DFR0 we accidentally used 'u', meaning that the top 32 bits of
this 64-bit ID register would end up always zero. Luckily at the
moment that's what they should be anyway, so this bug has no visible
effects.

Use the right-sized variable.

Backports commit 5a89dd2385a193aa954a7c9bf4e381f2ba6ae359 from qemu
2020-05-07 08:50:40 -04:00
Peter Maydell b427549ce4 target/arm: Implement ARMv8.2-TTS2UXN
The ARMv8.2-TTS2UXN feature extends the XN field in stage 2
translation table descriptors from just bit [54] to bits [54:53],
allowing stage 2 to control execution permissions separately for EL0
and EL1. Implement the new semantics of the XN field and enable
the feature for our 'max' CPU.

Backports commit ce3125bed935a12e619a8253c19340ecaa899347 from qemu
2020-05-07 08:49:18 -04:00
Peter Maydell 1e75276a89 target/arm: Add new 's1_is_el0' argument to get_phys_addr_lpae()
For ARMv8.2-TTS2UXN, the stage 2 page table walk wants to know
whether the stage 1 access is for EL0 or not, because whether
exec permission is given can depend on whether this is an EL0
or EL1 access. Add a new argument to get_phys_addr_lpae() so
the call sites can pass this information in.

Since get_phys_addr_lpae() doesn't already have a doc comment,
add one so we have a place to put the documentation of the
semantics of the new s1_is_el0 argument.

Backports commit ff7de2fc2c994030bfb83af9ddc9a3cd70ce3e88 from qemu
2020-05-07 08:45:23 -04:00
Peter Maydell bec9ee21b6 target/arm: Use enum constant in get_phys_addr_lpae() call
The access_type argument to get_phys_addr_lpae() is an MMUAccessType;
use the enum constant MMU_DATA_LOAD rather than a literal 0 when we
call it in S1_ptw_translate().

Backports commit 59dff859cd850876df2cfa561c7bcfc4bdda4599 from qemu
2020-05-07 08:42:41 -04:00
Peter Maydell 3df93e463d target/arm: Don't use a TLB for ARMMMUIdx_Stage2
We define ARMMMUIdx_Stage2 as being an MMU index which uses a QEMU
TLB. However we never actually use the TLB -- all stage 2 lookups
are done by direct calls to get_phys_addr_lpae() followed by a
physical address load via address_space_ld*().

Remove Stage2 from the list of ARM MMU indexes which correspond to
real core MMU indexes, and instead put it in the set of "NOTLB" ARM
MMU indexes.

This allows us to drop NB_MMU_MODES to 11. It also means we can
safely add support for the ARMv8.3-TTS2UXN extension, which adds
permission bits to the stage 2 descriptors which define execute
permission separatel for EL0 and EL1; supporting that while keeping
Stage2 in a QEMU TLB would require us to use separate TLBs for
"Stage2 for an EL0 access" and "Stage2 for an EL1 access", which is a
lot of extra complication given we aren't even using the QEMU TLB.

In the process of updating the comment on our MMU index use,
fix a couple of other minor errors:
* NS EL2 EL2&0 was missing from the list in the comment
* some text hadn't been updated from when we bumped NB_MMU_MODES
above 8

Backports commit bf05340cb655637451162c02dadcd6581a05c02c from qemu
2020-05-07 08:40:06 -04:00
Fredrik Strupe 65200d8aad target/arm: Make VQDMULL undefined when U=1
According to Arm ARM, VQDMULL is only valid when U=0, while having
U=1 is unallocated.

Backports commit ab553ef74ee52c0889679d0bd0da084aaf938f5c from qemu
2020-05-07 08:34:56 -04:00
Philippe Mathieu-Daudé 12cad29510 target/arm/cpu: Update coding style to make checkpatch.pl happy
We will move this code in the next commit. Clean it up
first to avoid checkpatch.pl errors.

Backports commit 51c510aa5876a681cd0059ed3bacaa17590dc2d5 from qemu
2020-04-30 21:40:07 -04:00
Thomas Huth 84f2729a29 target/arm: Make cpu_register() available for other files
Make cpu_register() (renamed to arm_cpu_register()) available
from internals.h so we can register CPUs also from other files
in the future.

Backports commit 37bcf244454f4efb82e2c0c64bbd7eabcc165a0c from qemu
2020-04-30 21:38:42 -04:00
Philippe Mathieu-Daudé afeb8ff2dc target/arm: Restrict the Address Translate write operation to TCG accel
Under KVM these registers are written by the hardware.
Restrict the writefn handlers to TCG to avoid when building
without TCG:

LINK aarch64-softmmu/qemu-system-aarch64
target/arm/helper.o: In function `do_ats_write':
target/arm/helper.c:3524: undefined reference to `raise_exception'

Backports commit 9fb005b02dbda7f47b789b7f19bf5f73622a4756 from qemu
2020-04-30 21:31:22 -04:00
Richard Henderson b26b4c06cd target/arm: Vectorize integer comparison vs zero
These instructions are often used in glibc's string routines.
They were the final uses of the 32-bit at a time neon helpers.

Backports commit 6b375d3546b009d1e63e07397ec9c6af256e15e9 from qemu
2020-04-30 21:29:17 -04:00
Peter Maydell a4a171a9c9 target/arm: Fix ID_MMFR4 value on AArch64 'max' CPU
In commit 41a4bf1feab098da4cd the added code to set the CNP
field in ID_MMFR4 for the AArch64 'max' CPU had a typo
where it used the wrong variable name, resulting in ID_MMFR4
fields AC2, XNX and LSM being wrong. Fix the typo.

Fixes: 41a4bf1feab098da4cd

Backports commit e73c4443473107ddf11ad3a7fea5bef2001ee802 from qemu
2020-04-30 07:29:06 -04:00
Peter Maydell 6a015761ac target/arm: Remove obsolete TODO note from get_phys_addr_lpae()
An old comment in get_phys_addr_lpae() claims that the code does not
support the different format TCR for VTCR_EL2. This used to be true
but it is not true now (in particular the aa64_va_parameters() and
aa32_va_parameters() functions correctly handle the different
register format by checking whether the mmu_idx is Stage2).
Remove the out of date parts of the comment.

Backports commit 07d1be3b3aac20c21ac4a95c7f3f01a3622a31a3 from qemu
2020-04-30 07:21:17 -04:00
Peter Maydell 4228e7f155 target/arm: PSTATE.PAN should not clear exec bits
Our implementation of the PSTATE.PAN bit incorrectly cleared all
access permission bits for privileged access to memory which is
user-accessible. It should only affect the privileged read and write
permissions; execute permission is dealt with via XN/PXN instead.

Fixes: 81636b70c226dc27d7ebc8d

Backports commit f4e1dbc578a051db08a40c05276ebf525b98f949 from qemu
2020-04-30 07:20:20 -04:00
Changbin Du 1e274425bd target/arm: fix incorrect current EL bug in aarch32 exception emulation
The arm_current_el() should be invoked after mode switching. Otherwise, we
get a wrong current EL value, since current EL is also determined by
current mode.

Fixes: 4a2696c0d4 ("target/arm: Set PAN bit as required on exception entry")

Backports commit 88828bf133b64b7a860c166af3423ef1a47c5d3b from qemu
2020-04-30 06:57:36 -04:00
Richard Henderson c9ee9a2729 target/arm: Move computation of index in handle_simd_dupe
Coverity reports a BAD_SHIFT with ctz32(imm5), with imm5 == 0.
This is an invalid encoding, but we diagnose that just below
by rejecting size > 3. Avoid the warning by sinking the
computation of index below the check.

Backports commit 550a04893c2bd4442211b353680b9a6408d94dba from qemu
2020-04-30 06:54:39 -04:00
Richard Henderson fd4ce2cba0 target/arm: Assert immh != 0 in disas_simd_shift_imm
Coverity raised a shed-load of errors cascading from inferring
that clz32(immh) might yield 32, from immh might be 0.

While immh cannot be 0 from encoding, it is not obvious even to
a human how we've checked that: via the filtering provided by
data_proc_simd[].

Backports commit 3944d58db3fc5bf131345a21a44013bc13849a12 from qemu
2020-04-30 06:53:54 -04:00
Richard Henderson d5234c8b3d target/arm: Rearrange disabled check for watchpoints
Coverity rightly notes that ctz32(bas) on 0 will return 32,
which makes the len calculation a BAD_SHIFT.

A value of 0 in DBGWCR<n>_EL1.BAS is reserved. Simply move
the existing check we have for this case

Backports commit ae1111d4def40c6f592c3a307c599272b778eb65 from qemu
2020-04-30 06:52:38 -04:00
Alex Bennée 46e1dab19e target/arm: don't bother with id_aa64pfr0_read for USER_ONLY
For system emulation we need to check the state of the GIC before we
report the value. However this isn't relevant to exporting of the
value to linux-user and indeed breaks the exported value as set by
modify_arm_cp_regs.

Backports commit 976b99b6ec2e15cd7c36d72fdb9b60c37c5494f8 from qemu
2020-04-30 06:24:10 -04:00
Richard Henderson 6c8172fd08 target/arm: Disable clean_data_tbi for system mode
We must include the tag in the FAR_ELx register when raising
an addressing exception. Which means that we should not clear
out the tag during translation.

We cannot at present comply with this for user mode, so we
retain the clean_data_tbi function for the moment, though it
no longer does what it says on the tin for system mode. This
function is to be replaced with MTE, so don't worry about the
slight misnaming.

Buglink: https://bugs.launchpad.net/qemu/+bug/1867072

Backports commit 38d931687fa196a7ef860f8583815abc7fd5521a from qemu
2020-04-30 06:18:31 -04:00
Richard Henderson e040675fbf target/arm: Clean address for DC ZVA
This data access was forgotten when we added support for cleaning
addresses of TBI information.

Fixes: 3a471103ac1823ba

Backports commit 597d61a3b1f94c53a3aaa77671697c0c5f797dbf from qemu.
2020-04-30 06:16:03 -04:00
Richard Henderson a37d9b2be5 target/arm: Use DEF_HELPER_FLAGS for helper_dc_zva
The function does not write registers, and only reads them by
implication via the exception path.

Backports commit 1371b02c5a060e423e70560dbca769b54e471ba9 from qemu
2020-04-30 06:14:45 -04:00
Richard Henderson 3cb68bc44e target/arm: Move helper_dc_zva to helper-a64.c
This is an aarch64-only function. Move it out of the shared file.
This patch is code movement only.

Backports commit 7b182eb2467af6c47c9c77c64bbbeed8ed53c330 from qemu
2020-04-30 06:12:26 -04:00
Richard Henderson a22a2a8b71 target/arm: Introduce core_to_aa64_mmu_idx
If by context we know that we're in AArch64 mode, we need not
test for M-profile when reconstructing the full ARMMMUIdx.

Backports commit 20dc67c947a691fa9df05e76aec6df50204b4b94 from qemu
2020-04-30 05:58:59 -04:00
Richard Henderson d3a5843aeb target/arm: Replicate TBI/TBID bits for single range regimes
Replicate the single TBI bit from TCR_EL2 and TCR_EL3 so that
we can unconditionally use pointer bit 55 to index into our
composite TBI1:TBI0 field.

Backports commit 3e270f67f0f05277021763af119a6ce195f8ed51 from qemu
2020-04-30 05:58:59 -04:00
Richard Henderson cc32a96183 target/arm: Honor the HCR_EL2.TTLB bit
This bit traps EL1 access to tlb maintenance insns.

Backports commit 30881b7353b5bb41210c32cd8e00421da757808c from qemu
2020-04-30 05:58:59 -04:00
Richard Henderson 74d6aa6012 target/arm: Honor the HCR_EL2.TPU bit
This bit traps EL1 access to cache maintenance insns that operate
to the point of unification. There are no longer any references to
plain aa64_cacheop_access, so remove it.

Backports commit 38262d8a732f8bd0e9ca3dc064f6e73d00c08b9a from qemu
2020-04-30 05:58:59 -04:00
Richard Henderson f35a83d5ff target/arm: Honor the HCR_EL2.TPCP bit
This bit traps EL1 access to cache maintenance insns that operate
to the point of coherency or persistence.

Backports commit 1bed4d2e55459129c19f5952bcfc65bd0c70db5b from qemu
2020-03-22 02:44:41 -04:00
Richard Henderson 8ff8ff0c4a target/arm: Honor the HCR_EL2.TACR bit
This bit traps EL1 access to the auxiliary control registers.

Backports commit 9960237769ada2faaaf1898b96da7a55e1691cf4 from qemu
2020-03-22 02:42:05 -04:00
Richard Henderson d252af2069 target/arm: Honor the HCR_EL2.TSW bit
These bits trap EL1 access to set/way cache maintenance insns.

Backports commit 1803d2713b29d85031cc964d545036bda9880f26 from qemu
2020-03-22 02:40:10 -04:00
Richard Henderson 7ee27e5d93 target/arm: Honor the HCR_EL2.{TVM,TRVM} bits
These bits trap EL1 access to various virtual memory controls.

Backports commit 84929218512c19ec9a296fbfd7b39219e0c592ae from qemu
2020-03-22 02:38:03 -04:00
Richard Henderson 7cd75be7c8 target/arm: Improve masking in arm_hcr_el2_eff
Update the {TGE,E2H} == '11' masking to ARMv8.6.
If EL2 is configured for aarch32, disable all of
the bits that are RES0 in aarch32 mode.

Backports commit 4990e1d3c128580dd2fa0bbb1a42b6d63ba1ac28 from qemu
2020-03-22 02:32:35 -04:00
Richard Henderson fa599a9538 target/arm: Add HCR_EL2 bit definitions from ARMv8.6
Backports commit e0a38bb35aa930c2d3b9982914297f0c0e8fd5c8 from qemu
2020-03-22 02:31:19 -04:00
Richard Henderson c4b2493c2e target/arm: Improve masking of HCR/HCR2 RES0 bits
Don't merely start with v8.0, handle v7VE as well. Ensure that writes
from aarch32 mode do not change bits in the other half of the register.
Protect reads of aa64 id registers with ARM_FEATURE_AARCH64.

Backports commit d1fb4da208411ce7b3dafb9f9e7726ebcec14edb from qemu
2020-03-22 02:28:41 -04:00
Peter Maydell 32b0e506e6 target/arm: Implement (trivially) ARMv8.2-TTCNP
The ARMv8.2-TTCNP extension allows an implementation to optimize by
sharing TLB entries between multiple cores, provided that software
declares that it's ready to deal with this by setting a CnP bit in
the TTBRn_ELx. It is mandatory from ARMv8.2 onward.

For QEMU's TLB implementation, sharing TLB entries between different
cores would not really benefit us and would be a lot of work to
implement. So we implement this extension in the "trivial" manner:
we allow the guest to set and read back the CnP bit, but don't change
our behaviour (this is an architecturally valid implementation
choice).

The only code path which looks at the TTBRn_ELx values for the
long-descriptor format where the CnP bit is defined is already doing
enough masking to not get confused when the CnP bit at the bottom of
the register is set, so we can simply add a comment noting why we're
relying on that mask.

Backports commit 41a4bf1feab098da4cd5495cd56a99b0339e2275 from qemu
2020-03-22 02:24:48 -04:00
Peter Maydell 7271ebf96d target/arm: Implement ARMv8.3-CCIDX
The ARMv8.3-CCIDX extension makes the CCSIDR_EL1 system ID registers
have a format that uses the full 64 bit width of the register, and
adds a new CCSIDR2 register so AArch32 can get at the high 32 bits.

QEMU doesn't implement caches, so we just treat these ID registers as
opaque values that are set to the correct constant values for each
CPU. The only thing we need to do is allow 64-bit values in our
cssidr[] array and provide the CCSIDR2 accessors.

We don't set the CCIDX field in our 'max' CPU because the CCSIDR
constant values we use are the same as the ones used by the
Cortex-A57 and they are in the old 32-bit format. This means
that the extra regdef added here is unused currently, but it
means that whenever in the future we add a CPU that does need
the new 64-bit format it will just work when we set the cssidr
values and the ID registers for it.

Backports commit 957e615503bd0de22393fd8dbcb22a5064fd2b5c from qemu
2020-03-22 00:17:37 -04:00
Peter Maydell 5416c5a672 target/arm: Implement v8.4-RCPC
The v8.4-RCPC extension implements some new instructions:
* LDAPUR, LDAPURB, LDAPURH, LDAPRSB, LDAPRSH, LDAPRSW
* STLUR, STLURB, STLURH

These are all in a new subgroup of encodings that sits below the
top-level "Loads and Stores" group in the Arm ARM.

The STLUR* instructions have standard store-release semantics; the
LDAPUR* have Load-AcquirePC semantics, but (as with LDAPR*) we choose
to implement them as the slightly stronger Load-Acquire.

Backports commit a1229109dec4375259d3fff99f362405aab7917a from qemu
2020-03-22 00:15:46 -04:00
Peter Maydell f72582bb7a target/arm: Implement v8.3-RCPC
The v8.3-RCPC extension implements three new load instructions
which provide slightly weaker consistency guarantees than the
existing load-acquire operations. For QEMU we choose to simply
implement them with a full LDAQ barrier.

Backports commit 2677cf9f92a5319bb995927f9225940414ce879d from qemu
2020-03-22 00:13:08 -04:00
Peter Maydell 8d12309fd8 target/arm: Fix wrong use of FIELD_EX32 on ID_AA64DFR0
We missed an instance of using FIELD_EX32 on a 64-bit ID
register, in isar_feature_aa64_pmu_8_4(). Fix it.

Backports commit 54117b90ffd8a3977917971c3bd99bb5242710d9 from qemu.
2020-03-22 00:10:52 -04:00
Richard Henderson c3eaaf7c33 target/arm: Split VMINMAXNM decode
Passing the raw op field from the manual is less instructive
than it might be. Do the full decode and use the existing
helpers to perform the expansion.

Since these are v8 insns, VECLEN+VECSTRIDE are already RES0.

Backports commit f2eafb75511e5d2ee601b43dc6ee0bcc6e453acd from qemu
2020-03-22 00:09:53 -04:00
Richard Henderson 303d922e5d target/arm: Split VFM decode
Passing the raw o1 and o2 fields from the manual is less
instructive than it might be. Do the full decode and let
the trans_* functions pass in booleans to a helper.

Backports commit d486f8308a13543bbcc4887f246e856df991a4bc from qemu
2020-03-22 00:07:53 -04:00
Richard Henderson a445b1dab9 target/arm: Add formats for some vfp 2 and 3-register insns
Those vfp instructions without extra opcode fields can
share a common @format for brevity.

Backports commit 906b60facc3d3dd3af56cb1a7860175d805e10a3 from qemu
2020-03-22 00:05:27 -04:00
Richard Henderson 3d2a091389 target/arm: Remove ARM_FEATURE_VFP*
We have converted all tests against these features
to ISAR tests.

Backports commit f9506e162c33e87b609549157dd8431fcc732085 from qemu
2020-03-22 00:02:13 -04:00
Richard Henderson 4ce91875e4 target/arm: Move the vfp decodetree calls next to the base isa
Have the calls adjacent as an intermediate step toward
actually merging the decodes.

Backports commit f0f6d5c81be47d593e5ece7f06df6fba4c15738b from qemu
2020-03-21 23:54:56 -04:00
Richard Henderson f1ce64857c target/arm: Move VLLDM and VLSTM to vfp.decode
Now that we no longer have an early check for ARM_FEATURE_VFP,
we can use the proper ISA check in trans_VLLDM_VLSTM.

Backports commit dc778a6873f534817a13257be2acba3ca87ec015 from qemu
2020-03-21 23:51:59 -04:00
Richard Henderson 7592564248 target/arm: Remove ARM_FEATURE_VFP check from disas_vfp_insn
We now have proper ISA checks within each trans_* function.

Backports commit 46c98019255b056f5dbc9676a6490951469ca661 from qemu
2020-03-21 23:49:14 -04:00
Richard Henderson 3f0ae7ccee target/arm: Replace ARM_FEATURE_VFP4 with isar_feature_aa32_simdfmac
All remaining tests for VFP4 are for fused multiply-add insns.

Since the MVFR1 field is used for both VFP and NEON, move its adjustment
from the !has_neon block to the (!has_vfp && !has_neon) block.

Test for vfp of the appropraite width alongside the test for simdfmac
within translate-vfp.inc.c. Within disas_neon_data_insn, we have
already tested for ARM_FEATURE_NEON.

Backports commit c52881bbc22b50db99a6c37171ad3eea7d959ae6 from qemu
2020-03-21 23:48:13 -04:00
Richard Henderson f6b5a9ef81 target/arm: Add missing checks for fpsp_v2
We will eventually remove the early ARM_FEATURE_VFP test,
so add a proper test for each trans_* that does not already
have another ISA test.

Backports commit 82f6abe16b9b951180657c5fe15942d5214aa12e from qemu
2020-03-21 23:42:27 -04:00
Richard Henderson ed1ce1437a target/arm: Replace ARM_FEATURE_VFP3 checks with fp{sp, dp}_v3
Sort this check to the start of a trans_* function.
Merge this with any existing test for fpdp_v2.

Backports commit 84774cc37f2c17e48a4867a8e8e055deb23bea69 from qemu
2020-03-21 23:33:13 -04:00
Richard Henderson 54e9ce5174 target/arm: Perform fpdp_v2 check first
Shuffle the order of the checks so that we test the ISA
before we test anything else, such as the register arguments.

Backports commit 799449abda137153a0e68b8788d8e1486f389490 from qemu
2020-03-21 23:29:08 -04:00
Richard Henderson 7e99995b7b target/arm: Add isar_feature_aa64_fp_simd, isar_feature_aa32_vfp
We cannot easily create "any" functions for these, because the
ID_AA64PFR0 fields for FP and SIMD signal "enabled" with zero.
Which means that an aarch32-only cpu will return incorrect results
when testing the aarch64 registers.

To use these, we must either have context or additionally test
vs ARM_FEATURE_AARCH64.

Backports commit 7d63183ff1a61b3f7934dc9b40b10e4fd5e100cd from qemu
2020-03-21 23:19:14 -04:00
Richard Henderson 39c49f488b target/arm: Add isar_feature_aa32_{fpsp_v2, fpsp_v3, fpdp_v3}
We will shortly use these to test for VFPv2 and VFPv3
in different situations.

Backports commit f67957e17cbf8fc3cc5d1146a2db2023404578b0 from qemu
2020-03-21 23:17:19 -04:00
Richard Henderson f73b360f8e target/arm: Rename isar_feature_aa32_fpdp_v2
The old name, isar_feature_aa32_fpdp, does not reflect
that the test includes VFPv2. We will introduce another
feature tests for VFPv3.

Backports commit c4ff873583834c8275586914fff714e3ae65dee4 from qemu
2020-03-21 23:16:00 -04:00
Richard Henderson 06b52d6660 target/arm: Add isar_feature_aa32_vfp_simd
Use this in the places that were checking ARM_FEATURE_VFP, and
are obviously testing for the existance of the register set
as opposed to testing for some particular instruction extension.

Backports commit 7fbc6a403a0aab834e764fa61d81ed8586cfe352 from qemu
2020-03-21 23:11:36 -04:00
Richard Henderson 764207865a target/arm: Set ID_MMFR4.HPDS for aarch64_max_initfn
We had set this for aarch32-only in arm_max_initfn, but
failed to set the same bit for aarch64.

Backports commit dac65ba1d7945c5d58ab63d8769103634adb2b01 from qemu
2020-03-21 19:45:09 -04:00
Richard Henderson 7c799d29db target/arm: Set MVFR0.FPSP for ARMv5 cpus
We are going to convert FEATURE tests to ISAR tests,
so FPSP needs to be set for these cpus, like we have
already for FPDP.

Backports commit 9eb4f58918a851fb46895fd9b7ce579afeac9d02 from qemu
2020-03-21 19:41:18 -04:00
Richard Henderson 833de589ed target/arm: Use isar_feature_aa32_simd_r32 more places
Many uses of ARM_FEATURE_VFP3 are testing for the number of simd
registers implemented. Use the proper test vs MVFR0.SIMDReg.

Backports commit a6627f5fc607939f7c8b9c3157fdcb2d368ba0ed from qemu
2020-03-21 19:39:35 -04:00
Richard Henderson c06fd38b57 target/arm: Rename isar_feature_aa32_simd_r32
The old name, isar_feature_aa32_fp_d32, does not reflect
the MVFR0 field name, SIMDReg.

Backports commit 0e13ba7889432c5e2f1bdb1b25e7076ca1b1dcba from qemu
2020-03-21 19:37:33 -04:00
Richard Henderson fcce8d4aa1 target/arm: Convert PMULL.8 to gvec
We still need two different helpers, since NEON and SVE2 get the
inputs from different locations within the source vector. However,
we can convert both to the same internal form for computation.

The sve2 helper is not used yet, but adding it with this patch
helps illustrate why the neon changes are helpful.

Backports commit e7e96fc5ec8c79dc77fef522d5226ac09f684ba5 from qemu
2020-03-21 19:35:46 -04:00
Richard Henderson c00f72f74f target/arm: Convert PMULL.64 to gvec
The gvec form will be needed for implementing SVE2.

Backports commit b9ed510e46f2f9e31e5e8adb4661d5d1cbe9a459 from qemu
2020-03-21 19:27:38 -04:00
Richard Henderson db8a935b44 target/arm: Convert PMUL.8 to gvec
The gvec form will be needed for implementing SVE2.

Extend the implementation to operate on uint64_t instead of uint32_t.
Use a counted inner loop instead of terminating when op1 goes to zero,
looking toward the required implementation for ARMv8.4-DIT.

Backports commit a21bb78e5817be3f494922e1dadd6455fe5d6318 from qemu
2020-03-21 19:22:18 -04:00
Richard Henderson d3139f2f0a target/arm: Vectorize USHL and SSHL
These instructions shift left or right depending on the sign
of the input, and 7 bits are significant to the shift. This
requires several masks and selects in addition to the actual
shifts to form the complete answer.

That said, the operation is still a small improvement even for
two 64-bit elements -- 13 vector operations instead of 2 * 7
integer operations.

Backports commit 87b74e8b6edd287ea2160caa0ebea725fa8f1ca1 from qemu
2020-03-21 19:14:17 -04:00
Peter Maydell 61cf5abc9e target/arm: Correctly implement ACTLR2, HACTLR2
The ACTLR2 and HACTLR2 AArch32 system registers didn't exist in ARMv7
or the original ARMv8. They were later added as optional registers,
whose presence is signaled by the ID_MMFR4.AC2 field. From ARMv8.2
they are mandatory (ie ID_MMFR4.AC2 must be non-zero).

We implemented HACTLR2 in commit 0e0456ab8895a5e85, but we
incorrectly made it exist for all v8 CPUs, and we didn't implement
ACTLR2 at all.

Sort this out by implementing both registers only when they are
supposed to exist, and setting the ID_MMFR4 bit for -cpu max.

Note that this removes HACTLR2 from our Cortex-A53, -A47 and -A72
CPU models; this is correct, because those CPUs do not implement
this register.

Fixes: 0e0456ab8895a5e85

Backports commit f6287c24c66d6b9187c1c2887e1c7cfa4d304b0c from qemu
2020-03-21 18:52:30 -04:00
Peter Maydell 1876feeede target/arm: Use FIELD_EX32 for testing 32-bit fields
Cut-and-paste errors mean we're using FIELD_EX64() to extract fields from
some 32-bit ID register fields. Use FIELD_EX32() instead. (This makes
no difference in behaviour, it's just more consistent.)

Backports commit b3a816f6ce1ec184ab6072f50bbe4479fc5116c3 from qemu
2020-03-21 18:50:14 -04:00
Peter Maydell 2ce106df33 target/arm: Use isar_feature function for testing AA32HPD feature
Now we have moved ID_MMFR4 into the ARMISARegisters struct, we
can define and use an isar_feature for the presence of the
ARMv8.2-AA32HPD feature, rather than open-coding the test.

While we're here, correct a comment typo which missed an 'A'
from the feature name.

Backports commit 4036b7d1cd9fb1097a5f4bc24d7d31744256260f from qemu
2020-03-21 18:48:57 -04:00
Peter Maydell 4693b2c011 target/arm: Test correct register in aa32_pan and aa32_ats1e1 checks
The isar_feature_aa32_pan and isar_feature_aa32_ats1e1 functions
are supposed to be testing fields in ID_MMFR3; but a cut-and-paste
error meant we were looking at MVFR0 instead.

Fix the functions to look at the right register; this requires
us to move at least id_mmfr3 to the ARMISARegisters struct; we
choose to move all the ID_MMFRn registers for consistency.

Backports commit 10054016eda1b13bdd8340d100fd029cc8b58f36 from qemu
2020-03-21 18:47:12 -04:00
Peter Maydell e72fa1cb33 target/arm: Correct handling of PMCR_EL0.LC bit
The LC bit in the PMCR_EL0 register is supposed to be:
* read/write
* RES1 on an AArch64-only implementation
* an architecturally UNKNOWN value on reset
(and use of LC==0 by software is deprecated).

We were implementing it incorrectly as read-only always zero,
though we do have all the code needed to test it and behave
accordingly.

Instead make it a read-write bit which resets to 1 always, which
satisfies all the architectural requirements above.

Backports commit 62d96ff48510f4bf648ad12f5d3a5507227b026f from qemu
2020-03-21 18:40:26 -04:00
Peter Maydell de428e4b45 target/arm: Correct definition of PMCRDP
The PMCR_EL0.DP bit is bit 5, which is 0x20, not 0x10. 0x10 is 'X'.
Correct our #define of PMCRDP and add the missing PMCRX.

We do have the correct behaviour for handling the DP bit being
set, so this fixes a guest-visible bug.

Fixes: 033614c47de

Backports commit a1ed04dd79aabb9dbeeb5fa7d49f1a3de0357553 from qemu
2020-03-21 18:39:37 -04:00
Peter Maydell 28b239adb9 target/arm: Provide ARMv8.4-PMU in '-cpu max'
Set the ID register bits to provide ARMv8.4-PMU (and implicitly
also ARMv8.1-PMU) in the 'max' CPU.

Backports commit 3bec78447a958d4819911252e056f29740ac25e4 from qemu
2020-03-21 18:38:53 -04:00
Peter Maydell 4dd57f7acc target/arm: Implement ARMv8.4-PMU extension
The ARMv8.4-PMU extension adds:
* one new required event, STALL
* one new system register PMMIR_EL1

(There are also some more L1-cache related events, but since
we don't implement any cache we don't provide these, in the
same way we don't provide the base-PMUv3 cache events.)

The STALL event "counts every attributable cycle on which no
attributable instruction or operation was sent for execution on this
PE". QEMU doesn't stall in this sense, so this is another
always-reads-zero event.

The PMMIR_EL1 register is a read-only register providing
implementation-specific information about the PMU; currently it has
only one field, SLOTS, which defines behaviour of the STALL_SLOT PMU
event. Since QEMU doesn't implement the STALL_SLOT event, we can
validly make the register read zero.

Backports commit 15dd1ebda4a6ef928d484c5a4f48b8ccb7438bb2 from qemu
2020-03-21 18:37:50 -04:00
Peter Maydell 5c93f43eb9 target/arm: Implement ARMv8.1-PMU extension
The ARMv8.1-PMU extension requires:
* the evtCount field in PMETYPER<n>_EL0 is 16 bits, not 10
* MDCR_EL2.HPMD allows event counting to be disabled at EL2
* two new required events, STALL_FRONTEND and STALL_BACKEND
* ID register bits in ID_AA64DFR0_EL1 and ID_DFR0

We already implement the 16-bit evtCount field and the
HPMD bit, so all that is missing is the two new events:
STALL_FRONTEND
"counts every cycle counted by the CPU_CYCLES event on which no
operation was issued because there are no operations available
to issue to this PE from the frontend"
STALL_BACKEND
"counts every cycle counted by the CPU_CYCLES event on which no
operation was issued because the backend is unable to accept
any available operations from the frontend"

QEMU never stalls in this sense, so our implementation is trivial:
always return a zero count.

Backports commit 0727f63b1ecf765ebc48266f616f8fc362dc7fbc from qemu
2020-03-21 18:34:33 -04:00
Peter Maydell 7dfc30b754 target/arm: Read debug-related ID registers from KVM
Backports 1548a7b2ad621a31b4216ed703b6d658a2ecf0d0 from qemu
2020-03-21 18:30:20 -04:00
Peter Maydell cef6f3e72c target/arm: Move DBGDIDR into ARMISARegisters
We're going to want to read the DBGDIDR register from KVM in
a subsequent commit, which means it needs to be in the
ARMISARegisters sub-struct. Move it.

Backports commit 4426d3617d64922d97b74ed22e67e33b6fb7de0a from qemu
2020-03-21 18:29:01 -04:00
Peter Maydell a6c9c87a5d target/arm: Stop assuming DBGDIDR always exists
The AArch32 DBGDIDR defines properties like the number of
breakpoints, watchpoints and context-matching comparators. On an
AArch64 CPU, the register may not even exist if AArch32 is not
supported at EL1.

Currently we hard-code use of DBGDIDR to identify the number of
breakpoints etc; this works for all our TCG CPUs, but will break if
we ever add an AArch64-only CPU. We also have an assert() that the
AArch32 and AArch64 registers match, which currently works only by
luck for KVM because we don't populate either of these ID registers
from the KVM vCPU and so they are both zero.

Clean this up so we have functions for finding the number
of breakpoints, watchpoints and context comparators which look
in the appropriate ID register.

This allows us to drop the "check that AArch64 and AArch32 agree
on the number of breakpoints etc" asserts:
* we no longer look at the AArch32 versions unless that's the
right place to be looking
* it's valid to have a CPU (eg AArch64-only) where they don't match
* we shouldn't have been asserting the validity of ID registers
in a codepath used with KVM anyway

Backports commit 88ce6c6ee85d902f59dc65afc3ca86b34f02b9ed from qemu
2020-03-21 18:26:24 -04:00
Peter Maydell afc28d9b2c target/arm: Add _aa64_ and _any_ versions of pmu_8_1 isar checks
Add the 64-bit version of the "is this a v8.1 PMUv3?"
ID register check function, and the _any_ version that
checks for either AArch32 or AArch64 support. We'll use
this in a later commit.

We don't (yet) do any isar_feature checks on ID_AA64DFR1_EL1,
but we move id_aa64dfr1 into the ARMISARegisters struct with
id_aa64dfr0, for consistency.

Backports commit 2a609df87d9b886fd38a190a754dbc241ff707e8 from qemu
2020-03-21 18:24:00 -04:00
Peter Maydell e64143966a target/arm: Define an aa32_pmu_8_1 isar feature test function
Instead of open-coding a check on the ID_DFR0 PerfMon ID register
field, create a standardly-named isar_feature for "does AArch32 have
a v8.1 PMUv3" and use it.

This entails moving the id_dfr0 field into the ARMISARegisters struct.

Backports commit a617953855b65a602d36364b9643f7e5bc31288e from qemu
2020-03-21 18:21:26 -04:00
Peter Maydell fd537585d7 target/arm: Use FIELD macros for clearing ID_DFR0 PERFMON field
We already define FIELD macros for ID_DFR0, so use them in the
one place where we're doing direct bit value manipulation.

Backports commit d52c061e541982a3663ad5c65bd3b518dbe85b87 from qemu
2020-03-21 18:17:55 -04:00
Peter Maydell fd6c635e03 target/arm: Add and use FIELD definitions for ID_AA64DFR0_EL1
Add FIELD() definitions for the ID_AA64DFR0_EL1 and use them
where we currently have hard-coded bit values.

Backports commit ceb2744b47a1ef4184dca56a158eb3156b6eba36 from qemu
2020-03-21 18:16:55 -04:00
Peter Maydell ebd7131c16 target/arm: Factor out PMU register definitions
Pull the code that defines the various PMU registers out
into its own function, matching the pattern we have
already for the debug registers.

Apart from one style fix to a multi-line comment, this
is purely movement of code with no changes to it.

Backports commit 24183fb6f00ecca8b508e245c95ff50ddde3f18b from qemu
2020-03-21 18:15:09 -04:00
Peter Maydell b1c088e2f2 target/arm: Define and use any_predinv isar_feature test
Instead of open-coding "ARM_FEATURE_AARCH64 ? aa64_predinv: aa32_predinv",
define and use an any_predinv isar_feature test function.

Backports commit 22e570730d15374453baa73ff2a699e01ef4e950 from qemu
2020-03-21 18:13:25 -04:00
Peter Maydell 62178626e4 target/arm: Add isar_feature_any_fp16 and document naming/usage conventions
Our current usage of the isar_feature feature tests almost always
uses an _aa32_ test when the code path is known to be AArch32
specific and an _aa64_ test when the code path is known to be
AArch64 specific. There is just one exception: in the vfp_set_fpscr
helper we check aa64_fp16 to determine whether the FZ16 bit in
the FP(S)CR exists, but this code is also used for AArch32.
There are other places in future where we're likely to want
a general "does this feature exist for either AArch32 or
AArch64" check (typically where architecturally the feature exists
for both CPU states if it exists at all, but the CPU might be
AArch32-only or AArch64-only, and so only have one set of ID
registers).

Introduce a new category of isar_feature_* functions:
isar_feature_any_foo() should be tested when what we want to
know is "does this feature exist for either AArch32 or AArch64",
and always returns the logical OR of isar_feature_aa32_foo()
and isar_feature_aa64_foo().

Backports commit 6e61f8391cc6cb0846d4bf078dbd935c2aeebff5 from qemu
2020-03-21 18:12:02 -04:00
Peter Maydell 778fcd9562 target/arm: Check aa32_pan in take_aarch32_exception(), not aa64_pan
In take_aarch32_exception(), we know we are dealing with a CPU that
has AArch32, so the right isar_feature test is aa32_pan, not aa64_pan.

Backports commit f8af1143ef93954e77cf59e09b5e004dafbd64fd from qemu
2020-03-21 18:09:27 -04:00
Peter Maydell e63f70f980 target/arm: Add _aa32_ to isar_feature functions testing 32-bit ID registers
Enforce a convention that an isar_feature function that tests a
32-bit ID register always has _aa32_ in its name, and one that
tests a 64-bit ID register always has _aa64_ in its name.
We already follow this except for three cases: thumb_div,
arm_div and jazelle, which all need _aa32_ adding.

(As noted in the comment, isar_feature_aa32_fp16_arith()
is an exception in that it currently tests ID_AA64PFR0_EL1,
but will switch to MVFR1 once we've properly implemented
FP16 for AArch32.)

Backports commit 873b73c0c891ec20adacc7bd1ae789294334d675 from qemu
2020-03-21 18:08:23 -04:00
Richard Henderson 0131e804fb target/arm: Split out aa64_va_parameter_tbi, aa64_va_parameter_tbid
For the purpose of rebuild_hflags_a64, we do not need to compute
all of the va parameters, only tbi. Moreover, we can compute them
in a form that is more useful to storing in hflags.

This eliminates the need for aa64_va_parameter_both, so fold that
in to aa64_va_parameter. The remaining calls to aa64_va_parameter
are in get_phys_addr_lpae and in pauth_helper.c.

This reduces the total cpu consumption of aa64_va_parameter in a
kernel boot plus a kvm guest kernel boot from 3% to 0.5%.

Backports commit b830a5ee82e66f54697dcc6450fe9239b7412d13 from qemu
2020-03-21 18:04:39 -04:00
Richard Henderson 2cce7e0dd0 target/arm: Remove ttbr1_valid check from get_phys_addr_lpae
Now that aa64_va_parameters_both sets select based on the number
of ranges in the regime, the ttbr1_valid check is redundant.

Backports commit 03f27724dff15633911e68a3906c30f57938ea45 from qemu
2020-03-21 18:01:24 -04:00
Richard Henderson f3fa39829d target/arm: Fix select for aa64_va_parameters_both
Select should always be 0 for a regime with one range.

Backports commit 71d181640a1a9470f074fa28600ca85587e2ca6b from qemu
2020-03-21 18:00:15 -04:00
Richard Henderson 3183349f1c target/arm: Use bit 55 explicitly for pauth
The psuedocode in aarch64/functions/pac/auth/Auth and
aarch64/functions/pac/strip/Strip always uses bit 55 for
extfield and do not consider if the current regime has 2 ranges.

Backports commit 7eeb4c2ce8dc0a5655526f3f39bd5d6cc02efb39 from qemu
2020-03-21 17:59:06 -04:00
Richard Henderson 51b6064ba4 target/arm: Flush high bits of sve register after AdvSIMD INS
Writes to AdvSIMD registers flush the bits above 128.

Backports commit 528dc354b6f3aa82d65141cc60bc0e725e6cae98 from qemu
2020-03-21 17:58:09 -04:00
Richard Henderson 74cbfceb56 target/arm: Flush high bits of sve register after AdvSIMD ZIP/UZP/TRN
Writes to AdvSIMD registers flush the bits above 128.

Backports commit 33649de62e40df0060a1c514574e4ef25c4e52e1 from qemu
2020-03-21 17:56:40 -04:00
Richard Henderson 6eb8472344 target/arm: Flush high bits of sve register after AdvSIMD TBL/TBX
Writes to AdvSIMD registers flush the bits above 128.

Backports commit 263273bc988e677ebadeaf7d0e49f6792a112db5 from qemu
2020-03-21 17:56:08 -04:00
Richard Henderson 18e9c4805f target/arm: Flush high bits of sve register after AdvSIMD EXT
Writes to AdvSIMD registers flush the bits above 128.

Backports commit 78cedfabd53b6f64e7e64fc84878d848e5df1d08 from qemu
2020-03-21 17:55:12 -04:00
Peter Maydell 96a96565db target/arm: Implement ARMv8.1-VMID16 extension
The ARMv8.1-VMID16 extension extends the VMID from 8 bits to 16 bits:

* the ID_AA64MMFR1_EL1.VMIDBits field specifies whether the VMID is
8 or 16 bits
* the VMID field in VTTBR_EL2 is extended to 16 bits
* VTCR_EL2.VS lets the guest specify whether to use the full 16 bits,
or use the backwards-compatible 8 bits

For QEMU implementing this is trivial:
* we do not track VMIDs in TLB entries, so we never use the VMID field
* we treat any write to VTTBR_EL2, not just a change to the VMID field
bits, as a "possible VMID change" that causes us to throw away TLB
entries, so that code doesn't need changing
* we allow the guest to read/write the VTCR_EL2.VS bit already

So all that's missing is the ID register part: report that we support
VMID16 in our 'max' CPU.

Backports commit dc7a88d0810ad272bdcd2e0869359af78fdd9114 from qemu
2020-03-21 17:52:43 -04:00
Richard Henderson 57f0aa3044 target/arm: Enable ARMv8.2-UAO in -cpu max
Backports commit e11f0eb6724571adb812a3ce5269c41586e0262b from qemu
2020-03-21 17:51:44 -04:00
Richard Henderson 18a86780ee target/arm: Implement UAO semantics
We need only override the current condition under which
TBFLAG_A64.UNPRIV is set.

Backports commit 7a8014ab871d5320effd737dfe88b2e80f16a509 from qemu
2020-03-21 17:50:29 -04:00
Richard Henderson 5b5050c6ca target/arm: Update MSR access to UAO
Backports commit 9eeb7a1c9531cb3574bfe2c36eb7624802c3ec00 from qemu
2020-03-21 17:48:01 -04:00
Richard Henderson 0630e66b5a target/arm: Add ID_AA64MMFR2_EL1
Add definitions for all of the fields, up to ARMv8.5.
Convert the existing RESERVED register to a full register.
Query KVM for the value of the register for the host.

Backports commit 64761e10af2742a916c08271828890274137b9e8 from qemu
2020-03-21 17:45:27 -04:00
Richard Henderson 7287bf16b8 target/arm: Enable ARMv8.2-ATS1E1 in -cpu max
This includes enablement of ARMv8.1-PAN.

Backports commit e0fe7309a7c21ef2386de50d37c86aea0d671c08 from qemu
2020-03-21 17:43:54 -04:00