Commit graph

2000 commits

Author SHA1 Message Date
Alistair Francis e4e41d3d87 target/riscv: Fix CSR perm checking for HS mode
Update the CSR permission checking to work correctly when we are in
HS-mode.

Backports commit 0a42f4c4408824dc7cb9ff60c9bdce6dcc0d24a5 from qemu
2020-03-22 01:19:04 -04:00
Alistair Francis ab7c1570d8 target/riscv: Add the force HS exception mode
Add a FORCE_HS_EXCEP mode to the RISC-V virtulisation status. This bit
specifies if an exeption should be taken to HS mode no matter the
current delegation status. This is used when an exeption must be taken
to HS mode, such as when handling interrupts.

Backports commit c7b1bbc80fc2af17395d3986c346fd2307e57829 from qemu
2020-03-22 01:16:55 -04:00
Alistair Francis 95537388c5 target/riscv: Add the virtulisation mode
Backports commit ef6bb7b62682badefdcb744831510aaa5971684f from qemu
2020-03-22 01:15:06 -04:00
Alistair Francis fa04e7ea45 target/riscv: Rename the H irqs to VS irqs
Backports commit 205377f8940898e4c53d1b44350a3d4934a2da72 from qemu
2020-03-22 01:09:04 -04:00
Alistair Francis 0198a09698 target/riscv: Add support for the new execption numbers
The v0.5 Hypervisor spec add new execption numbers, let's add support
for those.

Backports commit ab67a1d07a4f6f1b4d577c5c47013273b9804551 from qemu
2020-03-22 01:07:23 -04:00
Alistair Francis 870603305d target/riscv: Add the Hypervisor CSRs to CPUState
Add the Hypervisor CSRs to CPUState and at the same time (to avoid
bisect issues) update the CSR macros for the v0.5 Hyp spec.

Backports commit bd023ce33b85d73791b7bc78fd04a8115c60995e from qemu
2020-03-22 01:05:23 -04:00
Alistair Francis 40afe12008 target/riscv: Add the Hypervisor extension
Backports commit af1fa0039c799a350bcde07b3d8a71dfde07d11b from qemu
2020-03-22 01:01:07 -04:00
Alistair Francis 83ac8747a5 target/riscv: Convert MIP CSR to target_ulong
The MIP CSR is a xlen CSR, it was only 32-bits to allow atomic access.
Now that we don't use atomics for MIP we can change this back to a xlen
CSR.

Backports commit 028616130d5f0abc8a3b96f28963da51a875024b from qemu
2020-03-22 00:59:47 -04:00
Peter Maydell 7271ebf96d target/arm: Implement ARMv8.3-CCIDX
The ARMv8.3-CCIDX extension makes the CCSIDR_EL1 system ID registers
have a format that uses the full 64 bit width of the register, and
adds a new CCSIDR2 register so AArch32 can get at the high 32 bits.

QEMU doesn't implement caches, so we just treat these ID registers as
opaque values that are set to the correct constant values for each
CPU. The only thing we need to do is allow 64-bit values in our
cssidr[] array and provide the CCSIDR2 accessors.

We don't set the CCIDX field in our 'max' CPU because the CCSIDR
constant values we use are the same as the ones used by the
Cortex-A57 and they are in the old 32-bit format. This means
that the extra regdef added here is unused currently, but it
means that whenever in the future we add a CPU that does need
the new 64-bit format it will just work when we set the cssidr
values and the ID registers for it.

Backports commit 957e615503bd0de22393fd8dbcb22a5064fd2b5c from qemu
2020-03-22 00:17:37 -04:00
Peter Maydell 5416c5a672 target/arm: Implement v8.4-RCPC
The v8.4-RCPC extension implements some new instructions:
* LDAPUR, LDAPURB, LDAPURH, LDAPRSB, LDAPRSH, LDAPRSW
* STLUR, STLURB, STLURH

These are all in a new subgroup of encodings that sits below the
top-level "Loads and Stores" group in the Arm ARM.

The STLUR* instructions have standard store-release semantics; the
LDAPUR* have Load-AcquirePC semantics, but (as with LDAPR*) we choose
to implement them as the slightly stronger Load-Acquire.

Backports commit a1229109dec4375259d3fff99f362405aab7917a from qemu
2020-03-22 00:15:46 -04:00
Peter Maydell f72582bb7a target/arm: Implement v8.3-RCPC
The v8.3-RCPC extension implements three new load instructions
which provide slightly weaker consistency guarantees than the
existing load-acquire operations. For QEMU we choose to simply
implement them with a full LDAQ barrier.

Backports commit 2677cf9f92a5319bb995927f9225940414ce879d from qemu
2020-03-22 00:13:08 -04:00
Peter Maydell 8d12309fd8 target/arm: Fix wrong use of FIELD_EX32 on ID_AA64DFR0
We missed an instance of using FIELD_EX32 on a 64-bit ID
register, in isar_feature_aa64_pmu_8_4(). Fix it.

Backports commit 54117b90ffd8a3977917971c3bd99bb5242710d9 from qemu.
2020-03-22 00:10:52 -04:00
Richard Henderson c3eaaf7c33 target/arm: Split VMINMAXNM decode
Passing the raw op field from the manual is less instructive
than it might be. Do the full decode and use the existing
helpers to perform the expansion.

Since these are v8 insns, VECLEN+VECSTRIDE are already RES0.

Backports commit f2eafb75511e5d2ee601b43dc6ee0bcc6e453acd from qemu
2020-03-22 00:09:53 -04:00
Richard Henderson 303d922e5d target/arm: Split VFM decode
Passing the raw o1 and o2 fields from the manual is less
instructive than it might be. Do the full decode and let
the trans_* functions pass in booleans to a helper.

Backports commit d486f8308a13543bbcc4887f246e856df991a4bc from qemu
2020-03-22 00:07:53 -04:00
Richard Henderson a445b1dab9 target/arm: Add formats for some vfp 2 and 3-register insns
Those vfp instructions without extra opcode fields can
share a common @format for brevity.

Backports commit 906b60facc3d3dd3af56cb1a7860175d805e10a3 from qemu
2020-03-22 00:05:27 -04:00
Richard Henderson 3d2a091389 target/arm: Remove ARM_FEATURE_VFP*
We have converted all tests against these features
to ISAR tests.

Backports commit f9506e162c33e87b609549157dd8431fcc732085 from qemu
2020-03-22 00:02:13 -04:00
Richard Henderson 4ce91875e4 target/arm: Move the vfp decodetree calls next to the base isa
Have the calls adjacent as an intermediate step toward
actually merging the decodes.

Backports commit f0f6d5c81be47d593e5ece7f06df6fba4c15738b from qemu
2020-03-21 23:54:56 -04:00
Richard Henderson f1ce64857c target/arm: Move VLLDM and VLSTM to vfp.decode
Now that we no longer have an early check for ARM_FEATURE_VFP,
we can use the proper ISA check in trans_VLLDM_VLSTM.

Backports commit dc778a6873f534817a13257be2acba3ca87ec015 from qemu
2020-03-21 23:51:59 -04:00
Richard Henderson 7592564248 target/arm: Remove ARM_FEATURE_VFP check from disas_vfp_insn
We now have proper ISA checks within each trans_* function.

Backports commit 46c98019255b056f5dbc9676a6490951469ca661 from qemu
2020-03-21 23:49:14 -04:00
Richard Henderson 3f0ae7ccee target/arm: Replace ARM_FEATURE_VFP4 with isar_feature_aa32_simdfmac
All remaining tests for VFP4 are for fused multiply-add insns.

Since the MVFR1 field is used for both VFP and NEON, move its adjustment
from the !has_neon block to the (!has_vfp && !has_neon) block.

Test for vfp of the appropraite width alongside the test for simdfmac
within translate-vfp.inc.c. Within disas_neon_data_insn, we have
already tested for ARM_FEATURE_NEON.

Backports commit c52881bbc22b50db99a6c37171ad3eea7d959ae6 from qemu
2020-03-21 23:48:13 -04:00
Richard Henderson f6b5a9ef81 target/arm: Add missing checks for fpsp_v2
We will eventually remove the early ARM_FEATURE_VFP test,
so add a proper test for each trans_* that does not already
have another ISA test.

Backports commit 82f6abe16b9b951180657c5fe15942d5214aa12e from qemu
2020-03-21 23:42:27 -04:00
Richard Henderson ed1ce1437a target/arm: Replace ARM_FEATURE_VFP3 checks with fp{sp, dp}_v3
Sort this check to the start of a trans_* function.
Merge this with any existing test for fpdp_v2.

Backports commit 84774cc37f2c17e48a4867a8e8e055deb23bea69 from qemu
2020-03-21 23:33:13 -04:00
Richard Henderson 54e9ce5174 target/arm: Perform fpdp_v2 check first
Shuffle the order of the checks so that we test the ISA
before we test anything else, such as the register arguments.

Backports commit 799449abda137153a0e68b8788d8e1486f389490 from qemu
2020-03-21 23:29:08 -04:00
Richard Henderson 7e99995b7b target/arm: Add isar_feature_aa64_fp_simd, isar_feature_aa32_vfp
We cannot easily create "any" functions for these, because the
ID_AA64PFR0 fields for FP and SIMD signal "enabled" with zero.
Which means that an aarch32-only cpu will return incorrect results
when testing the aarch64 registers.

To use these, we must either have context or additionally test
vs ARM_FEATURE_AARCH64.

Backports commit 7d63183ff1a61b3f7934dc9b40b10e4fd5e100cd from qemu
2020-03-21 23:19:14 -04:00
Richard Henderson 39c49f488b target/arm: Add isar_feature_aa32_{fpsp_v2, fpsp_v3, fpdp_v3}
We will shortly use these to test for VFPv2 and VFPv3
in different situations.

Backports commit f67957e17cbf8fc3cc5d1146a2db2023404578b0 from qemu
2020-03-21 23:17:19 -04:00
Richard Henderson f73b360f8e target/arm: Rename isar_feature_aa32_fpdp_v2
The old name, isar_feature_aa32_fpdp, does not reflect
that the test includes VFPv2. We will introduce another
feature tests for VFPv3.

Backports commit c4ff873583834c8275586914fff714e3ae65dee4 from qemu
2020-03-21 23:16:00 -04:00
Richard Henderson 06b52d6660 target/arm: Add isar_feature_aa32_vfp_simd
Use this in the places that were checking ARM_FEATURE_VFP, and
are obviously testing for the existance of the register set
as opposed to testing for some particular instruction extension.

Backports commit 7fbc6a403a0aab834e764fa61d81ed8586cfe352 from qemu
2020-03-21 23:11:36 -04:00
Richard Henderson 764207865a target/arm: Set ID_MMFR4.HPDS for aarch64_max_initfn
We had set this for aarch32-only in arm_max_initfn, but
failed to set the same bit for aarch64.

Backports commit dac65ba1d7945c5d58ab63d8769103634adb2b01 from qemu
2020-03-21 19:45:09 -04:00
Paolo Bonzini 583cc7b8b1 target/i386: check for empty register in FXAM
The fxam instruction returns the wrong result after fdecstp or after
an underflow. Check fptags to handle this.

Backports commit 93c3593ad04f2610fd0a176dfa89a7e40b6afe1f from qemu
2020-03-21 19:43:24 -04:00
Richard Henderson 7c799d29db target/arm: Set MVFR0.FPSP for ARMv5 cpus
We are going to convert FEATURE tests to ISAR tests,
so FPSP needs to be set for these cpus, like we have
already for FPDP.

Backports commit 9eb4f58918a851fb46895fd9b7ce579afeac9d02 from qemu
2020-03-21 19:41:18 -04:00
Richard Henderson 833de589ed target/arm: Use isar_feature_aa32_simd_r32 more places
Many uses of ARM_FEATURE_VFP3 are testing for the number of simd
registers implemented. Use the proper test vs MVFR0.SIMDReg.

Backports commit a6627f5fc607939f7c8b9c3157fdcb2d368ba0ed from qemu
2020-03-21 19:39:35 -04:00
Richard Henderson c06fd38b57 target/arm: Rename isar_feature_aa32_simd_r32
The old name, isar_feature_aa32_fp_d32, does not reflect
the MVFR0 field name, SIMDReg.

Backports commit 0e13ba7889432c5e2f1bdb1b25e7076ca1b1dcba from qemu
2020-03-21 19:37:33 -04:00
Richard Henderson fcce8d4aa1 target/arm: Convert PMULL.8 to gvec
We still need two different helpers, since NEON and SVE2 get the
inputs from different locations within the source vector. However,
we can convert both to the same internal form for computation.

The sve2 helper is not used yet, but adding it with this patch
helps illustrate why the neon changes are helpful.

Backports commit e7e96fc5ec8c79dc77fef522d5226ac09f684ba5 from qemu
2020-03-21 19:35:46 -04:00
Richard Henderson c00f72f74f target/arm: Convert PMULL.64 to gvec
The gvec form will be needed for implementing SVE2.

Backports commit b9ed510e46f2f9e31e5e8adb4661d5d1cbe9a459 from qemu
2020-03-21 19:27:38 -04:00
Richard Henderson db8a935b44 target/arm: Convert PMUL.8 to gvec
The gvec form will be needed for implementing SVE2.

Extend the implementation to operate on uint64_t instead of uint32_t.
Use a counted inner loop instead of terminating when op1 goes to zero,
looking toward the required implementation for ARMv8.4-DIT.

Backports commit a21bb78e5817be3f494922e1dadd6455fe5d6318 from qemu
2020-03-21 19:22:18 -04:00
Richard Henderson d3139f2f0a target/arm: Vectorize USHL and SSHL
These instructions shift left or right depending on the sign
of the input, and 7 bits are significant to the shift. This
requires several masks and selects in addition to the actual
shifts to form the complete answer.

That said, the operation is still a small improvement even for
two 64-bit elements -- 13 vector operations instead of 2 * 7
integer operations.

Backports commit 87b74e8b6edd287ea2160caa0ebea725fa8f1ca1 from qemu
2020-03-21 19:14:17 -04:00
Peter Maydell 61cf5abc9e target/arm: Correctly implement ACTLR2, HACTLR2
The ACTLR2 and HACTLR2 AArch32 system registers didn't exist in ARMv7
or the original ARMv8. They were later added as optional registers,
whose presence is signaled by the ID_MMFR4.AC2 field. From ARMv8.2
they are mandatory (ie ID_MMFR4.AC2 must be non-zero).

We implemented HACTLR2 in commit 0e0456ab8895a5e85, but we
incorrectly made it exist for all v8 CPUs, and we didn't implement
ACTLR2 at all.

Sort this out by implementing both registers only when they are
supposed to exist, and setting the ID_MMFR4 bit for -cpu max.

Note that this removes HACTLR2 from our Cortex-A53, -A47 and -A72
CPU models; this is correct, because those CPUs do not implement
this register.

Fixes: 0e0456ab8895a5e85

Backports commit f6287c24c66d6b9187c1c2887e1c7cfa4d304b0c from qemu
2020-03-21 18:52:30 -04:00
Peter Maydell 1876feeede target/arm: Use FIELD_EX32 for testing 32-bit fields
Cut-and-paste errors mean we're using FIELD_EX64() to extract fields from
some 32-bit ID register fields. Use FIELD_EX32() instead. (This makes
no difference in behaviour, it's just more consistent.)

Backports commit b3a816f6ce1ec184ab6072f50bbe4479fc5116c3 from qemu
2020-03-21 18:50:14 -04:00
Peter Maydell 2ce106df33 target/arm: Use isar_feature function for testing AA32HPD feature
Now we have moved ID_MMFR4 into the ARMISARegisters struct, we
can define and use an isar_feature for the presence of the
ARMv8.2-AA32HPD feature, rather than open-coding the test.

While we're here, correct a comment typo which missed an 'A'
from the feature name.

Backports commit 4036b7d1cd9fb1097a5f4bc24d7d31744256260f from qemu
2020-03-21 18:48:57 -04:00
Peter Maydell 4693b2c011 target/arm: Test correct register in aa32_pan and aa32_ats1e1 checks
The isar_feature_aa32_pan and isar_feature_aa32_ats1e1 functions
are supposed to be testing fields in ID_MMFR3; but a cut-and-paste
error meant we were looking at MVFR0 instead.

Fix the functions to look at the right register; this requires
us to move at least id_mmfr3 to the ARMISARegisters struct; we
choose to move all the ID_MMFRn registers for consistency.

Backports commit 10054016eda1b13bdd8340d100fd029cc8b58f36 from qemu
2020-03-21 18:47:12 -04:00
Peter Maydell e72fa1cb33 target/arm: Correct handling of PMCR_EL0.LC bit
The LC bit in the PMCR_EL0 register is supposed to be:
* read/write
* RES1 on an AArch64-only implementation
* an architecturally UNKNOWN value on reset
(and use of LC==0 by software is deprecated).

We were implementing it incorrectly as read-only always zero,
though we do have all the code needed to test it and behave
accordingly.

Instead make it a read-write bit which resets to 1 always, which
satisfies all the architectural requirements above.

Backports commit 62d96ff48510f4bf648ad12f5d3a5507227b026f from qemu
2020-03-21 18:40:26 -04:00
Peter Maydell de428e4b45 target/arm: Correct definition of PMCRDP
The PMCR_EL0.DP bit is bit 5, which is 0x20, not 0x10. 0x10 is 'X'.
Correct our #define of PMCRDP and add the missing PMCRX.

We do have the correct behaviour for handling the DP bit being
set, so this fixes a guest-visible bug.

Fixes: 033614c47de

Backports commit a1ed04dd79aabb9dbeeb5fa7d49f1a3de0357553 from qemu
2020-03-21 18:39:37 -04:00
Peter Maydell 28b239adb9 target/arm: Provide ARMv8.4-PMU in '-cpu max'
Set the ID register bits to provide ARMv8.4-PMU (and implicitly
also ARMv8.1-PMU) in the 'max' CPU.

Backports commit 3bec78447a958d4819911252e056f29740ac25e4 from qemu
2020-03-21 18:38:53 -04:00
Peter Maydell 4dd57f7acc target/arm: Implement ARMv8.4-PMU extension
The ARMv8.4-PMU extension adds:
* one new required event, STALL
* one new system register PMMIR_EL1

(There are also some more L1-cache related events, but since
we don't implement any cache we don't provide these, in the
same way we don't provide the base-PMUv3 cache events.)

The STALL event "counts every attributable cycle on which no
attributable instruction or operation was sent for execution on this
PE". QEMU doesn't stall in this sense, so this is another
always-reads-zero event.

The PMMIR_EL1 register is a read-only register providing
implementation-specific information about the PMU; currently it has
only one field, SLOTS, which defines behaviour of the STALL_SLOT PMU
event. Since QEMU doesn't implement the STALL_SLOT event, we can
validly make the register read zero.

Backports commit 15dd1ebda4a6ef928d484c5a4f48b8ccb7438bb2 from qemu
2020-03-21 18:37:50 -04:00
Peter Maydell 5c93f43eb9 target/arm: Implement ARMv8.1-PMU extension
The ARMv8.1-PMU extension requires:
* the evtCount field in PMETYPER<n>_EL0 is 16 bits, not 10
* MDCR_EL2.HPMD allows event counting to be disabled at EL2
* two new required events, STALL_FRONTEND and STALL_BACKEND
* ID register bits in ID_AA64DFR0_EL1 and ID_DFR0

We already implement the 16-bit evtCount field and the
HPMD bit, so all that is missing is the two new events:
STALL_FRONTEND
"counts every cycle counted by the CPU_CYCLES event on which no
operation was issued because there are no operations available
to issue to this PE from the frontend"
STALL_BACKEND
"counts every cycle counted by the CPU_CYCLES event on which no
operation was issued because the backend is unable to accept
any available operations from the frontend"

QEMU never stalls in this sense, so our implementation is trivial:
always return a zero count.

Backports commit 0727f63b1ecf765ebc48266f616f8fc362dc7fbc from qemu
2020-03-21 18:34:33 -04:00
Peter Maydell 7dfc30b754 target/arm: Read debug-related ID registers from KVM
Backports 1548a7b2ad621a31b4216ed703b6d658a2ecf0d0 from qemu
2020-03-21 18:30:20 -04:00
Peter Maydell cef6f3e72c target/arm: Move DBGDIDR into ARMISARegisters
We're going to want to read the DBGDIDR register from KVM in
a subsequent commit, which means it needs to be in the
ARMISARegisters sub-struct. Move it.

Backports commit 4426d3617d64922d97b74ed22e67e33b6fb7de0a from qemu
2020-03-21 18:29:01 -04:00
Peter Maydell a6c9c87a5d target/arm: Stop assuming DBGDIDR always exists
The AArch32 DBGDIDR defines properties like the number of
breakpoints, watchpoints and context-matching comparators. On an
AArch64 CPU, the register may not even exist if AArch32 is not
supported at EL1.

Currently we hard-code use of DBGDIDR to identify the number of
breakpoints etc; this works for all our TCG CPUs, but will break if
we ever add an AArch64-only CPU. We also have an assert() that the
AArch32 and AArch64 registers match, which currently works only by
luck for KVM because we don't populate either of these ID registers
from the KVM vCPU and so they are both zero.

Clean this up so we have functions for finding the number
of breakpoints, watchpoints and context comparators which look
in the appropriate ID register.

This allows us to drop the "check that AArch64 and AArch32 agree
on the number of breakpoints etc" asserts:
* we no longer look at the AArch32 versions unless that's the
right place to be looking
* it's valid to have a CPU (eg AArch64-only) where they don't match
* we shouldn't have been asserting the validity of ID registers
in a codepath used with KVM anyway

Backports commit 88ce6c6ee85d902f59dc65afc3ca86b34f02b9ed from qemu
2020-03-21 18:26:24 -04:00
Peter Maydell afc28d9b2c target/arm: Add _aa64_ and _any_ versions of pmu_8_1 isar checks
Add the 64-bit version of the "is this a v8.1 PMUv3?"
ID register check function, and the _any_ version that
checks for either AArch32 or AArch64 support. We'll use
this in a later commit.

We don't (yet) do any isar_feature checks on ID_AA64DFR1_EL1,
but we move id_aa64dfr1 into the ARMISARegisters struct with
id_aa64dfr0, for consistency.

Backports commit 2a609df87d9b886fd38a190a754dbc241ff707e8 from qemu
2020-03-21 18:24:00 -04:00
Peter Maydell e64143966a target/arm: Define an aa32_pmu_8_1 isar feature test function
Instead of open-coding a check on the ID_DFR0 PerfMon ID register
field, create a standardly-named isar_feature for "does AArch32 have
a v8.1 PMUv3" and use it.

This entails moving the id_dfr0 field into the ARMISARegisters struct.

Backports commit a617953855b65a602d36364b9643f7e5bc31288e from qemu
2020-03-21 18:21:26 -04:00
Peter Maydell fd537585d7 target/arm: Use FIELD macros for clearing ID_DFR0 PERFMON field
We already define FIELD macros for ID_DFR0, so use them in the
one place where we're doing direct bit value manipulation.

Backports commit d52c061e541982a3663ad5c65bd3b518dbe85b87 from qemu
2020-03-21 18:17:55 -04:00
Peter Maydell fd6c635e03 target/arm: Add and use FIELD definitions for ID_AA64DFR0_EL1
Add FIELD() definitions for the ID_AA64DFR0_EL1 and use them
where we currently have hard-coded bit values.

Backports commit ceb2744b47a1ef4184dca56a158eb3156b6eba36 from qemu
2020-03-21 18:16:55 -04:00
Peter Maydell ebd7131c16 target/arm: Factor out PMU register definitions
Pull the code that defines the various PMU registers out
into its own function, matching the pattern we have
already for the debug registers.

Apart from one style fix to a multi-line comment, this
is purely movement of code with no changes to it.

Backports commit 24183fb6f00ecca8b508e245c95ff50ddde3f18b from qemu
2020-03-21 18:15:09 -04:00
Peter Maydell b1c088e2f2 target/arm: Define and use any_predinv isar_feature test
Instead of open-coding "ARM_FEATURE_AARCH64 ? aa64_predinv: aa32_predinv",
define and use an any_predinv isar_feature test function.

Backports commit 22e570730d15374453baa73ff2a699e01ef4e950 from qemu
2020-03-21 18:13:25 -04:00
Peter Maydell 62178626e4 target/arm: Add isar_feature_any_fp16 and document naming/usage conventions
Our current usage of the isar_feature feature tests almost always
uses an _aa32_ test when the code path is known to be AArch32
specific and an _aa64_ test when the code path is known to be
AArch64 specific. There is just one exception: in the vfp_set_fpscr
helper we check aa64_fp16 to determine whether the FZ16 bit in
the FP(S)CR exists, but this code is also used for AArch32.
There are other places in future where we're likely to want
a general "does this feature exist for either AArch32 or
AArch64" check (typically where architecturally the feature exists
for both CPU states if it exists at all, but the CPU might be
AArch32-only or AArch64-only, and so only have one set of ID
registers).

Introduce a new category of isar_feature_* functions:
isar_feature_any_foo() should be tested when what we want to
know is "does this feature exist for either AArch32 or AArch64",
and always returns the logical OR of isar_feature_aa32_foo()
and isar_feature_aa64_foo().

Backports commit 6e61f8391cc6cb0846d4bf078dbd935c2aeebff5 from qemu
2020-03-21 18:12:02 -04:00
Peter Maydell 778fcd9562 target/arm: Check aa32_pan in take_aarch32_exception(), not aa64_pan
In take_aarch32_exception(), we know we are dealing with a CPU that
has AArch32, so the right isar_feature test is aa32_pan, not aa64_pan.

Backports commit f8af1143ef93954e77cf59e09b5e004dafbd64fd from qemu
2020-03-21 18:09:27 -04:00
Peter Maydell e63f70f980 target/arm: Add _aa32_ to isar_feature functions testing 32-bit ID registers
Enforce a convention that an isar_feature function that tests a
32-bit ID register always has _aa32_ in its name, and one that
tests a 64-bit ID register always has _aa64_ in its name.
We already follow this except for three cases: thumb_div,
arm_div and jazelle, which all need _aa32_ adding.

(As noted in the comment, isar_feature_aa32_fp16_arith()
is an exception in that it currently tests ID_AA64PFR0_EL1,
but will switch to MVFR1 once we've properly implemented
FP16 for AArch32.)

Backports commit 873b73c0c891ec20adacc7bd1ae789294334d675 from qemu
2020-03-21 18:08:23 -04:00
Richard Henderson 0131e804fb target/arm: Split out aa64_va_parameter_tbi, aa64_va_parameter_tbid
For the purpose of rebuild_hflags_a64, we do not need to compute
all of the va parameters, only tbi. Moreover, we can compute them
in a form that is more useful to storing in hflags.

This eliminates the need for aa64_va_parameter_both, so fold that
in to aa64_va_parameter. The remaining calls to aa64_va_parameter
are in get_phys_addr_lpae and in pauth_helper.c.

This reduces the total cpu consumption of aa64_va_parameter in a
kernel boot plus a kvm guest kernel boot from 3% to 0.5%.

Backports commit b830a5ee82e66f54697dcc6450fe9239b7412d13 from qemu
2020-03-21 18:04:39 -04:00
Richard Henderson 2cce7e0dd0 target/arm: Remove ttbr1_valid check from get_phys_addr_lpae
Now that aa64_va_parameters_both sets select based on the number
of ranges in the regime, the ttbr1_valid check is redundant.

Backports commit 03f27724dff15633911e68a3906c30f57938ea45 from qemu
2020-03-21 18:01:24 -04:00
Richard Henderson f3fa39829d target/arm: Fix select for aa64_va_parameters_both
Select should always be 0 for a regime with one range.

Backports commit 71d181640a1a9470f074fa28600ca85587e2ca6b from qemu
2020-03-21 18:00:15 -04:00
Richard Henderson 3183349f1c target/arm: Use bit 55 explicitly for pauth
The psuedocode in aarch64/functions/pac/auth/Auth and
aarch64/functions/pac/strip/Strip always uses bit 55 for
extfield and do not consider if the current regime has 2 ranges.

Backports commit 7eeb4c2ce8dc0a5655526f3f39bd5d6cc02efb39 from qemu
2020-03-21 17:59:06 -04:00
Richard Henderson 51b6064ba4 target/arm: Flush high bits of sve register after AdvSIMD INS
Writes to AdvSIMD registers flush the bits above 128.

Backports commit 528dc354b6f3aa82d65141cc60bc0e725e6cae98 from qemu
2020-03-21 17:58:09 -04:00
Richard Henderson 74cbfceb56 target/arm: Flush high bits of sve register after AdvSIMD ZIP/UZP/TRN
Writes to AdvSIMD registers flush the bits above 128.

Backports commit 33649de62e40df0060a1c514574e4ef25c4e52e1 from qemu
2020-03-21 17:56:40 -04:00
Richard Henderson 6eb8472344 target/arm: Flush high bits of sve register after AdvSIMD TBL/TBX
Writes to AdvSIMD registers flush the bits above 128.

Backports commit 263273bc988e677ebadeaf7d0e49f6792a112db5 from qemu
2020-03-21 17:56:08 -04:00
Richard Henderson 18e9c4805f target/arm: Flush high bits of sve register after AdvSIMD EXT
Writes to AdvSIMD registers flush the bits above 128.

Backports commit 78cedfabd53b6f64e7e64fc84878d848e5df1d08 from qemu
2020-03-21 17:55:12 -04:00
Peter Maydell 96a96565db target/arm: Implement ARMv8.1-VMID16 extension
The ARMv8.1-VMID16 extension extends the VMID from 8 bits to 16 bits:

* the ID_AA64MMFR1_EL1.VMIDBits field specifies whether the VMID is
8 or 16 bits
* the VMID field in VTTBR_EL2 is extended to 16 bits
* VTCR_EL2.VS lets the guest specify whether to use the full 16 bits,
or use the backwards-compatible 8 bits

For QEMU implementing this is trivial:
* we do not track VMIDs in TLB entries, so we never use the VMID field
* we treat any write to VTTBR_EL2, not just a change to the VMID field
bits, as a "possible VMID change" that causes us to throw away TLB
entries, so that code doesn't need changing
* we allow the guest to read/write the VTCR_EL2.VS bit already

So all that's missing is the ID register part: report that we support
VMID16 in our 'max' CPU.

Backports commit dc7a88d0810ad272bdcd2e0869359af78fdd9114 from qemu
2020-03-21 17:52:43 -04:00
Richard Henderson 57f0aa3044 target/arm: Enable ARMv8.2-UAO in -cpu max
Backports commit e11f0eb6724571adb812a3ce5269c41586e0262b from qemu
2020-03-21 17:51:44 -04:00
Richard Henderson 18a86780ee target/arm: Implement UAO semantics
We need only override the current condition under which
TBFLAG_A64.UNPRIV is set.

Backports commit 7a8014ab871d5320effd737dfe88b2e80f16a509 from qemu
2020-03-21 17:50:29 -04:00
Richard Henderson 5b5050c6ca target/arm: Update MSR access to UAO
Backports commit 9eeb7a1c9531cb3574bfe2c36eb7624802c3ec00 from qemu
2020-03-21 17:48:01 -04:00
Richard Henderson 0630e66b5a target/arm: Add ID_AA64MMFR2_EL1
Add definitions for all of the fields, up to ARMv8.5.
Convert the existing RESERVED register to a full register.
Query KVM for the value of the register for the host.

Backports commit 64761e10af2742a916c08271828890274137b9e8 from qemu
2020-03-21 17:45:27 -04:00
Richard Henderson 7287bf16b8 target/arm: Enable ARMv8.2-ATS1E1 in -cpu max
This includes enablement of ARMv8.1-PAN.

Backports commit e0fe7309a7c21ef2386de50d37c86aea0d671c08 from qemu
2020-03-21 17:43:54 -04:00
Richard Henderson d196288b4f target/arm: Implement ATS1E1 system registers
This is a minor enhancement over ARMv8.1-PAN.
The *_PAN mmu_idx are used with the existing do_ats_write.

Backports commit 04b07d29722192926f467ea5fedf2c3b0996a2a5 from qemu
2020-03-21 17:42:01 -04:00
Richard Henderson 6576864930 target/arm: Set PAN bit as required on exception entry
The PAN bit is preserved, or set as per SCTLR_ELx.SPAN,
plus several other conditions listed in the ARM ARM.

Backports commit 4a2696c0d4d80e14a192b28148c6167bc5056f94 from qemu
2020-03-21 17:40:11 -04:00
Richard Henderson aad0621f96 target/arm: Enforce PAN semantics in get_S1prot
If we have a PAN-enforcing mmu_idx, set prot == 0 if user_rw != 0.

Backports commit 81636b70c226dc27d7ebc8dedbcec26166d23085 from qemu
2020-03-21 17:35:55 -04:00
Richard Henderson 41d03da852 target/arm: Update arm_mmu_idx_el for PAN
Examine the PAN bit for EL1, EL2, and Secure EL1 to
determine if it applies.

Backports commit 66412260cc1bee60a22d96e4ad8569b85745fea4 from qemu
2020-03-21 17:34:12 -04:00
Richard Henderson 35fab80c57 target/arm: Update MSR access for PAN
For aarch64, there's a dedicated msr (imm, reg) insn.
For aarch32, this is done via msr to cpsr. Writes from el0
are ignored, which is already handled by the CPSR_USER mask.

Backports commit 220f508f49c5f49fb771d5105f991c19ffede3f7 from qemu
2020-03-21 17:33:16 -04:00
Richard Henderson 50bb867a6f target/arm: Introduce aarch64_pstate_valid_mask
Use this along the exception return path, where we previously
accepted any values

Backports commit 140845111809cd6fd57ccde93867b48cc56ffc74 from qemu
2020-03-21 17:26:00 -04:00
Richard Henderson b6b69d7ac5 target/arm: Remove CPSR_RESERVED
The only remaining use was in op_helper.c. Use PSTATE_SS
directly, and move the commentary so that it is more obvious
what is going on.

Backports commit 70dae0d069c45250bbefd9424089383a8ac239de from qemu
2020-03-21 17:24:21 -04:00
Richard Henderson 2d3239d0a1 target/arm: Use aarch32_cpsr_valid_mask in helper_exception_return
Using ~0 as the mask on the aarch64->aarch32 exception return
was not even as correct as the CPSR_ERET_MASK that we had used
on the aarch32->aarch32 exception return.

Backports commit d203cabd1bd12f31c9df0b5737421ba67b96857b from qemu
2020-03-21 17:20:53 -04:00
Richard Henderson c450694f1a target/arm: Replace CPSR_ERET_MASK with aarch32_cpsr_valid_mask
CPSR_ERET_MASK was a useless renaming of CPSR_RESERVED.
The function also takes into account bits that the cpu
does not support.

Backports commit 437864216d63f052f3cd06ec8861d0e432496424 from qemu
2020-03-21 17:19:17 -04:00
Richard Henderson e4a7a089f0 target/arm: Mask CPSR_J when Jazelle is not enabled
The J bit signals Jazelle mode, and so of course is RES0
when the feature is not enabled.

Backports commit f062d1447f2a80e7a5f593b8cb5ac7cab5e16eb0 from qemu
2020-03-21 17:17:50 -04:00
Richard Henderson ca2bb77ab3 target/arm: Split out aarch32_cpsr_valid_mask
Split this helper out of msr_mask in translate.c. At the same time,
transform the negative reductive logic to positive accumulative logic.
It will be usable along the exception paths.

While touching msr_mask, fix up formatting.

Backports commit 4f9584ed4bba8a57a3cb2fa48a682725005d530a from qemu
2020-03-21 17:16:20 -04:00
Richard Henderson e9850834d5 target/arm: Move LOR regdefs to file scope
For static const regdefs, file scope is preferred.

Backports commit d8564ee4e5bce87ec1fdf23656df9367eb1bc571 from qemu
2020-03-21 17:13:58 -04:00
Richard Henderson 5c8c2ca505 target/arm: Add isar_feature tests for PAN + ATS1E1
Include definitions for all of the bits in ID_MMFR3.
We already have a definition for ID_AA64MMFR1.PAN.

Backports commit 3d6ad6bb466f487bcc861f99e2c9054230df1076 from qemu
2020-03-21 17:13:07 -04:00
Richard Henderson 7aaf0d442b target/arm: Add mmu_idx for EL1 and EL2 w/ PAN enabled
To implement PAN, we will want to swap, for short periods
of time, to a different privileged mmu_idx. In addition,
we cannot do this with flushing alone, because the AT*
instructions have both PAN and PAN-less versions.

Add the ARMMMUIdx*_PAN constants where necessary next to
the corresponding ARMMMUIdx* constant.

Backports commit 452ef8cb8c7b06f44a30a3c3a54d3be82c4aef59 from qemu
2020-03-21 17:12:16 -04:00
Richard Henderson ed5a4950fd target/arm: Add arm_mmu_idx_is_stage1_of_2
Use a common predicate for querying stage1-ness.

Backports commit fee7aa46edd76f06c3dc176abb8fd05b365efce6 from qemu
2020-03-21 16:56:03 -04:00
Richard Henderson eb0586f9cd target/arm: Raise only one interrupt in arm_cpu_exec_interrupt
The fall through organization of this function meant that we
would raise an interrupt, then might overwrite that with another.
Since interrupt prioritization is IMPLEMENTATION DEFINED, we
can recognize these in any order we choose.

Unify the code to raise the interrupt in a block at the end.

Backports commit d63d0ec59d87a698de5ed843288f90a23470cf2e from qemu
2020-03-21 16:42:52 -04:00
Richard Henderson d00e5ec47d target/arm: Use bool for unmasked in arm_excp_unmasked
The value computed is fully boolean; using int8_t is odd.

Backports commit 16e07f78df002067bc4bfb115ba1ee0ce278e9e5 from qemu
2020-03-21 16:40:36 -04:00
Richard Henderson 975f0a9bc5 target/arm: Pass more cpu state to arm_excp_unmasked
Avoid redundant computation of cpu state by passing it in
from the caller, which has already computed it for itself.

Backports commit be87955687446be152f366af543c9234eab78a7c from qemu
2020-03-21 16:39:16 -04:00
Richard Henderson 6023db20bc target/arm: Move arm_excp_unmasked to cpu.c
This inline function has one user in cpu.c, and need not be exposed
otherwise. Code movement only, with fixups for checkpatch.

Backports commit 310cedf39dea240a89f90729fd99481ff6158e90 from qemu
2020-03-21 16:37:12 -04:00
Richard Henderson ad5a3b2532 target/arm: Enable ARMv8.1-VHE in -cpu max
Backports commit cd3f80aba0c559a6291f7c3e686422b15381f6b7 from qemu
2020-03-21 16:36:04 -04:00
Richard Henderson 36407da586 target/arm: Update arm_cpu_do_interrupt_aarch64 for VHE
When VHE is enabled, the exception level below EL2 is not EL1,
but EL0, and so to identify the entry vector offset for exceptions
targeting EL2 we need to look at the width of EL0, not of EL1.

Backports commit cb092fbbaeb7b4e91b3f9c53150c8160f91577c7 from qemu
2020-03-21 16:35:07 -04:00
Richard Henderson 8f1201e392 target/arm: Update get_a64_user_mem_index for VHE
The EL2&0 translation regime is affected by Load Register (unpriv).

The code structure used here will facilitate later changes in this
area for implementing UAO and NV.

Backports commit cc28fc30e333dc2f20ebfde54444697e26cd8f6d from qemu
2020-03-21 16:33:52 -04:00
Alex Bennée 76ca1cd732 target/arm: check TGE and E2H flags for EL0 pauth traps
According to ARM ARM we should only trap from the EL1&0 regime.

Backports commit a7469a3c1edc7687d7d25967bc2c0280de202bca from qemu
2020-03-21 16:27:40 -04:00
Richard Henderson 01e1e7a3a0 target/arm: Update {fp,sve}_exception_el for VHE
When TGE+E2H are both set, CPACR_EL1 is ignored.

Backports commit c2ddb7cf963b3bea838266bfca62514dc9750a10 from qemu
2020-03-21 16:26:01 -04:00
Richard Henderson 86d0163465 target/arm: Update arm_phys_excp_target_el for TGE
The TGE bit routes all asynchronous exceptions to EL2.

Backports commit d1b31428fd522b725bc053c84b5fbc8764061363 from qemu
2020-03-21 16:23:52 -04:00
Richard Henderson 0c03fa2dac target/arm: Flush tlbs for E2&0 translation regime
Backports commit 85d0dc9fa205027554372367f6925749a2d2b4c4 from qemu
2020-03-21 16:22:46 -04:00
Richard Henderson 50ac89852a target/arm: Flush tlb for ASID changes in EL2&0 translation regime
Since we only support a single ASID, flush the tlb when it changes.

Note that TCR_EL2, like TCR_EL1, has the A1 bit that chooses between
the two TTBR* registers for the location of the ASID.

Backports commit d06dc93340825030b6297c61199a17c0067b0377 from qemu
2020-03-21 16:13:55 -04:00
Richard Henderson a2b8ebabfa target/arm: Add VHE timer register redirection and aliasing
Apart from the wholesale redirection that HCR_EL2.E2H performs
for EL2, there's a separate redirection specific to the timers
that happens for EL0 when running in the EL2&0 regime.

Backports commit bb5972e439dc0ac4d21329a9d97bad6760ec702d from qemu
2020-03-21 16:09:54 -04:00
Richard Henderson e41c51f6da target/arm: Add VHE system register redirection and aliasing
Several of the EL1/0 registers are redirected to the EL2 version when in
EL2 and HCR_EL2.E2H is set. Many of these registers have side effects.
Link together the two ARMCPRegInfo structures after they have been
properly instantiated. Install common dispatch routines to all of the
relevant registers.

The same set of registers that are redirected also have additional
EL12/EL02 aliases created to access the original register that was
redirected.

Omit the generic timer registers from redirection here, because we'll
need multiple kinds of redirection from both EL0 and EL2.

Backports commit e2cce18f5c1d0d55328c585c8372cdb096bbf528 from qemu
2020-03-21 15:57:03 -04:00