Commit graph

453 commits

Author SHA1 Message Date
Richard Henderson 06b52d6660 target/arm: Add isar_feature_aa32_vfp_simd
Use this in the places that were checking ARM_FEATURE_VFP, and
are obviously testing for the existance of the register set
as opposed to testing for some particular instruction extension.

Backports commit 7fbc6a403a0aab834e764fa61d81ed8586cfe352 from qemu
2020-03-21 23:11:36 -04:00
Richard Henderson 833de589ed target/arm: Use isar_feature_aa32_simd_r32 more places
Many uses of ARM_FEATURE_VFP3 are testing for the number of simd
registers implemented. Use the proper test vs MVFR0.SIMDReg.

Backports commit a6627f5fc607939f7c8b9c3157fdcb2d368ba0ed from qemu
2020-03-21 19:39:35 -04:00
Peter Maydell 61cf5abc9e target/arm: Correctly implement ACTLR2, HACTLR2
The ACTLR2 and HACTLR2 AArch32 system registers didn't exist in ARMv7
or the original ARMv8. They were later added as optional registers,
whose presence is signaled by the ID_MMFR4.AC2 field. From ARMv8.2
they are mandatory (ie ID_MMFR4.AC2 must be non-zero).

We implemented HACTLR2 in commit 0e0456ab8895a5e85, but we
incorrectly made it exist for all v8 CPUs, and we didn't implement
ACTLR2 at all.

Sort this out by implementing both registers only when they are
supposed to exist, and setting the ID_MMFR4 bit for -cpu max.

Note that this removes HACTLR2 from our Cortex-A53, -A47 and -A72
CPU models; this is correct, because those CPUs do not implement
this register.

Fixes: 0e0456ab8895a5e85

Backports commit f6287c24c66d6b9187c1c2887e1c7cfa4d304b0c from qemu
2020-03-21 18:52:30 -04:00
Peter Maydell 2ce106df33 target/arm: Use isar_feature function for testing AA32HPD feature
Now we have moved ID_MMFR4 into the ARMISARegisters struct, we
can define and use an isar_feature for the presence of the
ARMv8.2-AA32HPD feature, rather than open-coding the test.

While we're here, correct a comment typo which missed an 'A'
from the feature name.

Backports commit 4036b7d1cd9fb1097a5f4bc24d7d31744256260f from qemu
2020-03-21 18:48:57 -04:00
Peter Maydell 4693b2c011 target/arm: Test correct register in aa32_pan and aa32_ats1e1 checks
The isar_feature_aa32_pan and isar_feature_aa32_ats1e1 functions
are supposed to be testing fields in ID_MMFR3; but a cut-and-paste
error meant we were looking at MVFR0 instead.

Fix the functions to look at the right register; this requires
us to move at least id_mmfr3 to the ARMISARegisters struct; we
choose to move all the ID_MMFRn registers for consistency.

Backports commit 10054016eda1b13bdd8340d100fd029cc8b58f36 from qemu
2020-03-21 18:47:12 -04:00
Peter Maydell e72fa1cb33 target/arm: Correct handling of PMCR_EL0.LC bit
The LC bit in the PMCR_EL0 register is supposed to be:
* read/write
* RES1 on an AArch64-only implementation
* an architecturally UNKNOWN value on reset
(and use of LC==0 by software is deprecated).

We were implementing it incorrectly as read-only always zero,
though we do have all the code needed to test it and behave
accordingly.

Instead make it a read-write bit which resets to 1 always, which
satisfies all the architectural requirements above.

Backports commit 62d96ff48510f4bf648ad12f5d3a5507227b026f from qemu
2020-03-21 18:40:26 -04:00
Peter Maydell de428e4b45 target/arm: Correct definition of PMCRDP
The PMCR_EL0.DP bit is bit 5, which is 0x20, not 0x10. 0x10 is 'X'.
Correct our #define of PMCRDP and add the missing PMCRX.

We do have the correct behaviour for handling the DP bit being
set, so this fixes a guest-visible bug.

Fixes: 033614c47de

Backports commit a1ed04dd79aabb9dbeeb5fa7d49f1a3de0357553 from qemu
2020-03-21 18:39:37 -04:00
Peter Maydell 4dd57f7acc target/arm: Implement ARMv8.4-PMU extension
The ARMv8.4-PMU extension adds:
* one new required event, STALL
* one new system register PMMIR_EL1

(There are also some more L1-cache related events, but since
we don't implement any cache we don't provide these, in the
same way we don't provide the base-PMUv3 cache events.)

The STALL event "counts every attributable cycle on which no
attributable instruction or operation was sent for execution on this
PE". QEMU doesn't stall in this sense, so this is another
always-reads-zero event.

The PMMIR_EL1 register is a read-only register providing
implementation-specific information about the PMU; currently it has
only one field, SLOTS, which defines behaviour of the STALL_SLOT PMU
event. Since QEMU doesn't implement the STALL_SLOT event, we can
validly make the register read zero.

Backports commit 15dd1ebda4a6ef928d484c5a4f48b8ccb7438bb2 from qemu
2020-03-21 18:37:50 -04:00
Peter Maydell 5c93f43eb9 target/arm: Implement ARMv8.1-PMU extension
The ARMv8.1-PMU extension requires:
* the evtCount field in PMETYPER<n>_EL0 is 16 bits, not 10
* MDCR_EL2.HPMD allows event counting to be disabled at EL2
* two new required events, STALL_FRONTEND and STALL_BACKEND
* ID register bits in ID_AA64DFR0_EL1 and ID_DFR0

We already implement the 16-bit evtCount field and the
HPMD bit, so all that is missing is the two new events:
STALL_FRONTEND
"counts every cycle counted by the CPU_CYCLES event on which no
operation was issued because there are no operations available
to issue to this PE from the frontend"
STALL_BACKEND
"counts every cycle counted by the CPU_CYCLES event on which no
operation was issued because the backend is unable to accept
any available operations from the frontend"

QEMU never stalls in this sense, so our implementation is trivial:
always return a zero count.

Backports commit 0727f63b1ecf765ebc48266f616f8fc362dc7fbc from qemu
2020-03-21 18:34:33 -04:00
Peter Maydell cef6f3e72c target/arm: Move DBGDIDR into ARMISARegisters
We're going to want to read the DBGDIDR register from KVM in
a subsequent commit, which means it needs to be in the
ARMISARegisters sub-struct. Move it.

Backports commit 4426d3617d64922d97b74ed22e67e33b6fb7de0a from qemu
2020-03-21 18:29:01 -04:00
Peter Maydell a6c9c87a5d target/arm: Stop assuming DBGDIDR always exists
The AArch32 DBGDIDR defines properties like the number of
breakpoints, watchpoints and context-matching comparators. On an
AArch64 CPU, the register may not even exist if AArch32 is not
supported at EL1.

Currently we hard-code use of DBGDIDR to identify the number of
breakpoints etc; this works for all our TCG CPUs, but will break if
we ever add an AArch64-only CPU. We also have an assert() that the
AArch32 and AArch64 registers match, which currently works only by
luck for KVM because we don't populate either of these ID registers
from the KVM vCPU and so they are both zero.

Clean this up so we have functions for finding the number
of breakpoints, watchpoints and context comparators which look
in the appropriate ID register.

This allows us to drop the "check that AArch64 and AArch32 agree
on the number of breakpoints etc" asserts:
* we no longer look at the AArch32 versions unless that's the
right place to be looking
* it's valid to have a CPU (eg AArch64-only) where they don't match
* we shouldn't have been asserting the validity of ID registers
in a codepath used with KVM anyway

Backports commit 88ce6c6ee85d902f59dc65afc3ca86b34f02b9ed from qemu
2020-03-21 18:26:24 -04:00
Peter Maydell afc28d9b2c target/arm: Add _aa64_ and _any_ versions of pmu_8_1 isar checks
Add the 64-bit version of the "is this a v8.1 PMUv3?"
ID register check function, and the _any_ version that
checks for either AArch32 or AArch64 support. We'll use
this in a later commit.

We don't (yet) do any isar_feature checks on ID_AA64DFR1_EL1,
but we move id_aa64dfr1 into the ARMISARegisters struct with
id_aa64dfr0, for consistency.

Backports commit 2a609df87d9b886fd38a190a754dbc241ff707e8 from qemu
2020-03-21 18:24:00 -04:00
Peter Maydell e64143966a target/arm: Define an aa32_pmu_8_1 isar feature test function
Instead of open-coding a check on the ID_DFR0 PerfMon ID register
field, create a standardly-named isar_feature for "does AArch32 have
a v8.1 PMUv3" and use it.

This entails moving the id_dfr0 field into the ARMISARegisters struct.

Backports commit a617953855b65a602d36364b9643f7e5bc31288e from qemu
2020-03-21 18:21:26 -04:00
Peter Maydell fd6c635e03 target/arm: Add and use FIELD definitions for ID_AA64DFR0_EL1
Add FIELD() definitions for the ID_AA64DFR0_EL1 and use them
where we currently have hard-coded bit values.

Backports commit ceb2744b47a1ef4184dca56a158eb3156b6eba36 from qemu
2020-03-21 18:16:55 -04:00
Peter Maydell ebd7131c16 target/arm: Factor out PMU register definitions
Pull the code that defines the various PMU registers out
into its own function, matching the pattern we have
already for the debug registers.

Apart from one style fix to a multi-line comment, this
is purely movement of code with no changes to it.

Backports commit 24183fb6f00ecca8b508e245c95ff50ddde3f18b from qemu
2020-03-21 18:15:09 -04:00
Peter Maydell b1c088e2f2 target/arm: Define and use any_predinv isar_feature test
Instead of open-coding "ARM_FEATURE_AARCH64 ? aa64_predinv: aa32_predinv",
define and use an any_predinv isar_feature test function.

Backports commit 22e570730d15374453baa73ff2a699e01ef4e950 from qemu
2020-03-21 18:13:25 -04:00
Peter Maydell 778fcd9562 target/arm: Check aa32_pan in take_aarch32_exception(), not aa64_pan
In take_aarch32_exception(), we know we are dealing with a CPU that
has AArch32, so the right isar_feature test is aa32_pan, not aa64_pan.

Backports commit f8af1143ef93954e77cf59e09b5e004dafbd64fd from qemu
2020-03-21 18:09:27 -04:00
Peter Maydell e63f70f980 target/arm: Add _aa32_ to isar_feature functions testing 32-bit ID registers
Enforce a convention that an isar_feature function that tests a
32-bit ID register always has _aa32_ in its name, and one that
tests a 64-bit ID register always has _aa64_ in its name.
We already follow this except for three cases: thumb_div,
arm_div and jazelle, which all need _aa32_ adding.

(As noted in the comment, isar_feature_aa32_fp16_arith()
is an exception in that it currently tests ID_AA64PFR0_EL1,
but will switch to MVFR1 once we've properly implemented
FP16 for AArch32.)

Backports commit 873b73c0c891ec20adacc7bd1ae789294334d675 from qemu
2020-03-21 18:08:23 -04:00
Richard Henderson 0131e804fb target/arm: Split out aa64_va_parameter_tbi, aa64_va_parameter_tbid
For the purpose of rebuild_hflags_a64, we do not need to compute
all of the va parameters, only tbi. Moreover, we can compute them
in a form that is more useful to storing in hflags.

This eliminates the need for aa64_va_parameter_both, so fold that
in to aa64_va_parameter. The remaining calls to aa64_va_parameter
are in get_phys_addr_lpae and in pauth_helper.c.

This reduces the total cpu consumption of aa64_va_parameter in a
kernel boot plus a kvm guest kernel boot from 3% to 0.5%.

Backports commit b830a5ee82e66f54697dcc6450fe9239b7412d13 from qemu
2020-03-21 18:04:39 -04:00
Richard Henderson 2cce7e0dd0 target/arm: Remove ttbr1_valid check from get_phys_addr_lpae
Now that aa64_va_parameters_both sets select based on the number
of ranges in the regime, the ttbr1_valid check is redundant.

Backports commit 03f27724dff15633911e68a3906c30f57938ea45 from qemu
2020-03-21 18:01:24 -04:00
Richard Henderson f3fa39829d target/arm: Fix select for aa64_va_parameters_both
Select should always be 0 for a regime with one range.

Backports commit 71d181640a1a9470f074fa28600ca85587e2ca6b from qemu
2020-03-21 18:00:15 -04:00
Richard Henderson 18a86780ee target/arm: Implement UAO semantics
We need only override the current condition under which
TBFLAG_A64.UNPRIV is set.

Backports commit 7a8014ab871d5320effd737dfe88b2e80f16a509 from qemu
2020-03-21 17:50:29 -04:00
Richard Henderson 5b5050c6ca target/arm: Update MSR access to UAO
Backports commit 9eeb7a1c9531cb3574bfe2c36eb7624802c3ec00 from qemu
2020-03-21 17:48:01 -04:00
Richard Henderson 0630e66b5a target/arm: Add ID_AA64MMFR2_EL1
Add definitions for all of the fields, up to ARMv8.5.
Convert the existing RESERVED register to a full register.
Query KVM for the value of the register for the host.

Backports commit 64761e10af2742a916c08271828890274137b9e8 from qemu
2020-03-21 17:45:27 -04:00
Richard Henderson d196288b4f target/arm: Implement ATS1E1 system registers
This is a minor enhancement over ARMv8.1-PAN.
The *_PAN mmu_idx are used with the existing do_ats_write.

Backports commit 04b07d29722192926f467ea5fedf2c3b0996a2a5 from qemu
2020-03-21 17:42:01 -04:00
Richard Henderson 6576864930 target/arm: Set PAN bit as required on exception entry
The PAN bit is preserved, or set as per SCTLR_ELx.SPAN,
plus several other conditions listed in the ARM ARM.

Backports commit 4a2696c0d4d80e14a192b28148c6167bc5056f94 from qemu
2020-03-21 17:40:11 -04:00
Richard Henderson aad0621f96 target/arm: Enforce PAN semantics in get_S1prot
If we have a PAN-enforcing mmu_idx, set prot == 0 if user_rw != 0.

Backports commit 81636b70c226dc27d7ebc8dedbcec26166d23085 from qemu
2020-03-21 17:35:55 -04:00
Richard Henderson 41d03da852 target/arm: Update arm_mmu_idx_el for PAN
Examine the PAN bit for EL1, EL2, and Secure EL1 to
determine if it applies.

Backports commit 66412260cc1bee60a22d96e4ad8569b85745fea4 from qemu
2020-03-21 17:34:12 -04:00
Richard Henderson 35fab80c57 target/arm: Update MSR access for PAN
For aarch64, there's a dedicated msr (imm, reg) insn.
For aarch32, this is done via msr to cpsr. Writes from el0
are ignored, which is already handled by the CPSR_USER mask.

Backports commit 220f508f49c5f49fb771d5105f991c19ffede3f7 from qemu
2020-03-21 17:33:16 -04:00
Richard Henderson e9850834d5 target/arm: Move LOR regdefs to file scope
For static const regdefs, file scope is preferred.

Backports commit d8564ee4e5bce87ec1fdf23656df9367eb1bc571 from qemu
2020-03-21 17:13:58 -04:00
Richard Henderson 7aaf0d442b target/arm: Add mmu_idx for EL1 and EL2 w/ PAN enabled
To implement PAN, we will want to swap, for short periods
of time, to a different privileged mmu_idx. In addition,
we cannot do this with flushing alone, because the AT*
instructions have both PAN and PAN-less versions.

Add the ARMMMUIdx*_PAN constants where necessary next to
the corresponding ARMMMUIdx* constant.

Backports commit 452ef8cb8c7b06f44a30a3c3a54d3be82c4aef59 from qemu
2020-03-21 17:12:16 -04:00
Richard Henderson ed5a4950fd target/arm: Add arm_mmu_idx_is_stage1_of_2
Use a common predicate for querying stage1-ness.

Backports commit fee7aa46edd76f06c3dc176abb8fd05b365efce6 from qemu
2020-03-21 16:56:03 -04:00
Richard Henderson 36407da586 target/arm: Update arm_cpu_do_interrupt_aarch64 for VHE
When VHE is enabled, the exception level below EL2 is not EL1,
but EL0, and so to identify the entry vector offset for exceptions
targeting EL2 we need to look at the width of EL0, not of EL1.

Backports commit cb092fbbaeb7b4e91b3f9c53150c8160f91577c7 from qemu
2020-03-21 16:35:07 -04:00
Richard Henderson 8f1201e392 target/arm: Update get_a64_user_mem_index for VHE
The EL2&0 translation regime is affected by Load Register (unpriv).

The code structure used here will facilitate later changes in this
area for implementing UAO and NV.

Backports commit cc28fc30e333dc2f20ebfde54444697e26cd8f6d from qemu
2020-03-21 16:33:52 -04:00
Richard Henderson 01e1e7a3a0 target/arm: Update {fp,sve}_exception_el for VHE
When TGE+E2H are both set, CPACR_EL1 is ignored.

Backports commit c2ddb7cf963b3bea838266bfca62514dc9750a10 from qemu
2020-03-21 16:26:01 -04:00
Richard Henderson 86d0163465 target/arm: Update arm_phys_excp_target_el for TGE
The TGE bit routes all asynchronous exceptions to EL2.

Backports commit d1b31428fd522b725bc053c84b5fbc8764061363 from qemu
2020-03-21 16:23:52 -04:00
Richard Henderson 0c03fa2dac target/arm: Flush tlbs for E2&0 translation regime
Backports commit 85d0dc9fa205027554372367f6925749a2d2b4c4 from qemu
2020-03-21 16:22:46 -04:00
Richard Henderson 50ac89852a target/arm: Flush tlb for ASID changes in EL2&0 translation regime
Since we only support a single ASID, flush the tlb when it changes.

Note that TCR_EL2, like TCR_EL1, has the A1 bit that chooses between
the two TTBR* registers for the location of the ASID.

Backports commit d06dc93340825030b6297c61199a17c0067b0377 from qemu
2020-03-21 16:13:55 -04:00
Richard Henderson a2b8ebabfa target/arm: Add VHE timer register redirection and aliasing
Apart from the wholesale redirection that HCR_EL2.E2H performs
for EL2, there's a separate redirection specific to the timers
that happens for EL0 when running in the EL2&0 regime.

Backports commit bb5972e439dc0ac4d21329a9d97bad6760ec702d from qemu
2020-03-21 16:09:54 -04:00
Richard Henderson e41c51f6da target/arm: Add VHE system register redirection and aliasing
Several of the EL1/0 registers are redirected to the EL2 version when in
EL2 and HCR_EL2.E2H is set. Many of these registers have side effects.
Link together the two ARMCPRegInfo structures after they have been
properly instantiated. Install common dispatch routines to all of the
relevant registers.

The same set of registers that are redirected also have additional
EL12/EL02 aliases created to access the original register that was
redirected.

Omit the generic timer registers from redirection here, because we'll
need multiple kinds of redirection from both EL0 and EL2.

Backports commit e2cce18f5c1d0d55328c585c8372cdb096bbf528 from qemu
2020-03-21 15:57:03 -04:00
Richard Henderson ff720b7fd3 target/arm: Update define_one_arm_cp_reg_with_opaque for VHE
For ARMv8.1, op1 == 5 is reserved for EL2 aliases of
EL1 and EL0 registers.

Backports commit b4ecf60f7eee88cbfe5700044790cb7494c5dd37 from qemu
2020-03-21 15:39:54 -04:00
Richard Henderson 8c7795dc04 target/arm: Update timer access for VHE
Backports commit 5bc8437136fb1e7bc8b566f4f2f7269b0f990fad from qemu
2020-03-21 15:38:47 -04:00
Richard Henderson d6150127b4 target/arm: Add the hypervisor virtual counter
Backports commit 8c94b071a09c2183f032febff3112f2b7662156c from qemu
2020-03-21 15:35:36 -04:00
Richard Henderson 8e2ac48ad0 target/arm: Update ctr_el0_access for EL2
Update to include checks against HCR_EL2.TID2.

Backports commit 97475a89375d62a7722e04ced9fbdf0b992f4b83 from qemu
2020-03-21 15:31:48 -04:00
Richard Henderson 6886ba66d0 target/arm: Update aa64_zva_access for EL2
The comment that we don't support EL2 is somewhat out of date.
Update to include checks against HCR_EL2.TDZ.

Backports commit 4351cb72fb65926136ab618c9e40c1f5a8813251 from qemu
2020-03-21 15:30:37 -04:00
Richard Henderson 3a5135473f target/arm: Update arm_sctlr for VHE
Use the correct sctlr for EL2&0 regime. Due to header ordering,
and where arm_mmu_idx_el is declared, we need to move the function
out of line. Use the function in many more places in order to
select the correct control.

Backports commit aaec143212bb70ac9549cf73203d13100bd5c7c2 from qemu
2020-03-21 15:29:21 -04:00
Richard Henderson 6073542afc target/arm: Update arm_mmu_idx for VHE
Return the indexes for the EL2&0 regime when the appropriate bits
are set within HCR_EL2.

Backports commit 6003d9800ee38aa11eefb5cd64ae55abb64bef16 from qemu
2020-03-21 15:23:38 -04:00
Richard Henderson 3d583cd45f target/arm: Split out arm_mmu_idx_el
Backports commit 164690b29f9eaf69fe641859bc9f8954f12e691d from qemu
2020-03-21 15:22:01 -04:00
Richard Henderson f4397b0212 target/arm: Add regime_has_2_ranges
Create a predicate to indicate whether the regime has
both positive and negative addresses.

Backports commit 339370b90d067345b69585ddf4b668fa01f41d67 from qemu
2020-03-21 15:14:11 -04:00
Richard Henderson 0318d7af99 target/arm: Reorganize ARMMMUIdx
Prepare for, but do not yet implement, the EL2&0 regime.
This involves adding the new MMUIdx enumerators and adjusting
some of the MMUIdx related predicates to match.

Backports commit b9f6033c1a5fb7da55ed353794db8ec064f78bb2 from qemu.
2020-03-21 15:10:05 -04:00
Richard Henderson be3c71fb8b target/arm: Recover 4 bits from TBFLAGs
We had completely run out of TBFLAG bits.
Split A- and M-profile bits into two overlapping buckets.
This results in 4 free bits.

We used to initialize all of the a32 and m32 fields in DisasContext
by assignment, in arm_tr_init_disas_context. Now we only initialize
either the a32 or m32 by assignment, because the bits overlap in
tbflags. So zero the entire structure in gen_intermediate_code.

Backports commit 79cabf1f473ca6e9fa0727f64ed9c2a84a36f0aa from qemu
2020-03-21 14:51:46 -04:00
Richard Henderson 153d7aadd5 target/arm: Rename ARMMMUIdx_S1E2 to ARMMMUIdx_E2
This is part of a reorganization to the set of mmu_idx.
The non-secure EL2 regime only has a single stage translation;
there is no point in pointing out that the idx is for stage1.

Backports commit e013b7411339342aac8d986c5d5e329e1baee8e1 from qemu
2020-03-21 14:42:23 -04:00
Richard Henderson f45ab0614e target/arm: Rename ARMMMUIdx*_S1E3 to ARMMMUIdx*_SE3
This is part of a reorganization to the set of mmu_idx.
The EL3 regime only has a single stage translation, and
is always secure.

Backports commit 127b2b086303296289099a6fb10bbc51077f1d53 from qemu
2020-03-21 14:38:44 -04:00
Richard Henderson 1a672fc3b1 target/arm: Rename ARMMMUIdx_S1SE[01] to ARMMMUIdx_SE10_[01]
This is part of a reorganization to the set of mmu_idx.
This emphasizes that they apply to the Secure EL1&0 regime.

Backports commit fba37aedecb82506c62a1f9e81d066b4fd04e443 from qemu
2020-03-21 14:35:28 -04:00
Richard Henderson 31837384b3 target/arm: Rename ARMMMUIdx_S1NSE* to ARMMMUIdx_Stage1_E*
This is part of a reorganization to the set of mmu_idx.
The EL1&0 regime is the only one that uses 2-stage translation.
Spelling out Stage avoids confusion with Secure.

Backports commit 2859d7b590760283a7b5aef40b723e9dfd7c98ba from qemu
2020-03-21 14:20:31 -04:00
Richard Henderson b62b4c4f35 target/arm: Rename ARMMMUIdx_S2NS to ARMMMUIdx_Stage2
The EL1&0 regime is the only one that uses 2-stage translation.

Backports commit 97fa9350017e647151dd1dc212f1bbca0294dba7 from qemu
2020-03-21 14:15:35 -04:00
Richard Henderson ec05f22e82 target/arm: Rename ARMMMUIdx*_S12NSE* to ARMMMUIdx*_E10_*
This is part of a reorganization to the set of mmu_idx.
This emphasizes that they apply to the EL1&0 regime.

The ultimate goal is

-- Non-secure regimes:
ARMMMUIdx_E10_0,
ARMMMUIdx_E20_0,
ARMMMUIdx_E10_1,
ARMMMUIdx_E2,
ARMMMUIdx_E20_2,

-- Secure regimes:
ARMMMUIdx_SE10_0,
ARMMMUIdx_SE10_1,
ARMMMUIdx_SE3,

-- Helper mmu_idx for non-secure EL1&0 stage1 and stage2
ARMMMUIdx_Stage2,
ARMMMUIdx_Stage1_E0,
ARMMMUIdx_Stage1_E1,

The 'S' prefix is reserved for "Secure". Unless otherwise specified,
each mmu_idx represents all stages of translation.

Backports commit 01b98b686460b3a0fb47125882e4f8d4268ac1b6 from qemu
2020-03-21 14:09:15 -04:00
Richard Henderson 270d557a99 target/arm: Split out alle1_tlbmask
No functional change, but unify code sequences.

Backports commit 90c19cdf1de440d7d9745cf255168999071b3a31 from qemu
2020-03-21 13:57:04 -04:00
Richard Henderson 6d4a7b84b5 target/arm: Split out vae1_tlbmask
No functional change, but unify code sequences.

Backports commit b7e0730de32d7079a1447ecbb5616d89de77b823 from qemu
2020-03-21 13:53:39 -04:00
Richard Henderson 4f4c385a8e target/arm: Update CNTVCT_EL0 for VHE
The virtual offset may be 0 depending on EL, E2H and TGE.

Backports commit 53d1f85608f83d645491eba6581d1f300dba2384 from qemu
2020-03-21 13:50:35 -04:00
Richard Henderson 215b4a9851 target/arm: Add TTBR1_EL2
At the same time, add writefn to TTBR0_EL2 and TCR_EL2.
A later patch will update any ASID therein.

Backports commit ed30da8eee6906032b38a84e4807e2142b09d8ec from qemu
2020-03-21 13:47:56 -04:00
Richard Henderson 35508d46c7 target/arm: Add CONTEXTIDR_EL2
Not all of the breakpoint types are supported, but those that
only examine contextidr are extended to support the new register.

Backports commit e2a1a4616c86159eb4c07659a02fff8bb25d3729 from qemu
2020-03-21 13:39:20 -04:00
Richard Henderson fe6825ca4d target/arm: Enable HCR_E2H for VHE
Backports commit 03c76131bc494366a4357a1d265c5eb5cc820754 from qemu
2020-03-21 13:35:00 -04:00
Alex Bennée 8f275077b0 target/arm: only update pc after semihosting completes
Before we introduce blocking semihosting calls we need to ensure we
can restart the system on semi hosting exception. To be able to do
this the EXCP_SEMIHOST operation should be idempotent until it finally
completes. Practically this means ensureing we only update the pc
after the semihosting call has completed.

Backports commit 4ff5ef9e911c670ca10cdd36dd27c5395ec2c753 from qemu
2020-01-14 08:28:25 -05:00
Alex Bennée ea2714796f target/arm: remove unused EXCP_SEMIHOST leg
All semihosting exceptions are dealt with earlier in the common code
so we should never get here.

Backports commit b906acbb3aceed5b1eca30d9d365d5bd7431400b from qemu
2020-01-14 08:18:51 -05:00
Alex Bennée 639c5c4fe2 target/arm: ensure we use current exception state after SCR update
A write to the SCR can change the effective EL by droppping the system
from secure to non-secure mode. However if we use a cached current_el
from before the change we'll rebuild the flags incorrectly. To fix
this we introduce the ARM_CP_NEWEL CP flag to indicate the new EL
should be used when recomputing the flags.

Backports partof commit f80741d107673f162e3b097fc76a1590036cc9d1 from
qemu
2020-01-14 07:51:10 -05:00
Beata Michalska 81c14bb595 target/arm: Add support for DC CVAP & DC CVADP ins
ARMv8.2 introduced support for Data Cache Clean instructions
to PoP (point-of-persistence) - DC CVAP and PoDP (point-of-deep-persistence)
- DV CVADP. Both specify conceptual points in a memory system where all writes
that are to reach them are considered persistent.
The support provided considers both to be actually the same so there is no
distinction between the two. If none is available (there is no backing store
for given memory) both will result in Data Cache Clean up to the point of
coherency. Otherwise sync for the specified range shall be performed.

Backports commit 0d57b49992200a926c4436eead97ecfc8cc710be from qemu
2020-01-14 07:47:48 -05:00
Marc Zyngier 9c3e512479 target/arm: Add support for missing Jazelle system registers
QEMU lacks the minimum Jazelle implementation that is required
by the architecture (everything is RAZ or RAZ/WI). Add it
together with the HCR_EL2.TID0 trapping that goes with it.

Backports commit f96f3d5f09973ef40f164cf2d5fd98ce5498b82a from qemu
2020-01-07 18:09:13 -05:00
Marc Zyngier 457934855b target/arm: Handle AArch32 CP15 trapping via HSTR_EL2
HSTR_EL2 offers a way to trap ranges of CP15 system register
accesses to EL2, and it looks like this register is completely
ignored by QEMU.

To avoid adding extra .accessfn filters all over the place (which
would have a direct performance impact), let's add a new TB flag
that gets set whenever HSTR_EL2 is non-zero and that QEMU translates
a context where this trap has a chance to apply, and only generate
the extra access check if the hypervisor is actively using this feature.

Tested with a hand-crafted KVM guest accessing CBAR.

Backports commit 5bb0a20b74ad17dee5dae38e3b8b70b383ee7c2d from qemu
2020-01-07 18:07:21 -05:00
Marc Zyngier 51062d3fc2 target/arm: Honor HCR_EL2.TID1 trapping requirements
HCR_EL2.TID1 mandates that access from EL1 to REVIDR_EL1, AIDR_EL1
(and their 32bit equivalents) as well as TCMTR, TLBTR are trapped
to EL2. QEMU ignores it, making it harder for a hypervisor to
virtualize the HW (though to be fair, no known hypervisor actually
cares).

Do the right thing by trapping to EL2 if HCR_EL2.TID1 is set.

Backports commit 93fbc983b29a2eb84e2f6065929caf14f99c3681 from qemu
2020-01-07 18:00:01 -05:00
Marc Zyngier d1e981c44b target/arm: Honor HCR_EL2.TID2 trapping requirements
HCR_EL2.TID2 mandates that access from EL1 to CTR_EL0, CCSIDR_EL1,
CCSIDR2_EL1, CLIDR_EL1, CSSELR_EL1 are trapped to EL2, and QEMU
completely ignores it, making it impossible for hypervisors to
virtualize the cache hierarchy.

Do the right thing by trapping to EL2 if HCR_EL2.TID2 is set.

Backports commit 630fcd4d2ba37050329e0adafdc552d656ebe2f3 from qemu
2020-01-07 17:55:40 -05:00
Marc Zyngier 145d58c367
target/arm: Honor HCR_EL2.TID3 trapping requirements
HCR_EL2.TID3 mandates that access from EL1 to a long list of id
registers traps to EL2, and QEMU has so far ignored this requirement.

This breaks (among other things) KVM guests that have PtrAuth enabled,
while the hypervisor doesn't want to expose the feature to its guest.
To achieve this, KVM traps the ID registers (ID_AA64ISAR1_EL1 in this
case), and masks out the unsupported feature.

QEMU not honoring the trap request means that the guest observes
that the feature is present in the HW, starts using it, and dies
a horrible death when KVM injects an UNDEF, because the feature
*really* isn't supported.

Do the right thing by trapping to EL2 if HCR_EL2.TID3 is set.

Note that this change does not include trapping of the MVFR
registers from AArch32 (they are accessed via the VMRS
instruction and need to be handled in a different way).

Backports commit 6a4ef4e5d1084ce41fafa7d470a644b0fd3d9317 from qemu
2019-11-28 03:46:32 -05:00
Marc Zyngier 2e8c8b5a7c
target/arm: Fix ISR_EL1 tracking when executing at EL2
The ARMv8 ARM states when executing at EL2, EL3 or Secure EL1,
ISR_EL1 shows the pending status of the physical IRQ, FIQ, or
SError interrupts.

Unfortunately, QEMU's implementation only considers the HCR_EL2
bits, and ignores the current exception level. This means a hypervisor
trying to look at its own interrupt state actually sees the guest
state, which is unexpected and breaks KVM as of Linux 5.3.

Instead, check for the running EL and return the physical bits
if not running in a virtualized context.

Backports commit 7cf95aed53c8770a338617ef40d5f37d2c197853 from qemu
2019-11-28 03:41:38 -05:00
Peter Maydell 9fb54a7f72
target/arm: Take exceptions on ATS instructions when needed
The translation table walk for an ATS instruction can result in
various faults. In general these are just reported back via the
PAR_EL1 fault status fields, but in some cases the architecture
requires that the fault is turned into an exception:
* synchronous stage 2 faults of any kind during AT S1E0* and
AT S1E1* instructions executed from NS EL1 fault to EL2 or EL3
* synchronous external aborts are taken as Data Abort exceptions

(This is documented in the v8A Arm ARM DDI0487A.e D5.2.11 and
G5.13.4.)

Backports commit 0710b2fa84a4aeb925422e1e88edac49ed407c79 from qemu
2019-11-20 17:24:44 -05:00
Andrew Jones bb8b3bc42b
target/arm/helper: zcr: Add build bug next to value range assumption
The current implementation of ZCR_ELx matches the architecture, only
implementing the lower four bits, with the rest RAZ/WI. This puts
a strict limit on ARM_MAX_VQ of 16. Make sure we don't let ARM_MAX_VQ
grow without a corresponding update here.

Backports commit 7b351d98709d3f77d6bb18562e1bf228862b0d57 from qemu
2019-11-18 20:14:42 -05:00
Peter Maydell c61e22627d
target/arm: Fix routing of singlestep exceptions
When generating an architectural single-step exception we were
routing it to the "default exception level", which is to say
the same exception level we execute at except that EL0 exceptions
go to EL1. This is incorrect because the debug exception level
can be configured by the guest for situations such as single
stepping of EL0 and EL1 code by EL2.

We have to track the target debug exception level in the TB
flags, because it is dependent on CPU state like HCR_EL2.TGE
and MDCR_EL2.TDE. (That we were previously calling the
arm_debug_target_el() function to determine dc->ss_same_el
is itself a bug, though one that would only have manifested
as incorrect syndrome information.) Since we are out of TB
flag bits unless we want to expand into the cs_base field,
we share some bits with the M-profile only HANDLER and
STACKCHECK bits, since only A-profile has this singlestep.

Fixes: https://bugs.launchpad.net/qemu/+bug/1838913

Backports commit 8bd587c1066f4456ddfe611b571d9439a947d74c from qemu
2019-11-18 16:50:15 -05:00
Philippe Mathieu-Daudé 9bd010263a
target/arm: Add missing break statement for Hypervisor Trap Exception
Reported by GCC9 when building with -Wimplicit-fallthrough=2:

target/arm/helper.c: In function ‘arm_cpu_do_interrupt_aarch32_hyp’:
target/arm/helper.c:7958:14: error: this statement may fall through [-Werror=implicit-fallthrough=]
7958 | addr = 0x14;
| ~~~~~^~~~~~
target/arm/helper.c:7959:5: note: here
7959 | default:
| ^~~~~~~
cc1: all warnings being treated as errors

Backports commit 9bbb4ef991fa93323f87769a6e217c2b9273a128 from qemu
2019-08-08 19:43:01 -04:00
Richard Henderson c687259bf6
target/arm: Fix sve_zcr_len_for_el
Off by one error in the EL2 and EL3 tests. Remove the test
against EL3 entirely, since it must always be true.

Backports commit 6a02a73211c5bc634fccd652777230954b83ccba from qemu
2019-08-08 19:20:35 -04:00
Philippe Mathieu-Daudé bde186433d
target/arm/helper: Move M profile routines to m_helper.c
In preparation for supporting TCG disablement on ARM, we move most
of TCG related v7m/v8m helpers and APIs into their own file.

Note: It is easier to review this commit using the 'histogram'
diff algorithm:

$ git diff --diff-algorithm=histogram ...
or
$ git diff --histogram ...

Backports commit 7aab5a8c8bb525ea390b4ebc17ab82c0835cfdb6 from qemu
2019-08-08 18:04:08 -04:00
Philippe Mathieu-Daudé 199e2f8a7d
target/arm: Restrict semi-hosting to TCG
Semihosting hooks either SVC or HLT instructions, and inside KVM
both of those go to EL1, ie to the guest, and can't be trapped to
KVM.

Let check_for_semihosting() return False when not running on TCG.

backports commit 91f78c58da9ba78c8ed00f5d822b701765be8499 from qemu
2019-08-08 17:48:34 -04:00
Philippe Mathieu-Daudé fa2a772c7b
target/arm: Declare some M-profile functions publicly
In the next commit we will split the M-profile functions from this
file. Some function will be called out of helper.c. Declare them in
the "internals.h" header.

Backports commit 787a7e76c2e93a48c47b324fea592c9910a70483 from qemu
2019-08-08 15:37:01 -04:00
Philippe Mathieu-Daudé 91e264823e
target/arm: Move TLB related routines to tlb_helper.c
These routines are TCG specific.
The arm_deliver_fault() function is only used within the new
helper. Make it static.

Backports commit e21b551cb652663f2f2405a64d63ef6b4a1042b7 from qemu
2019-08-08 15:24:26 -04:00
Philippe Mathieu-Daudé 1af5deaf52
target/arm: Declare get_phys_addr() function publicly
In the next commit we will split the TLB related routines of
this file, and this function will also be called in the new
file. Declare it in the "internals.h" header.

Backports commit ebae861fc6c385a7bcac72dde4716be06e6776f1 from qemu
2019-08-08 15:14:45 -04:00
Samuel Ortiz 9296465289
target/arm: Move the DC ZVA helper into op_helper
Those helpers are a software implementation of the ARM v8 memory zeroing
op code. They should be moved to the op helper file, which is going to
eventually be built only when TCG is enabled.

Backports commit 6cdca173ef81a9dbcee9e142f1a5a34ad9c44b75 from qemu
2019-08-08 15:10:46 -04:00
Philippe Mathieu-Daudé 4f71266524
target/arm: Fix multiline comment syntax
Since commit 8c06fbdf36b checkpatch.pl enforce a new multiline
comment syntax. Since we'll move this code around, fix its style
first.

Backports commit 9a223097e44d5320f5e0546710263f22d11f12fc from qemu
2019-08-08 15:05:37 -04:00
Philippe Mathieu-Daudé 0a5152caf8
target/arm/helper: Remove unused include
Backports commit 2c8ec397f8438eea5e52be4898dfcf12a1f88267 from qemu
2019-08-08 14:43:57 -04:00
Philippe Mathieu-Daudé 8a0bae93ff
target/arm: Add copyright boilerplate
Backports commit ed3baad15b0b0edc480373f9c1d805d6b8e7e78c from qemu
2019-08-08 14:43:32 -04:00
Peter Maydell 230f8a091a
target/arm: Implement NSACR gating of floating point
The NSACR register allows secure code to configure the FPU
to be inaccessible to non-secure code. If the NSACR.CP10
bit is set then:
* NS accesses to the FPU trap as UNDEF (ie to NS EL1 or EL2)
* CPACR.{CP10,CP11} behave as if RAZ/WI
* HCPTR.{TCP11,TCP10} behave as if RAO/WI

Note that we do not implement the NSACR.NSASEDIS bit which
gates only access to Advanced SIMD, in the same way that
we don't implement the equivalent CPACR.ASEDIS and HCPTR.TASE.

Backports commit fc1120a7f5f2d4b601003205c598077d3eb11ad2 from qemu
2019-06-13 16:15:28 -04:00
Richard Henderson b8bd543390
target/arm: Use env_cpu, env_archcpu
Cleanup in the boilerplate that each target must define.
Replace arm_env_get_cpu with env_archcpu. The combination
CPU(arm_env_get_cpu) should have used ENV_GET_CPU to begin;
use env_cpu now.

Backports commit 2fc0cc0e1e034582f4718b1a2d57691474ccb6aa from qemu
2019-06-12 11:34:08 -04:00
Richard Henderson fbf91a6535
cpu: Replace ENV_GET_CPU with env_cpu
Now that we have both ArchCPU and CPUArchState, we can define
this generically instead of via macro in each target's cpu.h.

Backports commit 29a0af618ddd21f55df5753c3e16b0625f534b3c from qemu
2019-06-12 11:16:16 -04:00
Richard Henderson 3dd7358a53
target/arm: Implement ARMv8.5-RNG
Use the newly introduced infrastructure for guest random numbers.

Backports commit de390645675966cce113bf5394445bc1f8d07c85 from qemu

(with the actual RNG portion disabled to preserve determinism for the
time being).
2019-05-23 15:03:23 -04:00
Richard Henderson a8df33c37c
target/arm: Put all PAC keys into a structure
This allows us to use a single syscall to initialize them all.

Backports commit 108b3ba891408c4dce93df78261ec4aca38c0e2e from qemu
2019-05-23 14:54:06 -04:00
Richard Henderson dab0061a0d
tcg: Use CPUClass::tlb_fill in cputlb.c
We can now use the CPUClass hook instead of a named function.

Create a static tlb_fill function to avoid other changes within
cputlb.c. This also isolates the asserts within. Remove the
named tlb_fill function from all of the targets.

Backports commit c319dc13579a92937bffe02ad2c9f1a550e73973 from qemu
2019-05-16 17:35:37 -04:00
Richard Henderson 31ecdb5341
target/arm: Convert to CPUClass::tlb_fill
Backports commit 7350d553b5066abdc662045d7db5cdb73d0f9d53 from qemu
2019-05-16 16:55:12 -04:00
Peter Maydell 26cb1b8767
target/arm: Stop using variable length array in dc_zva
Currently the dc_zva helper function uses a variable length
array. In fact we know (as the comment above remarks) that
the length of this array is bounded because the architecture
limits the block size and QEMU limits the target page size.
Use a fixed array size and assert that we don't run off it.

Backports commit 63159601fb3e396b28da14cbb71e50ed3f5a0331 from qemu
2019-05-09 17:48:25 -04:00
Peter Maydell 7861820e94
target/arm: Implement XPSR GE bits
In the M-profile architecture, if the CPU implements the DSP extension
then the XPSR has GE bits, in the same way as the A-profile CPSR. When
we added DSP extension support we forgot to add support for reading
and writing the GE bits, which are stored in env->GE. We did put in
the code to add XPSR_GE to the mask of bits to update in the v7m_msr
helper, but forgot it in v7m_mrs. We also must not allow the XPSR we
pull off the stack on exception return to set the nonexistent GE bits.
Correct these errors:
* read and write env->GE in xpsr_read() and xpsr_write()
* only set GE bits on exception return if DSP present
* read GE bits for MRS if DSP present

Backports commit f1e2598c46d480c9e21213a244bc514200762828 from qemu
2019-05-09 17:46:31 -04:00
Peter Maydell 77ae3982b4
target/arm: Implement VLLDM for v7M CPUs with an FPU
Implement the VLLDM instruction for v7M for the FPU present cas.

Backports commit 956fe143b4f254356496a0a1c479fa632376dfec from qemu
2019-04-30 11:27:54 -04:00
Peter Maydell b483951046
target/arm: Implement VLSTM for v7M CPUs with an FPU
Implement the VLSTM instruction for v7M for the FPU present case.

Backports commit 019076b036da4444494de38388218040d9d3a26c from qemu
2019-04-30 11:25:44 -04:00
Peter Maydell a976d7642a
target/arm: Implement M-profile lazy FP state preservation
The M-profile architecture floating point system supports
lazy FP state preservation, where FP registers are not
pushed to the stack when an exception occurs but are instead
only saved if and when the first FP instruction in the exception
handler is executed. Implement this in QEMU, corresponding
to the check of LSPACT in the pseudocode ExecuteFPCheck().

Backports commit e33cf0f8d8c9998a7616684f9d6aa0d181b88803 from qemu
2019-04-30 11:21:50 -04:00
Peter Maydell 72e5ae480d
target/arm: Add lazy-FP-stacking support to v7m_stack_write()
Pushing registers to the stack for v7M needs to handle three cases:
* the "normal" case where we pend exceptions
* an "ignore faults" case where we set FSR bits but
do not pend exceptions (this is used when we are
handling some kinds of derived exception on exception entry)
* a "lazy FP stacking" case, where different FSR bits
are set and the exception is pended differently

Implement this by changing the existing flag argument that
tells us whether to ignore faults or not into an enum that
specifies which of the 3 modes we should handle.

Backports commit a356dacf647506bccdf8ecd23574246a8bf615ac from qemu
2019-04-30 10:59:53 -04:00