Commit graph

482 commits

Author SHA1 Message Date
Peter Collingbourne de7bcbae57 target/arm: Use TCF0 and TFSRE0 for unprivileged tag checks
Section D6.7 of the ARM ARM states:

For the purpose of determining Tag Check Fault handling, unprivileged
load and store instructions are treated as if executed at EL0 when
executed at either:
- EL1, when the Effective value of PSTATE.UAO is 0.
- EL2, when both the Effective value of HCR_EL2.{E2H, TGE} is {1, 1}
and the Effective value of PSTATE.UAO is 0.

ARM has confirmed a defect in the pseudocode function
AArch64.TagCheckFault that makes it inconsistent with the above
wording. The remedy is to adjust references to PSTATE.EL in that
function to instead refer to AArch64.AccessUsesEL(acctype), so
that unprivileged instructions use SCTLR_EL1.TCF0 and TFSRE0_EL1.
The exception type for synchronous tag check faults remains unchanged.

This patch implements the described change by partially reverting
commits 50244cc76abc and cc97b0019bb5.

Backports 2d928adf8a9148510e1b2041145b8a873f4d26df
2021-03-08 11:34:03 -05:00
Rebecca Cran 01105515c7 target/arm: Add support for FEAT_SSBS, Speculative Store Bypass Safe
Add support for FEAT_SSBS. SSBS (Speculative Store Bypass Safe) is an
optional feature in ARMv8.0, and mandatory in ARMv8.5.

Backports f2f68a78b793808b84367bc708d632969d4440aa
2021-03-08 11:23:50 -05:00
Daniel Müller 642a683d7a target/arm: Correctly initialize MDCR_EL2.HPMN
When working with performance monitoring counters, we look at
MDCR_EL2.HPMN as part of the check whether a counter is enabled. This
check fails, because MDCR_EL2.HPMN is reset to 0, meaning that no
counters are "enabled" for < EL2.
That's in violation of the Arm specification, which states that

> On a Warm reset, this field [MDCR_EL2.HPMN] resets to the value in
> PMCR_EL0.N

That's also what a comment in the code acknowledges, but the necessary
adjustment seems to have been forgotten when support for more counters
was added.
This change fixes the issue by setting the reset value to PMCR.N, which
is four.

Backports d3c1183ffeb71ca3a783eae3d7e1c51e71e8a621
2021-03-04 18:34:06 -05:00
Rebecca Cran f7424d89e2 target/arm: Support AA32 DIT by moving PSTATE_SS from cpsr into env->pstate
cpsr has been treated as being the same as spsr, but it isn't.
Since PSTATE_SS isn't in cpsr, remove it and move it into env->pstate.

This allows us to add support for CPSR_DIT, adding helper functions
to merge SPSR_ELx to and from CPSR.

Backports f944a854ce4007000accf7c191b5b52916947198
2021-03-04 18:24:57 -05:00
Rebecca Cran d8458f14af target/arm: Add support for FEAT_DIT, Data Independent Timing
Add support for FEAT_DIT. DIT (Data Independent Timing) is a required
feature for ARMv8.4. Since virtual machine execution is largely
nondeterministic and TCG is outside of the security domain, it's
implemented as a NOP.

Backports dc8b18534ea1dcc90d80ad9a61a3b0aa7eb312fb
2021-03-04 18:19:32 -05:00
Mike Nawrocki 4e482764e2 target/arm: Fix SCR RES1 handling
The FW and AW bits of SCR_EL3 are RES1 only in some contexts. Force them
to 1 only when there is no support for AArch32 at EL1 or above.

The reset value will be 0x30 only if the CPU is AArch64-only; if there
is support for AArch32 at EL1 or above, it will be reset to 0.

Also adds helper function isar_feature_aa64_aa32_el1 to check if AArch32
is supported at EL1 or above.

Backports 10d0ef3e6cfe228df4b2d3e27325f1b0e2b71fd5
2021-03-04 18:15:39 -05:00
Claudio Fontana 11ae599cb8 target/arm: do not use cc->do_interrupt for KVM directly
cc->do_interrupt is in theory a TCG callback used in accel/tcg only,
to prepare the emulated architecture to take an interrupt as defined
in the hardware specifications,

but in reality the _do_interrupt style of functions in targets are
also occasionally reused by KVM to prepare the architecture state in a
similar way where userspace code has identified that it needs to
deliver an exception to the guest.

In the case of ARM, that includes:

1) the vcpu thread got a SIGBUS indicating a memory error,
and we need to deliver a Synchronous External Abort to the guest to
let it know about the error.
2) the kernel told us about a debug exception (breakpoint, watchpoint)
but it is not for one of QEMU's own gdbstub breakpoints/watchpoints
so it must be a breakpoint the guest itself has set up, therefore
we need to deliver it to the guest.

So in order to reuse code, the same arm_do_interrupt function is used.
This is all fine, but we need to avoid calling it using the callback
registered in CPUClass, since that one is now TCG-only.

Fortunately this is easily solved by replacing calls to
CPUClass::do_interrupt() with explicit calls to arm_do_interrupt().

Backports 853bfef4e6d60244fd131ec55bbf1e7caa52599b. We don't support
KVM, so we just bring the comment addition over.
2021-03-04 16:33:23 -05:00
Philippe Mathieu-Daudé daafb0ba17 target/arm: Replace magic value by MMU_DATA_LOAD definition
cpu_get_phys_page_debug() uses 'DATA LOAD' MMU access type.

Backports a9dd161ff2f54446f0b0547447d8196699aca3e1
2021-03-04 15:43:47 -05:00
Richard Henderson 2c8f7b1fbc target/arm: Conditionalize DBGDIDR
Only define the register if it exists for the cpu.

Backports 54a78718be6dd5fc6b6201f84bef8de5ac3b3802
2021-03-04 15:42:03 -05:00
Richard Henderson 073923709f target/arm: Implement ID_PFR2
This was defined at some point before ARMv8.4, and will
shortly be used by new processor descriptions.

Backports 1d51bc96cc4a9b2d31a3f4cb8442ce47753088e2
2021-03-04 15:40:49 -05:00
Rémi Denis-Courmont 9dfa469976 target/arm: refactor vae1_tlbmask()
Backports bc944d3a8b305029196a5e1406702a92fa0b94cf
2021-03-04 15:05:54 -05:00
Rémi Denis-Courmont e6d32dc2e0 target/arm: Implement SCR_EL2.EEL2
This adds handling for the SCR_EL3.EEL2 bit.

Backports 926c1b97895879b78ca14bca2831c08740ed1c38
2021-03-04 15:03:08 -05:00
Rémi Denis-Courmont ce8872709f target/arm: set HPFAR_EL2.NS on secure stage 2 faults
Backport 9861248f637ecf11113b04b0b5c7b13c9aa06f09
2021-03-04 14:54:33 -05:00
Rémi Denis-Courmont b49531cfef target/arm: secure stage 2 translation regime
b1a10c868f9b2b09e64009b43450e9a86697d9f3
2021-03-04 14:49:33 -05:00
Rémi Denis-Courmont eeefc3c4a2 target/arm: generalize 2-stage page-walk condition
The stage_1_mmu_idx() already effectively keeps track of which
translation regimes have two stages. Don't hard-code another test.

Backports 7879460a6149ed5e80c29cac85449191d9c5754a
2021-03-04 14:26:22 -05:00
Rémi Denis-Courmont 07ebb7f7ba target/arm: translate NS bit in page-walks
588c6dd113b27b8db393c7264297b9d33261692e
2021-03-04 14:25:13 -05:00
Rémi Denis-Courmont 6f57520b1d target/arm: do S1_ptw_translate() before address space lookup
In the secure stage 2 translation regime, the VSTCR.SW and VTCR.NSW
bits can invert the secure flag for pagetable walks. This patchset
allows S1_ptw_translate() to change the non-secure bit.

Backports 3d4bd397433b12b148d150c8bc5655a696389bd1
2021-03-04 14:23:43 -05:00
Rémi Denis-Courmont ce50ba6d07 target/arm: handle VMID change in secure state
The VTTBR write callback so far assumes that the underlying VM lies in
non-secure state. This handles the secure state scenario.

backports c4f060e89effd70ebdb23d3315495d33af377a09
2021-03-04 14:20:47 -05:00
Rémi Denis-Courmont a78c31e36a target/arm: add ARMv8.4-SEL2 system registers
Backports e9152ee91cc39ed8a53d03607e6e980a7e9444e6
2021-03-04 14:20:10 -05:00
Rémi Denis-Courmont edd5f021e6 target/arm: add MMU stage 1 for Secure EL2
This adds the MMU indices for EL2 stage 1 in secure state.

To keep code contained, which is largelly identical between secure and
non-secure modes, the MMU indices are reassigned. The new assignments
provide a systematic pattern with a non-secure bit.

Backports b6ad6062f1e55bd5b9407ce89e55e3a08b83827c
2021-03-04 14:16:31 -05:00
Rémi Denis-Courmont fbdcef3ca5 target/arm: add 64-bit S-EL2 to EL exception table
With the ARMv8.4-SEL2 extension, EL2 is a legal exception level in
secure mode, though it can only be AArch64.

This patch adds the target EL for exceptions from 64-bit S-EL2.

It also fixes the target EL to EL2 when HCR.{A,F,I}MO are set in secure
mode. Those values were never used in practice as the effective value of
HCR was always 0 in secure mode.

Backports 6c85f906261226e87211506bd9f787fd48a09f17
2021-03-04 14:00:23 -05:00
Rémi Denis-Courmont b42e6d6036 target/arm: factor MDCR_EL2 common handling
This adds a common helper to compute the effective value of MDCR_EL2.
That is the actual value if EL2 is enabled in the current security
context, or 0 elsewise.

Backports 59dd089cf9e4a9cddee596c8a1378620df51b9bb
2021-03-04 13:57:34 -05:00
Rémi Denis-Courmont b657bfc59b target/arm: use arm_hcr_el2_eff() where applicable
This will simplify accessing HCR conditionally in secure state.

Backports e04a5752cb03e066d7b1e583e340c7982fcd5e4e
2021-03-04 13:53:30 -05:00
Rémi Denis-Courmont 58af3e76e6 target/arm: use arm_is_el2_enabled() where applicable
Do not assume that EL2 is available in and only in non-secure context.
That equivalence is broken by ARMv8.4-SEL2.

Backports e6ef0169264b00cce552404f689ce137018ff290
2021-03-04 13:49:19 -05:00
Rémi Denis-Courmont 7402645436 target/arm: remove redundant tests
In this context, the HCR value is the effective value, and thus is
zero in secure mode. The tests for HCR.{F,I}MO are sufficient.

Backports cc974d5cd84ea60a3dad59752aea712f3d47f8ce
2021-03-04 13:42:12 -05:00
Rémi Denis-Courmont c7415c92d5 target/arm: ARMv8.4-TTST extension
This adds for the Small Translation tables extension in AArch64 state.

Backports c36c65ea3c35b309d524c05a1c05fdeabf83ddd5
2021-03-03 20:09:01 -05:00
Richard Henderson 85b417d438 target/arm: Fix MTE0_ACTIVE
In 50244cc76abc we updated mte_check_fail to match the ARM
pseudocode, using the correct EL to select the TCF field.
But we failed to update MTE0_ACTIVE the same way, which led
to g_assert_not_reached().

Backports cc97b0019bb590b9b3c2a623e9ebee48831e0ce3
2021-03-03 19:56:23 -05:00
Peter Maydell cfefada296 target/arm: Implement v8.1M PXN extension
In v8.1M the PXN architecture extension adds a new PXN bit to the
MPU_RLAR registers, which forbids execution of code in the region
from a privileged mode.

This is another feature which is just in the generic "in v8.1M" set
and has no ID register field indicating its presence.

Backports cad8e2e3160dd10371552fce6cd8c6e171503e13
2021-03-03 17:50:26 -05:00
Rémi Denis-Courmont d9592046ef target/arm: fix stage 2 page-walks in 32-bit emulation
Using a target unsigned long would limit the Input Address to a LPAE
page-walk to 32 bits on AArch32 and 64 bits on AArch64. This is okay
for stage 1 or on AArch64, but it is insufficient for stage 2 on
AArch32. In that later case, the Input Address can have up to 40 bits.

Backports commit 98e8779770c40901ed585745aacc9a8e2b934a28
2021-03-02 13:37:02 -05:00
Xinhao Zhang 492fbc4d2c target/arm: add spaces around operator
Fix code style. Operator needs spaces both sides.

Backports bdc3b6f570e8bd219aa6a24a149b35a691e6986c
2021-03-02 13:15:12 -05:00
Rémi Denis-Courmont a4053565d6 target/arm: fix LORID_EL1 access check
Secure mode is not exempted from checking SCR_EL3.TLOR, and in the
future HCR_EL2.TLOR when S-EL2 is enabled.

Backports 9bd268bae5c4760870522292fb1d46e7da7e372a
2021-03-02 13:06:50 -05:00
Rémi Denis-Courmont df4413edc7 target/arm: fix handling of HCR.FB
HCR should be applied when NS is set, not when it is cleared.

Backports 373e7ffde9bae90a20fb5db21b053f23091689f4
2021-03-02 13:05:01 -05:00
Richard Henderson f7e831a7e4 target/arm: Ignore HCR_EL2.ATA when {E2H,TGE} != 11
Unlike many other bits in HCR_EL2, the description for this
bit does not contain the phrase "if ... this field behaves
as 0 for all purposes other than", so do not squash the bit
in arm_hcr_el2_eff.

Instead, replicate the E2H+TGE test in the two places that
require it.

Backports 4301acd7d7d455792ea873ced75c0b5d653618b1
2021-03-01 20:12:36 -05:00
Peter Maydell 09a7d6381e target/arm: Move id_pfr0, id_pfr1 into ARMISARegisters
Move the id_pfr0 and id_pfr1 fields into the ARMISARegisters
sub-struct. We're going to want id_pfr1 for an isar_features
check, and moving both at the same time avoids an odd
inconsistency.

Changes other than the ones to cpu.h and kvm64.c made
automatically with:
perl -p -i -e 's/cpu->id_pfr/cpu->isar.id_pfr/' target/arm/*.c hw/intc/armv7m_nvic.c

Backports commit 8a130a7be6e222965641e1fd9469fd3ee752c7d4
2021-03-01 19:15:10 -05:00
Peter Maydell ed92f3c42b target/arm: Replace ARM_FEATURE_PXN with ID_MMFR0.VMSA check
The ARM_FEATURE_PXN bit indicates whether the CPU supports the PXN
bit in short-descriptor translation table format descriptors. This
is indicated by ID_MMFR0.VMSA being at least 0b0100. Replace the
feature bit with an ID register check, in line with our preference
for ID register checks over feature bits.

Backports commit 0ae0326b984e77a55c224b7863071bd3d8951231
2021-03-01 19:06:15 -05:00
Aaron Lindsay 97702da7ad target/arm: Count PMU events when MDCR.SPME is set
This check was backwards when introduced in commit
033614c47de78409ad3fb39bb7bd1483b71c6789:

target/arm: Filter cycle counter based on PMCCFILTR_EL0

Backports commit db1f3afb17269cf2bd86c222e1bced748487ef71
2021-03-01 18:25:25 -05:00
Philippe Mathieu-Daudé d4c59cce4e target/arm: Clarify HCR_EL2 ARMCPRegInfo type
In commit ce4afed839 ("target/arm: Implement AArch32 HCR and HCR2")
the HCR_EL2 register has been changed from type NO_RAW (no underlying
state and does not support raw access for state saving/loading) to
type CONST (TCG can assume the value to be constant), removing the
read/write accessors.
We forgot to remove the previous type ARM_CP_NO_RAW. This is not
really a problem since the field is overwritten. However it makes
code review confuse, so remove it.

Backports 0e5aac18bc31dbdfab51f9784240d0c31a4c5579
2021-02-26 12:18:15 -05:00
Peter Maydell fc4cc9d95f target/arm: Convert A32 coprocessor insns to decodetree
Convert the A32 coprocessor instructions to decodetree.

Note that this corrects an underdecoding: for the 64-bit access case
(MRRC/MCRR) we did not check that bits [24:21] were 0b0010, so we
would incorrectly treat LDC/STC as MRRC/MCRR rather than UNDEFing
them.

The decodetree versions of these insns assume the coprocessor
is in the range 0..7 or 14..15. This is architecturally sensible
(as per the comments) and OK in practice for QEMU because the only
uses of the ARMCPRegInfo infrastructure we have that aren't
for coprocessors 14 or 15 are the pxa2xx use of coprocessor 6.
We add an assertion to the define_one_arm_cp_reg_with_opaque()
function to catch any accidental future attempts to use it to
define coprocessor registers for invalid coprocessors.

Backports commit cd8be50e58f63413c033531d3273c0e44851684f from qemu
2021-02-26 10:57:00 -05:00
Peter Maydell 0718459fb3 target/arm: Fix Rt/Rt2 in ESR_ELx for copro traps from AArch32 to 64
When a coprocessor instruction in an AArch32 guest traps to AArch32
Hyp mode, the syndrome register (HSR) includes Rt and Rt2 fields
which are simply copies of the Rt and Rt2 fields from the trapped
instruction. However, if the instruction is trapped from AArch32 to
an AArch64 higher exception level, the Rt and Rt2 fields in the
syndrome register (ESR_ELx) must be the AArch64 view of the register.
This makes a difference if the AArch32 guest was in a mode other than
User or System and it was using r13 or r14, or if it was in FIQ mode
and using r8-r14.

We don't know at translate time which AArch32 CPU mode we are in, so
we leave the values we generate in our prototype syndrome register
value at translate time as the raw Rt/Rt2 from the instruction, and
instead correct them to the AArch64 view when we find we need to take
an exception from AArch32 to AArch64 with one of these syndrome
values.

Fixes: https://bugs.launchpad.net/qemu/+bug/1879587

Backports commit a65dabf71a9f9b949d556b1b57fd72595df92398 from qemu
2021-02-25 23:50:18 -05:00
Richard Henderson 5e1316a92e target/arm: Always pass cacheattr in S1_ptw_translate
When we changed the interface of get_phys_addr_lpae to require
the cacheattr parameter, this spot was missed. The compiler is
unable to detect the use of NULL vs the nonnull attribute here.

Fixes: 7e98e21c098

Backports commit a6d6f37aed4b171d121cd4a9363fbb41e90dcb53 from qemu
2021-02-25 23:40:32 -05:00
Aaron Lindsay e532ce610e target/arm: Don't do raw writes for PMINTENCLR
Raw writes to this register when in KVM mode can cause interrupts to be
raised (even when the PMU is disabled). Because the underlying state is
already aliased to PMINTENSET (which already provides raw write
functions), we can safely disable raw accesses to PMINTENCLR entirely.

Backports commit 887c0f1544991f567543b7c214aa11ab0cea0a29 from qemu
2021-02-25 23:27:47 -05:00
Richard Henderson 2ea0b53c1a target/arm: Cache the Tagged bit for a page in MemTxAttrs
This "bit" is a particular value of the page's MemAttr.

Backports commit 337a03f07ff0f9e6295662f4094e03a045b60bdc from qemu
2021-02-25 22:48:04 -05:00
Richard Henderson 28cd096d67 target/arm: Always pass cacheattr to get_phys_addr
We need to check the memattr of a page in order to determine
whether it is Tagged for MTE. Between Stage1 and Stage2,
this becomes simpler if we always collect this data, instead
of occasionally being presented with NULL.

Use the nonnull attribute to allow the compiler to check that
all pointer arguments are non-null.

Backports commit 7e98e21c09871cddc20946c8f3f3595e93154ecb from qemu
2021-02-25 22:46:00 -05:00
Richard Henderson e2456a83a4 target/arm: Set PSTATE.TCO on exception entry
D1.10 specifies that exception handlers begin with tag checks overridden.

Backports commit 34669338bd9d66255fceaa84c314251ca49ca8d5 from qemu
2021-02-25 22:41:26 -05:00
Richard Henderson 35d0443056 target/arm: Implement data cache set allocation tags
This is DC GVA and DC GZVA, and the tag check for DC ZVA.

Backports commit eb821168db798302bd124a3b000cebc23bd0a395 from qemu
2021-02-25 22:40:08 -05:00
Richard Henderson 84012be55c target/arm: Add arm_tlb_bti_gp
Introduce an lvalue macro to wrap target_tlb_bit0.

Backports commit 149d3b31f3f0f7f9e1c3a77043450a95c7a7e93d from qemu
2021-02-25 17:45:50 -05:00
Richard Henderson 91e2f55b69 target/arm: Implement helper_mte_check1
Fill out the stub that was added earlier.

Backports commit 2e34ff45f32cb032883616a1cc5ea8ac96f546d5 from qemu
2021-02-25 17:02:34 -05:00
Richard Henderson 4488858072 target/arm: Move regime_tcr to internals.h
We will shortly need this in mte_helper.c as well.

Backports commit 38659d311d05e6c5feff6bddcc1c33b60d3b86a1 from qemu
2021-02-25 16:04:54 -05:00
Richard Henderson e46cec8543 target/arm: Move regime_el to internals.h
We will shortly need this in mte_helper.c as well.

Backports commit 9c7ab8fc8cb6d6e2fb7a82c1088691c7c23fa1b9 from qemu
2021-02-25 16:04:12 -05:00
Richard Henderson 7147f5f28a target/arm: Implement the access tag cache flushes
Like the regular data cache flushes, these are nops within qemu.

Backports commit 5463df160ecee510e78493993eb1bd38b4838a10 from qemu
2021-02-25 16:02:30 -05:00