Partial cleanup from the CONFIG_VECTOR16 removal.
Replace the vec* types with their scalar expansions.
Backports commit 6c7ab3015ac498181444deff55dcc8fd43ad468c from qemu
The comment in tcg-runtime-gvec.c about CONFIG_VECTOR16 says that
tcg-op-gvec.c has eliminated size 8 vectors, and only passes on
multiples of 16. This may have been true of the first few operations,
but is not true of all operations.
In particular, multiply, shift by scalar, and compare of 8- and 16-bit
elements are not expanded inline if host vector operations are not
supported.
For an x86_64 host that does not support AVX, this means that we will
fall back to the helper, which will attempt to use SSE instructions,
which will SEGV on an invalid 8-byte aligned memory operation.
This patch simply removes the CONFIG_VECTOR16 code and configuration
without further simplification.
Buglink: https://bugs.launchpad.net/bugs/1863508
Backports commit 43d1ccd2a02fadf3c36b46a8c31125a28864d141 from qemu
A given RISU testcase for SVE can produce
tcg-op-vec.c:511: do_shifti: Assertion `i >= 0 && i < (8 << vece)' failed.
because expand_vec_sari gave a shift count of 32 to a MO_32
vector shift.
In 44f1441dbe1, we changed from direct expansion of vector opcodes
to re-use of the tcg expanders. So while the comment correctly notes
that the hw will handle such a shift count, we now have to take our
own sanity checks into account. Which is easy in this particular case.
Fixes: 44f1441dbe1
Backports commit 312b426fea4d6dd322d7472c80010a8ba7a166d2 from qemu
For system emulation we need to check the state of the GIC before we
report the value. However this isn't relevant to exporting of the
value to linux-user and indeed breaks the exported value as set by
modify_arm_cp_regs.
Backports commit 976b99b6ec2e15cd7c36d72fdb9b60c37c5494f8 from qemu
Currently riscv_cpu_local_irq_pending is used to find out pending
interrupt and VS mode interrupts are being shifted to represent
S mode interrupts in this function. So when the cause returned by
this function is passed to riscv_cpu_do_interrupt to actually
forward the interrupt, the VS mode forwarding check does not work
as intended and interrupt is actually forwarded to hypervisor. This
patch fixes this issue.
Backports commit c5969a3a3c2cb9ea02ffb7e86acb059d3cf8c264 from qemu
As reported in: https://bugs.launchpad.net/qemu/+bug/1851939 we weren't
correctly handling illegal instructions based on the value of MSTATUS_TSR
and the current privledge level.
This patch fixes the issue raised in the bug by raising an illegal
instruction if TSR is set and we are in S-Mode.
Backports commit ed5abf46b3c414ef58e647145f19b3966700b206 from qemu
We must include the tag in the FAR_ELx register when raising
an addressing exception. Which means that we should not clear
out the tag during translation.
We cannot at present comply with this for user mode, so we
retain the clean_data_tbi function for the moment, though it
no longer does what it says on the tin for system mode. This
function is to be replaced with MTE, so don't worry about the
slight misnaming.
Buglink: https://bugs.launchpad.net/qemu/+bug/1867072
Backports commit 38d931687fa196a7ef860f8583815abc7fd5521a from qemu
This data access was forgotten when we added support for cleaning
addresses of TBI information.
Fixes: 3a471103ac1823ba
Backports commit 597d61a3b1f94c53a3aaa77671697c0c5f797dbf from qemu.
The function does not write registers, and only reads them by
implication via the exception path.
Backports commit 1371b02c5a060e423e70560dbca769b54e471ba9 from qemu
This is an aarch64-only function. Move it out of the shared file.
This patch is code movement only.
Backports commit 7b182eb2467af6c47c9c77c64bbbeed8ed53c330 from qemu
If by context we know that we're in AArch64 mode, we need not
test for M-profile when reconstructing the full ARMMMUIdx.
Backports commit 20dc67c947a691fa9df05e76aec6df50204b4b94 from qemu
Replicate the single TBI bit from TCR_EL2 and TCR_EL3 so that
we can unconditionally use pointer bit 55 to index into our
composite TBI1:TBI0 field.
Backports commit 3e270f67f0f05277021763af119a6ce195f8ed51 from qemu
This bit traps EL1 access to cache maintenance insns that operate
to the point of unification. There are no longer any references to
plain aa64_cacheop_access, so remove it.
Backports commit 38262d8a732f8bd0e9ca3dc064f6e73d00c08b9a from qemu
This bit traps EL1 access to cache maintenance insns that operate
to the point of coherency or persistence.
Backports commit 1bed4d2e55459129c19f5952bcfc65bd0c70db5b from qemu
Update the {TGE,E2H} == '11' masking to ARMv8.6.
If EL2 is configured for aarch32, disable all of
the bits that are RES0 in aarch32 mode.
Backports commit 4990e1d3c128580dd2fa0bbb1a42b6d63ba1ac28 from qemu
Don't merely start with v8.0, handle v7VE as well. Ensure that writes
from aarch32 mode do not change bits in the other half of the register.
Protect reads of aa64 id registers with ARM_FEATURE_AARCH64.
Backports commit d1fb4da208411ce7b3dafb9f9e7726ebcec14edb from qemu
The ARMv8.2-TTCNP extension allows an implementation to optimize by
sharing TLB entries between multiple cores, provided that software
declares that it's ready to deal with this by setting a CnP bit in
the TTBRn_ELx. It is mandatory from ARMv8.2 onward.
For QEMU's TLB implementation, sharing TLB entries between different
cores would not really benefit us and would be a lot of work to
implement. So we implement this extension in the "trivial" manner:
we allow the guest to set and read back the CnP bit, but don't change
our behaviour (this is an architecturally valid implementation
choice).
The only code path which looks at the TTBRn_ELx values for the
long-descriptor format where the CnP bit is defined is already doing
enough masking to not get confused when the CnP bit at the bottom of
the register is set, so we can simply add a comment noting why we're
relying on that mask.
Backports commit 41a4bf1feab098da4cd5495cd56a99b0339e2275 from qemu
Currently, TIME CSRs are emulated only for user-only mode. This
patch add TIME CSRs emulation for privileged mode.
For privileged mode, the TIME CSRs will return value provided
by rdtime callback which is registered by QEMU machine/platform
emulation (i.e. CLINT emulation). If rdtime callback is not
available then the monitor (i.e. OpenSBI) will trap-n-emulate
TIME CSRs in software.
We see 25+% performance improvement in hackbench numbers when
TIME CSRs are not trap-n-emulated.
Backports commit c695724868ce4049fd79c5a509880dbdf171e744 from qemu
Add a helper macro MSTATUS_MPV_ISSET() which will determine if the
MSTATUS_MPV bit is set for both 32-bit and 64-bit RISC-V.
Backports commit e44b50b5b2e508fdd24915ab0e44ac49685e1de3 from qemu
mark_fs_dirty() is the only place in translate.c that uses the
virt_enabled bool. Let's respect the contents of MSTATUS.MPRV and
HSTATUS.SPRV when setting the bool as this is used for performing
floating point operations when V=0.
Backports commit ae84dd0ab7eaf7e98cd6ee05b2063cce8ff9bc02 from qemu
When the Hypervisor extension is in use we only enable floating point
support when both status and vsstatus have enabled floating point
support.
Backports commit 29409c1d921d607873268671bf11a088efb5558e from qemu
The hret instruction does not exist in the new spec versions, so remove
it from QEMU.
Backports commit 0736febb2d0e1bb503ca07091c16a16e78480366 from qemu
To ensure our TLB isn't out-of-date we flush it on all virt mode
changes. Unlike priv mode this isn't saved in the mmu_idx as all
guests share V=1. The easiest option is just to flush on all changes.
Backports commit eccc5a12c2fd1c646c69a1e7de29183b7a559973 from qemu
Add a FORCE_HS_EXCEP mode to the RISC-V virtulisation status. This bit
specifies if an exeption should be taken to HS mode no matter the
current delegation status. This is used when an exeption must be taken
to HS mode, such as when handling interrupts.
Backports commit c7b1bbc80fc2af17395d3986c346fd2307e57829 from qemu
Add the Hypervisor CSRs to CPUState and at the same time (to avoid
bisect issues) update the CSR macros for the v0.5 Hyp spec.
Backports commit bd023ce33b85d73791b7bc78fd04a8115c60995e from qemu
The MIP CSR is a xlen CSR, it was only 32-bits to allow atomic access.
Now that we don't use atomics for MIP we can change this back to a xlen
CSR.
Backports commit 028616130d5f0abc8a3b96f28963da51a875024b from qemu
The ARMv8.3-CCIDX extension makes the CCSIDR_EL1 system ID registers
have a format that uses the full 64 bit width of the register, and
adds a new CCSIDR2 register so AArch32 can get at the high 32 bits.
QEMU doesn't implement caches, so we just treat these ID registers as
opaque values that are set to the correct constant values for each
CPU. The only thing we need to do is allow 64-bit values in our
cssidr[] array and provide the CCSIDR2 accessors.
We don't set the CCIDX field in our 'max' CPU because the CCSIDR
constant values we use are the same as the ones used by the
Cortex-A57 and they are in the old 32-bit format. This means
that the extra regdef added here is unused currently, but it
means that whenever in the future we add a CPU that does need
the new 64-bit format it will just work when we set the cssidr
values and the ID registers for it.
Backports commit 957e615503bd0de22393fd8dbcb22a5064fd2b5c from qemu
The v8.4-RCPC extension implements some new instructions:
* LDAPUR, LDAPURB, LDAPURH, LDAPRSB, LDAPRSH, LDAPRSW
* STLUR, STLURB, STLURH
These are all in a new subgroup of encodings that sits below the
top-level "Loads and Stores" group in the Arm ARM.
The STLUR* instructions have standard store-release semantics; the
LDAPUR* have Load-AcquirePC semantics, but (as with LDAPR*) we choose
to implement them as the slightly stronger Load-Acquire.
Backports commit a1229109dec4375259d3fff99f362405aab7917a from qemu
The v8.3-RCPC extension implements three new load instructions
which provide slightly weaker consistency guarantees than the
existing load-acquire operations. For QEMU we choose to simply
implement them with a full LDAQ barrier.
Backports commit 2677cf9f92a5319bb995927f9225940414ce879d from qemu
We missed an instance of using FIELD_EX32 on a 64-bit ID
register, in isar_feature_aa64_pmu_8_4(). Fix it.
Backports commit 54117b90ffd8a3977917971c3bd99bb5242710d9 from qemu.
Passing the raw op field from the manual is less instructive
than it might be. Do the full decode and use the existing
helpers to perform the expansion.
Since these are v8 insns, VECLEN+VECSTRIDE are already RES0.
Backports commit f2eafb75511e5d2ee601b43dc6ee0bcc6e453acd from qemu
Passing the raw o1 and o2 fields from the manual is less
instructive than it might be. Do the full decode and let
the trans_* functions pass in booleans to a helper.
Backports commit d486f8308a13543bbcc4887f246e856df991a4bc from qemu
Those vfp instructions without extra opcode fields can
share a common @format for brevity.
Backports commit 906b60facc3d3dd3af56cb1a7860175d805e10a3 from qemu
Have the calls adjacent as an intermediate step toward
actually merging the decodes.
Backports commit f0f6d5c81be47d593e5ece7f06df6fba4c15738b from qemu
Now that we no longer have an early check for ARM_FEATURE_VFP,
we can use the proper ISA check in trans_VLLDM_VLSTM.
Backports commit dc778a6873f534817a13257be2acba3ca87ec015 from qemu
All remaining tests for VFP4 are for fused multiply-add insns.
Since the MVFR1 field is used for both VFP and NEON, move its adjustment
from the !has_neon block to the (!has_vfp && !has_neon) block.
Test for vfp of the appropraite width alongside the test for simdfmac
within translate-vfp.inc.c. Within disas_neon_data_insn, we have
already tested for ARM_FEATURE_NEON.
Backports commit c52881bbc22b50db99a6c37171ad3eea7d959ae6 from qemu
We will eventually remove the early ARM_FEATURE_VFP test,
so add a proper test for each trans_* that does not already
have another ISA test.
Backports commit 82f6abe16b9b951180657c5fe15942d5214aa12e from qemu
Sort this check to the start of a trans_* function.
Merge this with any existing test for fpdp_v2.
Backports commit 84774cc37f2c17e48a4867a8e8e055deb23bea69 from qemu
Shuffle the order of the checks so that we test the ISA
before we test anything else, such as the register arguments.
Backports commit 799449abda137153a0e68b8788d8e1486f389490 from qemu
We cannot easily create "any" functions for these, because the
ID_AA64PFR0 fields for FP and SIMD signal "enabled" with zero.
Which means that an aarch32-only cpu will return incorrect results
when testing the aarch64 registers.
To use these, we must either have context or additionally test
vs ARM_FEATURE_AARCH64.
Backports commit 7d63183ff1a61b3f7934dc9b40b10e4fd5e100cd from qemu
The old name, isar_feature_aa32_fpdp, does not reflect
that the test includes VFPv2. We will introduce another
feature tests for VFPv3.
Backports commit c4ff873583834c8275586914fff714e3ae65dee4 from qemu
Use this in the places that were checking ARM_FEATURE_VFP, and
are obviously testing for the existance of the register set
as opposed to testing for some particular instruction extension.
Backports commit 7fbc6a403a0aab834e764fa61d81ed8586cfe352 from qemu
We had set this for aarch32-only in arm_max_initfn, but
failed to set the same bit for aarch64.
Backports commit dac65ba1d7945c5d58ab63d8769103634adb2b01 from qemu
The fxam instruction returns the wrong result after fdecstp or after
an underflow. Check fptags to handle this.
Backports commit 93c3593ad04f2610fd0a176dfa89a7e40b6afe1f from qemu
We are going to convert FEATURE tests to ISAR tests,
so FPSP needs to be set for these cpus, like we have
already for FPDP.
Backports commit 9eb4f58918a851fb46895fd9b7ce579afeac9d02 from qemu
Many uses of ARM_FEATURE_VFP3 are testing for the number of simd
registers implemented. Use the proper test vs MVFR0.SIMDReg.
Backports commit a6627f5fc607939f7c8b9c3157fdcb2d368ba0ed from qemu
The old name, isar_feature_aa32_fp_d32, does not reflect
the MVFR0 field name, SIMDReg.
Backports commit 0e13ba7889432c5e2f1bdb1b25e7076ca1b1dcba from qemu
We still need two different helpers, since NEON and SVE2 get the
inputs from different locations within the source vector. However,
we can convert both to the same internal form for computation.
The sve2 helper is not used yet, but adding it with this patch
helps illustrate why the neon changes are helpful.
Backports commit e7e96fc5ec8c79dc77fef522d5226ac09f684ba5 from qemu
The gvec form will be needed for implementing SVE2.
Extend the implementation to operate on uint64_t instead of uint32_t.
Use a counted inner loop instead of terminating when op1 goes to zero,
looking toward the required implementation for ARMv8.4-DIT.
Backports commit a21bb78e5817be3f494922e1dadd6455fe5d6318 from qemu
These instructions shift left or right depending on the sign
of the input, and 7 bits are significant to the shift. This
requires several masks and selects in addition to the actual
shifts to form the complete answer.
That said, the operation is still a small improvement even for
two 64-bit elements -- 13 vector operations instead of 2 * 7
integer operations.
Backports commit 87b74e8b6edd287ea2160caa0ebea725fa8f1ca1 from qemu
The ACTLR2 and HACTLR2 AArch32 system registers didn't exist in ARMv7
or the original ARMv8. They were later added as optional registers,
whose presence is signaled by the ID_MMFR4.AC2 field. From ARMv8.2
they are mandatory (ie ID_MMFR4.AC2 must be non-zero).
We implemented HACTLR2 in commit 0e0456ab8895a5e85, but we
incorrectly made it exist for all v8 CPUs, and we didn't implement
ACTLR2 at all.
Sort this out by implementing both registers only when they are
supposed to exist, and setting the ID_MMFR4 bit for -cpu max.
Note that this removes HACTLR2 from our Cortex-A53, -A47 and -A72
CPU models; this is correct, because those CPUs do not implement
this register.
Fixes: 0e0456ab8895a5e85
Backports commit f6287c24c66d6b9187c1c2887e1c7cfa4d304b0c from qemu
Cut-and-paste errors mean we're using FIELD_EX64() to extract fields from
some 32-bit ID register fields. Use FIELD_EX32() instead. (This makes
no difference in behaviour, it's just more consistent.)
Backports commit b3a816f6ce1ec184ab6072f50bbe4479fc5116c3 from qemu
Now we have moved ID_MMFR4 into the ARMISARegisters struct, we
can define and use an isar_feature for the presence of the
ARMv8.2-AA32HPD feature, rather than open-coding the test.
While we're here, correct a comment typo which missed an 'A'
from the feature name.
Backports commit 4036b7d1cd9fb1097a5f4bc24d7d31744256260f from qemu
The isar_feature_aa32_pan and isar_feature_aa32_ats1e1 functions
are supposed to be testing fields in ID_MMFR3; but a cut-and-paste
error meant we were looking at MVFR0 instead.
Fix the functions to look at the right register; this requires
us to move at least id_mmfr3 to the ARMISARegisters struct; we
choose to move all the ID_MMFRn registers for consistency.
Backports commit 10054016eda1b13bdd8340d100fd029cc8b58f36 from qemu
The LC bit in the PMCR_EL0 register is supposed to be:
* read/write
* RES1 on an AArch64-only implementation
* an architecturally UNKNOWN value on reset
(and use of LC==0 by software is deprecated).
We were implementing it incorrectly as read-only always zero,
though we do have all the code needed to test it and behave
accordingly.
Instead make it a read-write bit which resets to 1 always, which
satisfies all the architectural requirements above.
Backports commit 62d96ff48510f4bf648ad12f5d3a5507227b026f from qemu
The PMCR_EL0.DP bit is bit 5, which is 0x20, not 0x10. 0x10 is 'X'.
Correct our #define of PMCRDP and add the missing PMCRX.
We do have the correct behaviour for handling the DP bit being
set, so this fixes a guest-visible bug.
Fixes: 033614c47de
Backports commit a1ed04dd79aabb9dbeeb5fa7d49f1a3de0357553 from qemu
Set the ID register bits to provide ARMv8.4-PMU (and implicitly
also ARMv8.1-PMU) in the 'max' CPU.
Backports commit 3bec78447a958d4819911252e056f29740ac25e4 from qemu
The ARMv8.4-PMU extension adds:
* one new required event, STALL
* one new system register PMMIR_EL1
(There are also some more L1-cache related events, but since
we don't implement any cache we don't provide these, in the
same way we don't provide the base-PMUv3 cache events.)
The STALL event "counts every attributable cycle on which no
attributable instruction or operation was sent for execution on this
PE". QEMU doesn't stall in this sense, so this is another
always-reads-zero event.
The PMMIR_EL1 register is a read-only register providing
implementation-specific information about the PMU; currently it has
only one field, SLOTS, which defines behaviour of the STALL_SLOT PMU
event. Since QEMU doesn't implement the STALL_SLOT event, we can
validly make the register read zero.
Backports commit 15dd1ebda4a6ef928d484c5a4f48b8ccb7438bb2 from qemu
The ARMv8.1-PMU extension requires:
* the evtCount field in PMETYPER<n>_EL0 is 16 bits, not 10
* MDCR_EL2.HPMD allows event counting to be disabled at EL2
* two new required events, STALL_FRONTEND and STALL_BACKEND
* ID register bits in ID_AA64DFR0_EL1 and ID_DFR0
We already implement the 16-bit evtCount field and the
HPMD bit, so all that is missing is the two new events:
STALL_FRONTEND
"counts every cycle counted by the CPU_CYCLES event on which no
operation was issued because there are no operations available
to issue to this PE from the frontend"
STALL_BACKEND
"counts every cycle counted by the CPU_CYCLES event on which no
operation was issued because the backend is unable to accept
any available operations from the frontend"
QEMU never stalls in this sense, so our implementation is trivial:
always return a zero count.
Backports commit 0727f63b1ecf765ebc48266f616f8fc362dc7fbc from qemu
We're going to want to read the DBGDIDR register from KVM in
a subsequent commit, which means it needs to be in the
ARMISARegisters sub-struct. Move it.
Backports commit 4426d3617d64922d97b74ed22e67e33b6fb7de0a from qemu
The AArch32 DBGDIDR defines properties like the number of
breakpoints, watchpoints and context-matching comparators. On an
AArch64 CPU, the register may not even exist if AArch32 is not
supported at EL1.
Currently we hard-code use of DBGDIDR to identify the number of
breakpoints etc; this works for all our TCG CPUs, but will break if
we ever add an AArch64-only CPU. We also have an assert() that the
AArch32 and AArch64 registers match, which currently works only by
luck for KVM because we don't populate either of these ID registers
from the KVM vCPU and so they are both zero.
Clean this up so we have functions for finding the number
of breakpoints, watchpoints and context comparators which look
in the appropriate ID register.
This allows us to drop the "check that AArch64 and AArch32 agree
on the number of breakpoints etc" asserts:
* we no longer look at the AArch32 versions unless that's the
right place to be looking
* it's valid to have a CPU (eg AArch64-only) where they don't match
* we shouldn't have been asserting the validity of ID registers
in a codepath used with KVM anyway
Backports commit 88ce6c6ee85d902f59dc65afc3ca86b34f02b9ed from qemu
Add the 64-bit version of the "is this a v8.1 PMUv3?"
ID register check function, and the _any_ version that
checks for either AArch32 or AArch64 support. We'll use
this in a later commit.
We don't (yet) do any isar_feature checks on ID_AA64DFR1_EL1,
but we move id_aa64dfr1 into the ARMISARegisters struct with
id_aa64dfr0, for consistency.
Backports commit 2a609df87d9b886fd38a190a754dbc241ff707e8 from qemu
Instead of open-coding a check on the ID_DFR0 PerfMon ID register
field, create a standardly-named isar_feature for "does AArch32 have
a v8.1 PMUv3" and use it.
This entails moving the id_dfr0 field into the ARMISARegisters struct.
Backports commit a617953855b65a602d36364b9643f7e5bc31288e from qemu
We already define FIELD macros for ID_DFR0, so use them in the
one place where we're doing direct bit value manipulation.
Backports commit d52c061e541982a3663ad5c65bd3b518dbe85b87 from qemu
Add FIELD() definitions for the ID_AA64DFR0_EL1 and use them
where we currently have hard-coded bit values.
Backports commit ceb2744b47a1ef4184dca56a158eb3156b6eba36 from qemu
