With the reduction operations, we intentionally increase maxsz to
the next power of 2, so as to fill out the reduction tree correctly.
Since e2e7168a214b, oprsz must equal maxsz, with exceptions for small
vectors, so this triggers an assertion for vector sizes > 32 that are
not themselves a power of 2.
Pass the power-of-two value in the simd_data field instead.
Backports c648c9b7e1ccff94b51ecbebe86a206952c47e75
Wrote too much with low-half zip (zip1) with vl % 512 != 0.
Adjust all of the x + (y << s) to x | (y << s) as a style fix.
We only ever have exact overlap between D, M, and N. Therefore
we only need a single temporary, and we do not need to check for
partial overlap.
Backports 8e7fefed1bdcc0f7e722ccf2a2fc2b4f79fe725e
Missed out on compressing the second half of a predicate
with length vl % 512 > 256.
Adjust all of the x + (y << s) to x | (y << s) as a
general style fix. Drop the extract64 because the input
uint64_t are known to be already zero-extended from the
current size of the predicate.
Backports 226e6c046c0fce8da32575aad020ca56a5a8064d
clang's C11 atomic_fetch_*() functions only take a C11 atomic type
pointer argument. QEMU uses direct types (int, etc) and this causes a
compiler error when a QEMU code calls these functions in a source file
that also included <stdatomic.h> via a system header file:
$ CC=clang CXX=clang++ ./configure ... && make
../util/async.c:79:17: error: address argument to atomic operation must be a pointer to _Atomic type ('unsigned int *' invalid)
Avoid using atomic_*() names in QEMU's atomic.h since that namespace is
used by <stdatomic.h>. Prefix QEMU's APIs with 'q' so that atomic.h
and <stdatomic.h> can co-exist. I checked /usr/include on my machine and
searched GitHub for existing "qatomic_" users but there seem to be none.
This patch was generated using:
$ git grep -h -o '\<atomic\(64\)\?_[a-z0-9_]\+' include/qemu/atomic.h | \
sort -u >/tmp/changed_identifiers
$ for identifier in $(</tmp/changed_identifiers); do
sed -i "s%\<$identifier\>%q$identifier%g" \
$(git grep -I -l "\<$identifier\>")
done
I manually fixed line-wrap issues and misaligned rST tables.
Backports d73415a315471ac0b127ed3fad45c8ec5d711de1
Section D6.7 of the ARM ARM states:
For the purpose of determining Tag Check Fault handling, unprivileged
load and store instructions are treated as if executed at EL0 when
executed at either:
- EL1, when the Effective value of PSTATE.UAO is 0.
- EL2, when both the Effective value of HCR_EL2.{E2H, TGE} is {1, 1}
and the Effective value of PSTATE.UAO is 0.
ARM has confirmed a defect in the pseudocode function
AArch64.TagCheckFault that makes it inconsistent with the above
wording. The remedy is to adjust references to PSTATE.EL in that
function to instead refer to AArch64.AccessUsesEL(acctype), so
that unprivileged instructions use SCTLR_EL1.TCF0 and TFSRE0_EL1.
The exception type for synchronous tag check faults remains unchanged.
This patch implements the described change by partially reverting
commits 50244cc76abc and cc97b0019bb5.
Backports 2d928adf8a9148510e1b2041145b8a873f4d26df
Always perform one call instead of two for 16-byte operands.
Use byte loads/stores directly into the vector register file
instead of extractions and deposits to a 64-bit local variable.
In order to easily receive pointers into the vector register file,
convert the helper to the gvec out-of-line signature. Move the
helper into vec_helper.c, where it can make use of H1 and clear_tail.
Backports 519183d3fee58e52f7b51cf146c9dc9edc565059
Add support for FEAT_SSBS. SSBS (Speculative Store Bypass Safe) is an
optional feature in ARMv8.0, and mandatory in ARMv8.5.
Backports f2f68a78b793808b84367bc708d632969d4440aa
We were fudging TBI1 enabled to speed up the generated code.
Now that we've improved the code generation, remove this.
Also, tidy the comment to reflect the current code.
The pauth test was testing a kernel address (-1) and making
incorrect assumptions about TBI1; stick to userland addresses.
Backports 16c849784873d10d0da257d698e391fddea1f0e4
When working with performance monitoring counters, we look at
MDCR_EL2.HPMN as part of the check whether a counter is enabled. This
check fails, because MDCR_EL2.HPMN is reset to 0, meaning that no
counters are "enabled" for < EL2.
That's in violation of the Arm specification, which states that
> On a Warm reset, this field [MDCR_EL2.HPMN] resets to the value in
> PMCR_EL0.N
That's also what a comment in the code acknowledges, but the necessary
adjustment seems to have been forgotten when support for more counters
was added.
This change fixes the issue by setting the reset value to PMCR.N, which
is four.
Backports d3c1183ffeb71ca3a783eae3d7e1c51e71e8a621
cpsr has been treated as being the same as spsr, but it isn't.
Since PSTATE_SS isn't in cpsr, remove it and move it into env->pstate.
This allows us to add support for CPSR_DIT, adding helper functions
to merge SPSR_ELx to and from CPSR.
Backports f944a854ce4007000accf7c191b5b52916947198
Add support for FEAT_DIT. DIT (Data Independent Timing) is a required
feature for ARMv8.4. Since virtual machine execution is largely
nondeterministic and TCG is outside of the security domain, it's
implemented as a NOP.
Backports dc8b18534ea1dcc90d80ad9a61a3b0aa7eb312fb
The FW and AW bits of SCR_EL3 are RES1 only in some contexts. Force them
to 1 only when there is no support for AArch32 at EL1 or above.
The reset value will be 0x30 only if the CPU is AArch64-only; if there
is support for AArch32 at EL1 or above, it will be reset to 0.
Also adds helper function isar_feature_aa64_aa32_el1 to check if AArch32
is supported at EL1 or above.
Backports 10d0ef3e6cfe228df4b2d3e27325f1b0e2b71fd5
commit 568496c0c0f1 ("cpu: Add callback to check architectural") and
commit 3826121d9298 ("target-arm: Implement checking of fired")
introduced an ARM-specific hack for cpu_check_watchpoint.
Make debug_check_watchpoint optional, and move it to tcg_ops.
Backports c73bdb35a91fb6b17c2c93b1ba381fc88a406f8d
commit 40612000599e ("arm: Correctly handle watchpoints for BE32 CPUs")
introduced this ARM-specific, TCG-specific hack to adjust the address,
before checking it with cpu_check_watchpoint.
Make adjust_watchpoint_address optional and move it to tcg_ops.
Backports 9ea9087bb4a86893e4ac6ff643837937dc9e5849
The TCG-specific CPU methods will be moved to a separate struct,
to make it easier to move accel-specific code outside generic CPU
code in the future. Start by moving tcg_initialize().
The new CPUClass.tcg_opts field may eventually become a pointer,
but keep it an embedded struct for now, to make code conversion
easier.
Backports e9e51b7154404efc9af8735ab87c658a9c434cfd
cc->do_interrupt is in theory a TCG callback used in accel/tcg only,
to prepare the emulated architecture to take an interrupt as defined
in the hardware specifications,
but in reality the _do_interrupt style of functions in targets are
also occasionally reused by KVM to prepare the architecture state in a
similar way where userspace code has identified that it needs to
deliver an exception to the guest.
In the case of ARM, that includes:
1) the vcpu thread got a SIGBUS indicating a memory error,
and we need to deliver a Synchronous External Abort to the guest to
let it know about the error.
2) the kernel told us about a debug exception (breakpoint, watchpoint)
but it is not for one of QEMU's own gdbstub breakpoints/watchpoints
so it must be a breakpoint the guest itself has set up, therefore
we need to deliver it to the guest.
So in order to reuse code, the same arm_do_interrupt function is used.
This is all fine, but we need to avoid calling it using the callback
registered in CPUClass, since that one is now TCG-only.
Fortunately this is easily solved by replacing calls to
CPUClass::do_interrupt() with explicit calls to arm_do_interrupt().
Backports 853bfef4e6d60244fd131ec55bbf1e7caa52599b. We don't support
KVM, so we just bring the comment addition over.
This was defined at some point before ARMv8.4, and will
shortly be used by new processor descriptions.
Backports 1d51bc96cc4a9b2d31a3f4cb8442ce47753088e2
When building with GCC 10.2 configured with --extra-cflags=-Os, we get:
target/arm/m_helper.c: In function ‘arm_v7m_cpu_do_interrupt’:
target/arm/m_helper.c:1811:16: error: ‘restore_s16_s31’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
1811 | if (restore_s16_s31) {
| ^
target/arm/m_helper.c:1350:10: note: ‘restore_s16_s31’ was declared here
1350 | bool restore_s16_s31;
| ^~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
Initialize the 'restore_s16_s31' variable to silence the warning.
Backports 0ae4f11ee57350dac0e705ba79516310400ff43c
These two were odd, in that do_pfirst_pnext passed the
count of 64-bit words rather than bytes. Change to pass
the standard pred_full_reg_size to avoid confusion.
Backports 86300b5d044064046395ae8ed605cc19e63f2a7c
SVE predicate operations cannot use the "usual" simd_desc
encoding, because the lengths are not a multiple of 8.
But we were abusing the SIMD_* fields to store values anyway.
This abuse broke when SIMD_OPRSZ_BITS was modified in e2e7168a214.
Introduce a new set of field definitions for exclusive use
of predicates, so that it is obvious what kind of predicate
we are manipulating. To be used in future patches
Backports b64ee454a4a086ed459bcda4c0bbb54e197841e4
On ARMv8-A, accesses by 32-bit secure EL1 to monitor registers trap to
the upper (64-bit) EL. With Secure EL2 support, we can no longer assume
that that is always EL3, so make room for the value to be computed at
run-time.
Backports 6b340aeb48e4f7f983e1c38790de65ae93079840
