Commit graph

224 commits

Author SHA1 Message Date
Richard Henderson e040675fbf target/arm: Clean address for DC ZVA
This data access was forgotten when we added support for cleaning
addresses of TBI information.

Fixes: 3a471103ac1823ba

Backports commit 597d61a3b1f94c53a3aaa77671697c0c5f797dbf from qemu.
2020-04-30 06:16:03 -04:00
Richard Henderson a22a2a8b71 target/arm: Introduce core_to_aa64_mmu_idx
If by context we know that we're in AArch64 mode, we need not
test for M-profile when reconstructing the full ARMMMUIdx.

Backports commit 20dc67c947a691fa9df05e76aec6df50204b4b94 from qemu
2020-04-30 05:58:59 -04:00
Peter Maydell 5416c5a672 target/arm: Implement v8.4-RCPC
The v8.4-RCPC extension implements some new instructions:
* LDAPUR, LDAPURB, LDAPURH, LDAPRSB, LDAPRSH, LDAPRSW
* STLUR, STLURB, STLURH

These are all in a new subgroup of encodings that sits below the
top-level "Loads and Stores" group in the Arm ARM.

The STLUR* instructions have standard store-release semantics; the
LDAPUR* have Load-AcquirePC semantics, but (as with LDAPR*) we choose
to implement them as the slightly stronger Load-Acquire.

Backports commit a1229109dec4375259d3fff99f362405aab7917a from qemu
2020-03-22 00:15:46 -04:00
Peter Maydell f72582bb7a target/arm: Implement v8.3-RCPC
The v8.3-RCPC extension implements three new load instructions
which provide slightly weaker consistency guarantees than the
existing load-acquire operations. For QEMU we choose to simply
implement them with a full LDAQ barrier.

Backports commit 2677cf9f92a5319bb995927f9225940414ce879d from qemu
2020-03-22 00:13:08 -04:00
Richard Henderson fcce8d4aa1 target/arm: Convert PMULL.8 to gvec
We still need two different helpers, since NEON and SVE2 get the
inputs from different locations within the source vector. However,
we can convert both to the same internal form for computation.

The sve2 helper is not used yet, but adding it with this patch
helps illustrate why the neon changes are helpful.

Backports commit e7e96fc5ec8c79dc77fef522d5226ac09f684ba5 from qemu
2020-03-21 19:35:46 -04:00
Richard Henderson c00f72f74f target/arm: Convert PMULL.64 to gvec
The gvec form will be needed for implementing SVE2.

Backports commit b9ed510e46f2f9e31e5e8adb4661d5d1cbe9a459 from qemu
2020-03-21 19:27:38 -04:00
Richard Henderson db8a935b44 target/arm: Convert PMUL.8 to gvec
The gvec form will be needed for implementing SVE2.

Extend the implementation to operate on uint64_t instead of uint32_t.
Use a counted inner loop instead of terminating when op1 goes to zero,
looking toward the required implementation for ARMv8.4-DIT.

Backports commit a21bb78e5817be3f494922e1dadd6455fe5d6318 from qemu
2020-03-21 19:22:18 -04:00
Richard Henderson d3139f2f0a target/arm: Vectorize USHL and SSHL
These instructions shift left or right depending on the sign
of the input, and 7 bits are significant to the shift. This
requires several masks and selects in addition to the actual
shifts to form the complete answer.

That said, the operation is still a small improvement even for
two 64-bit elements -- 13 vector operations instead of 2 * 7
integer operations.

Backports commit 87b74e8b6edd287ea2160caa0ebea725fa8f1ca1 from qemu
2020-03-21 19:14:17 -04:00
Richard Henderson 51b6064ba4 target/arm: Flush high bits of sve register after AdvSIMD INS
Writes to AdvSIMD registers flush the bits above 128.

Backports commit 528dc354b6f3aa82d65141cc60bc0e725e6cae98 from qemu
2020-03-21 17:58:09 -04:00
Richard Henderson 74cbfceb56 target/arm: Flush high bits of sve register after AdvSIMD ZIP/UZP/TRN
Writes to AdvSIMD registers flush the bits above 128.

Backports commit 33649de62e40df0060a1c514574e4ef25c4e52e1 from qemu
2020-03-21 17:56:40 -04:00
Richard Henderson 6eb8472344 target/arm: Flush high bits of sve register after AdvSIMD TBL/TBX
Writes to AdvSIMD registers flush the bits above 128.

Backports commit 263273bc988e677ebadeaf7d0e49f6792a112db5 from qemu
2020-03-21 17:56:08 -04:00
Richard Henderson 18e9c4805f target/arm: Flush high bits of sve register after AdvSIMD EXT
Writes to AdvSIMD registers flush the bits above 128.

Backports commit 78cedfabd53b6f64e7e64fc84878d848e5df1d08 from qemu
2020-03-21 17:55:12 -04:00
Richard Henderson 5b5050c6ca target/arm: Update MSR access to UAO
Backports commit 9eeb7a1c9531cb3574bfe2c36eb7624802c3ec00 from qemu
2020-03-21 17:48:01 -04:00
Richard Henderson 35fab80c57 target/arm: Update MSR access for PAN
For aarch64, there's a dedicated msr (imm, reg) insn.
For aarch32, this is done via msr to cpsr. Writes from el0
are ignored, which is already handled by the CPSR_USER mask.

Backports commit 220f508f49c5f49fb771d5105f991c19ffede3f7 from qemu
2020-03-21 17:33:16 -04:00
Richard Henderson 7aaf0d442b target/arm: Add mmu_idx for EL1 and EL2 w/ PAN enabled
To implement PAN, we will want to swap, for short periods
of time, to a different privileged mmu_idx. In addition,
we cannot do this with flushing alone, because the AT*
instructions have both PAN and PAN-less versions.

Add the ARMMMUIdx*_PAN constants where necessary next to
the corresponding ARMMMUIdx* constant.

Backports commit 452ef8cb8c7b06f44a30a3c3a54d3be82c4aef59 from qemu
2020-03-21 17:12:16 -04:00
Richard Henderson 8f1201e392 target/arm: Update get_a64_user_mem_index for VHE
The EL2&0 translation regime is affected by Load Register (unpriv).

The code structure used here will facilitate later changes in this
area for implementing UAO and NV.

Backports commit cc28fc30e333dc2f20ebfde54444697e26cd8f6d from qemu
2020-03-21 16:33:52 -04:00
Richard Henderson f4397b0212 target/arm: Add regime_has_2_ranges
Create a predicate to indicate whether the regime has
both positive and negative addresses.

Backports commit 339370b90d067345b69585ddf4b668fa01f41d67 from qemu
2020-03-21 15:14:11 -04:00
Richard Henderson 1a672fc3b1 target/arm: Rename ARMMMUIdx_S1SE[01] to ARMMMUIdx_SE10_[01]
This is part of a reorganization to the set of mmu_idx.
This emphasizes that they apply to the Secure EL1&0 regime.

Backports commit fba37aedecb82506c62a1f9e81d066b4fd04e443 from qemu
2020-03-21 14:35:28 -04:00
Richard Henderson b62b4c4f35 target/arm: Rename ARMMMUIdx_S2NS to ARMMMUIdx_Stage2
The EL1&0 regime is the only one that uses 2-stage translation.

Backports commit 97fa9350017e647151dd1dc212f1bbca0294dba7 from qemu
2020-03-21 14:15:35 -04:00
Richard Henderson ec05f22e82 target/arm: Rename ARMMMUIdx*_S12NSE* to ARMMMUIdx*_E10_*
This is part of a reorganization to the set of mmu_idx.
This emphasizes that they apply to the EL1&0 regime.

The ultimate goal is

-- Non-secure regimes:
ARMMMUIdx_E10_0,
ARMMMUIdx_E20_0,
ARMMMUIdx_E10_1,
ARMMMUIdx_E2,
ARMMMUIdx_E20_2,

-- Secure regimes:
ARMMMUIdx_SE10_0,
ARMMMUIdx_SE10_1,
ARMMMUIdx_SE3,

-- Helper mmu_idx for non-secure EL1&0 stage1 and stage2
ARMMMUIdx_Stage2,
ARMMMUIdx_Stage1_E0,
ARMMMUIdx_Stage1_E1,

The 'S' prefix is reserved for "Secure". Unless otherwise specified,
each mmu_idx represents all stages of translation.

Backports commit 01b98b686460b3a0fb47125882e4f8d4268ac1b6 from qemu
2020-03-21 14:09:15 -04:00
Alex Bennée ced8834737 target/arm: fix TCG leak for fcvt half->double
When support for the AHP flag was added we inexplicably only freed the
new temps in one of the two legs. Move those tcg_temp_free to the same
level as the allocation to fix that leak.

Backports commit aeab8e5eb220cc5ff84b0b68b9afccc611bf0fcd from qemu
2020-03-21 13:14:47 -04:00
Pan Nengyuan 134a026e6b arm/translate-a64: fix uninitialized variable warning
Fixes:
target/arm/translate-a64.c: In function 'disas_crypto_three_reg_sha512':
target/arm/translate-a64.c:13625:9: error: 'genfn' may be used uninitialized in this function [-Werror=maybe-uninitialized]
genfn(tcg_rd_ptr, tcg_rn_ptr, tcg_rm_ptr);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
qemu/target/arm/translate-a64.c:13609:8: error: 'feature' may be used uninitialized in this function [-Werror=maybe-uninitialized]
if (!feature) {

Backports commit c7a5e7910517e2711215a9e869a733ffde696091 from qemu
2020-01-14 08:46:42 -05:00
Alex Bennée 8f275077b0 target/arm: only update pc after semihosting completes
Before we introduce blocking semihosting calls we need to ensure we
can restart the system on semi hosting exception. To be able to do
this the EXCP_SEMIHOST operation should be idempotent until it finally
completes. Practically this means ensureing we only update the pc
after the semihosting call has completed.

Backports commit 4ff5ef9e911c670ca10cdd36dd27c5395ec2c753 from qemu
2020-01-14 08:28:25 -05:00
Lioncash eadeae183d
target/arm: Amend bad merge 2019-11-28 03:29:56 -05:00
Tony Nguyen f75368cd0f
tcg: TCGMemOp is now accelerator independent MemOp
Preparation for collapsing the two byte swaps, adjust_endianness and
handle_bswap, along the I/O path.

Target dependant attributes are conditionalized upon NEED_CPU_H.

Backports commit 14776ab5a12972ea439c7fb2203a4c15a09094b4 from qemu
2019-11-28 03:01:12 -05:00
Peter Maydell 56b54f361e
target/arm: Allow ARMCPRegInfo read/write functions to throw exceptions
Currently the only part of an ARMCPRegInfo which is allowed to cause
a CPU exception is the access function, which returns a value indicating
that some flavour of UNDEF should be generated.

For the ATS system instructions, we would like to conditionally
generate exceptions as part of the writefn, because some faults
during the page table walk (like external aborts) should cause
an exception to be raised rather than returning a value.

There are several ways we could do this:
* plumb the GETPC() value from the top level set_cp_reg/get_cp_reg
helper functions through into the readfn and writefn hooks
* add extra readfn_with_ra/writefn_with_ra hooks that take the GETPC()
value
* require the ATS instructions to provide a dummy accessfn,
which serves no purpose except to cause the code generation
to emit TCG ops to sync the CPU state
* add an ARM_CP_ flag to mark the ARMCPRegInfo as possibly
throwing an exception in its read/write hooks, and make the
codegen sync the CPU state before calling the hooks if the
flag is set

This patch opts for the last of these, as it is fairly simple
to implement and doesn't require invasive changes like updating
the readfn/writefn hook function prototype signature.

Backports commit 37ff584c15bc3e1dd2c26b1998f00ff87189538c from qemu
2019-11-20 17:24:37 -05:00
Richard Henderson 1f59a43544
Revert "target/arm: Use unallocated_encoding for aarch32"
Despite the fact that the text for the call to gen_exception_insn
is identical for aarch64 and aarch32, the implementation inside
gen_exception_insn is totally different.

This fixes exceptions raised from aarch64.

This reverts commit fb2d3c9a9a.
2019-11-18 23:49:47 -05:00
Richard Henderson fb2d3c9a9a
target/arm: Use unallocated_encoding for aarch32
Promote this function from aarch64 to fully general use.
Use it to unify the code sequences for generating illegal
opcode exceptions.

Backports commit 3cb36637157088892e9e33ddb1034bffd1251d3b from qemu
2019-11-18 20:10:50 -05:00
Richard Henderson d562bea784
target/arm: Remove offset argument to gen_exception_bkpt_insn
Unlike the other more generic gen_exception{,_internal}_insn
interfaces, breakpoints always refer to the current instruction.

Backports commit 06bcbda3f64d464b6ecac789bce4bd69f199cd68 from qemu
2019-11-18 20:05:45 -05:00
Richard Henderson f19b4df20d
target/arm: Replace offset with pc in gen_exception_internal_insn
The offset is variable depending on the instruction set.
Passing in the actual value is clearer in intent.

Backpors commit aee828e7541a5895669ade3a4b6978382b6b094a from qemu
2019-11-18 20:05:23 -05:00
Richard Henderson 00fbadf637
target/arm: Replace s->pc with s->base.pc_next
We must update s->base.pc_next when we return from the translate_insn
hook to the main translator loop. By incrementing s->base.pc_next
immediately after reading the insn word, "pc_next" contains the address
of the next instruction throughout translation.

All remaining uses of s->pc are referencing the address of the next insn,
so this is now a simple global replacement. Remove the "s->pc" field.

Backports commit a04159166b880b505ccadc16f2fe84169806883d from qemu
2019-11-18 17:32:53 -05:00
Richard Henderson 0048f3e887
target/arm: Introduce pc_curr
Add a new field to retain the address of the instruction currently
being translated. The 32-bit uses are all within subroutines used
by a32 and t32. This will become less obvious when t16 support is
merged with a32+t32, and having a clear definition will help.

Convert aarch64 as well for consistency. Note that there is one
instance of a pre-assert fprintf that used the wrong value for the
address of the current instruction.

Backports commit 43722a6d4f0c92f7e7e1e291580039b0f9789df1 from qemu
2019-11-18 16:58:40 -05:00
Peter Maydell c61e22627d
target/arm: Fix routing of singlestep exceptions
When generating an architectural single-step exception we were
routing it to the "default exception level", which is to say
the same exception level we execute at except that EL0 exceptions
go to EL1. This is incorrect because the debug exception level
can be configured by the guest for situations such as single
stepping of EL0 and EL1 code by EL2.

We have to track the target debug exception level in the TB
flags, because it is dependent on CPU state like HCR_EL2.TGE
and MDCR_EL2.TDE. (That we were previously calling the
arm_debug_target_el() function to determine dc->ss_same_el
is itself a bug, though one that would only have manifested
as incorrect syndrome information.) Since we are out of TB
flag bits unless we want to expand into the cs_base field,
we share some bits with the M-profile only HANDLER and
STACKCHECK bits, since only A-profile has this singlestep.

Fixes: https://bugs.launchpad.net/qemu/+bug/1838913

Backports commit 8bd587c1066f4456ddfe611b571d9439a947d74c from qemu
2019-11-18 16:50:15 -05:00
Peter Maydell 3f531fac61
target/arm: Factor out 'generate singlestep exception' function
Factor out code to 'generate a singlestep exception', which is
currently repeated in four places.

To do this we need to also pull the identical copies of the
gen-exception() function out of translate-a64.c and translate.c
into translate.h.

(There is a bug in the code: we're taking the exception to the wrong
target EL. This will be simpler to fix if there's only one place to
do it.)

Backports commit c1d5f50f094ab204accfacc2ee6aafc9601dd5c4 from qemu
2019-11-18 16:47:08 -05:00
Peter Maydell b2dc290454
target/arm: Move vfp_expand_imm() to translate.[ch]
We want to use vfp_expand_imm() in the AArch32 VFP decode;
move it from the a64-only header/source file to the
AArch32 one (which is always compiled even for AArch64).

Backports commit d6a092d479333b5f20a647a912a31b0102d37335 from qemu
2019-06-25 18:17:49 -05:00
Richard Henderson 7c32498b7f
target/arm: Use tcg_gen_gvec_bitsel
This replaces 3 target-specific implementations for BIT, BIF, and BSL.

Backports commit 3a7a2b4e5cf0d49cd8b14e8225af0310068b7d20 from qemu
2019-06-13 16:12:56 -04:00
Richard Henderson b8bd543390
target/arm: Use env_cpu, env_archcpu
Cleanup in the boilerplate that each target must define.
Replace arm_env_get_cpu with env_archcpu. The combination
CPU(arm_env_get_cpu) should have used ENV_GET_CPU to begin;
use env_cpu now.

Backports commit 2fc0cc0e1e034582f4718b1a2d57691474ccb6aa from qemu
2019-06-12 11:34:08 -04:00
Richard Henderson 9287750362
target/arm: Simplify BFXIL expansion
The mask implied by the extract is redundant with the one
implied by the deposit. Also, fix spelling of BFXIL.

Backports commit 87eb65a3c45c788a309986d48170a54a0d1c0705 from qemu
2019-05-24 18:01:26 -04:00
Richard Henderson 1778828644
target/arm: Use extract2 for EXTR
This is, after all, how we implement extract2 in tcg/aarch64.

Backports commit 80ac954c369e7e61bd1ed00cef07b63e11f9c734 from qemu
2019-05-24 17:58:58 -04:00
Richard Henderson 552e48f14e
target/arm: Use tcg_gen_abs_i64 and tcg_gen_gvec_abs
Backports commit 4e027a710673f5d4dc6cff88728bcfd32e4c47b0 from qemu
2019-05-16 16:43:02 -04:00
Emilio G. Cota 1715f382b4
target/arm: check CF_PARALLEL instead of parallel_cpus
Thereby decoupling the resulting translated code from the current state
of the system.

Backports commit 2399d4e7cec22ecf1c51062d2ebfd45220dbaace from qemu
2019-05-04 22:44:32 -04:00
Lioncash d844d7cc9d
exec: Backport tb_cflags accessor 2019-04-22 06:12:59 -04:00
Lioncash 3521e72580
target/arm: Sychronize with qemu
Synchronizes with bits and pieces that were missed due to merging
incorrectly (sorry :<)
2019-04-18 04:49:11 -04:00
Richard Henderson 2427ace0c0
target/arm: Fix non-parallel expansion of CASP
The second word has been loaded from the unincremented
address since the first commit.

Backports commit a036f5302c13634f3d375615b2949fd1fa1657b6 from qemu
2019-03-26 20:31:01 -04:00
Richard Henderson f116560d2c
target/arm: Implement ARMv8.5-FRINT
Backports 6bea25631af92531027d3bf3ef972a4d51d62e7c from qemu.
2019-03-05 23:17:33 -05:00
Richard Henderson f855ac073d
target/arm: Restructure handle_fp_1src_{single, double}
This will allow sharing code that adjusts rmode beyond
the existing users.

Backports commit 0e4db23d1fdbfed4fc1ec19b6e59820209600358 from qemu
2019-03-05 23:09:48 -05:00
Richard Henderson 94b5aab8f8
target/arm: Implement ARMv8.5-CondM
Backports commit 5ef84f111483e3f7b57efc690e22081ca8f99544 from qemu
2019-03-05 23:04:06 -05:00
Richard Henderson 1dfa15a683
target/arm: Implement ARMv8.4-CondM
Backports commit b89d9c988a988d5547c73e2bc43f59b0c07420a5 from qemu
2019-03-05 22:59:51 -05:00
Richard Henderson 65a3f3be5b
target/arm: Rearrange disas_data_proc_reg
This decoding more closely matches the ARMv8.4 Table C4-6,
Encoding table for Data Processing - Register Group.

In particular, op2 == 0 is now more than just Add/sub (with carry).

Backports commit 2fba34f70d9a81bab56e61bb99a4d6632bdfe531 from qemu
2019-03-05 22:55:27 -05:00
Richard Henderson 45c297c99b
target/arm: Add set/clear_pstate_bits, share gen_ss_advance
We do not need an out-of-line helper for manipulating bits in pstate.
While changing things, share the implementation of gen_ss_advance.

Backports commit 22ac3c49641f6eed93dca5b852030b4d3eacf6c4 from qemu
2019-03-05 22:55:22 -05:00