Use the correct sctlr for EL2&0 regime. Due to header ordering,
and where arm_mmu_idx_el is declared, we need to move the function
out of line. Use the function in many more places in order to
select the correct control.
Backports commit aaec143212bb70ac9549cf73203d13100bd5c7c2 from qemu
Prepare for, but do not yet implement, the EL2&0 regime.
This involves adding the new MMUIdx enumerators and adjusting
some of the MMUIdx related predicates to match.
Backports commit b9f6033c1a5fb7da55ed353794db8ec064f78bb2 from qemu.
Implement emulation of GINVT instruction. As QEMU doesn't support
caches and virtualization, this implementation covers only one
instruction (GINVT - Global Invalidate TLB) among all TLB-related
MIPS instructions.
Backports commit 99029be1c2875cd857614397674bbf563ddb6f91 from qemu
WatchHi is extended by the field MemoryMapID with the GINVT instruction.
The field is accessible by MTHC0/MFHC0 in 32-bit architectures and DMTC0/
DMFC0 in 64-bit architectures.
Backports commit feafe82cc2289a31b3e3f11dc76f3539ea22d670 from qemu
Add an option to trigger memory writeback to sync given memory region
with the corresponding backing store, case one is available.
This extends the support for persistent memory, allowing syncing on-demand.
Backports commit 61c490e25e081af39ff40556f6c1229b8b011585 from qemu
The raising of exceptions from check_watchpoint, buried inside
of the I/O subsystem, is fundamentally broken. We do not have
the helper return address with which we can unwind guest state.
Replace PHYS_SECTION_WATCH and io_mem_watch with TLB_WATCHPOINT.
Move the call to cpu_check_watchpoint into the cputlb helpers
where we do have the helper return address.
This allows watchpoints on RAM to bypass the full i/o access path.
Backports commit 50b107c5d617eaf93301cef20221312e7a986701 from qemu
Preparation for collapsing the two byte swaps adjust_endianness and
handle_bswap into the former.
Call memory_region_dispatch_{read|write} with endianness encoded into
the "MemOp op" operand.
This patch does not change any behaviour as
memory_region_dispatch_{read|write} is yet to handle the endianness.
Once it does handle endianness, callers with byte swaps can collapse
them into adjust_endianness.
Backports commit d5d680cacc66ef7e3c02c81dc8f3a34eabce6dfe from qemu
HCR_EL2.TID3 requires that AArch32 reads of MVFR[012] are trapped to
EL2, and HCR_EL2.TID0 does the same for reads of FPSID.
In order to handle this, introduce a new TCG helper function that
checks for these control bits before executing the VMRC instruction.
Tested with a hacked-up version of KVM/arm64 that sets the control
bits for 32bit guests.
Backports commit 9ca1d776cb49c09b09579d9edd0447542970c834 from qemu
Switch the SPARC target from the old unassigned_access hook to the
new do_transaction_failed hook.
This will cause the "if transaction failed" code paths added in
the previous commits to become active if the access is to an
unassigned address. In particular we'll now handle bus errors
during page table walks correctly (generating a translation
error with the right kind of fault status).
Backports commit f8c3db33a5e863291182f8862ddf81618a7c6194 from qemu
Switch the MIPS target from the old unassigned_access hook to the new
do_transaction_failed hook.
Unlike the old hook, do_transaction_failed is only ever called from
the TCG memory access paths, so there is no need for the "ignore this
if we're using KVM" hack that we were previously using to work around
the way unassigned_access was called for all kinds of memory accesses
to unassigned physical addresses.
The MIPS target does not ever do direct memory reads by physical
address (via either ldl_phys etc or address_space_ldl etc), so the
only memory accesses this affects are the 'normal' guest loads and
stores, which will be handled by the new hook; their behaviour is
unchanged.
Backports commit 4f02a06d50ef0081089ed8cb3ec7c7986e3c95f8 from qemu
Despite the fact that the text for the call to gen_exception_insn
is identical for aarch64 and aarch32, the implementation inside
gen_exception_insn is totally different.
This fixes exceptions raised from aarch64.
This reverts commit fb2d3c9a9a.
Replace x = double_saturate(y) with x = add_saturate(y, y).
There is no need for a separate more specialized helper.
Backports commit 640581a06d14e2d0d3c3ba79b916de6bc43578b0 from qemu
Promote this function from aarch64 to fully general use.
Use it to unify the code sequences for generating illegal
opcode exceptions.
Backports commit 3cb36637157088892e9e33ddb1034bffd1251d3b from qemu
This patch adds support for the riscv_cpu_unassigned_access call
and will raise a load or store access fault.
Backports commit cbf5827693addaff4e4d2102afedbf078a204eb2 from qemu
In the next commit we will split the M-profile functions from this
file. Some function will be called out of helper.c. Declare them in
the "internals.h" header.
Backports commit 787a7e76c2e93a48c47b324fea592c9910a70483 from qemu
These routines are TCG specific.
The arm_deliver_fault() function is only used within the new
helper. Make it static.
Backports commit e21b551cb652663f2f2405a64d63ef6b4a1042b7 from qemu
In the next commit we will split the TLB related routines of
this file, and this function will also be called in the new
file. Declare it in the "internals.h" header.
Backports commit ebae861fc6c385a7bcac72dde4716be06e6776f1 from qemu
The old version of the helper for the INSERT.<B|H|W|D> MSA instructions
has been replaced with four helpers that don't use switch, and change
the endianness of the given index, when executed on a big endian host.
Backports commit c1c9a10fb1f7a6782711817c167a2c20b000fc12 from qemu
The old version of the helper for the COPY_U.<B|H|W> MSA instructions
has been replaced with four helpers that don't use switch, and change
the endianness of the given index, when executed on a big endian host.
Backports commit 41d288582782cf8d63241ecb6efa1e4160fe78f7 from qemu
The old version of the helper for the COPY_S.<B|H|W|D> MSA instructions
has been replaced with four helpers that don't use switch, and change
the endianness of the given index, when executed on a big endian host.
Backports commit 631c467461496dcf6d6a3e4c3d27a1433e96868e from qemu
Perform a per-element conditional move. This combination operation is
easier to implement on some host vector units than plain cmp+bitsel.
Omit the usual gvec interface, as this is intended to be used by
target-specific gvec expansion call-backs.
Backports commit f75da2988eb2457fa23d006d573220c5c680ec4e from qemu
This operation performs d = (b & a) | (c & ~a), and is present
on a majority of host vector units. Include gvec expanders.
Backports commit 38dc12947ec9106237f9cdbd428792c985cd86ae from qemu
We can now use the CPUClass hook instead of a named function.
Create a static tlb_fill function to avoid other changes within
cputlb.c. This also isolates the asserts within. Remove the
named tlb_fill function from all of the targets.
Backports commit c319dc13579a92937bffe02ad2c9f1a550e73973 from qemu
Note that env->pc is removed from the qemu_log as that value is garbage.
The PC isn't recovered until cpu_restore_state, called from
cpu_loop_exit_restore, called from riscv_raise_exception.
Backports commit 8a4ca3c10a96be6ed7f023b685b688c4d409bbcb from qemu
Note that env->active_tc.PC is removed from the qemu_log as that value
is garbage. The PC isn't recovered until cpu_restore_state, called from
cpu_loop_exit_restore, called from do_raise_exception_err.
Backports commit 931d019f5b2e7bbacb162869497123be402ddd86 from qemu
Remove a function of the same name from target/arm/.
Use a branchless implementation of abs gleaned from gcc.
Backports commit ff1f11f7f8710a768f9313f24bd7f509d3db27e5 from qemu
Allow expansion either via shift by scalar or by replicating
the scalar for shift by vector.
Backports commit b4578cd91cda4cef1c413304353ca6dc5b957b60 from qemu
The gvec expanders perform a modulo on the shift count. If the target
requires alternate behaviour, then it cannot use the generic gvec
expanders anyway, and will have to have its own custom code.
Backports commit 5ee5c14cacda27e904cd6b0d9e7ffe1acff42838 from qemu
Allow the backend to expand dup from memory directly, instead of
forcing the value into a temp first. This is especially important
if integer/vector register moves do not exist.
Note that officially tcg_out_dupm_vec is allowed to fail.
If it did, we could fix this up relatively easily:
VECE == 32/64:
Load the value into a vector register, then dup.
Both of these must work.
VECE == 8/16:
If the value happens to be at an offset such that an aligned
load would place the desired value in the least significant
end of the register, go ahead and load w/garbage in high bits.
Load the value w/INDEX_op_ld{8,16}_i32.
Attempt a move directly to vector reg, which may fail.
Store the value into the backing store for OTS.
Load the value into the vector reg w/TCG_TYPE_I32, which must work.
Duplicate from the vector reg into itself, which must work.
All of which is well and good, except that all supported
hosts can support dupm for all vece, so all of the failure
paths would be dead code and untestable.
Backports commit 37ee55a081b7863ffab2151068dd1b2f11376914 from qemu
Replace the single opcode in .opc with a null-terminated
array in .opt_opc. We still require that all opcodes be
used with the same .vece.
Validate the contents of this list with CONFIG_DEBUG_TCG.
All tcg_gen_*_vec functions will check any list active
during .fniv expansion. Swap the active list in and out
as we expand other opcodes, or take control away from the
front-end function.
Convert all existing vector aware front ends.
Backports commit 53229a7703eeb2bbe101a19a33ef22aaf960c65b from qemu
Let's add tcg_gen_gvec_3i(), similar to tcg_gen_gvec_2i(), however
without introducing "gen_helper_gvec_3i *fnoi", as it isn't needed
for now.
Backports commit e1227bb6e59173117f094a6a13b998587b45c928 from qemu
Thereby decoupling the resulting translated code from the current state
of the system.
Backports commit 2399d4e7cec22ecf1c51062d2ebfd45220dbaace from qemu
The M-profile architecture floating point system supports
lazy FP state preservation, where FP registers are not
pushed to the stack when an exception occurs but are instead
only saved if and when the first FP instruction in the exception
handler is executed. Implement this in QEMU, corresponding
to the check of LSPACT in the pseudocode ExecuteFPCheck().
Backports commit e33cf0f8d8c9998a7616684f9d6aa0d181b88803 from qemu
Add a new helper function which returns the MMU index to use
for v7M, where the caller specifies all of the security
state, privilege level and whether the execution priority
is negative, and reimplement the existing
arm_v7m_mmu_idx_for_secstate_and_priv() in terms of it.
We are going to need this for the lazy-FP-stacking code.
Backports commit fa6252a988dbe440cd6087bf93cbe0887f0c401b from qemu
Will be helpful for s390x. Input 128 bit and output 64 bit only,
which is sufficient for now.
Backports commit 2089fcc9e7b4174d1c351eaa7d277c02188a6dd2 from qemu
We can't allow the supervisor to control SEIP as this would allow the
supervisor to clear a pending external interrupt which will result in
lost a interrupt in the case a PLIC is attached. The SEIP bit must be
hardware controlled when a PLIC is attached.
This logic was previously hard-coded so SEIP was always masked even
if no PLIC was attached. This patch adds riscv_cpu_claim_interrupts
so that the PLIC can register control of SEIP. In the case of models
without a PLIC (spike), the SEIP bit remains software controlled.
This interface allows for hardware control of supervisor timer and
software interrupts by other interrupt controller models.
Backports commit e3e7039cc24ecf47d81c091e8bb04552d6564ad8 from qemu
Add a debugger field to CPURISCVState. Add riscv_csrrw_debug function
to set it. Disable mode checks when debugger field true.
Backports commit 753e3fe207db08ce0ef0405e8452c3397c9b9308 from qemu
for now only LUI & AUIPC are decoded and translated. If decodetree fails, we
fall back to the old decoder.
Backports commit 2a53cff418335ccb4719e9a94fde55f6ebcc895d from qemu
This ports over the RISC-V architecture from Qemu. This is currently a
very barebones transition. No code hooking or any fancy stuff.
Currently, you can feed it instructions and query the CPU state itself.
This also allows choosing whether or not RISC-V 32-bit or RISC-V 64-bit
is desirable through Unicorn's interface as well.
Extremely basic examples of executing a single instruction have been
added to the samples directory to help demonstrate how to use the basic
functionality.
We do not need an out-of-line helper for manipulating bits in pstate.
While changing things, share the implementation of gen_ss_advance.
Backports commit 22ac3c49641f6eed93dca5b852030b4d3eacf6c4 from qemu
The EL0+UMA check is unique to DAIF. While SPSel had avoided the
check by nature of already checking EL >= 1, the other post v8.0
extensions to MSR (imm) allow EL0 and do not require UMA. Avoid
the unconditional write to pc and use raise_exception_ra to unwind.
Backports commit ff730e9666a716b669ac4a8ca7c521177d1d2b15 from qemu
Note that float16_to_float32 rightly squashes SNaN to QNaN.
But of course pickNaNMulAdd, for ARM, selects SNaNs first.
So we have to preserve SNaN long enough for the correct NaN
to be selected. Thus float16_to_float32_by_bits.
Backports commit a4e943a716d5fac923d82df3eabc65d1e3624019 from qemu
For same-sign saturation, we have tcg vector operations. We can
compute the QC bit by comparing the saturated value against the
unsaturated value.
Backports commit 89e68b575e138d0af1435f11a8ffcd8779c237bd from qemu
A number of CPUID registers are exposed to userspace by modern Linux
kernels thanks to the "ARM64 CPU Feature Registers" ABI. For QEMU's
user-mode emulation we don't need to emulate the kernels trap but just
return the value the trap would have done. To avoid too much #ifdef
hackery we process ARMCPRegInfo with a new helper (modify_arm_cp_regs)
before defining the registers. The modify routine is driven by a
simple data structure which describes which bits are exported and
which are fixed.
Backports commit 6c5c0fec29bbfe36c64eca1edfd8455be46b77c6 from qemu
A bug was introduced during a respin of:
commit 57a4a11b2b281bb548b419ca81bfafb214e4c77a
target/arm: Add array for supported PMU events, generate PMCEID[01]_EL0
This patch introduced two calls to get_pmceid() during CPU
initialization - one each for PMCEID0 and PMCEID1. In addition to
building the register values, get_pmceid() clears an internal array
mapping event numbers to their implementations (supported_event_map)
before rebuilding it. This is an optimization since much of the logic is
shared. However, since it was called twice, the contents of
supported_event_map reflect only the events in PMCEID1 (the second call
to get_pmceid()).
Fix this bug by moving the initialization of PMCEID0 and PMCEID1 back
into a single function call, and name it more appropriately since it is
doing more than simply generating the contents of the PMCEID[01]
registers.
Backports commit bf8d09694ccc07487cd73d7562081fdaec3370c8 from qemu
This commit doesn't add any supported events, but provides the framework
for adding them. We store the pm_event structs in a simple array, and
provide the mapping from the event numbers to array indexes in the
supported_event_map array. Because the value of PMCEID[01] depends upon
which events are supported at runtime, generate it dynamically.
Backports commit 57a4a11b2b281bb548b419ca81bfafb214e4c77a from qemu
Rename arm_ccnt_enabled to pmu_counter_enabled, and add logic to only
return 'true' if the specified counter is enabled and neither prohibited
or filtered.
Backports commit 033614c47de78409ad3fb39bb7bd1483b71c6789 from qemu
pmccntr_read and pmccntr_write contained duplicate code that was already
being handled by pmccntr_sync. Consolidate the duplicated code into two
functions: pmccntr_op_start and pmccntr_op_finish. Add a companion to
c15_ccnt in CPUARMState so that we can simultaneously save both the
architectural register value and the last underlying cycle count - this
ensures time isn't lost and will also allow us to access the 'old'
architectural register value in order to detect overflows in later
patches.
Backports commit 5d05b9d462666ed21b7fef61aa45dec9aaa9f0ff from qemu
The arm_regime_tbi{0,1} functions are replacable with the new function
by giving the lowest and highest address.
Backports commit 5d8634f5a3a8474525edcfd581a659830e9e97c0 from qemu
We need to reuse this from helper-a64.c. Provide a stub
definition for CONFIG_USER_ONLY. This matches the stub
definitions that we removed for arm_regime_tbi{0,1} before.
Backports commit bf0be433878935e824479e8ae890493e1fb646ed from qemu
While we could expose stage_1_mmu_idx, the combination is
probably going to be more useful.
Backports commit 64be86ab1b5ef10b660a4230ee7f27c0da499043 from qemu
The pattern
ARMMMUIdx mmu_idx = core_to_arm_mmu_idx(env, cpu_mmu_index(env, false));
is computing the full ARMMMUIdx, stripping off the ARM bits,
and then putting them back.
Avoid the extra two steps with the appropriate helper function.
Backports commit 50494a279dab22a015aba9501a94fcc3cd52140e from qemu
The cryptographic internals are stubbed out for now,
but the enable and trap bits are checked.
Backports commit 0d43e1a2d29a05f7b0d5629caaff18733cbdf3bb from qemu
Replace arm_hcr_el2_{fmo,imo,amo} with a more general routine
that also takes SCR_EL3.NS (aka arm_is_secure_below_el3) into
account, as documented for the plethora of bits in HCR_EL2.
Backports commit f77784446045231f7dfa46c9b872091241fa1557 from qemu
In commit 8a0fc3a29fc2315325400 we tried to implement HCR_EL2.{VI,VF},
but we got it wrong and had to revert it.
In that commit we implemented them as simply tracking whether there
is a pending virtual IRQ or virtual FIQ. This is not correct -- these
bits cause a software-generated VIRQ/VFIQ, which is distinct from
whether there is a hardware-generated VIRQ/VFIQ caused by the
external interrupt controller. So we need to track separately
the HCR_EL2 bit state and the external virq/vfiq line state, and
OR the two together to get the actual pending VIRQ/VFIQ state.
Fixes: 8a0fc3a29fc2315325400c738f807d0d4ae0ab7f
Backports commit 89430fc6f80a5aef1d4cbd6fc26b40c30793786c from qemu
Add a new flag to mark memory region that are used as non-volatile, by
NVDIMM for example. That bit is propagated down to the flat view, and
reflected in HMP info mtree with a "nv-" prefix on the memory type.
This way, guest_phys_blocks_region_add() can skip the NV memory
regions for dumps and TCG memory clear in a following patch.
Backports commit c26763f8ec70b1011098cab0da9178666d8256a5 from qemu
Implement hardware page table walker. This implementation is
limiter only to MIPS32.
Backports commit 074cfcb4daedf59ccbbbc83c24eee80e0e8f4c71 from qemu
Add PWCtl register (CP0 Register 5, Select 6).
The PWCtl register configures hardware page table walking for TLB
refills.
This register is required for the hardware page walker feature. It
exists only if Config3 PW bit is set to 1. It contains following
fields:
PWEn (31) - Hardware Page Table walker enable
PWDirExt (30) - If 1, 4-th level implemented (MIPS64 only)
XK (28) - If 1, walker handles xkseg (MIPS64 only)
XS (27) - If 1, walker handles xsseg (MIPS64 only)
XU (26) - If 1, walker handles xuseg (MIPS64 only)
DPH (7) - Dual Page format of Huge Page support
HugePg (6) - Huge Page PTE supported in Directory levels
PSn (5..0) - Bit position of PTEvld in Huge Page PTE
Backports commit 103be64c26c166f12b3e1308edadef3443723ff1 from qemu
Add PWSize register (CP0 Register 5, Select 7).
The PWSize register configures hardware page table walking for TLB
refills.
This register is required for the hardware page walker feature. It
exists only if Config3 PW bit is set to 1. It contains following
fields:
BDW (37..32) Base Directory index width (MIPS64 only)
GDW (29..24) Global Directory index width
UDW (23..18) Upper Directory index width
MDW (17..12) Middle Directory index width
PTW (11..6 ) Page Table index width
PTEW ( 5..0 ) Left shift applied to the Page Table index
Backports commit 20b28ebc49945583d7191b57755cfd92433de9ff from qemu
Add PWField register (CP0 Register 5, Select 6).
The PWField register configures hardware page table walking for TLB
refills.
This register is required for the hardware page walker feature. It
exists only if Config3 PW bit is set to 1. It contains following
fields:
MIPS64:
BDI (37..32) - Base Directory index
GDI (29..24) - Global Directory index
UDI (23..18) - Upper Directory index
MDI (17..12) - Middle Directory index
PTI (11..6 ) - Page Table index
PTEI ( 5..0 ) - Page Table Entry shift
MIPS32:
GDW (29..24) - Global Directory index
UDW (23..18) - Upper Directory index
MDW (17..12) - Middle Directory index
PTW (11..6 ) - Page Table index
PTEW ( 5..0 ) - Page Table Entry shift
Backports commit fa75ad1459f4f6abbeb6d375a812dfad61320f58 from qemu
Add code to insert calls to a helper function to do the stack
limit checking when we handle these forms of instruction
that write to SP:
* ADD (SP plus immediate)
* ADD (SP plus register)
* SUB (SP minus immediate)
* SUB (SP minus register)
* MOV (register)
Backports commit 5520318939fea5d659bf808157cd726cb967b761 from qemu
This implements the feature for softmmu, and moves the
main loop out of a macro and into a function.
Backports commit 116347ce20bb7b5cac17bf2b0e6f607530b50862 from qemu
We can choose the endianness at translation time, rather than
re-computing it at execution time.
Backports commit 28d57f2dc59c287e1c40239509b0a325fd00e32f from qemu
We can choose the endianness at translation time, rather than
re-computing it at execution time.
Backports commit 7d0a57a2e1cea188b9023261a404d7a211117230 from qemu
This fixes the endianness problem for softmmu, and moves
the main loop out of a macro and into an inlined function.
Backports commit 78cf1b886aa1b95c97fc5114641515c2892bb240 from qemu
This fixes the endianness problem for softmmu, and moves
the main loop out of a macro and into an inlined function.
Backports commit d4f75f25b43041e7a46d12352b3c70ae457d8cea from qemu
Use the existing helpers to determine if (1) the fpu is enabled,
(2) sve state is enabled, and (3) the current sve vector length.
Backports commit ced3155141755ba244c988c72c4bde32cc819670 from qemu
SVE vector length can change when changing EL, or when writing
to one of the ZCR_ELn registers.
For correctness, our implementation requires that predicate bits
that are inaccessible are never set. Which means noticing length
changes and zeroing the appropriate register bits.
Backports commit 0ab5953b00b3165877d00cf75de628c51670b550 from qemu
It has not had users since f83311e476 ("target-m68k: use floatx80
internally", 2017-06-21).
Note that no other bit-width has floatX_trunc_to_int.
Backports commit c953da8f0be5e026d1c9128660736d72294feb3e from qemu
if MemoryRegion intialization fails it's left in semi-initialized state,
where it's size is not 0 and attached as child to owner object.
And this leds to crash in following use-case:
(monitor) object_add memory-backend-file,id=mem1,size=99999G,mem-path=/tmp/foo,discard-data=yes
memory.c:2083: memory_region_get_ram_ptr: Assertion `mr->ram_block' failed
Aborted (core dumped)
it happens due to assumption that memory region is intialized when
memory_region_size() != 0
and therefore it's ok to access it in
file_backend_unparent()
if (memory_region_size() != 0)
memory_region_get_ram_ptr()
which happens when object_add fails and unparents failed backend making
file_backend_unparent() access invalid memory region.
Fix it by making sure that memory_region_init_foo() APIs cleanup externally
visible side effects on failure (like set size to 0 and unparenting object)
Added a helper for ROTX based on the pseudocode from the
architecture spec. This instraction was not present in previous
MIPS instruction sets.
Backports commit e222f5067269392af489731221750976d0cf3c05 from qemu
The API for cpu_transaction_failed() says that it takes the physical
address for the failed transaction. However we were actually passing
it the offset within the target MemoryRegion. We don't currently
have any target CPU implementations of this hook that require the
physical address; fix this bug so we don't get confused if we ever
do add one.
Backports commit 2d54f19401bc54b3b56d1cc44c96e4087b604b97 from qemu
