The cpu_physical_memory_reset_dirty() function is sometimes used
together with cpu_physical_memory_get_dirty(). This is not atomic since
two separate accesses to the dirty memory bitmap are made.
Turn cpu_physical_memory_reset_dirty() and
cpu_physical_memory_clear_dirty_range_type() into the atomic
cpu_physical_memory_test_and_clear_dirty().
Backports commit 03eebc9e3246b9b3f5925aa41f7dfd7c1e467875 from qemu
Use set_bit_atomic() and bitmap_set_atomic() so that multiple threads
can dirty memory without race conditions.
Backports commit d114875b9a1c21162f69a12d72f69a22e7bab376 from qemu
Most of the time, not all bitmaps have to be marked as dirty;
do not do anything if the interesting ones are already dirty.
Previously, any clean bitmap would have cause all the bitmaps to be
marked dirty.
In fact, unless running TCG most of the time bitmap operations need
not be done at all, because memory_region_is_logging returns zero.
In this case, skip the call to cpu_physical_memory_range_includes_clean
altogether as well.
With this patch, cpu_physical_memory_set_dirty_range is called
unconditionally, so there need not be anymore a separate call to
xen_modified_memory.
Backports commit e87f7778b64d4a6a78e16c288c7fdc6c15317d5f from qemu
cpu_physical_memory_set_dirty_lebitmap unconditionally syncs the
DIRTY_MEMORY_CODE bitmap. This however is unused unless TCG is
enabled.
Backports commit 9460dee4b2258e3990906fb34099481c8334c267 from qemu
The new bitmap_test_and_clear_atomic() function clears a range and
returns whether or not the bits were set.
Backports commit 36546e5b803f6e363906607307f27c489441fd15 from qemu
Use atomic_or() for atomic bitmaps where several threads may set bits at
the same time. This avoids the race condition between threads loading
an element, bitwise ORing, and then storing the element.
When setting all bits in a word we can avoid atomic ops and instead just
use an smp_mb() at the end.
Most bitmap users don't need atomicity so introduce new functions.
Backports commit 9f02cfc84b85929947b32fe1674fbc6a429f332a from qemu
While it is obvious that cpu_physical_memory_get_dirty returns true even if
a single page is dirty, the same is not true for cpu_physical_memory_get_clean;
one would expect that it returns true only if all the pages are clean, but
it actually looks for even one clean page. (By contrast, the caller of that
function, cpu_physical_memory_range_includes_clean, has a good name).
To clarify, rename the function to cpu_physical_memory_all_dirty and return
true if _all_ the pages are dirty. This is the opposite of the previous
meaning, because "all are 1" is the same as "not (any is 0)", so we have to
modify cpu_physical_memory_range_includes_clean as well
Backports commit 72b47e79cef36ed6ffc718f10e21001d7ec2a66f from qemu
is_cpu_write_access is only set if tb_invalidate_phys_page_range is called
from tb_invalidate_phys_page_fast, and hence from notdirty_mem_write.
However:
- the code bitmap can be built directly in tb_invalidate_phys_page_fast
(unconditionally, since is_cpu_write_access would always be passed as 1);
- the virtual address is not needed to mark the page as "not containing
code" (dirty code bitmap = 1), so we can also remove that use of
is_cpu_write_access. For calls of tb_invalidate_phys_page_range
that do not come from notdirty_mem_write, the next call to
notdirty_mem_write will notice that the page does not contain code
anymore, and will fix up the TLB entry.
The parameter needs to remain in order to guard accesses to cpu->mem_io_pc.
Backports commit fc377bcf617a48233a99a9fe0a26247c38b5cb76 from qemu
These days modification of the TLB is done in notdirty_mem_write,
so the virtual address and env pointer as unnecessary.
The new name of the function, tlb_unprotect_code, is consistent with
tlb_protect_code.
Backports commit 9564f52da7eb061326956ed9a468935e3352512d from qemu
Remove them from the sundry exec-all.h header, since they are only used by
the TCG runtime in exec.c and user-exec.c.
Backports commit 1652b974766401743879d78f796f44b8929b0787 from qemu
DIRTY_MEMORY_CODE is only needed for TCG. By adding it directly to
mr->dirty_log_mask, we avoid testing for TCG everywhere a region is
checked for the enabled/disabled state of dirty logging.
Backports commit 677e7805cf95f3b2bca8baf0888d1ebed7f0c606 from qemu
When the dirty log mask will also cover other bits than DIRTY_MEMORY_VGA,
some listeners may be interested in the overall zero/non-zero value of
the dirty log mask; others may be interested in the value of single bits.
For this reason, always call log_start/log_stop if bits have respectively
appeared or disappeared, and pass the old and new values of the dirty log
mask so that listeners can distinguish the kinds of change.
For example, KVM checks if dirty logging used to be completely disabled
(in log_start) or is now completely disabled (in log_stop). On the
other hand, Xen has to check manually if DIRTY_MEMORY_VGA changed,
since that is the only bit it cares about.
Backports commit b2dfd71c4843a762f2befe702adb249cf55baf66 from qemu
For now memory regions only track DIRTY_MEMORY_VGA individually, but
this will change soon. To support this, split memory_region_is_logging
in two functions: one that returns a given bit from dirty_log_mask,
and one that returns the entire mask. memory_region_is_logging gets an
extra parameter so that the compiler flags misuse.
While VGA-specific users (including the Xen listener!) will want to keep
checking that bit, KVM and vhost check for "any bit except migration"
(because migration is handled via the global start/stop listener
callbacks).
Backports commit 2d1a35bef0ed96b3f23535e459c552414ccdbafd from qemu
phys_page_set_level is writing zeroes to a struct that has just been
filled in by phys_map_node_alloc. Instead, tell phys_map_node_alloc
whether to fill in the page "as a leaf" or "as a non-leaf".
memcpy is faster than struct assignment, which copies each bitfield
individually. A compiler bug (https://gcc.gnu.org/PR66391), and
small memcpys like this one are special-cased anyway, and optimized
to a register move, so just use the memcpy.
This cuts the cost of phys_page_set_level from 25% to 5% when
booting qboot.
Backports commit db94604b20278c1dc227a04e4c564d80230e6c3f from qemu
At 8k per TLB (for 64-bit host or target), 8 or more modes
make the TLBs bigger than 64k, and some RISC TCG backends do
not like that. On the affected hosts, cut the TLB size in
half---there is still a measurable speedup on PPC with the
next patch.
Backports commit 1de29aef17a7d70dbc04a7fe51e18942e3ebe313 from qemu
This will be used to size the TLB when more than 8 MMU modes are
used by the target. Limitations come from the limited size of
the immediate fields (which sometimes, as in the case of Aarch64,
extend to instructions that shift the immediate).
Backports commit 006f8638c62bca2b0caf609485f47fa5e14d8a3c from qemu
Existing definition triggers the following when using clang
-fsanitize=undefined:
hw/intc/apic_common.c:314:55: runtime error: left shift of 1048575 by 12
places cannot be represented in type 'int'
Fix it so we won't try to shift a 1 to the sign bit of a signed integer.
Backports commit 458cf469f4a1cb520b07092f5537c5a6d2389d23 from qemu
The ARMCPRegInfo arrays v8_el3_no_el2_cp_reginfo and v8_el2_cp_reginfo
are actually used on non-v8 CPUs as well. Remove the incorrect v8_
prefix from their names.
Backports commit 4771cd01daaccb2a8929fa04c88c608e378cf814 from qemu
Add support for trapping WFI and WFE instructions to the proper EL when
SCTLR/SCR/HCR settings apply.
Backports commit b1eced713d9913a5c58ba9daa795f10e4c856c49 from qemu
Just NOP the WFI instruction if we have work to do.
This doesn't make much difference currently (though it does avoid
jumping out to the top level loop and immediately restarting),
but the distinction between "halt" and "don't halt" will become
more important when the decision to halt requires us to trap
to a higher exception level instead.
Backport commit 84549b6dcf9147559ec08b066de673587be6b763 from qemu
Deleting the now-unused ARM_TBFLAG_CPACR_FPEN left a gap in the
bit usage; move the following ARM_TBFLAG_XSCALE_CPAR and
ARM_TBFLAG_NS_SHIFT down 3 bits to fill the gap.
Backports commit 647f767ba3b37fb229275086187e96242248a4ac from qemu
Extend the ARM disassemble context to take a target exception EL instead of a
boolean enable. This change reverses the polarity of the check making a value
of 0 indicate floating point enabled (no exception).
Backports commit 9dbbc748d671c70599101836cd1c2719d92f3017 from qemu
Currently we keep the TB flags PSTATE_SS and SS_ACTIVE in different
bit positions for AArch64 and AArch32. Replace these separate
definitions with a single common flag in the upper part of the
flags word.
Backports commit 3cf6a0fcedd429693d439556543400d5f0e31e1d from qemu
Adds CPTR_EL2/3 system registers definitions and access function.
Signed-off-by: Greg Bellows <greg.bellows@linaro.org>
[PMM: merge CPTR_EL2 and HCPTR definitions into a single
def using STATE_BOTH;
don't use readfn/writefn to implement RAZ/WI registers;
don't use accessfn for the no-EL2 CPTR_EL2;
fix cpacr_access logic to catch EL2 accesses to CPACR being
trapped to EL3;
use new CP_ACCESS_TRAP_EL[23] rather than setting
exception.target_el directly]
Backports commit c6f191642a4027909813b4e6e288411f8371e951 from qemu
Some coprocessor access functions will need to indicate that the
instruction should trap to EL2 or EL3 rather than the default
target exception level; add corresponding CPAccessResult enum
entries and handling code.
Backports commit 38836a2cd47c20daaaa84873e3d6020f19e4bfca from qemu
Updated the interrupt handling to utilize and report through the target EL
exception field. This includes consolidating and cleaning up code where
needed. Target EL is now calculated once in arm_cpu_exec_interrupt() and
do_interrupt was updated to use the target_el exception field. The
necessary code from arm_excp_target_el() was merged in where needed and the
function removed.
Backports commit 012a906b19e99b126403ff4a257617dab9b34163 from qemu
Rather than making every caller of raise_exception set the
syndrome and target EL by hand, make these arguments to
raise_exception() and have that do the job.
Backports commit c63285991b371c031147ad620dd7671662a90303 from qemu
Move the code which sets exception information out of
arm_cpu_handle_mmu_fault and into tlb_fill. tlb_fill
is the only caller which wants to raise_exception()
so it makes more sense for it to handle the whole of
the exception setup.
As part of this cleanup, move the user-mode-only
implementation function for the handle_mmu_fault CPU
method into cpu.c so we don't need to make it globally
visible, and rename the softmmu-only utility function
arm_cpu_handle_mmu_fault to arm_tlb_fill so it's clear
that it's not the same thing.
Backports commit 8c6084bf10fe721929ca94cf16acd6687e61d3ec from qemu
If the SCTLR.UMA trap bit is set then attempts by EL0 to update
the PSTATE DAIF bits via "MSR DAIFSet, imm" and "MSR DAIFClr, imm"
instructions will raise an exception. We were failing to set
the syndrome information for this exception, which meant that
it would be reported as a repeat of whatever the previous
exception was. Set the correct syndrome information.
Backports commit f2932df777dace044719dc2f394f5a5a8aa1b1cd from qemu
Updated the various helper routines to set the target EL as needed using a
dedicated function.
Backports commit e3b1d480995f6e2e86ef062038e618c1234dbcf1 from qemu
Add a CPU state exception target EL field that will be used for communicating
the EL to which an exception should be routed.
Add a disassembly context field for tracking the EL3 architecture needed for
determining the target exception EL.
Add a target EL argument to the generic exception helper for callers to specify
the EL to which the exception should be routed. Extended the helper to set
the newly added CPU state exception target el.
Added a function for setting the target exception EL and updated calls to helpers
to call it.
Backports commit 737103619869600668cc7e8700e4f6eab3943896 from qemu
Updated get_phys_addr_lpae to check the appropriate TTBCR/TCR depending on the
current EL. Support includes using the different TCR format as well as checks to
insure TTBR1 is not used when in EL2 or EL3.
Backports commit 88e8add8b6656c349a96b447b074688d02dc5415 from qemu
Add a utility function for choosing the correct TTBR system register based on
the specified MMU index. Add use of function on physical address lookup.
Backports commit aef878be4e7ab1bdb30b408007320400b0a29c83 from qemu
Add the ARM Cortex-A53 processor definition. Similar to A57, but with
different L1 I cache policy, phys addr size and different cache
geometries. The cache sizes is implementation configurable, but use
these values (from Xilinx Zynq MPSoC) as a default until cache size
configurability is added.
Backports commit e35310260ec57d20301c65a5714ca55369e971cc from qemu
Rename some A57 CP register variables in preparation for support for
Cortex A53. Use "a57_a53" to describe the shareable features. Some of
the CP15 registers (such as ACTLR) are specific to implementation, but
we currently just RAZ them so continue with that as the policy for both
A57 and A53 processors under a shared definition.
Backports commit ee804264ddc4d3cd36a5183a09847e391da0fc66 from qemu
No code uses the cpu_pc_from_tb() function. Delete from tricore and
arm which each provide an unused implementation. Update the comment
in tcg.h to reflect that this is obsoleted by synchronize_from_tb.
Backports commit fee068e4f190a36ef3bda9aa7c802f90434ef8e5 from qemu
Due to old Seabios bug, QEMU reenable LINT0 after reset. This bug is long gone
and therefore this hack is no longer needed. Since it violates the
specifications, it is removed.
Backports commit b8eb5512fd8a115f164edbbe897cdf8884920ccb from qemu
address_space_translate_internal will clamp the *plen length argument
based on the size of the memory region being queried. The iommu walker
logic in addresss_space_translate was ignoring this by discarding the
post fn call value of *plen. Fix by just always using *plen as the
length argument throughout the fn, removing the len local variable.
This fixes a bootloader bug when a single elf section spans multiple
QEMU memory regions.
Backports commit 23820dbfc79d1c9dce090b4c555994f2bb6a69b3 from qemu
There could be a race condition when two processes call
address_space_map concurrently and both want to use the bounce buffer.
Add an in_use flag in BounceBuffer to sync it.
Backports commit c2cba0ffe495b60c4cc58080281e99c7a6580d4b from qemu
When CPU vendor is AMD, the AMD feature alias bits on
CPUID[0x80000001].EDX are already automatically copied from CPUID[1].EDX
on x86_cpu_realizefn(). When CPU vendor is Intel, those bits are
reserved and should be zero. On either case, those bits shouldn't be set
in the CPU model table.
Backports commit 726a8ff68677d8d5fba17eb0ffb85076bfb598dc from qemu
Static properties require only 1 line of code, much simpler than the
existing code that requires writing new getters/setters.
As a nice side-effect, this fixes an existing bug where the setters were
incorrectly allowing the properties to be changed after the CPU was
already realized.
Backports commit b9472b76d273c7796d877c49af50969c0a879c50 from qemu
Updated scr_write to always allow updates to the SCR.SMD bit on ARMv8
regardless of whether virtualization (EL2) is enabled or not.
Backports commit 4eb276408363aef5435a72a8e818f24220b5edd0 from qemu
Add a transaction attribute indicating that a memory access is being
done from user-mode (unprivileged). This corresponds to an equivalent
signal in ARM AMBA buses.
Backports commit 0995bf8cd91b81ec9c1078e37b808794080dc5c0 from qemu
Factor out the page table walk memory accesses into their own function,
so that we can specify the correct S/NS memory attributes for them.
This will also provide a place to use the correct endianness and
handle the need for a stage-2 translation when virtualization is
supported.
Backports commit ebca90e4c3aaaae5ed1ee7c569dea00d5d6ed476 from qemu
Honour the NS bit in ARM page tables:
* when adding entries to the TLB, include the Secure/NonSecure
transaction attribute
* set the NS bit in the PAR when doing ATS operations
Note that we don't yet correctly use the NSTable bit to
cause the page table walk itself to use the right attributes.
Backports commit 8bf5b6a9c1911d2c8473385fc0cebfaaeef42dbc from qem
This patch makes the following changes to the determination of
whether an address is executable, when translating addresses
using LPAE.
1. No longer assumes that PL0 can't execute when it can't read.
It can in AArch64, a difference from AArch32.
2. Use va_size == 64 to determine we're in AArch64, rather than
arm_feature(env, ARM_FEATURE_V8), which is insufficient.
3. Add additional XN determinants
- NS && is_secure && (SCR & SCR_SIF)
- WXN && (prot & PAGE_WRITE)
- AArch64: (prot_PL0 & PAGE_WRITE)
- AArch32: UWXN && (prot_PL0 & PAGE_WRITE)
- XN determination should also work in secure mode (untested)
- XN may even work in EL2 (currently impossible to test)
4. Cleans up the bloated PAGE_EXEC condition - by removing it.
The helper get_S1prot is introduced. It may even work in EL2,
when support for that comes, but, as the function name implies,
it only works for stage 1 translations.
Backports commit d8e052b387635639a6ba4a09a7874fd2f113b218 from qemu
Introduce simple_ap_to_rw_prot(), which has the same behavior as
ap_to_rw_prot(), but takes the 2-bit simple AP[2:1] instead of
the 3-bit AP[2:0]. Use this in get_phys_addr_v6 when SCTLR_AFE
is set, as that bit indicates we should be using the simple AP
format.
It's unlikely this path is getting used. I don't see CR_AFE
getting used by Linux, so possibly not. If it had been, then
the check would have been wrong for all but AP[2:1] = 0b11.
Anyway, this should fix it up, in case it ever does get used.
Backports commit d76951b65dfb1be4e41cfae6abebf8db7a1243a3 from qemu
Instead of mixing access permission checking with access permissions
to page protection flags translation, just do the translation, and
leave it to the caller to check the protection flags against the access
type. Also rename to ap_to_rw_prot to better describe the new behavior.
Backports commit 0fbf5238203041f734c51b49778223686f14366b from qemu
Switch all the uses of ld/st*_phys to address_space_ld/st*,
except for those cases where the address space is the CPU's
(ie cs->as). This was done with the following script which
generates a Coccinelle patch.
A few over-80-columns lines in the result were rewrapped by
hand where Coccinelle failed to do the wrapping automatically,
as well as one location where it didn't put a line-continuation
'\' when wrapping lines on a change made to a match inside
a macro definition.
===begin===
for FN in ub uw_le uw_be l_le l_be q_le q_be uw l q; do
cat <<EOF
@ cpu_matches_ld_${FN} @
expression E1,E2;
identifier as;
@@
ld${FN}_phys(E1->as,E2)
@ other_matches_ld_${FN} depends on !cpu_matches_ld_${FN} @
expression E1,E2;
@@
-ld${FN}_phys(E1,E2)
+address_space_ld${FN}(E1,E2, MEMTXATTRS_UNSPECIFIED, NULL)
EOF
done
for FN in b w_le w_be l_le l_be q_le q_be w l q; do
cat <<EOF
@ cpu_matches_st_${FN} @
expression E1,E2,E3;
identifier as;
@@
st${FN}_phys(E1->as,E2,E3)
@ other_matches_st_${FN} depends on !cpu_matches_st_${FN} @
expression E1,E2,E3;
@@
-st${FN}_phys(E1,E2,E3)
+address_space_st${FN}(E1,E2,E3, MEMTXATTRS_UNSPECIFIED, NULL)
EOF
done
===endit===
Backports commit 42874d3a8c6267ff7789a0396843c884b1d0933a from qemu
Add new address_space_ld*/st* functions which allow transaction
attributes and error reporting for basic load and stores. These
are named to be in line with the address_space_read/write/rw
buffer operations.
The existing ld/st*_phys functions are now wrappers around
the new functions.
Backports commit 500131154d677930fce35ec3a6f0b5a26bcd2973 from qemu
Make address_space_rw take transaction attributes, rather
than always using the 'unspecified' attributes.
Backports commit 5c9eb0286c819c1836220a32f2e1a7b5004ac79a from qemu
Convert the subpage memory ops to _with_attrs; this will allow
us to pass the attributes through to the underlying access
functions. (Nothing uses the attributes yet.)
Backports commit f25a49e0057bbfcc2b1111f60785d919b6ddaeea from qemu
Add a MemTxAttrs field to the IOTLB, and allow target-specific
code to set it via a new tlb_set_page_with_attrs() function;
pass the attributes through to the device when making IO accesses.
Backports commit fadc1cbe85c6b032d5842ec0d19d209f50fcb375 from qemu
Make the CPU iotlb a structure rather than a plain hwaddr;
this will allow us to add transaction attributes to it.
Backports commit e469b22ffda40188954fafaf6e3308f58d50f8f8 from qemu
Rather than retaining io_mem_read/write as simple wrappers around
the memory_region_dispatch_read/write functions, make the latter
public and change all the callers to use them, since we need to
touch all the callsites anyway to add MemTxAttrs and MemTxResult
support. Delete io_mem_read and io_mem_write entirely.
(All the callers currently pass MEMTXATTRS_UNSPECIFIED
and convert the return value back to bool or ignore it.)
Backports commit 3b6434953934e6d4a776ed426d8c6d6badee176f from qemu
Define an API so that devices can register MemoryRegionOps whose read
and write callback functions are passed an arbitrary pointer to some
transaction attributes and can return a success-or-failure status code.
This will allow us to model devices which:
* behave differently for ARM Secure/NonSecure memory accesses
* behave differently for privileged/unprivileged accesses
* may return a transaction failure (causing a guest exception)
for erroneous accesses
This patch defines the new API and plumbs the attributes parameter through
to the memory.c public level functions io_mem_read() and io_mem_write(),
where it is currently dummied out.
The success/failure response indication is also propagated out to
io_mem_read() and io_mem_write(), which retain the old-style
boolean true-for-error return.
Backports commit cc05c43ad942165ecc6ffd39e41991bee43af044 from qemu
A LDRD or STRD where rd is not an even number is UNPREDICTABLE.
We were letting this fall through, which is OK unless rd is 15,
in which case we would attempt to do a load_reg or store_reg
to a nonexistent r16 for the second half of the double-word.
Catch the odd-numbered-rd cases and UNDEF them instead.
To do this we rearrange the structure of the code a little
so we can put the UNDEF catches at the top before we've
allocated TCG temporaries.
Backports commit a4bb522ee51087af61998f290d12ba2e14c7910e from qemu
Since the BSP bit is writable on real hardware, during reset all the CPUs which
were not chosen to be the BSP should have their BSP bit cleared. This fix is
required for KVM to work correctly when it changes the BSP bit.
An additional fix is required for QEMU tcg to allow software to change the BSP
bit.
Backports commit 9cb11fd7539b5b787d8fb3834004804a58dd16ae from qemu
The AArch64 SPSR_EL1 register is architecturally mandated to
be mapped to the AArch32 SPSR_svc register. This means its
state should live in QEMU's env->banked_spsr[1] field.
Correct the various places in the code that incorrectly
put it in banked_spsr[0].
Backports commit 7847f9ea9fce15a9ecfb62ab72c1e84ff516b0db from qemu
Commit 8074264 (qom: Add description field in ObjectProperty struct)
introduced property descriptions and copied them for alias properties.
Instead of using the caller-supplied property name, use the returned
property name for setting the description. This avoids an Error when
setting a property description for a property with literal "[*]" that
doesn't exist due to automatic property naming in object_property_add().
Backports commit a18bb417e954ceea0a30b46c38b0d58c3a7ca6a1 from qemu
PC needs to be saved if an exception can be generated by an helper.
This fixes a problem related to resuming the execution at unexpected address
after an exception (caused by MSA load/store instruction) has been serviced.
Backports commit 0af7a37054310384e00209e0a43efe95b7c19ef0 from qemu
All instructions which may change hflags terminate tb. However, this doesn't
work if such an instruction is placed in delay or forbidden slot.
gen_branch() clears MIPS_HFLAG_BMASK in ctx->hflags and then generates code
to overwrite hflags with ctx->hflags, consequently we loose any execution-time
hflags modifications. For example, in the following scenario hflag related to
Status.CU1 will not be updated:
/* Set Status.CU1 in delay slot */
mfc0 $24, $12, 0
lui $25, 0x2000
or $25, $25, $24
b check_Status_CU1
mtc0 $25, $12, 0
With this change we clear MIPS_HFLAG_BMASK in execution-time hflags if
instruction in delay or forbidden slot wants to terminate tb for some reason
(i.e. ctx->bstate != BS_NONE).
Also, die early and loudly if "unknown branch" is encountered as this should
never happen.
Backports commit a5f533909e746ca6e534b232fb42c9c6fd81b468 from qemu
CP0.BadVAddr is supposed to capture the most recent virtual address that caused
the exception. Currently this does not work correctly for unaligned instruction
fetch as translation is not stopped and CP0.BadVAddr is updated with subsequent
addresses.
Backports commit 62c688693bf2f0355fc5bad5dcc59c1cd2a51f1a from qemu
For the ARM M-profile cores, exception return pops various registers
including the PC from the stack. The architecture defines that if the
lowest bit in the new PC value is set (ie the PC is not halfword
aligned) then behaviour is UNPREDICTABLE. In practice hardware
implementations seem to simply ignore the low bit, and some buggy
RTOSes incorrectly rely on this. QEMU's behaviour was architecturally
permitted, but bringing QEMU into line with the hardware behaviour
allows more guest code to run. We log the situation as a guest error.
This was reported as LP:1428657.
Backports commit fcf83ab103dce6d2951f24f48e30820e7dbb3622 from qemu
The A32 encoding of LDM distinguishes LDM (user) from LDM (exception
return) based on whether r15 is in the register list. However for
STM (user) there is no equivalent distinction. We were incorrectly
treating "r15 in list" as indicating exception return for both LDM
and STM, with the result that an STM (user) involving r15 went into
an infinite loop. Fix this; note that the value stored for r15
in this case is the current PC regardless of our current mode.
Backports commit da3e53ddcb0ca924da97ca5a35605fc554aa3e05 from qemu
Save MSACSR state. Also remove fp_status, msa_fp_status, hflags and restore
them in post_load() from the architectural registers.
Float exception flags are not present in vmstate. Information they carry
is used only by softfloat caller who translates them into MIPS FCSR.Cause,
FCSR.Flags and then they are cleared. Therefore there is no need for saving
them in vmstate.
Backports commit 644511117e7ca9f26d633a59c202a297113a796c from qemu
The documentation for sextract64() claims that the return type is
an int64_t, but the code itself disagrees. Fix the return type to
conform to the documentation and to bring it into line with
sextract32(), which returns int32_t.
Backports commit 4f9950520a115acf9c0a209f0befa45758ad0215 from qemu
According to my reading of the Intel documentation, the SYSRET instruction
is supposed to force the RPL bits of the %ss register to 3 when returning
to user mode. The actual sequence is:
SS.Selector <-- (IA32_STAR[63:48]+8) OR 3; (* RPL forced to 3 *)
However, the code in helper_sysret() leaves them at 0 (in other words, the "OR
3" part of the above sequence is missing). It does set the privilege level
bits of %cs correctly though.
This has caused me trouble with some of my VxWorks development: code that runs
okay on real hardware will crash on QEMU, unless I apply the patch below.
Backports commit ac57622985220de064059971f9ccb00905e9bd04 from qemu
The APIC ID compatibility code is required only for PC, and now that
x86_cpu_initfn() doesn't use x86_cpu_apic_id_from_index() anymore, that
code can be moved to pc.c.
Backports commit de13197a38cf45c990802661a057f64a05426cbc from qemu
Instead of setting APIC ID automatically when creating a X86CPU, require
the property to be set before realizing the object (which all callers of
cpu_x86_create() already do).
Backports commit e1356dd70aef11425883dd4d2885f1d208eb9d57 from qemu
The PC CPU initialization code already sets apic-id based on the CPU
topology, and CONFIG_USER doesn't need the topology-based APIC ID
calculation code.
Make CONFIG_USER set apic-id before realizing the CPU (just like PC
already does), so we can simplify x86_cpu_initfn later. As there is no
CPU topology configuration in CONFIG_USER, just use cpu_index as the
APIC ID.
Backports commit 9c235e83f1c3437be6ca45755909efb745c10deb from qemu
The field doesn't need to be inside CPUState, and it is not specific for
the CPUID instruction, so move and rename it.
Backports commit 9e9d3863adcbd1ffeca30f240f49805b00ba0d87 from qemu
Instead of putting extra logic inside cpu.h, just do everything inside
cpu_x86_init_user().
Backports commit 15258d46baef5f8265ad5f1002905664cf58f051 from qem
The function was used in only two places. In one of them, the function
made the code less readable by requiring temporary te[bcd]x variables.
In the other one we can simply inline the existing code.
Backports commit 08e1a1e5a175ecbfdb761db5a62090498f736969 from qemu
After the previous patch, TLBs will be flushed on every change to
the memory mapping. This patch augments that with synchronization
of the MemoryRegionSections referred to in the iotlb array.
With this change, it is guaranteed that iotlb_to_region will access
the correct memory map, even once the TLB will be accessed outside
the BQL.
Backports commit 9d82b5a792236db31a75b9db5c93af69ac07c7c5 from qemu
This for now is a simple TLB flush. This can change later for two
reasons:
1) an AddressSpaceDispatch will be cached in the CPUState object
2) it will not be possible to do tlb_flush once the TCG-generated code
runs outside the BQL.
Backports commit 76e5c76f2e2e0d20bab2cd5c7a87452f711654fb from qemu
Avoid shifting potentially negative signed offset values in
disas_ldst_pair() by keeping the offset in a uint64_t rather
than an int64_t.
Backports commit c2ebd862a54b7e12175d65c03ba259926cb2237a from qemu
Shifting a negative integer left is undefined behaviour in C.
Avoid it by assembling and shifting the offset fields as
unsigned values and then sign extending as the final action.
Backports commit 037e1d009e2fcb80784d37f0e12aa999787d46d4 from qemu
The code in logic_imm_decode_wmask attempts to rotate a mask
value within the bottom 'e' bits of the value with
mask = (mask >> r) | (mask << (e - r));
This has two issues:
* if the element size is 64 then a rotate by zero results
in a shift left by 64, which is undefined behaviour
* if the element size is smaller than 64 then this will
leave junk in the value at bit 'e' and above, which is
not valid input to bitfield_replicate(). As it happens,
the bits at bit 'e' to '2e - r' are exactly the ones
which bitfield_replicate is going to copy in there,
so this isn't a "wrong code generated" bug, but it's
confusing and if we ever put an assert in
bitfield_replicate it would fire on valid guest code.
Fix the former by not doing anything if r is zero, and
the latter by masking with bitmask64(e).
Backports commit e167adc9d9f5df4f8109aecd4552c407fdce094a from qemu
Fix attempts to shift into the sign bit of an int, which is undefined
behaviour in C and warned about by the clang sanitizer.
Backports commit 1743d55c8b38bcee632cf6eb2de81131635bb3d2 from qemu
Add AArch32 to AArch64 register sychronization functions.
Replace manual register synchronization with new functions in
aarch64_cpu_do_interrupt() and HELPER(exception_return)().
Backports commit ce02049dbf1828b4bc77d921b108a9d84246e5aa from qemu
Adds registration and get/set functions for enabling/disabling the AArch64
execution state on AArch64 CPUs. By default AArch64 execution state is enabled
on AArch64 CPUs, setting the property to off, will disable the execution state.
The below QEMU invocation would have AArch64 execution state disabled.
$ ./qemu-system-aarch64 -machine virt -cpu cortex-a57,aarch64=off
Also adds stripping of features from CPU model string in acquiring the ARM CPU
by name.
Backports part of commit fb8d6c24b095c426151b9bba8c8b0e58b03d6503 from qemu
The code in the softfloat source files is under a mixture of
licenses: the original code and many changes from QEMU contributors
are under the base SoftFloat-2a license; changes from Stefan Weil
and RedHat employees are GPLv2-or-later; changes from Fabrice Bellard
are under the BSD license. Clarify this in the comments at the
top of each affected source file, including a statement about
the assumed licensing for future contributions, so we don't need
to remember to ask patch submitters explicitly to pick a license.
Backports commit 16017c48547960539fcadb1f91d252124f442482 from qemu
Revert the parts of commits b645bb4885 and 5a6932d51d which are still
in the codebase and under a SoftFloat-2b license.
Reimplement support for architectures where the most significant bit
in the mantissa is 1 for a signaling NaN rather than a quiet NaN,
by adding handling for SNAN_BIT_IS_ONE being set to the functions
which test values for NaN-ness.
This includes restoring the bugfixes lost in the reversion where
some of the float*_is_quiet_nan() functions were returning true
for both signaling and quiet NaNs.
[This is a mechanical squashing together of two separate "revert"
and "reimplement" patches.]
Backports commit 332d5849708d11b835e0b36f4e26e8b36bfb3f5a from qemu
Revert the remaining portions of commits 75d62a5856 and 3430b0be36f
which are under a SoftFloat-2b license, ie the functions
uint64_to_float32() and uint64_to_float64(). (The float64_to_uint64()
and float64_to_uint64_round_to_zero() functions were completely
rewritten in commits fb3ea83aa and 0a87a3107d so can stay.)
Reimplement from scratch the uint64_to_float64() and uint64_to_float32()
conversion functions.
[This is a mechanical squashing together of two separate "revert"
and "reimplement" patches.]
Backports commit 6bb8e0f130bd4aecfe835a0caa94390fa2235fde from qemu
This commit applies the changes to master which correspond to
replacing commit 158142c2c2df with a set of changes made by:
* taking the SoftFloat-2a release
* mechanically transforming the block comment style
* reapplying Fabrice's original changes from 158142c2c2df
This commit was created by:
diff -u 158142c2c2df import-sf-2a
patch -p1 --fuzz 10 <../relicense-patch.txt
(where import-sf-2a is the branch resulting from the changes above).
Backports commit a7d1ac78e0f1101df2ff84502029a4b0da6024ae from qemu
Right now, the AVX512 registers are split in many different fields:
xmm_regs for the low 128 bits of the first 16 registers, ymmh_regs
for the next 128 bits of the same first 16 registers, zmmh_regs
for the next 256 bits of the same first 16 registers, and finally
hi16_zmm_regs for the full 512 bits of the second 16 bit registers.
This makes it simple to move data in and out of the xsave region,
but would be a nightmare for a hypothetical TCG implementation and
leads to a proliferation of [XYZ]MM_[BWLSQD] macros. Instead,
this patch marshals data manually from the xsave region to a single
32x512-bit array, simplifying the macro jungle and clarifying which
bits are in which vmstate subsection.
The migration format is unaffected.
Backports commit b7711471f551aa4419f9d46a11121f48ced422da from qemu
f64 exponent in HELPER(recpe_f64) should be compared to 2045 rather than 1023
(FPRecipEstimate in ARMV8 spec). This fixes incorrect underflow handling when
flushing denormals to zero in the FRECPE instructions operating on 64-bit
values.
Backports commit fc1792e9aa36227ee9994757974f9397684e1a48 from qemu
This patch implements a fucntion pointer "virtio_is_big_endian"
from "CPUClass" structure for arm/arm64.
Function arm_cpu_is_big_endian() is added to determine and
return the guest cpu endianness to virtio.
This is required for running cross endian guests with virtio on ARM/ARM64.
Backports commit 84f2bed3cf505f90b7918e2de32e11da27160563 from qemu
A few of the oldest parts of the page-table-walk code have broken indent
(either hardcoded tabs or two-spaces). Reindent these sections.
For ease of review, this patch does not touch the brace style and
so is a whitespace-only change.
Backports commit 554b0b09aec4579c8164f363b18a263150e91a2c from qemu
Now we have the mmu_idx in get_phys_addr(), use it correctly to
determine the behaviour of virtual to physical address translations,
rather than using just an is_user flag and the current CPU state.
Some TODO comments have been added to indicate where changes will
need to be made to add EL2 and 64-bit EL3 support.
Backports commit 0480f69abf849ca0d48928cc6c669c1c7264239b from qemu
Make all the callers of get_phys_addr() pass it the correct
mmu_idx rather than just a simple "is_user" flag. This includes
properly decoding the AT/ATS system instructions; we include the
logic for handling all the opc1/opc2 cases because we'll need
them later for supporting EL2/EL3, even if we don't have the
regdef stanzas yet.
Backports commit d364970287c0ba68979711928c15e5d37414f87f from qemu
Instead of simply reusing ats_write() as the handler for both AArch32
and AArch64 address translation operations, use a different function
for each with the common code in a third function. This is necessary
because the semantics for selecting the right translation regime are
different; we are only getting away with sharing currently because
we don't support EL2 and only support EL3 in AArch32.
Backports commit 060e8a48cb84d41d4ac36e4bb29d9c14ed7168b6 from qemu
target-arm doesn't use any of the MMU-mode specific cpu ldst
accessor functions. Suppress their generation by not defining
any of the MMU_MODE*_SUFFIX macros. ("user" and "kernel" are
too simplistic as descriptions of indexes 0 and 1 anyway.)
Backports commit 0dfef7b58f0c24b463e36630f08a45e93012b33a from qemu
The MMU index to use for unprivileged loads and stores is more
complicated than we currently implement:
* for A64, it should be "if at EL1, access as if EL0; otherwise
access at current EL"
* for A32/T32, it should be "if EL2, UNPREDICTABLE; otherwise
access as if at EL0".
In both cases, if we want to make the access for Secure EL0
this is not the same mmu_idx as for Non-Secure EL0.
Backports commit 579d21cce63f3dd2f6ee49c0b02a14e92cb4a836 from qemu
We currently claim that for ARM the mmu_idx should simply be the current
exception level. However this isn't actually correct -- secure EL0 and EL1
should have separate indexes from non-secure EL0 and EL1 since their
VA->PA mappings may differ. We also will want an index for stage 2
translations when we properly support EL2.
Define and document all seven mmu index values that we require, and
pass the mmu index in the TB flags rather than exception level or
priv/user bit.
This change doesn't update the get_phys_addr() code, so our page
table walking still assumes a simplistic "user or priv?" model for
the moment.
Backports commit c1e3781090b9d36c60e1a254ba297cb34011d3d4 from qemu
Support guest CPUs which need 7 MMU index values.
Add a comment about what would be required to raise the limit
further (trivial for 8, TCG backend rework for 9 or more).
Backports commit 8f3ae2ae2d02727f6d56610c09d7535e43650dd4 from qemu
Although M profile doesn't have the same concept of exception level
as A profile, it does have a notion of privileged versus not, which
we currently track in the privmode TB flag. Support returning this
information if arm_current_el() is called on an M profile core, so
that we can identify the correct MMU index to use (and put the MMU
index in the TB flags) without having to special-case M profile.
Backports commit 6d54ed3c93f1e05a483201b087142998381c9be8 from qemu
The documentation states that if LSB > MSB in BFI instruction behaviour
is unpredictable. Currently QEMU crashes because of assertion failure in
this case:
tcg/tcg-op.h:2061: tcg_gen_deposit_i32: Assertion `len <= 32' failed.
While assertion failure may meet the "unpredictable" definition this
behaviour is undesirable because it allows an unprivileged guest program
to crash the emulator with the OS and other programs.
This patch addresses the issue by throwing illegal instruction exception
if LSB > MSB. Only ARM decoder is affected because Thumb decoder already
has this check in place.
To reproduce issue run the following program
int main(void) {
asm volatile (".long 0x07c00c12" :: );
return 0;
}
compiled with
gcc -marm -static badop_arm.c -o badop_arm
Backports commit 45140a57675ecb4b0daee71bf145c24dbdf9429c from qemu
The helper functions for FRECPS and FRSQRTS have special case
handling that includes checks for zero inputs, so squash input
denormals if necessary before those checks. This fixes incorrect
output when the FPCR DZ bit is set to enable squashing of input
denormals.
Backports commit a8eb6e19991d1a7a6a7b04ac447548d30d75eb4a from qemu
Add assertion checking when cpreg structures are registered that they
either forbid raw-access attempts or at least make an attempt at
handling them. Also add an assert in the raw-accessor-of-last-resort,
to avoid silently doing a read or write from offset zero, which is
actually AArch32 CPU register r0.
Backports commit 375421ccaeebae8212eb8f9a36835ad4d9dc60a8 from qemu
We currently mark ARM coprocessor/system register definitions with
the flag ARM_CP_NO_MIGRATE for two different reasons:
1) register is an alias on to state that's also visible via
some other register, and that other register is the one
responsible for migrating the state
2) register is not actually state at all (for instance the TLB
or cache maintenance operation "registers") and it makes no
sense to attempt to migrate it or otherwise access the raw state
This works fine for identifying which registers should be ignored
when performing migration, but we also use the same functions for
synchronizing system register state between QEMU and the kernel
when using KVM. In this case we don't want to try to sync state
into registers in category 2, but we do want to sync into registers
in category 1, because the kernel might have picked a different
one of the aliases as its choice for which one to expose for
migration. (In particular, on 32 bit hosts the kernel will
expose the state in the AArch32 version of the register, but
TCG's convention is to mark the AArch64 version as the version
to migrate, even if the CPU being emulated happens to be 32 bit,
so almost all system registers will hit this issue now that we've
added AArch64 system emulation.)
Fix this by splitting the NO_MIGRATE flag in two (ALIAS and NO_RAW)
corresponding to the two different reasons we might not want to
migrate a register. When setting up the TCG list of registers to
migrate we honour both flags; when populating the list from KVM,
only ignore registers which are NO_RAW.
Backports commit 7a0e58fa648736a75f2a6943afd2ab08ea15b8e0 from qemu
Update to arm_cpu_reset() to reset into the highest available exception level
based on the set ARM features.
Backports commit 5097227c15aa89baec1123aac25dd9500a62684d from qemu
Added RVBAR_EL2 and RVBAR_EL3 CP register support. All RVBAR_EL# registers
point to the same location and only the highest EL version exists at any one
time.
Backports commit be8e8128595b41b9f609c1507e67d121e65e7173 from qemu
The crypto emulation code in target-arm/crypto_helper.c never worked
correctly on big endian hosts, due to the fact that it uses a union
of array types to convert between the native VFP register size (64
bits) and the types used in the algorithms (bytes and 32 bit words)
We cannot just swab between LE and BE when reading and writing the
registers, as the SHA code performs word additions, so instead, add
array accessors for the CRYPTO_STATE type whose LE and BE specific
implementations ensure that the correct array elements are referenced.
Backports commit b449ca3c1874418d948878d5417a32fc0dbf9fea from qemu
Added a "has_el3" state property to the ARMCPU descriptor. This property
indicates whether the ARMCPU has security extensions enabled (EL3) or not.
By default it is disabled at this time.
Backports commit 51942aee3c51ca23b0dd78f95534a57e8dc1e582 from qemu
Add an unset_feature() function to compliment the set_feature() function. This
will be used to disable functions after they have been enabled during
initialization.
Backports commit 08828484a5c1ec55a6cbb4b4d377bfcf41199b5c from qemu
Merge of the v8_el2_cp_reginfo and el3_cp_reginfo ARMCPRegInfo lists.
Previously, some EL3 registers were restricted to the ARMv8 list under the
impression that they were not needed on ARMv7. However, this is not the case
as the ARMv7/32-bit variants rely on the ARMv8/64-bit variants to handle
migration and reset. For this reason they must always exist.
Backports commit 60fb1a87b47b14e4ea67043aa56f353e77fbd70a from qemu
When EL3 is running in AArch32 (or ARMv7 with Security Extensions)
FCSEIDR, CONTEXTIDR, TPIDRURW, TPIDRURO and TPIDRPRW have a secure
and a non-secure instance.
Backports commit 54bf36ed351c526cde0c853079f9ff1ab7e2ff89 from qemu
When EL3 is running in Aarch32 (or ARMv7 with Security Extensions)
VBAR has a secure and a non-secure instance, which are mapped to
VBAR_EL1 and VBAR_EL3.
Backports commit fb6c91ba2bb0b1c1b8662ceeeeb9474a025f9a6b from qemu
When EL3 is running in AArch32 (or ARMv7 with Security Extensions)
PAR has a secure and a non-secure instance.
Backports commit 01c097f7960b330c4bf038d34bae17ad6c1ba499 from qemu
When EL3 is running in AArch32 (or ARMv7 with Security Extensions)
IFAR and DFAR have a secure and a non-secure instance.
Backports commit b848ce2b9cbd38da3f2530fd93dba76dba0621c0 from qemu
When EL3 is running in AArch32 (or ARMv7 with Security Extensions)
DFSR has a secure and a non-secure instance.
Backports commit 4a7e2d7315bd2ce28e49ccd0bde73eabdfd7437b from qemu
When EL3 is running in AArch32 (or ARMv7 with Security Extensions)
IFSR has a secure and a non-secure instance. Adds IFSR32_EL2 definition and
storage.
Backports commit 88ca1c2d70523486a952065f3ed7b8fc823b5863 from qemu
When EL3 is running in AArch32 (or ARMv7 with Security Extensions)
DACR has a secure and a non-secure instance. Adds definition for DACR32_EL2.
Backports commit 0c17d68c1d3d6c35f37f5692042d2edb65c8bcc0 from qemu
Adds secure and non-secure bank register suport for TTBCR.
Added new struct to compartmentalize the TCR data and masks. Removed old
tcr/ttbcr data and added a 4 element array of the new structs in cp15. This
allows for one entry per EL. Added a CP register definition for TCR_EL3.
Backports commit 11f136ee25232a00f433cefe98ee33cd614ecccc from qemu
Adds secure and non-secure bank register suport for TTBR0 and TTBR1.
Changes include adding secure and non-secure instances of ttbr0 and ttbr1 as
well as a CP register definition for TTBR0_EL3. Added a union containing
both EL based array fields and secure and non-secure fields mapped to them.
Updated accesses to use A32_BANKED_CURRENT_REG_GET macro.
Backports commit 7dd8c9af0d9d18fb3e54a4843b3bb1398bd330bc to qemu
Add checks of SCR AW/FW bits when performing writes of CPSR. These SCR bits
are used to control whether the CPSR masking bits can be adjusted from
non-secure state.
Backports commit 6e8801f9dea9e10449f4fd7d85dbe8cab708a686 from qemu
Use MVBAR register as exception vector base address for
exceptions taken to CPU monitor mode.
Backports commit e89e51a17ea0d8aef9bf9b766c98f963e835fbf2 from qemu
Added CP register defintions for SDER and SDER32_EL3 as well as cp15.sder for
register storage.
Backports commit 144634ae6c1618dcee6aced9c0d4427844154091 from qemu
Implements NSACR register with corresponding read/write functions
for ARMv7 and ARMv8.
Backports commit 770225764f831031d2e1453f69c365eb1b647d87 from qemu
SCR.{IRQ/FIQ} bits allow to route IRQ/FIQ exceptions to monitor CPU
mode. When taking IRQ exception to monitor mode FIQ exception is
additionally masked.
Backports commit de38d23b542efca54108ef28bcc0efe96f378d2e from qemu
Define a new ARM CP register info list for the ARMv7 Security Extension
feature. Register that list only for ARM cores with Security Extension/EL3
support. Moving AArch32 SCR into Security Extension register group.
Backports commit 0f1a3b2470d798ad5335eb9d6236f02ff64e31a8 from qemu
Prepare for cp register banking by inserting every cp register twice,
once for secure world and once for non-secure world.
Backports commit 3f3c82a57d128aa3ec823aa8032867c3a6e2e795 from qemu
Added additional NS-bit to CPREG hash encoding. Updated hash lookup
locations to specify hash bit currently set to non-secure.
Backports commit 51a79b039728277e35fd19f7a7b4bc6cb323697f from qemu
Prepare ARMCPRegInfo to support specifying two fieldoffsets per
register definition. This will allow us to keep one register
definition for banked registers (different offsets for secure/
non-secure world).
Also added secure state tracking field and flags. This allows for
identification of the register info secure state.
Backports commit c3e302606253a17568dc3ef30238f102468f7ee1 from qemu
This patch is based on idea found in patch at
git://github.com/jowinter/qemu-trustzone.git
f3d955c6c0ed8c46bc0eb10b634201032a651dd2 by
Johannes Winter <johannes.winter@iaik.tugraz.at>.
The TBFLAG captures the SCR NS secure state at the time when a TB is created so
the correct bank is accessed on system register accesses.
Backports commit 3f342b9e0e64ad681cd39840bfa75ef12d2807c1 from qemu
If EL3 is in AArch32 state certain cp registers are banked (secure and
non-secure instance). When reading or writing to coprocessor registers
the following macros can be used.
- A32_BANKED macros are used for choosing the banked register based on provided
input security argument. This macro is used to choose the bank during
translation of MRC/MCR instructions that are dependent on something other
than the current secure state.
- A32_BANKED_CURRENT macros are used for choosing the banked register based on
current secure state. This is NOT to be used for choosing the bank used
during translation as it breaks monitor mode.
If EL3 is operating in AArch64 state coprocessor registers are not
banked anymore. The macros use the non-secure instance (_ns) in this
case, which is architecturally mapped to the AArch64 EL register.
Backports commit ea30a4b824ecc3c829b70eb9999ac5457dc5790f from qemu
Adds a dedicated function and a lookup table for determining the target
exception level of IRQ and FIQ exceptions. The lookup table is taken from the
ARMv7 and ARMv8 specification exception routing tables.
Backports commit 0eeb17d618361a0f4faddc160e33598b23da6dd5 from qemu
This patch extends arm_excp_unmasked() to use lookup tables for determining
whether IRQ and FIQ exceptions are masked. The lookup tables are based on the
ARMv8 and ARMv7 specification physical interrupt masking tables.
If EL3 is using AArch64 IRQ/FIQ masking is ignored in all exception levels
other than EL3 if SCR.{FIQ|IRQ} is set to 1 (routed to EL3).
Backports commit 57e3a0c7cb0ac2f0288890482e0a463adce2080a from qemu
Using rs = -1 in gen_logic_imm() for microMIPS LUI instruction is dangerous
and may bite us when implementing microMIPS R6 because in R6 AUI and LUI
are distinguished by rs value. Therefore use 0 for safety.
Backports commit 5e88759a52934a32502298f2c78c6dfaa144364b from qemu
The test is supposed to terminate TB if the end of the page is reached.
However, with current implementation it may never succeed for microMIPS or
mips16.
Backports commit fe2372910a09034591fd2cfc2d70cca43fccaa95 from qemu
Commit fecd264 added a number of fall-throughs, but neglected to
properly document them as intentional. Commit d922445 cleaned that up
for many, but not all cases. Take care of the remaining ones.
Backports commit b6f3b233eabb4df5d65ae9fbfb3d3c8befea0de7 from qemu
Reduce line wrapping throughout MSA helper macros by using a local float
status pointer rather than referring to the float status through the
environment each time. No functional change.
Backports commit 1a4d570017bf35d99340781ecb59dd3772464031 from qemu
Add missing calls to synchronise the SoftFloat status with the CP1.FSCR:
+ for the rounding and flush-to-zero modes upon processor reset,
+ for the flush-to-zero mode on FSCR updates through the GDB stub.
Refactor code accordingly and remove the redundant RESTORE_ROUNDING_MODE
macro.
Backports commit bb962386b82c1b0e9e12fdb6b9bb62106bf1f822 from qemu
Make CP0.Status writes made with the MTTC0 instruction respect this
register's mask just like all the other places. Also preserve the
current values of masked out bits.
Backports commit 1d725ae952a14b30c84b7bc81b218b8ba77dd311 from qemu
Make sure the address space is unconditionally wrapped on 32-bit
processors, that is ones that do not implement at least the MIPS III
ISA.
Also make MIPS16 SAVE and RESTORE instructions use address calculation
rather than plain arithmetic operations for stack pointer manipulation
so that their semantics for stack accesses follows the architecture
specification. That in particular applies to user software run on
64-bit processors with the CP0.Status.UX bit clear where the address
space is wrapped to 32 bits.
Backports commit c48245f0c62405f27266fcf08722d8c290520418 from qemu
Tighten ISA level checks down to MIPS II that many of our instructions
are missing. Also make sure any 64-bit instruction enables are only
applied to 64-bit processors, that is ones that implement at least the
MIPS III ISA.
Backports commit d9224450208e0de62323b64ace91f98bc31d6e2c from qemu
Fix CP0.Config3.ISAOnExc write accesses on microMIPS processors. This
bit is mandatory for any processor that implements the microMIPS
instruction set. This bit is r/w for processors that implement both the
standard MIPS and the microMIPS instruction set. This bit is r/o and
hardwired to 1 if only the microMIPS instruction set is implemented.
There is no other bit ever writable in CP0.Config3 so defining a
corresponding `CP0_Config3_rw_bitmask' member in `CPUMIPSState' is I
think an overkill. Therefore make the ability to write the bit rely on
the presence of ASE_MICROMIPS set in the instruction flags.
The read-only case of the microMIPS instruction set being implemented
only can be added when we add support for such a configuration. We do
not currently have such support, we have no instruction flag that would
control the presence of the standard MIPS instruction set nor any
associated code in instruction decoding.
This change is needed to boot a microMIPS Linux kernel successfully,
otherwise it hangs early on as interrupts are enabled and then the
exception handler invoked loops as its first instruction is interpreted
in the wrong execution mode and triggers another exception right away.
And then over and over again.
We already check the current setting of the CP0.Config3.ISAOnExc in
`set_hflags_for_handler' to set the ISA bit correctly on the exception
handler entry so it is the ability to set it that is missing only.
Backports commit 90f12d735d66ac1196d9a2bced039a432eefc03d from qemu
Fix microMIPS MOVE16 and MOVEP instructions on 64-bit processors by
using register addition operations.
This copies the approach taken with MIPS16 MOVE instructions (I8_MOV32R
and I8_MOVR32 opcodes) and follows the observation that OPC_ADDU expands
to tcg_gen_mov_tl whenever `rt' is 0 and `rs' is not, therefore copying
`rs' to `rd' verbatim. This is not the case with OPC_ADDIU where a
sign-extension from bit #31 is made, unless in the uninteresting case of
`rs' being 0, losing the upper 32 bits of the value copied for any
proper 64-bit values.
This also serves as an optimization as one op is produced in generated
code rather than two (again, unless `rs' is 0, where it doesn't change
anything).
Backports commit 7215d7e7aea85699bf516c3e8d84f6a22584da35 from qemu
Make writes to CP0.Status and CP0.Cause have the same effect as
executing corresponding MTC0 instructions would in Kernel Mode. Also
ignore writes in the user emulation mode.
Currently for requests from the GDB stub we write all the bits across
both registers, ignoring any read-only locations, and do not synchronise
the environment to evaluate side effects. We also write these registers
in the user emulation mode even though a real kernel presents them as
read only.
Backports commit 81a423e6c6d3ccaa79de4e58024369c660c1eeb4 from qemu
Correct these issues with the handling of CP0.Status for MIPSr6:
* only ignore the bit pattern of 0b11 on writes to CP0.Status.KSU, that
is for processors that do implement Supervisor Mode, let the bit
pattern be written to CP0.Status.UM:R0 freely (of course the value
written to read-only CP0.Status.R0 will be discarded anyway); this is
in accordance to the relevant architecture specification[1],
* check the newly written pattern rather than the current contents of
CP0.Status for the KSU bits being 0b11,
* use meaningful macro names to refer to CP0.Status bits rather than
magic numbers.
References:
[1] "MIPS Architecture For Programmers, Volume III: MIPS64 / microMIPS64
Privileged Resource Architecture", MIPS Technologies, Inc., Document
Number: MD00091, Revision 6.00, March 31, 2014, Table 9.45 "Status
Register Field Descriptions", pp. 210-211.
Backports commit f88f79ec9df06d26d84e1d2e0c02d2634b4d8583 from qemu
Correct MIPS16/microMIPS branch size calculation in PC adjustment
needed:
- to set the value of CP0.ErrorEPC at the entry to the reset exception,
- for the purpose of branch reexecution in the context of device I/O.
Follow the approach taken in `exception_resume_pc' for ordinary, Debug
and NMI exceptions.
MIPS16 and microMIPS branches can be 2 or 4 bytes in size and that has
to be reflected in calculation. Original MIPS ISA branches, which is
where this code originates from, are always 4 bytes long, just as all
original MIPS ISA instructions.
Backports commit c3577479815f5bcf9d38993967bca2115af245d8 from qemu
Restore the order of helpers that used to be: unary operations (generic,
then MIPS-specific), binary operations (generic, then MIPS-specific),
compare operations. At one point FMA operations were inserted at a
random place in the file, disregarding the preexisting order, and later
on even more operations sprinkled across the file. Revert the mess by
moving FMA operations to a new ternary class inserted after the binary
class and move the misplaced unary and binary operations to where they
belong.
Backports commit 8fc605b8aa257feb3e69d44794a765bd492b573b from qemu
Remove the `FLOAT_OP' macro, unused since commit
b6d96beda3a6cbf20a2d04a609eff78adebd8859 [Use temporary registers for
the MIPS FPU emulation.].
Backports commit 51fdea945ae7adae8d7e4a1624e35bb7f714b58f from qemu
Move the call to `update_fcr31' in `helper_float_cvtw_s' after the
exception flag check, for consistency with the remaining helpers that do
it last too.
Backports commit 2b09f94cdbf5c54e2278d7f3aed2eceff3494790 from qemu
Backports commits d75de74967f631a7d0b538d4b88f96f9c426bfe2, 6225a4a0e39cb24e7b9e1d4d2c1a3e6eaee18e85, and d2bfa6e6222baa0218bd0658499d38bac56ac34c from qemu
Add the M14K and M14Kc processors from MIPS Technologies that are the
original implementation of the microMIPS ISA. They are dual instruction
set processors, implementing both the microMIPS and the standard MIPSr32
ISA.
These processors correspond to the M4K and 4KEc CPUs respectively,
except with support for the microMIPS instruction set added, support for
the MCU ASE added and two extra interrupt lines, making a total of 8
hardware interrupts plus 2 software interrupts. The remaining parts of
the microarchitecture, in particular the pipeline, stayed unchanged.
The presence of the microMIPS ASE is is reflected in the configuration
added. We currently have no support for the MCU ASE, including in
particular the ACLR, ASET and IRET instructions in either encoding, and
we have no support for the extra interrupt lines, including bits in
CP0.Status and CP0.Cause registers, so these features are not marked,
making our support diverge from real hardware.
Backports commit 11f5ea105c06bec72e9bc9a700fa65d60afb5ec3 from qemu
Make the data type used for the CP0.Config4 and CP0.Config5 registers
and their mask signed, for consistency with the remaining 32-bit CP0
registers, like CP0.Config0, etc.
Backports commit 8280b12c0e4b515d707509dde4ddde05d9bda4ef from qemu
Add the 5KEc and 5KEf processors from MIPS Technologies that are the
original implementation of the MIPS64r2 ISA.
Silicon for these processors has never been taped out and no soft cores
were released even. They do exist though, a CP0.PRId value has been
assigned and experimental RTLs produced at the time the MIPS64r2 ISA has
been finalized. The settings introduced here faithfully reproduce that
hardware.
As far the implementation goes these processors are the same as the 5Kc
and the 5Kf CPUs respectively, except implementing the MIPS64r2 rather
than the original MIPS64 instruction set. There must have been some
updates to the CP0 architecture as mandated by the ISA, such as the
addition of the EBase register, although I am not sure about the exact
details, no documentation has ever been produced for these processors.
The remaining parts of the microarchitecture, in particular the
pipeline, stayed unchanged. Or to put it another way, the difference
between a 5K and a 5KE CPU corresponds to one between a 4K and a 4KE
CPU, except for the 64-bit rather than 32-bit ISA.
Backports commit 36b86e0dc2be93fc538fe7e11e0fda1a198f0135 from qemu
With an eye toward having this data replace the gen_opc_* arrays
that each target collects in order to enable restore_state_from_tb.
Backports commit 9aef40ed1f6e2bd794bbb3ba8c8b773e506334c9 from qemu
While we're at it, emit the opcode adjacent to where we currently
record data for search_pc. This puts gen_io_start et al on the
"correct" side of the marker.
Backports commit 667b8e29c5b1d8c5b4e6ad5f780ca60914eb6e96 from qemu
Usually, eliminate an operation from the translator by combining
a shift with an extract.
In the case of gen_set_NZ64, we don't need a boolean value for cpu_ZF,
merely a non-zero value. Given that we can extract both halves of a
64-bit input in one call, this simplifies the code.
Backports commit 7cb36e18b2f1c1f971ebdc2121de22a8c2e94fd6 from qemu
For !SF, this initial ext32u can't be optimized away by the
current TCG code generator. (It would require backward bit
liveness propagation.)
Backports commit d3a77b42decd0cbfa62a5526e67d1d6d380c83a9 from qemu
This can allow much of a ccmp to be elided when particular
flags are subsequently dead.
Backports commit 7dd03d773e0dafae9271318fc8d6b2b14de74403 from qemu
Handling this with TCG_COND_ALWAYS will allow these unlikely
cases to be handled without special cases in the rest of the
translator. The TCG optimizer ought to be able to reduce
these ALWAYS conditions completely.
Backports commit 9305eac09e61d857c9cc11e20db754dfc25a82db from qemu
Split arm_gen_test_cc into 3 functions, so that it can be reused
for non-branch TCG comparisons.
Backports commit 6c2c63d3a02c79e9035ca0370cc549d0f938a4dd from qemu
In ffc6372851d8631a9f9fa56ec613b3244dc635b9, we swapped the guest
base to the address base register from the address index register.
Except that 31 in the base slot is SP not XZR, so we need to be
more intelligent about which reg gets placed in which slot.
Backports commit 352bcb0a2b816ff9ab9d75d0f2384650d9e9ab19 from qemu
Rather than allow arbitrary shift+trunc, only concern ourselves
with low and high parts. This is all that was being used anyway.
Backports commit 609ad70562793937257c89d07bf7c1370b9fc9aa from qemu
They behave the same as ext32s_i64 and ext32u_i64 from the constant
folding and zero propagation point of view, except that they can't
be replaced by a mov, so we don't compute the affected value.
Backports commit 8bcb5c8f34f9215d4f88f388c7ff14c9bd5cecd3 from qemu
Implement real ext_i32_i64 and extu_i32_i64 ops. They ensure that a
32-bit value is always converted to a 64-bit value and not propagated
through the register allocator or the optimizer.
Backports commit 4f2331e5b67af8172419eb1c8db510b497b30a7b from qemu
The op is sometimes named trunc_shr_i32 and sometimes trunc_shr_i64_i32,
and the name in the README doesn't match the name offered to the
frontends.
Always use the long name to make it clear it is a size changing op.
Backports commit 0632e555fc4d281d69cb08d98d500d96185b041f from qemu
Instead of using an enum which could be either a copy or a const, track
them separately. This will be used in the next patch.
Constants are tracked through a bool. Copies are tracked by initializing
temp's next_copy and prev_copy to itself, allowing to simplify the code
a bit.
Backports commit b41059dd9deec367a4ccd296659f0bc5de2dc705 from qemu
Add two accessor functions temp_is_const and temp_is_copy, to make the
code more readable and make code change easier.
Backports commit d9c769c60948815ee03b2684b1c1c68ee4375149 from qemu
The tcg_temp_info structure uses 24 bytes per temp. Now that we emulate
vector registers on most guests, it's not uncommon to have more than 100
used temps. This means we have initialize more than 2kB at least twice
per TB, often more when there is a few goto_tb.
Instead used a TCGTempSet bit array to track which temps are in used in
the current basic block. This means there are only around 16 bytes to
initialize.
This improves the boot time of a MIPS guest on an x86-64 host by around
7% and moves out tcg_optimize from the the top of the profiler list.
Backports commit 1208d7dd5fddc1fbd98de800d17429b4e5578848 from qemu
By convention, on a 64-bit host TCG internally stores 32-bit constants
as sign-extended. This is not the case in the optimizer when a 32-bit
constant is folded.
This doesn't seem to have more consequences than suboptimal code
generation. For instance the x86 backend assumes sign-extended constants,
and in some rare cases uses a 32-bit unsigned immediate 0xffffffff
instead of a 8-bit signed immediate 0xff for the constant -1. This is
with a ppc guest:
before
------
---- 0x9f29cc
movi_i32 tmp1,$0xffffffff
movi_i32 tmp2,$0x0
add2_i32 tmp0,CA,CA,tmp2,r6,tmp2
add2_i32 tmp0,CA,tmp0,CA,tmp1,tmp2
mov_i32 r10,tmp0
0x7fd8c7dfe90c: xor %ebp,%ebp
0x7fd8c7dfe90e: mov %ebp,%r11d
0x7fd8c7dfe911: mov 0x18(%r14),%r9d
0x7fd8c7dfe915: add %r9d,%r10d
0x7fd8c7dfe918: adc %ebp,%r11d
0x7fd8c7dfe91b: add $0xffffffff,%r10d
0x7fd8c7dfe922: adc %ebp,%r11d
0x7fd8c7dfe925: mov %r11d,0x134(%r14)
0x7fd8c7dfe92c: mov %r10d,0x28(%r14)
after
-----
---- 0x9f29cc
movi_i32 tmp1,$0xffffffffffffffff
movi_i32 tmp2,$0x0
add2_i32 tmp0,CA,CA,tmp2,r6,tmp2
add2_i32 tmp0,CA,tmp0,CA,tmp1,tmp2
mov_i32 r10,tmp0
0x7f37010d490c: xor %ebp,%ebp
0x7f37010d490e: mov %ebp,%r11d
0x7f37010d4911: mov 0x18(%r14),%r9d
0x7f37010d4915: add %r9d,%r10d
0x7f37010d4918: adc %ebp,%r11d
0x7f37010d491b: add $0xffffffffffffffff,%r10d
0x7f37010d491f: adc %ebp,%r11d
0x7f37010d4922: mov %r11d,0x134(%r14)
0x7f37010d4929: mov %r10d,0x28(%r14)
Backports commit 29f3ff8d6cbc28f79933aeaa25805408d0984a8f from qemu
Due to a copy&paste, the new op value is tested against mov_i32 instead
of movi_i32. The test is therefore always false. Fix that.
Backports commit 961521261a3d600b0695b2e6d2b0f490076f7e90 from qemu
The tcg_constant_folding folding ends up doing all the optimizations
(which is a good thing to avoid looping on all ops multiple time), so
make it clear and just rename it tcg_optimize.
Backports commit 36e60ef6ac5d8a262d0fbeedfdb2b588514cb1ea from qemu
Most of the calls to tcg_opt_gen_mov are preceeded by a test to check if
the source temp is a constant. Fold that into the tcg_opt_gen_mov
function.
Backports commit 97a79eb70dd35a24fda87d86196afba5e6f21c5d from qemu
Each call to tcg_opt_gen_mov is preceeded by a test to check if the
source and destination temps are copies. Fold that into the
tcg_opt_gen_mov function.
Backports commit 5365718a9afeeabde3784d82a542f8ad909b18cf from qemu
We can get the opcode using the TCGOp pointer. It needs to be
dereferenced, but it's anyway done a few lines below to write
the new value.
Backports commit 8d6a91602ea824ef4435ea38fd475387eecc098c from qemu
We can get the opcode using the TCGOp pointer. It needs to be
dereferenced, but it's anyway done a few lines below to write
the new value.
Backports commit ebd27391b00cdafc81e0541a940686137b3b48df from qemu
The checks in dins is required to avoid triggering an assertion
in tcg_gen_deposit_tl. The check in dext is just for completeness.
Fold the other D cases in via fallthru.
Backports commit b7f26e523914b982a1c1bfa8295f77ff9787c33c from qemu
Similar to the same fix for user-mode, except this instance
occurs on the softmmu path. Again, the tlb addend must be
the base register, while the guest address is the index.
Backports commit 80adb8fcad4778376a11d394a9e01516819e2327 from qemu
Thanks to the previous patch, it is now easy for tcg_out_qemu_ld and
tcg_out_qemu_st to use a 32-bit zero extended offset. However, the
guest base register x28 must be the base and addr_reg must be the
index.
Backports commit ffc6372851d8631a9f9fa56ec613b3244dc635b9 from qemu
The new argument lets you pick uxtw or uxtx mode for the offset
register. For now, all callers pass TCG_TYPE_I64 so that uxtx
is generated. The bits for uxtx are removed from I3312_TO_I3310.
Backports commit 6c0f0c0f124718650a8d682ba275044fc02f6fe2 from qemu
The addition of MO_AMASK means that places that used inverted masks
need to be changed to use positive masks, and places that failed to
mask the intended bits need updating.
Backports commit 2b7ec66f025263a5331f37d5ad78a625496fd7bd from qemu
These modifiers control, on a per-memory-op basis, whether
unaligned memory accesses are allowed. The default setting
reflects the target's definition of ALIGNED_ONLY.
Backports commit dfb36305626636e2e07e0c5acd3a002a5419399e from qemu
The extra information is not yet used but it is now available.
This requires minor changes through all of the tcg backends.
Backports commit 3972ef6f830d65e9bacbd31257abedc055fd6dc8 from qemu
At the tcg opcode level, not at the tcg-op.h generator level.
This requires minor changes through all of the tcg backends,
but none of the cpu translators.
Backports commit 59227d5d45bb3c31dc2118011691c35b3c00879c from qemu
This is less about improved type checking than enabling a
subsequent change to the representation of labels.
Backports commit bec1631100323fac0900aea71043d5c4e22fc2fa from qemu
This is improved type checking for the translators -- it's no longer
possible to accidentally swap arguments to the branch functions.
Note that the code generating backends still manipulate labels as int.
With notable exceptions, the scope of the change is just a few lines
for each target, so it's not worth building extra machinery to do this
change in per-target increments.
Backports commit 42a268c241183877192c376d03bd9b6d527407c7 from qemu
We no longer need INDEX_op_end to terminate the list, nor do we
need 5 forms of nop, since we just remove the TCGOp instead.
Backports commit 15fc7daa770764cc795158cbb525569f156f3659 from qemu
Rather reserving space in the op stream for optimization,
let the optimizer add ops as necessary.
Backports commit a4ce099a7a4b4734c372f6bf28f3362e370f23c1 from qemu
With the linked list scheme we need not leave nops in the stream
that we need to process later.
Backports commit 0c627cdca20155753a536c51385abb73941a59a0 from qemu
The method by which we count the number of ops emitted
is going to change. Abstract that away into some inlines.
Backports commit fe700adb3db5b028b504423b946d4ee5200a8f2f from qemu.
Almost completely eliminates the ifdefs in this file, improving
confidence in the lesser used 32-bit builds.
Backports commit 3a13c3f34ce2058e0c2decc3b0f9f56be24c9400 from qemu
Some of these functions are really quite large. We have a number of
things that ought to be circularly dependent, but we duplicated code
to break that chain for the inlines.
This saved 25% of the code size of one of the translators I examined.
Chain the temporaries together via pointers intstead of indices.
The mem_reg value is now mem_base->reg. This will be important later.
This does require that the frame pointer have a global temporary
allocated for it. This is simple bar the existing reserved_regs check.
Backports commit b3a62939561e07bc34493444fa926b6137cba4e8 from qemu
Thus, use cpu_env as the parameter, not TCG_AREG0 directly.
Update all uses in the translators.
Backports commit e1ccc05444676b92c63708096e36582be27fbee1 from qemu
* arm64eb: arm64 big endian also using little endian instructions.
* arm64: using another example that depends on endians.
example:
1. store a word: 0x12345678
2. load a byte:
* little endian : 0x78
* big endian : 0x12
