These include page table walks, SVM accesses and SMM state save accesses.
The bulk of the patch is obtained with
sed -i 's/\(\<[a-z_]*_phys\(_notdirty\)\?\>(cs\)->as,/x86_\1,/'
Backports commit b216aa6c0fcbaa8ff4128969c14594896a5485a4 from qemu
mr->terminates alone doesn't guarantee that we are looking at a RAM region.
mr->ram_addr also has to be checked, in order to distinguish RAM and I/O
regions.
So, do the following:
1) add a new define RAM_ADDR_INVALID, and test it in the assertions
instead of mr->terminates
2) IOMMU regions were not setting mr->ram_addr to a bogus value, initialize
it in the instance_init function so that the new assertions would fire
for IOMMU regions as well.
Backports commit ec05ec26f940564b1e07bf88857035ec27e21dd8 from qemu
The cpu_physical_memory_reset_dirty() function is sometimes used
together with cpu_physical_memory_get_dirty(). This is not atomic since
two separate accesses to the dirty memory bitmap are made.
Turn cpu_physical_memory_reset_dirty() and
cpu_physical_memory_clear_dirty_range_type() into the atomic
cpu_physical_memory_test_and_clear_dirty().
Backports commit 03eebc9e3246b9b3f5925aa41f7dfd7c1e467875 from qemu
Use set_bit_atomic() and bitmap_set_atomic() so that multiple threads
can dirty memory without race conditions.
Backports commit d114875b9a1c21162f69a12d72f69a22e7bab376 from qemu
Most of the time, not all bitmaps have to be marked as dirty;
do not do anything if the interesting ones are already dirty.
Previously, any clean bitmap would have cause all the bitmaps to be
marked dirty.
In fact, unless running TCG most of the time bitmap operations need
not be done at all, because memory_region_is_logging returns zero.
In this case, skip the call to cpu_physical_memory_range_includes_clean
altogether as well.
With this patch, cpu_physical_memory_set_dirty_range is called
unconditionally, so there need not be anymore a separate call to
xen_modified_memory.
Backports commit e87f7778b64d4a6a78e16c288c7fdc6c15317d5f from qemu
cpu_physical_memory_set_dirty_lebitmap unconditionally syncs the
DIRTY_MEMORY_CODE bitmap. This however is unused unless TCG is
enabled.
Backports commit 9460dee4b2258e3990906fb34099481c8334c267 from qemu
The new bitmap_test_and_clear_atomic() function clears a range and
returns whether or not the bits were set.
Backports commit 36546e5b803f6e363906607307f27c489441fd15 from qemu
Use atomic_or() for atomic bitmaps where several threads may set bits at
the same time. This avoids the race condition between threads loading
an element, bitwise ORing, and then storing the element.
When setting all bits in a word we can avoid atomic ops and instead just
use an smp_mb() at the end.
Most bitmap users don't need atomicity so introduce new functions.
Backports commit 9f02cfc84b85929947b32fe1674fbc6a429f332a from qemu
While it is obvious that cpu_physical_memory_get_dirty returns true even if
a single page is dirty, the same is not true for cpu_physical_memory_get_clean;
one would expect that it returns true only if all the pages are clean, but
it actually looks for even one clean page. (By contrast, the caller of that
function, cpu_physical_memory_range_includes_clean, has a good name).
To clarify, rename the function to cpu_physical_memory_all_dirty and return
true if _all_ the pages are dirty. This is the opposite of the previous
meaning, because "all are 1" is the same as "not (any is 0)", so we have to
modify cpu_physical_memory_range_includes_clean as well
Backports commit 72b47e79cef36ed6ffc718f10e21001d7ec2a66f from qemu
is_cpu_write_access is only set if tb_invalidate_phys_page_range is called
from tb_invalidate_phys_page_fast, and hence from notdirty_mem_write.
However:
- the code bitmap can be built directly in tb_invalidate_phys_page_fast
(unconditionally, since is_cpu_write_access would always be passed as 1);
- the virtual address is not needed to mark the page as "not containing
code" (dirty code bitmap = 1), so we can also remove that use of
is_cpu_write_access. For calls of tb_invalidate_phys_page_range
that do not come from notdirty_mem_write, the next call to
notdirty_mem_write will notice that the page does not contain code
anymore, and will fix up the TLB entry.
The parameter needs to remain in order to guard accesses to cpu->mem_io_pc.
Backports commit fc377bcf617a48233a99a9fe0a26247c38b5cb76 from qemu
These days modification of the TLB is done in notdirty_mem_write,
so the virtual address and env pointer as unnecessary.
The new name of the function, tlb_unprotect_code, is consistent with
tlb_protect_code.
Backports commit 9564f52da7eb061326956ed9a468935e3352512d from qemu
Remove them from the sundry exec-all.h header, since they are only used by
the TCG runtime in exec.c and user-exec.c.
Backports commit 1652b974766401743879d78f796f44b8929b0787 from qemu
DIRTY_MEMORY_CODE is only needed for TCG. By adding it directly to
mr->dirty_log_mask, we avoid testing for TCG everywhere a region is
checked for the enabled/disabled state of dirty logging.
Backports commit 677e7805cf95f3b2bca8baf0888d1ebed7f0c606 from qemu
When the dirty log mask will also cover other bits than DIRTY_MEMORY_VGA,
some listeners may be interested in the overall zero/non-zero value of
the dirty log mask; others may be interested in the value of single bits.
For this reason, always call log_start/log_stop if bits have respectively
appeared or disappeared, and pass the old and new values of the dirty log
mask so that listeners can distinguish the kinds of change.
For example, KVM checks if dirty logging used to be completely disabled
(in log_start) or is now completely disabled (in log_stop). On the
other hand, Xen has to check manually if DIRTY_MEMORY_VGA changed,
since that is the only bit it cares about.
Backports commit b2dfd71c4843a762f2befe702adb249cf55baf66 from qemu
For now memory regions only track DIRTY_MEMORY_VGA individually, but
this will change soon. To support this, split memory_region_is_logging
in two functions: one that returns a given bit from dirty_log_mask,
and one that returns the entire mask. memory_region_is_logging gets an
extra parameter so that the compiler flags misuse.
While VGA-specific users (including the Xen listener!) will want to keep
checking that bit, KVM and vhost check for "any bit except migration"
(because migration is handled via the global start/stop listener
callbacks).
Backports commit 2d1a35bef0ed96b3f23535e459c552414ccdbafd from qemu
phys_page_set_level is writing zeroes to a struct that has just been
filled in by phys_map_node_alloc. Instead, tell phys_map_node_alloc
whether to fill in the page "as a leaf" or "as a non-leaf".
memcpy is faster than struct assignment, which copies each bitfield
individually. A compiler bug (https://gcc.gnu.org/PR66391), and
small memcpys like this one are special-cased anyway, and optimized
to a register move, so just use the memcpy.
This cuts the cost of phys_page_set_level from 25% to 5% when
booting qboot.
Backports commit db94604b20278c1dc227a04e4c564d80230e6c3f from qemu
At 8k per TLB (for 64-bit host or target), 8 or more modes
make the TLBs bigger than 64k, and some RISC TCG backends do
not like that. On the affected hosts, cut the TLB size in
half---there is still a measurable speedup on PPC with the
next patch.
Backports commit 1de29aef17a7d70dbc04a7fe51e18942e3ebe313 from qemu
This will be used to size the TLB when more than 8 MMU modes are
used by the target. Limitations come from the limited size of
the immediate fields (which sometimes, as in the case of Aarch64,
extend to instructions that shift the immediate).
Backports commit 006f8638c62bca2b0caf609485f47fa5e14d8a3c from qemu
Existing definition triggers the following when using clang
-fsanitize=undefined:
hw/intc/apic_common.c:314:55: runtime error: left shift of 1048575 by 12
places cannot be represented in type 'int'
Fix it so we won't try to shift a 1 to the sign bit of a signed integer.
Backports commit 458cf469f4a1cb520b07092f5537c5a6d2389d23 from qemu
The ARMCPRegInfo arrays v8_el3_no_el2_cp_reginfo and v8_el2_cp_reginfo
are actually used on non-v8 CPUs as well. Remove the incorrect v8_
prefix from their names.
Backports commit 4771cd01daaccb2a8929fa04c88c608e378cf814 from qemu
Add support for trapping WFI and WFE instructions to the proper EL when
SCTLR/SCR/HCR settings apply.
Backports commit b1eced713d9913a5c58ba9daa795f10e4c856c49 from qemu
Just NOP the WFI instruction if we have work to do.
This doesn't make much difference currently (though it does avoid
jumping out to the top level loop and immediately restarting),
but the distinction between "halt" and "don't halt" will become
more important when the decision to halt requires us to trap
to a higher exception level instead.
Backport commit 84549b6dcf9147559ec08b066de673587be6b763 from qemu
Deleting the now-unused ARM_TBFLAG_CPACR_FPEN left a gap in the
bit usage; move the following ARM_TBFLAG_XSCALE_CPAR and
ARM_TBFLAG_NS_SHIFT down 3 bits to fill the gap.
Backports commit 647f767ba3b37fb229275086187e96242248a4ac from qemu
Extend the ARM disassemble context to take a target exception EL instead of a
boolean enable. This change reverses the polarity of the check making a value
of 0 indicate floating point enabled (no exception).
Backports commit 9dbbc748d671c70599101836cd1c2719d92f3017 from qemu
Currently we keep the TB flags PSTATE_SS and SS_ACTIVE in different
bit positions for AArch64 and AArch32. Replace these separate
definitions with a single common flag in the upper part of the
flags word.
Backports commit 3cf6a0fcedd429693d439556543400d5f0e31e1d from qemu
Adds CPTR_EL2/3 system registers definitions and access function.
Signed-off-by: Greg Bellows <greg.bellows@linaro.org>
[PMM: merge CPTR_EL2 and HCPTR definitions into a single
def using STATE_BOTH;
don't use readfn/writefn to implement RAZ/WI registers;
don't use accessfn for the no-EL2 CPTR_EL2;
fix cpacr_access logic to catch EL2 accesses to CPACR being
trapped to EL3;
use new CP_ACCESS_TRAP_EL[23] rather than setting
exception.target_el directly]
Backports commit c6f191642a4027909813b4e6e288411f8371e951 from qemu
Some coprocessor access functions will need to indicate that the
instruction should trap to EL2 or EL3 rather than the default
target exception level; add corresponding CPAccessResult enum
entries and handling code.
Backports commit 38836a2cd47c20daaaa84873e3d6020f19e4bfca from qemu
Updated the interrupt handling to utilize and report through the target EL
exception field. This includes consolidating and cleaning up code where
needed. Target EL is now calculated once in arm_cpu_exec_interrupt() and
do_interrupt was updated to use the target_el exception field. The
necessary code from arm_excp_target_el() was merged in where needed and the
function removed.
Backports commit 012a906b19e99b126403ff4a257617dab9b34163 from qemu
Rather than making every caller of raise_exception set the
syndrome and target EL by hand, make these arguments to
raise_exception() and have that do the job.
Backports commit c63285991b371c031147ad620dd7671662a90303 from qemu
Move the code which sets exception information out of
arm_cpu_handle_mmu_fault and into tlb_fill. tlb_fill
is the only caller which wants to raise_exception()
so it makes more sense for it to handle the whole of
the exception setup.
As part of this cleanup, move the user-mode-only
implementation function for the handle_mmu_fault CPU
method into cpu.c so we don't need to make it globally
visible, and rename the softmmu-only utility function
arm_cpu_handle_mmu_fault to arm_tlb_fill so it's clear
that it's not the same thing.
Backports commit 8c6084bf10fe721929ca94cf16acd6687e61d3ec from qemu
If the SCTLR.UMA trap bit is set then attempts by EL0 to update
the PSTATE DAIF bits via "MSR DAIFSet, imm" and "MSR DAIFClr, imm"
instructions will raise an exception. We were failing to set
the syndrome information for this exception, which meant that
it would be reported as a repeat of whatever the previous
exception was. Set the correct syndrome information.
Backports commit f2932df777dace044719dc2f394f5a5a8aa1b1cd from qemu
Updated the various helper routines to set the target EL as needed using a
dedicated function.
Backports commit e3b1d480995f6e2e86ef062038e618c1234dbcf1 from qemu
Add a CPU state exception target EL field that will be used for communicating
the EL to which an exception should be routed.
Add a disassembly context field for tracking the EL3 architecture needed for
determining the target exception EL.
Add a target EL argument to the generic exception helper for callers to specify
the EL to which the exception should be routed. Extended the helper to set
the newly added CPU state exception target el.
Added a function for setting the target exception EL and updated calls to helpers
to call it.
Backports commit 737103619869600668cc7e8700e4f6eab3943896 from qemu
