Hi Everyone,
I'm developing a bare-metal application to run on an ARM Cortex A53 core.
I'm using u-boot compiled with the respective flags to start the application at EL3, so that we have full control of the processor configuration.
Right now, I'm trying to read a memory block of 181 bytes at address 0x8028000 while on EL2 but, most of the times, the processor is reading what looks like a memory block with some corrupted bytes.
The EL3 registers are configured as follows:
TCR_EL3 - 0x80823518
MAIR_EL3 - 0x000000ff440c0400
SCTLR_EL3 - 0x00C5183D
the L0 table (at address 0x8000c000):
0x8000c000: 0x000000008000d003 0x00000000000000000x8000c010: 0x0000000000000000 0x0000000000000000
the L1 table
0x8000d000: 0x000000008000e003 0x000000008000f0030x8000d010: 0x0000000080000611 0x00000000c00006110x8000d020: 0x0000000000000000 0x0000000000000000
As I understand, address 0x80280000 would match index 0 and index 2 of table L0 and L1 respectively, which will lead to a block descriptor that configures the access to the 1GB block.
Also, analyzing the NS bit of the Table (bit 63) and Block (bit 5) descriptors, we can see that those accesses are being translated into secure IPA or PA space.
I've tried to configure both descriptors to have the NS bit enabled and configured the EL2 MMU to access the block I wanted, but failed to read it without corruption.
Does anyone have a suggestion as to what I might be missing?
Thanks in advance.
Kind Regards,
pcarmo
If I'm reading your values correctly, the region of memory is marked as Normal Write-Back Cacheable for EL3, and EL3 has the MMU and caches enabled.
So question... do you have the same cache ability set up in EL2? (i.e. same decriptior/MAIR/SCTLR set up)
Hi Martin,
Your suggestion to check if the EL2 registers were properly configured has made me detect an issue I was making when setting those registers, which was causing the MMU on EL2 not being properly enabled.
Unfortunately, after enabling the MMU, I'm unable to proceed with any instruction, as an exception with ESR_EL2 = 0x8600000e, indicating a Permission fault level 2, is being raised. I'm trying to fix this one first and then I'll see if in fact I have proper access.
Thanks for the help
Ho have got SCTLR_EL2.WXN configured?
https://developer.arm.com/documentation/ddi0601/2023-12/AArch64-Registers/SCTLR-EL2--System-Control-Register--EL2-
If I've read you're descriptors correctly, you have the address region marked as R/W and with XN=0. However, if SCTLR_EL2.WXN is 1, then all write-able locations are treated as execute-never
The permission fault I was having was because the MMU tables were not well configured.
To avoid further complications, I've simplified the MMU configuration at the EL2 so that all configurations are the same as in EL3:
- TTBR0_EL2: 0x8000c000
- SCTLR_EL2:0x30c5183d
- TCR_EL2:0x80823518
- MAIR_EL2: 0x000000ff_440c0400
The L0 table:
0x8000c000: 0x000000008000d003 0x00000054ffff8188
The L1 table:
0x8000d000: 0x000000008000e003 0x000000008000f0030x8000d010: 0x0000000080000611 0x00000000c0000611
I tried reading 0x80280000 at EL2 with and without EL3 MMU disabled, but have been having random memory corruptions anyways.
By random, I mean that the size and place of memory blocks that get corrupted is different everytime I reboot the board (I also have had situations where readings did not have any corruptions).
Assuming that there isn't some other core/device randomly writing into those locations, my suspicion would it's something to do with the cache/MMU setup. If, for whatever reason, you're using different attributes from EL3 and EL2 to access the same address that can lead to incoherency.
Are you invalidating the TLBs before the you enable the MMU in each EL?
How are you setting up the translation tables prior to enabling the MMU? Do you have a DSB in the sequence?
Yeah, I agree with you, but even with the same attributes in both EL3 and EL2 I get this same issue.
I am invalidating the TLB before enabling the MMU at EL2.
At EL3, the bootloader (u-boot) sets up the EL3 tables and enables the MMU before handing off to my bare-metal application.
In this case, I just configure the sections I want as non-secure and invalidate the TLBI, without disabling and enabling the MMU.
I do this because I have tested to configure the MMU tables with MMU disabled at EL3, and the MMU configuration is lost as soon as I enable the MMU at EL3.
In all situations, I invalidate the TLBs using the sequence suggested by the documentation:
dsb ishst tlbi alle3is dsb ish isb
In any case, I found a work around to this situation:
I've changed the EL3 MMU table from
to:
0x8000d000: 0x000000008000e003 0x000000008000f0030x8000d010: 0x0000000080000611 0x00000000800006310x8000d020: 0x0000000000000000 0x0000000000000000
This way, at EL3:
1. the block with virtual address 0x80000000 is pointing to the physical secure address 0x80000000
2. the block with virtual address 0xC0000000 is pointing to the physical non-secure address 0x80000000
At this point, I just copy the memory block starting at v_address 0x80000000 to the memory block starting at v_address 0xC0000000.
As soon as I enable the MMU at EL2 with the MMU tables configured to allow access to the v_address 0x80000000, I am able to properly read the memory I want, without any incoherence.
I am not sure if this is how it should be done, but it works fine and the copy loop runs quite fast, we don't even see a difference in the loading time, even though we are copying a 1GB memory block.
Would appreciate any other suggestion, if there's a simpler approach.
In any case, thanks for the help.