I am using Trusted Firmware Image(LSK and Android filesystem) provided by Linaro on Juno board r1. In my case, I just want to some trivial test in EL3, e.g., reading specific memory.
To make things easier, I didn't do anything until the system is completely booted. Then I load a kernel module which sends a SMC instruction and the SMC exception will be handled in BL3-1 by a custom handler. In the handler, I disabled MMU for EL3 and attempted to access a physical address directly. But I found that if the physical address is larger than 0xffffffff(4G), the content I got will be all 0. The physical address lower than 0xffffffff works perfectly. And If I mapped that physical address into a virtual address smaller than 0xffffffff(Linaro's EL3 only support virtual address lower than 0xffffffff), it also works.
So, why I can not get the correct content of a physical address larger than 0xffffffff after I disabled MMU in EL3?
Is there anyone know the details?
Thanks a lot for helping me!
Thanks for your reply.
Yes, you are right. In normal case, we just use it as you described. But if I want to do some job like introspection of the normal world, I have to access certain memory in kernel space of normal os. If dynamic mapping is difficult, then the only way I can do it is to temporarily disable MMU.
But the system always halts after I disabled MMU. I tried everything I could image, disable instruction cache, invalidate instruction cache, disable data cache, invalidate data cache, invalidate tlb. But it do now works. I am really disappointed.
Anyway, thanks very much for all the information you kindly provided. At least I can access all the physical memory now.
Hi,
This seems like an odd use case - Why would you want to be listing kernel modules from EL3? The usual way for EL1 to communicate with EL3 would be to make an SMC instruction using the defined interface, i.e. have X0 hold the function ID, and X1-X7 hold arguments specific to that function. The function might need more information, which can be stored in a mailbox.
While it's possible to use architectural features such as the translation table descriptor NSTable and NS bits to share the virtual address mapping of a mailbox between the Secure and Non-secure address spaces, I'm not sure how you would do this without heavily modifying the Trusted Firmware sources, which goes beyond the scope of what we'd be able to help you with on this forum (you can email support@arm.com if you have an entitlement to support).
Thanks for your patient.
In fact I just want to be familiar with the Trust Firmware and try to find out if there are any potential research topics. At this moment, as a test task, I want to list all the kernel modules from the secure domain.
So I found the smc exception handler in EL3 and add some test code in the handler. In the handler, I tried to list all the kernel modules by the double linked list of linux kernel. As the physical memory address of module structure in the linked list is dynamic changing and I failed to dynamic add memory map regions to MMU, so I disabled the MMU.
With your help, now I can read all the physical memory from secure domain and so I can list all the kernel modules. But it seems that I still have troubles. My codes looks like this,
disable_mmu_el3();
lsmod();
enable_mmu_el3(0);
As you can see, I attempt to enable MMU again to reduce the impact on other codes. But the system halts after that. With the debugging tool, I checked the pc register and found it is in a endless loop. I guess that is in some exception handler. I checked the source code and ensured that the data cache and instruction cache has been disabled while disabling MMU and enabled again while enabling MMU. I have no idea why this happened.
Thanks for the suggestion about baremetal development. In my thought, I just think now I am not very familiar with how it works. So maybe write all the system startup codes by myself maybe a little difficult at this moment.
No problem
As far as I know, the code in Trusted Firmware for mapping memory assumes that the MMU is disabled while the translation tables are generated, and that the MMU will be enabled afterwards. While this does mean that the memory map is generated at runtime, it's actually not possible to dynamically change the memory map while the MMU is enabled, i.e. you'd need to disable the MMU, reprogram your translation tables, and then re-enable the MMU. It's been a while since I looked at the sources for this though so I could be wrong.
May I ask what exactly you're trying to do?
As you've mentioned, disabling the MMU at EL3 will break the other code running at EL3, such as the SMC handlers and PSCI interface for hotplugging and suspending cores.
If you just want to do some baremetal development, we actually recommend running your code at EL2 for exactly this reason - It will not break the Trusted Firmware code resident at EL3 and your baremetal code will even be able to make SMC calls to hotplug or suspend other cores.
We have a tutorial for baremetal development on the Juno here, with this section outlining how to safely take control of the board at EL2 using DS-5's debugger commands. Note that by taking control of the board in this way, you can simply interrupt UEFI's automatic countdown and then enter the commands (or make a Debug Configuration that automatically executes the commands on connecting), so there's no need to even boot Linux.
Would this better suit your needs?
Thanks very much for you help! That really helps, I will get through the manual more carefully.
In fact, at the very beginning, I wasn't going to disable MMU because I guess that will disturb the executing of other codes in EL3. I tried to dynamically add some memory map region into the translation table, but always failed. I think it should be possible to do this, but I can not find out the details. Do you know whether it is possible to do this?
Thanks again.
Just adding some additional info to my post:
LDR X0, =0xC0000001LDR X2, =0x2A4A0150 //; Region 2 attrsLDR X3, =0x2A4A0170 //; Region 3 attrsSTR X0, [X2]STR X0, [X3]DSB SY
LDR X0, =0xC0000001
LDR X2, =0x2A4A0150 //; Region 2 attrs
LDR X3, =0x2A4A0170 //; Region 3 attrs
STR X0, [X2]STR X0, [X3]
DSB SY
You'll only really need the DSB if you're planning to enable the MMU so that you can map some areas of DRAM as Normal-type, to ensure that those accesses are made after the TZC-400 has been reprogrammed.
Hello,
First of all keep in mind that address 0x100000000 is reserved - The second region of DRAM actually starts at 0x880000000.
This issue is because of how the TrustZone Address Space Controller (TZC-400) has been programmed in order to prevent Secure accesses to certain regions of Non-secure memory, as mentioned in this tutorial here. The ARM Trusted Firmware uses a 32-bit VA space and 32-bit PA space, so it programs the TZC-400 to prevent Secure accesses to the upper region of DRAM.
You can see the various regions programmed by the TZC-400 in this Trusted Firmware source file.
You'll notice that regions 2 and 3 (3 being the one you're interested in) are both programmed with security attributes TZC_REGION_S_NONE.
So you need to reprogram the security attributes to be TZC_REGION_S_RDWR.
Digging through the sources, this macro collapses to 0x3, and is shifted left by 30 and bitwise OR'd with PLAT_ARM_TZC_FILTERS, which collapses to 0x1. So we need to reprogram the attribute registers with value 0xC0000001.
From the TZC-400 TRM, we see that the attribute registers for regions 2 and 3 are at offsets 0x150 and 0x170 respectively, so we can enter these commands in the DS-5 debugger command window to reprogram them:
memory set_typed AXI<PROT=1>:0x000000002A4A0150 (unsigned int) 0xC0000001memory set_typed AXI<PROT=1>:0x000000002A4A0170 (unsigned int) 0xC0000001
memory set_typed AXI<PROT=1>:0x000000002A4A0150 (unsigned int) 0xC0000001
memory set_typed AXI<PROT=1>:0x000000002A4A0170 (unsigned int) 0xC0000001
I tested this and it works - Before entering the commands I am unable to see the second region of DRAM (starting at 0x880000000), and then I can see it after entering the commands.
I hope that helps,
Ash.
View all questions in Arm Development Platforms forum