Hello experts,
In my project I need to write some bare metal code in order to boot my software (A VxWorks image), and would like to make the absolute minimum configurations before loading the VxWorks image, which then does the major part of the configurations.
I'm running on a Cortex A53, specifically on an LS1043ARDB board.
After POR I'm in S.EL3 of course, in which I configure all the SCTLR VBAR and SCR registers (I also initilze the RAM controller of course). I leave cache disabled as the VxWorks image will afterwards take care of cache settings.
After jumping to the VxWorks code, it will come up just fine, do the settings and bring itself to NS.EL1, then at some point it makes an smc #0 call in order to use some secure world functionality. Now instead of being vectored to EL3 offset 0x400 ("sync from lower level with current level sp") as expected, it stays in NS.EL1. and is vectored to offset 0 in the vector table which is the "sync with sp0" entry.
If I use U-Boot for boot loading my image all is fine and dandy, so appearantly U-Boot is doing something I don't.
Can anyone please help and let me know if I'm missing any configuration? Is there anything that needs to be configured in order for the CPU to be able to take an exception from NS.EL1 to EL3 via an smc call?
Thanks in advance.
What about SCR_EL3.SMD? Is it 1?
This is also my bet. I use the below to switch the system from EL3 to EL2when having no Secure Monitor in place.
Note "mov \xreg1, #0x5b1; msr scr_el3, \xreg1" - this amonf the others writes 1 to SMD.
.macro armv8_switch_to_el2_m, xreg1 /* 64bit EL2 | HCE | SMD | RES1 (Bits[5:4]) | Non-secure EL0/EL1 */ mov \xreg1, #0x5b1 msr scr_el3, \xreg1 msr cptr_el3, xzr /* Disable coprocessor traps to EL3 */ mov \xreg1, #0x33ff msr cptr_el2, \xreg1 /* Disable coprocessor traps to EL2 */ /* Initialize Generic Timers */ msr cntvoff_el2, xzr /* Initialize SCTLR_EL2 * * setting RES1 bits (29,28,23,22,18,16,11,5,4) to 1 * and RES0 bits (31,30,27,26,24,21,20,17,15-13,10-6) + * EE,WXN,I,SA,C,A,M to 0 */ mov \xreg1, #0x0830 movk \xreg1, #0x30C5, lsl #16 msr sctlr_el2, \xreg1 /* Return to the EL2_SP2 mode from EL3 */ mov \xreg1, sp msr sp_el2, \xreg1 /* Migrate SP */ mrs \xreg1, vbar_el3 msr vbar_el2, \xreg1 /* Migrate VBAR */ mov \xreg1, #0x3c9 msr spsr_el3, \xreg1 /* EL2_SP2 | D | A | I | F */ msr elr_el3, lr eret.endm
Also a digression to "appearantly U-Boot is doing something I don't.". Probably you have 3x BootLoaders (Uboot, SPL Uboot and ATF/Secure Monitor) if I can refer to Secure Monitor as a BootLoader as it services the system throughout the system life through SMC calls. Typically the bare minimum in EL3 is PSCI services.
What usually happens during the system boot-up is SPL Uboot passes execution to Secure Monitor that sets up the SMC services and exception returns ERET to Uboot switching the system from EL3, secure to EL2, non-secure.
If I don't want or have Secure Monitor then I do the system switch from EL3 to EL2 with the macro I pasted below. However without System Monitor the kernel to release the remaining CPUs (if you have > 1) out of the reset needs to go with "spin-table" in contrast to "psci"
The SMD's reset value is 0. I will check but I dont think I change it.
Hi, many thanks for the ellaborated reply.
The SMD's reset value is 0 and I don't think I change it right up to jumping to vxworks. I'll try checking right before it calls SMC if it somehow set the bit.
Would smd being 1 cause going to sync exception on the same level? Could there be any other reason in case it is 0?
Also what is the disabling of grpping of the coprocessor to EL3?
Thanks
What kind of exception (instead of entering EL3) do you get? This could be a hint.
I am getting a sync exception at the same level(stays at el1) with sp0 (offset 0 at the exception vector table instead of 0x400)
Yes, you wrote this, but what kind of exception? Illegal instruction, address exception ... ?
The ESR says 'unknown exception'
Yaron, what does the EL switch from EL3 through EL1? VMWorks?
Do you have the Secure Monitor in place to service the SMC calls?
If you don't have the Secure Monitor then SMC goes to EL3 but looks at the service not implemented not to mention other seetitngs such VBAR_EL3, SP_EL3, SPsel, MMU at EL3 and the results could vary but all together it cannot work.
So my question is again by saying you use Uboot what this Uboot means? Is it the 3x Bootloaders: SPL Uboot, Uboot, and ATF/SecureMonitor?
Bottom line if you want to have the SMC you must have the Secure Monitor (Arm Trusted Firmware) somewhere (in my case this is in a static memory, or any counterpart of it) implementing the SMC services. Setting the SMC services and ERET from EL3 to EL2 during a boot is called a "cold boot" of Secure Monitor.
Hi Marek, U-boot for sure loads secure monitor in place. So the answer is yes it's the 3x uboot
My boot code on the other hand. still doesnt but it will in the future.aJust to make sure we're on the same page, my boot comes instead of uboot. At the moment all my exception handlers are just busy loops so the cpu spins in one place. But I expect smc to vector to el3 regardless if I have secure monitor code in place or not.
My boot code initializes all vbars and sp and leaves the cpu at el3 none secure mode before jumping to vxworks.I have to check about spsel. Also caches and MMU are not initialized by me since vxworks will set them up for el1. For el3 mmu is disabled and I have instruction cache enabled and data cache disabled before jumping.
Vxworks will then bring the cpu all the way to el2 and eventually el1 before calling the smc. It expects then to go to secure monitor code, but I will settle for it trapping to my busy loop at el3 which it doesnt, meaning I need to set something more before jumping to vxworks.
All you wrote look fine to me. As the devil may be in a detail I don't think I can help without inspecting your code.
I can also suggest that you look and compare your boot code against the Secure Monitor/ATF in the "cold boot" path.
I will be in the office only next week so I will be able to send a code snippet.
Where can I find the secure monitor/ATF source code?
github.com/.../arm-trusted-firmware.git