Hi,
I have written two bare-metal programs to run on EL2 and EL1 on Raspberry Pi 3B. The board is boot loaded with U-Boot. The EL2 binary is loaded at 0x1000000. The EL1 binary is loaded at 0x3000000.
The EL2 code initializes the UART and prints a message, then it switches to EL1. The EL1 binary won't print any message if it makes use of the stack. If the stack is not used, it prints the message.
May I know what is wrong with my code?
The EL2 code. It switches to EL1
//Initialisation part Start.S mrs x1, mpidr_el1 and x1, x1, #3 cbz x1, 2f // cpu id > 0, stop 1: wfe b 1b mrs x0, cnthctl_el2 orr x0, x0, #3 msr cnthctl_el2, x0 msr cntvoff_el2, xzr // enable AArch64 in EL1 mov x0, #(1 << 31) // AArch64 orr x0, x0, #(1 << 1) // SWIO hardwired on Pi3 msr hcr_el2, x0 mrs x0, hcr_el2 mov x2, #0x3c4 msr spsr_el2, x2 bl test //test.c void test() { init_uart(); print_message(); __asm volatile ( "movz x2, 0x0000 \n\t" "movk x2, 0x0300, lsl #16 \n\t" "msr elr_el2, x2 \n\t" "movz x0, 0xFFF0 \n\t" "movk x0, 0x2FF, lsl 16 \n\t" "msr sp_el1, x0 \n\t" "eret \n\t" ); }
The code of the EL1 program is given below
.global _start _start: bl el1_test el1_test.c void el1_test() { print_uart(); while(1) { asm volatile("nop"); } }
The disassembly of the el1_test() code is given below
0000000003000280 <el1_test>: 3000280: a9bf7bfd stp x29, x30, [sp, #-16]! 3000284: 910003fd mov x29, sp 3000288: 97ffffa4 bl 3000118 <uart_send1> 300028c: d503201f nop 3000290: d503201f nop 3000294: 17fffffe b 300028c 3000298: d503201f nop
Any hint or help is greatly appreciated
You don't say in what way it doesn't work. Does it through some kind of exception? Or just not print?
Looking at the code, the only thing that immediately sprang out is how you set up SPSR_EL2. The code sets SPSR_EL2.M to 0x4. If you look at the description of the register:
https://developer.arm.com/documentation/ddi0601/2023-03/AArch64-Registers/SPSR-EL2--Saved-Program-Status-Register--EL2-?lang=en
That means EL1t, or "EL1 with SP_EL0 selected". But the stack pointer you then set up is SP_EL1. Have you tried setting SPSR_EL2.M to EL1h instead?
Thank you very much for pointing the mistake. I have set SPSR_EL2.M to EL1h (0101). Still the problem persists.
When the code switches to EL1, the EL1 memory address contains random data. If I inspect that memory location just before "ERET" from EL2, it contains the code I have loaded. If the execution returns to EL2 from EL1 again "unexpectedly", the EL1 memory location contains the valid code (when inspecting from EL2).
Can there be some memory security settings issue? Is there a remote chance for my linker script to have error? I am using "arm-gnu-toolchain-11.3.rel1-x86_64-aarch64-none-elf" . I have not turned on MMU (U-boot does something?). I am loading the stripped binaries.
I am stuck and any hint is greatly appreciated...
For debugging I have setup OpenOCD and JTAG. The debug steps followed are given below
1) Raspberry Pi 3B board boots in to U-Boot prompt.
2) Through TFTP, the EL2 and EL1 binaries are loaded in the memory
3) Started OpenOCD from Host
4) started gdb from host, Loaded debug symbols of EL1 code, necessary break points are set
5) using go command of U-Boot, EL2 code is executed
Observed that EL1 memory contains random data
I'm not familiar with that software stack, my experience is more bare-metal, but I have some thoughts that might help.
Lloyd said:When the code switches to EL1, the EL1 memory address contains random data. If I inspect that memory location just before "ERET" from EL2, it contains the code I have loaded. If the execution returns to EL2 from EL1 again "unexpectedly", the EL1 memory location contains the valid code (when inspecting from EL2).
The fact that the contents of memory appears to be different when viewed from EL2 vs EL1 is likely one of the causes.
Some possibilities that come to mind:
I think your problem is related to uboot's 'go' command as it does not flush the caches before jumping to new address and keeps the mmu enabled. You can verify through reading SCTRL_EL2 register.U-boot also support 'dcache' command. Flushing it before jumping to your code might help.