• State Suggested Answer
  • Locked Locked
  • Replies 7 replies
  • Answers 2 answers
  • Subscribers 350 subscribers
  • Views 2856 views
  • Users 0 members are here
Options
Related
How was your experience today?
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Bare metal - EL2 to EL1 - SP behaviour

Lloyd

Hi,

I have written two bare-metal programs to run on EL2 and EL1 on Raspberry Pi 3B. The board is boot loaded with U-Boot. The EL2 binary is loaded at 0x1000000. The EL1 binary is loaded at 0x3000000.

The EL2 code initializes the UART and prints a message, then it switches to EL1. The EL1 binary won't print any message if it makes use of the stack. If the stack is not used, it prints the message.

May I know what is wrong with my code?

The EL2 code. It switches to EL1

//Initialisation part Start.S

mrs     x1, mpidr_el1
and     x1, x1, #3
cbz     x1, 2f
// cpu id > 0, stop
1:  wfe
    b       1b
    
    
mrs     x0, cnthctl_el2
orr     x0, x0, #3
msr     cnthctl_el2, x0
msr     cntvoff_el2, xzr
// enable AArch64 in EL1
mov     x0, #(1 << 31)      // AArch64
orr     x0, x0, #(1 << 1)   // SWIO hardwired on Pi3
msr     hcr_el2, x0
mrs     x0, hcr_el2

mov     x2, #0x3c4
msr     spsr_el2, x2

bl test

//test.c
void test()
{
  init_uart();
  print_message();
  
  __asm volatile (
    "movz x2, 0x0000 \n\t"
    "movk x2, 0x0300, lsl #16 \n\t"
    "msr elr_el2, x2 \n\t"

    "movz x0, 0xFFF0 \n\t"
    "movk x0, 0x2FF, lsl 16 \n\t"
    "msr sp_el1, x0 \n\t"

    "eret \n\t"
  );
}

The code of the EL1 program is given below

.global _start

_start:

bl el1_test

el1_test.c

void el1_test()
{
  print_uart();
  while(1)
  {
     asm volatile("nop");
  }
}

The disassembly of the el1_test() code is given below

0000000003000280 <el1_test>:
 3000280:	a9bf7bfd 	stp	x29, x30, [sp, #-16]!
 3000284:	910003fd 	mov	x29, sp
 3000288:	97ffffa4 	bl	3000118 <uart_send1>
 300028c:	d503201f 	nop
 3000290:	d503201f 	nop
 3000294:	17fffffe 	b	300028c 
 3000298:	d503201f 	nop

Any hint or help is greatly appreciated 

Parents
  • 0 in reply to Lloyd

    I'm not familiar with that software stack, my experience is more bare-metal, but I have some thoughts that might help.

    Lloyd said:
    When the code switches to EL1, the EL1 memory address contains random data. If I inspect that memory location just before "ERET" from EL2, it contains the code I have loaded. If the execution returns to EL2 from EL1 again "unexpectedly", the EL1 memory location contains the valid code (when inspecting from EL2).

    The fact that the contents of memory appears to be different when viewed from EL2 vs EL1 is likely one of the causes.  

    Some possibilities that come to mind:

    • Some earlier piece of firmware has enabled the MMU for EL2
      • That could mean the virtual address you're writing the EL1 payload to doesn't map the physical address you expect it to. Which could explain why when execution moves to EL1 the payload isn't waiting for you.
      • Check the value of SCTLR_EL2.M to be sure.
    • Your EL2 code is not initialising the EL1 control  registers
      • In AArch64, only the system registers that effect execution at the highest implemented Exception level (EL3 in your case) have known reset values.  All the other registers contain UNKNOWN values.  It's software's job to initialise the system registers to useful (or safe) values before entering that EL for the first time.
      • If your code is not initialising the EL1 regs (especially SCTLR_EL1) it could that EL1 is running with the MMU enabled, the mapping isn't what you expect.  This seems a little less likely to me, as the chance of random values giving you a config that doesn't just blow up is pretty low.  But still worth checking that you are setting the EL1 regs to safe values
    • Caches
      • What I think you're doing is doing stores at EL2 (so data) which you then plan to execute at EL1 as instructors - right?  
      • If so, you'll also need to check cache ability and thus where the data might be ending up.  I suspect you'll need to a data cache clean followed by an instruction cache invalidate.

Reply
  • 0 in reply to Lloyd

    I'm not familiar with that software stack, my experience is more bare-metal, but I have some thoughts that might help.

    Lloyd said:
    When the code switches to EL1, the EL1 memory address contains random data. If I inspect that memory location just before "ERET" from EL2, it contains the code I have loaded. If the execution returns to EL2 from EL1 again "unexpectedly", the EL1 memory location contains the valid code (when inspecting from EL2).

    The fact that the contents of memory appears to be different when viewed from EL2 vs EL1 is likely one of the causes.  

    Some possibilities that come to mind:

    • Some earlier piece of firmware has enabled the MMU for EL2
      • That could mean the virtual address you're writing the EL1 payload to doesn't map the physical address you expect it to. Which could explain why when execution moves to EL1 the payload isn't waiting for you.
      • Check the value of SCTLR_EL2.M to be sure.
    • Your EL2 code is not initialising the EL1 control  registers
      • In AArch64, only the system registers that effect execution at the highest implemented Exception level (EL3 in your case) have known reset values.  All the other registers contain UNKNOWN values.  It's software's job to initialise the system registers to useful (or safe) values before entering that EL for the first time.
      • If your code is not initialising the EL1 regs (especially SCTLR_EL1) it could that EL1 is running with the MMU enabled, the mapping isn't what you expect.  This seems a little less likely to me, as the chance of random values giving you a config that doesn't just blow up is pretty low.  But still worth checking that you are setting the EL1 regs to safe values
    • Caches
      • What I think you're doing is doing stores at EL2 (so data) which you then plan to execute at EL1 as instructors - right?  
      • If so, you'll also need to check cache ability and thus where the data might be ending up.  I suspect you'll need to a data cache clean followed by an instruction cache invalidate.

Children
No data