I'm creating a simple kernel for educational purposes. I managed to enable memory mapping and currently have the kernel in high memory, whereas the low memory is mapped to an user program loaded from an ELF file.All goes well until I try to execute loaded instructions.I replace the mapping in TTBR0, do some bariers and TLBI invalidations, setup ELR and SPSR, and then ERET-urn to EL0 @ address 0x0. Then, for seemingly no reason whatsoever, CPU raises a synchronous exception, with ESR = 0x2000000 (Unknown reason, IL=1), on an STR instruction @ 0x40. It does this consistently.IIRC by changing some instructions around cache invalidation / bariers right before the ERET to EL0, I could make it consistently fire the same exception on something like 0x340, on an LDR, but I'm not sure if I remember the chain of events correctly.Edit: if I disable the instruction cache, the exception happens right after ERET, at address 0x0. In that case FAR_EL1 is set to a really weird value of 0xffff_ffff_ffff_fffc. In each case the address used in the instruction operand is valid, as evidenced by store & load instructions that happen right before, and by the output from JTAG.
(gdb) p $pc $1 = (*mut fn ()) 0x40 (gdb) p $sp $2 = (*mut ()) 0x10000f90
(gdb) x/17i 0x0 0x0: str x30, [sp, #-16]! 0x4: bl 0x14 0x8: mov x0, xzr 0xc: bl 0x1a4 0x10: brk #0x1 0x14: sub sp, sp, #0x50 0x18: stp xzr, x30, [sp, #56] 0x1c: add x8, sp, #0x48 0x20: str x8, [sp, #8] 0x24: adrp x8, 0x0 0x28: add x8, x8, #0x5b0 0x2c: mov w9, #0x1 0x30: stp x8, x9, [sp, #16] 0x34: adrp x8, 0x0 0x38: add x8, x8, #0x5c0 0x3c: stp xzr, xzr, [sp, #32] 0x40: str x8, [sp, #48]
Similar incident here
It was cache(s). https://shouldiblamecaching.com/After #1 invalidating the data cache, #2 invalidating the instructions cache - the loaded program seems to work.The invalidation was done similarily to the gcc's __cache_invalidate - see https://code.woboq.org/llvm/compiler-rt/lib/builtins/clear_cache.c.html (aarch64 section of it)