Hello,
I've been bringing up a hypervisor kernel on the Morello SDP which has prompted some questions about the role of CPTR_EL2, in particular the TC bit, and how it controls the behaviour of capabilities.
The code I'm debugging is completely unaware of capabilities (capability support will be added later). It is entered from the UEFI boot loader in AArch64 at EL2. At some point during its execution it performs an exception return (ERET) from EL2 to a new context also at EL2. In the original code this triggered a capability fault when the target instruction of the ERET was fetched. More specifically, ARM DS tells me this is a synchronous abort with type 0x28 indicating capability tag fault. It seems that during the ERET instruction, the capability tag and metadata are stripped from PCC leaving a "bare" target address, which may have a bearing on why the capability fault occurs.
By experiment I discovered that the setting of CPTR_EL2 bit 9 (0x200) affects whether or not the capability fault occurs. In the original failing code, CPTR_EL2 had the value 3, but setting it to 0x32ff prevents the exception from occurring. EL2H is disabled. I noted that the definition of CPTR_EL2 includes several RES1 bits which were not correctly set in the original code, but it seems to be the setting of bit 9 specifically which determines whether or not the capability fault occurs.
This seems counter-intuitive (to me at least) because according to the Morello architecture supplement (DDI0606 rev. A.k) setting bit 9, TC in the Morello architecture, "causes Morello instructions to be trapped". Looking carefully at the registers while single-stepping the instruction which sets CPTR_EL2, when it's written with TC set it seems to strip the capability bit from the PCC at that point, but in such a way that no capability fault is triggered (unlike the ERET case). That doesn't seem to relate to anything in the documentation for CPTR_EL2, unless I missed it.
In short, I am confused about why setting CPTR_EL2.TC is necessary and how/why it has the effect it does. If anyone can explain this and/or point me to the relevant section of the Morello architecture supplement, please, I'd be very grateful. Sorry if I've missed something obvious.
Thanks in anticipation,
Ian
Hi Ian,
This area is indeed a little counter-intuitive at first, but it does make sense considering two things: 1. Morello is disabled at ELx by default, and 2. capability exception entry and return is used when Morello is enabled. To be clear, by "Morello is disabled", I mean that the appropriate bit is set to "enable trapping", in other words CPTR_EL2.TC == 1 for EL2.
1. means that Morello-unaware software that sets reserved bits as appropriate runs unimpeded.
2. means (amongst other things) that if Morello is enabled at EL2, then it is imperative to set CVBAR_EL2 and CELR_EL2 to a valid capability, as PCC gets set to CVBAR_EL2 on exception entry and CELR_EL2 on exception return (ERET).
These are the reasons why CPTR_EL2.TC not being treated as RES1 resulted in a capability fault: Morello being thus enabled, PCC got set to CELR_EL2 on ERET, but that was presumably not a valid capability (as only the 64-bit ELR_EL2 got set).
For more information about all this, see section 2.13 ("Exception model") of the Morello architecture manual. For reference, the way Linux enables Morello at EL2 may also be of interest: https://git.morello-project.org/morello/kernel/linux/-/blob/morello/master/arch/arm64/kernel/head.S#L596
Cheers,Kevin
Hello Kevin,
Thanks for your quick and very helpful reply, appreciated.
To make sure I've understood: I gather that CPTR_EL2.TC set to 1 means Morello is disabled (at EL2), and that is deliberately arranged because the corresponding bit is RES1 in non-Morello Armv8, thereby allowing standard Arm64 software to run without being aware of capabilities.
The terminology does seem a bit strange: "enable trapping" sounds as though it is likely to result in an exception rather than supressing one, and that's what threw me (no pun intended). But I now see it is consistent with 2.13 in the Morello architecture document.
Cheers,
Glad it helped! Indeed the part where disabling trapping actually causes a trap is a little ironic. The confusion is likely caused by the fact that the description of TC pertains to the behaviour of Morello instructions (whether executing one causes a trap or not), while its effect on exception entry / return is not made explicit.