Debug the Hafnium in TC platform

Overview

Hafnium is a reference Secure Partition Manager (SPM) for systems that implement the Armv8.4-A Secure-EL2 extension. It enables multiple, isolated Secure Partitions (SPs) to run at Secure-EL1.More information about Hafnium can be found here.

Regarding the TotalCompute software stack which also include the BL32(SPM), the BL32(SPM) is implemented as Hafnium software component. You can find the introduction Here

The TC software stack can both support buildroot and Android build, since the hafnium is part of fimrware, we take buildroot as example.

Getting started

Firstly, let's setup the workspace following the user guide here to build out the whole software stack, let's take the buildroot file system as example as below.

Download and setup the workspace code
Then build the software stack, for example, build cmd as "./run_docker.sh ./build-all.sh build", once build successfully done, the output images is as below:
After the related image is built out, then run the related image to boot up successfully
```
./run-scripts/tc2/run_model.sh -m <model binary path> -d buildroot
```

AP firmware Images and boot

The AP firmware bootflow is as below:

BL1 --> BL2 --> BL1 --> BL31 --> BL32(SPM) --> BL31 --> BL33

Modify and build Hafnium seperatly

The build scirpt "build-scripts/build-hafnium.sh" can be used to build the Hafnium seperately, but the "hafnium.bin" image will be added into FIP package, so the modify and build source code, need to re-package again, the steps can be as below:

build-scripts/run_docker.sh build-scripts/build-hafnium.sh build
build-scripts/run_docker.sh build-scripts/build-hafnium.sh deploy
build-scripts/run_docker.sh build-scripts/build-tfa.sh build
build-scripts/run_docker.sh build-scripts/build-tfa.sh deploy
build-scripts/run_docker.sh build-scripts/build-flash-image.sh build
build-scripts/run_docker.sh build-scripts/build-flash-image.sh deploy

It needs to add the debug build option to make sure the symble file can be used for debugging as below:

diff --git a/build/BUILD.gn b/build/BUILD.gn
index 1baac1ba..80b9f44d 100644
--- a/build/BUILD.gn
+++ b/build/BUILD.gn
@@ -20,6 +20,7 @@ config("compiler_defaults") {
"-fstack-protector-all",
]

+ asmflags = ["-g"]
cflags_c = [ "-std=c11" ]

cflags_cc = [ "-std=c++2a" ]

Setup ARM DS to debug Hafnium

Import the TC22 FVP model

From the ARM DS Debug control pannel, navigate to:

Debug Congtrol--> New Debug Connection... -->Choose "Model Connection --> Debug connection name, such as "TC22-Hafnium-debug" -> Next --> Target Selection --> Add a new model ...

Then, "Select Model Interface(Iris)" --> Next--> choose "Launch and connect to specific model" --> Next --> Browse the Model Path to select the downloaded file path of “TC22 model” --> Finish

At last, the model is imported as below:

Connect the model

Step1: Open the debug configration

Step2: Setup the model parameters

For example, copy below model paramters into the label:

-C pctl.startup=0.0.0.0 -C bp.secure_memory=1 -C cluster0.NUM_CORES=4 -C cache_state_modelled=0 -C bp.pl011_uart0.untimed_fifos=1 -C bp.ve_sysregs.mmbSiteDefault=0 -C bp.ve_sysregs.exit_on_shutdown=1 -C bp.secureflashloader.fname=/home/jett/jetzho01_share/trusted-firmware-a/build/fvp/debug/bl1.bin -C bp.flashloader0.fname=/home/jett/jetzho01_share/trusted-firmware-a/build/fvp/debug/fip.bin -I -p

Step3: Loaded the symbol files to apply and debug

On the window that opens, navigate to the "Debugger" tab, tick "Connect only", tick "Execute debugger commands", and copy-paste the following into the text box to automatically load all symbols into the correct virtual address space each time you connect to the model:

add-symbol-file <workspace>/arm-tf/build/fvp/debug/bl1/bl1.elf EL3:0
add-symbol-file <workspace>/arm-tf/build/fvp/debug/bl2/bl2.elf EL1S:0
add-symbol-file <workspace>/arm-tf/build/fvp/debug/bl31/bl31.elf EL3:0

Replacing <workspace> with the path to your workspace directory.

The EL and number at the end of each command (e.g. `EL3:0') ensure the symbols are loaded into the correct virtual address space and at the correct memory offset; TF-A uses absolute addresses for its symbols so we ensure an offset of 0.

Click "Apply" and then "Debug" to connect to the paused model. You can now step through the TF-A code or set a breakpoint on the symbol corresponding to the functionality that you are interested.

Debugging BL1 and hand over to BL2

For the FVP, the BL1 start from address 0, when debuging with ARM DS debugger, mis-matching source and disassembly views in the Debugger, when debugging startup code around address 0x0, is caused by “dangling” debug data.Some information about this topic can be found at link.

So for the BL1, we can add some change into Makefile as below for BL1 to avoid this issue.

diff --git a/bl1/bl1.mk b/bl1/bl1.mk
index 95fe50ed0..cc793142a 100644
--- a/bl1/bl1.mk
+++ b/bl1/bl1.mk
@@ -30,5 +30,6 @@ ifneq ($(findstring gcc,$(notdir $(LD))),)
 else ifneq ($(findstring ld,$(notdir $(LD))),)
 BL1_LDFLAGS += --sort-section=alignment
 endif
-
+ BL1_LDFLAGS += --no-gc-sections
+ BL1_LDFLAGS += -z undefs
 BL1_DEFAULT_LINKER_SCRIPT_SOURCE := bl1/bl1.ld.S

Run into BL1 entry
The PC is 0x0 address for the Armv8 Base RevC FVP model platform:

The PSTATE can be dumped into BL1
BL1 boot into BL2
BL1 run at EL3, and BL2 run into Secure EL1, the BL1 need to setup SPSR_EL3 and ELR_EL3 accordingly, then jump to BL2.

BL2 hand over to BL31

Run into BL2 entry
BL2 SMC into EL3 of BL1 again
BL1 jump into BL31

BL1 has installed the vector table, the SMC exception which is triggerd from BL2 will be captrured by the exception handler of BL1 vecter table

diff --git a/build/BUILD.gn b/build/BUILD.gn
index 1baac1ba..80b9f44d 100644
--- a/build/BUILD.gn
+++ b/build/BUILD.gn
@@ -20,6 +20,7 @@ config("compiler_defaults") {
"-fstack-protector-all",
]

+ asmflags = ["-g"]
cflags_c = [ "-std=c11" ]

cflags_cc = [ "-std=c++2a" ]