Guide to Debug Hafnium on Total Compute Platform

Overview

Hafnium is a reference Secure Partition Manager (SPM) for systems that implement the Armv8.4-A Secure-EL2 extension. It enables multiple, isolated Secure Partitions (SPs) to run at Secure-EL1.

The Total Compute software stack also includes the BL32 (SPM), which is implemented as a Hafnium software component. You can find the introduction information at Total Compute Platform Software Components.

The following figure shows the software stack of the TC platform:

The TC software stack can support both the buildroot and Android file systems, because the hafnium is part of fimrware. This guide uess the buildroot file system as an example.

Step1: Setup the workspace for the TC software stack

Perform the following steps to build the whole software stack. Consider the buildroot file system as an example. For more information about seting up workspace, see the Total Compute (TC) User Guide.

Download and set up the workspace as follows:
Build the software stack. For example, you can use build command ./run_docker.sh ./build-all.sh build. After the build is successfully complete, the output images is as follows:

Run the related image to boot up successfully

./run-scripts/tc2/run_model.sh -m <model binary path> -d buildroot

Step2: Work on the Hafnium component of TC software stack

The boot flow of AP firmware is as follows:

BL1 --> BL2 --> BL1 --> BL31 --> BL32(SPM) --> BL31 --> BL33

1 We need to add the debug build option to ensure that the symbol file can be used for later debugging as follows:

diff --git a/build/BUILD.gn b/build/BUILD.gn
index 1baac1ba..80b9f44d 100644
--- a/build/BUILD.gn
+++ b/build/BUILD.gn
@@ -20,6 +20,7 @@ config("compiler_defaults") {
     "-fstack-protector-all",
   ]

+  asmflags = ["-g"]
   cflags_c = [ "-std=c11" ]

   cflags_cc = [ "-std=c++2a" ]

2 Modify and build Hafnium seperatly

The build scirpt build-scripts/build-hafnium.sh can be used to build the Hafnium seperately, but the hafnium.bin image is added into the FIP package. Therefore, when you modify and build the source code of hafnium, you must re-package again. The steps can be as follows:

build-scripts/run_docker.sh build-scripts/build-hafnium.sh build
build-scripts/run_docker.sh build-scripts/build-hafnium.sh deploy
build-scripts/run_docker.sh build-scripts/build-tfa.sh build
build-scripts/run_docker.sh build-scripts/build-tfa.sh deploy
build-scripts/run_docker.sh build-scripts/build-flash-image.sh build
build-scripts/run_docker.sh build-scripts/build-flash-image.sh deploy

Step3: Setup Arm DS to debug Hafnium

Import the TC22 FVP model (如果此处也是有步骤的话，也需要使用1，2. 3，表现出来）

From the ARM DS Debug control pannel, navigate to:

Debug Congtrol--> New Debug Connection... -->Choose "Model Connection --> Debug connection name, such as "TC22-Hafnium-debug" -> Next --> Target Selection --> Add a new model ...

Then, "Select Model Interface(Iris)" --> Next--> choose "Launch and connect to specific model" --> Next --> Browse the Model Path to select the downloaded file path of “TC22 model” --> Finish

At last, the model is imported as below:

Connect the model

Step1: Open the debug configration

Step2: Setup the model parameters

For example, copy below model paramters into the label:

-C board.flashloader0.fname=/data/jetzho01/tc22-2023.04.21/build-scripts/output/deploy/tc2/fip_gpt-tc.bin -C css.rss.rom.raw_image=/data/jetzho01/tc22-2023.04.21/build-scripts/output/deploy/tc2/rss_rom.bin -C css.scp.c0_pik.rvbaraddr_lw=0x1000 -C css.scp.c0_pik.rvbaraddr_up=0x0000 -C css.rss.VMADDRWIDTH=23 -C css.rss.CMU0_NUM_DB_CH=16 -C displayController=2 -C board.smsc_91c111.enabled=1 -C board.hostbridge.userNetworking=1 -C board.hostbridge.userNetPorts="5555=5555,8080=80,8022=22" --data board.dram=/data/jetzho01/tc22-2023.04.21/build-scripts/output/deploy/tc2/tc-fitImage.bin@0x20000000

Step3: Loaded the symbol files to apply and debug

On the window that opens, navigate to the "Debugger" tab, tick "Connect only", tick "Execute debugger commands", and copy-paste the following into the text box to automatically load all symbols into the correct virtual address space each time you connect to the model:

add-symbol-file "<workspace>/build-scripts/output/tmp_build/tfa/build/tc/debug/bl1/bl1.elf" EL3:0x0
add-symbol-file "<workspace>/build-scripts/output/tmp_build/tfa/build/tc/debug/bl2/bl2.elf" EL1S:0x0
add-symbol-file "<workspace>/build-scripts/output/tmp_build/tfa/build/tc/debug/bl31/bl31.elf" EL3:0x0
add-symbol-file "<workspace>/build-scripts/output/tmp_build/hafnium/secure_tc_clang/hafnium.elf" EL2S:0x0

Replacing <workspace> with the path to your workspace directory.

The EL and number at the end of each command (e.g. `EL2S:0') ensure the symbols are loaded into the correct virtual address space and at the correct memory offset, for software uses absolute addresses for its symbols so we ensure an offset of 0.

Click "Apply" and then "Debug" to connect to the paused model. You can now step through the Hafnium code or set a breakpoint on the symbol corresponding to the functionality that you are interested.

Debugging entrypoint for Hafnium

Entry point of Hafnium code piece is here

.section .init.entry, "ax"
.global entry
entry:
	/* Linux aarch64 image header. */
	b 0f
	.word 0
	.quad 0x1000      /* text_offset */
	.quad image_size  /* image_size */
	.quad 0           /* flags */
	.quad 0           /* res2 */
	.quad 0           /* res3 */
	.quad 0           /* res4 */
	.word 0x644d5241  /* magic */
	.word 0

	/*
	 * Calculate the difference between the actual load address and the
	 * preferred one. We'll use this to relocate.
	 */
0:	adrp x25, entry
	add x25, x25, :lo12:entry

How to decide the entrypoint(loading address) of Hafnium for TC22 platform?
The Hafnium(BL32) is loaded by AP BL2, the image address is defined as "BL32_BASE" which can be found at here, specifically for the TC22 platform, the address is redefined as:

"#define TC_EL2SPMC_LOAD_ADDR (TC_TZC_DRAM1_BASE + 0x04000000)"

The TC_TZC_DRAM1_BASE is defined as 0xF900_0000, so the loading address for Hafnium(BL32_BASE) is 0xFD00_0000.

More plaform definition about this can be found here.
Stop at the entrypoint

Set the breakpoint of "entry" which is the entrypoint of Hafnium, then Run, it will stop the first instruction.

Debuging other related functions

The initialize flow of Hafnium
For example, set breakpoint of "one_time_init", jump into it