Debug the Hafnium in TC platform

Preface

This article outlines how to use ARM DS to debug mainline Trusted Firmware-A from cold reset through to normal world handover.
These instructions are primarily written targeting the Armv8 Base RevC FVP model which the host is ubuntu linux host.

Getting started

Install required tools and component

• Install ARM DS,can be download here
• Install the Armv-A Base RevC AEM FVP here 
• Download the GCC toolchain from here 
  

Download Trusted Firware-A mainline source code and build

The mainline repo of Trusted Firmware-A is hosted at  https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/
Generally, it is also refer to some of the prerequisite for host enviroment.

• Download the latest mainline code

      git clone "https://review.trustedfirmware.org/TF-A/trusted-firmware-a"

• Build the code as Debug mode, we include the pre-built uboot-image as BL33, the BL33 is running into non-secure world.
  

  export CROSS_COMPILE=~/tools/arm-gnu-toolchain-12.2.rel1-x86_64-aarch64-none-elf/bin/aarch64-none-elf-
  make PLAT=fvp DEBUG=1 BL33=~/uboot.bin all fip
  

• Run the software image based on Armv-A Base RevC AEM FVP model by below cmds:
  

  /path/to/FVP_Base_RevC-2xAEMvA \
  -C pctl.startup=0.0.0.0 \
  -C bp.secure_memory=1 \
  -C cluster0.NUM_CORES=4 \
  -C cache_state_modelled=0 \
  -C bp.pl011_uart0.untimed_fifos=1 \
  -C bp.ve_sysregs.mmbSiteDefault=0 \
  -C bp.ve_sysregs.exit_on_shutdown=1 \
  -C bp.secureflashloader.fname=<workspace>/trusted-firmware-a/build/fvp/debug/bl1.bin \
  -C bp.flashloader0.fname=<workspace>/trusted-firmware-a/build/fvp/debug/fip.bin

  Note that the command references in `<workspace>' is the work directory for the code base.

Trusted Firmware-A overview

The TF-A cold boot flow comprises up to five individual boot stages running at different exception levels:

With these stages run in the following order:

BL1  -->  BL2  -->  BL1  -->  BL31  -->  BL32  -->  BL31  -->  BL33

We recommend reading the Arm Trusted Firmware Design document for more information 

This article outlines how to debug TF-A:

• From the BL1 entrypoint through to the BL33 entrypoint i.e. "normal world handover"
• In a system without a trusted operating system i.e. no BL32 present
• Using the official reference implementation sources of BL1, BL2, and BL31 (*)

(*) The `bl1/', `bl2/', and `bl31/' directories in `<workspace>/trusted-firmware-a/'.

When debugging TF-A it is important to know which boot stage(s) contain the functionality that you are interested in; this way the correct symbols and debug information can be loaded, allowing us to set breakpoints on textual symbol names rather than raw addresses, see function call target names rather than PC relative offsets, and so on. It also means we can skip unnecessary parts of the boot flow.

Setup ARM DS to debug TF-A

Import the Armv-A Base RevC AEM FVP model

From the ARM DS  Debug control pannel, navigate to:

Debug Congtrol--> New Debug Connection... -->Choose "Model Connection --> Debug connection name, such as "TF-A" -> Next --> Target Selection --> Add a new model ...
 



Then, "Select Model Interface(Iris)" --> Next--> choose "Launch and connect to specific model" --> Next --> Browse the Model Path to select the downloaded file path of “Armv-A Base RevC AEM FVP model” --> Finish

At last, the model is imported as below:

Connect the model

• Step1: Open the debug configration
  
• Step2: Setup the model parameters
  
  For example, copy below model paramters into the label:
  

  -C pctl.startup=0.0.0.0 -C bp.secure_memory=1 -C cluster0.NUM_CORES=4 -C cache_state_modelled=0 -C bp.pl011_uart0.untimed_fifos=1 -C bp.ve_sysregs.mmbSiteDefault=0 -C bp.ve_sysregs.exit_on_shutdown=1 -C bp.secureflashloader.fname=/home/jett/jetzho01_share/trusted-firmware-a/build/fvp/debug/bl1.bin -C bp.flashloader0.fname=/home/jett/jetzho01_share/trusted-firmware-a/build/fvp/debug/fip.bin -I -p
  
  

  
  
• Step3: Loaded the symbol files to apply and debug
  

On the window that opens, navigate to the "Debugger" tab, tick "Connect only", tick "Execute debugger commands", and copy-paste the following into the text box to automatically load all symbols into the correct virtual address space each time you connect to the model:

add-symbol-file <workspace>/arm-tf/build/fvp/debug/bl1/bl1.elf EL3:0
add-symbol-file <workspace>/arm-tf/build/fvp/debug/bl2/bl2.elf EL1S:0
add-symbol-file <workspace>/arm-tf/build/fvp/debug/bl31/bl31.elf EL3:0

Replacing <workspace> with the path to your workspace directory.

The EL and number at the end of each command (e.g. `EL3:0') ensure the symbols are loaded into the correct virtual address space and at the correct memory offset; TF-A uses absolute addresses for its symbols so we ensure an offset of 0.

Click "Apply" and then "Debug" to connect to the paused model. You can now step through the TF-A code or set a breakpoint on the symbol corresponding to the functionality that you are interested.

Debugging BL1 and hand over to BL2

For the FVP, the BL1 start from address 0, when debuging with ARM DS debugger, mis-matching source and disassembly views in the Debugger, when debugging startup code around address 0x0, is caused by “dangling” debug data.Some information about this topic can be found at  link.

So for the BL1, we can add some change into Makefile as below for BL1 to avoid this issue.

diff --git a/bl1/bl1.mk b/bl1/bl1.mk
index 95fe50ed0..cc793142a 100644
--- a/bl1/bl1.mk
+++ b/bl1/bl1.mk
@@ -30,5 +30,6 @@ ifneq ($(findstring gcc,$(notdir $(LD))),)
 else ifneq ($(findstring ld,$(notdir $(LD))),)
 BL1_LDFLAGS += --sort-section=alignment
 endif
-
+ BL1_LDFLAGS += --no-gc-sections
+ BL1_LDFLAGS += -z undefs
 BL1_DEFAULT_LINKER_SCRIPT_SOURCE := bl1/bl1.ld.S

• Run into BL1 entry
  The PC is 0x0 address for the Armv8 Base RevC FVP model platform:
  
  
  The PSTATE can be dumped into BL1
• BL1 boot into BL2
  BL1 run at EL3, and BL2 run into Secure EL1, the BL1 need to setup SPSR_EL3 and ELR_EL3 accordingly, then jump to BL2.
  
   

BL2 hand over to BL31

• Run into BL2 entry
  
  
• BL2 SMC into EL3 of BL1 again
  
  
  
• BL1 jump into BL31
  
  BL1 has installed the vector table, the SMC exception which is triggerd from BL2 will be captrured by the exception handler of BL1 vecter table
  
  
  BL1 SMC handler will hand over the flow to BL31, the ELR_EL3 will point to the BL31 image entery point address.
  
  

BL31 hand over to BL33