You are currently reviewing an older revision of this page.

History View current version

Debug EDK II UEFI on RD_N2 FVP with DS-5

Arm Development Studio Platinum 2021.a is required to debug RD_N2 FVP.

Download Software Stack

Select a Reference Design (RD) platform. There is a list of platforms here. For this guide we are using RD-N2 release tag RD-INFRA-2021.05.27.

For this guide, we have installed the software stack to ~/rd-infra.

Download the software stack using the guide here.

Setup Software Stack for Debugging

Make sure model-scripts/sgi/uefi.sh contains platforms_rdinfra[rdn2]=1

~/rd-infra$ grep "platforms_rdinfra\[rdn2\]" model-scripts/sgi/uefi.sh platforms_rdinfra[rdn2]=1

Modify PlatformDxe.inf

In uefi/edk2/edk2-platforms/Platform/ARM/SgiPkg/Drivers/PlatformDxe/PlatformDxe.inf under section Protocols add gHestTableProtocolGuid.

[Protocols] 
  gEfiAcpiTableProtocolGuid          ## CONSUMES 
  gEfiAcpiSdtProtocolGuid            ## CONSUMES 
  gHestTableProtocolGuid             ## SOMETIMES CONSUMES

Build UEFI edk2

Now build the software stack. This generates the config files we need to modify.

~/rd-infra$ ./build-scripts/build-test-uefi.sh -p rdn2 all

Modify tools_def

In uefi/edk2/Conf/tools_def.txt and uefi/edk2/BaseTool/Conf/tools_def.template remove -Os from line DEFINE GCC_ALL_CC_FLAGS. This makes debugging much easier by not space optimising the firmware.

uefi/edk2/Conf/tools_def.txt:

DEFINE GCC_ALL_CC_FLAGS         = -g -fshort-wchar -fno-builtin -fno-strict-aliasing -Wall -Werror -Wno-array-bounds -include AutoGen.h -fno-common

Add –g to DEFINE_GCC_ASM_FLAGS to generate debug information.

DEFINE GCC_ASM_FLAGS              = -g -c -x assembler -imacros AutoGen.h

Optional: Update tools_def.template to reflect these changes in the future.

~/rd-infra$ cp uefi/edk2/Conf/tools_def.txt uefi/edk2/BaseTools/Conf/tools_def.template

Disable SPM. The build will fail unless it’s disabled.

In build-scripts/configs/rdn2/rdn2 make ARM_TF_ENABLE_SPM=0

Set DebugPrintErrorLevel

In uefi/edk2/edk2-platforms/Platform/ARM/SgiPkg/RdN2/RdN2.dsc add the following line in section PcdsFixedAtBuild.common to enable verbose printing.

gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x00400000

Set 0x00400000 to a combination of these flags:

#define DEBUG_INIT      0x00000001  // Initialization
#define DEBUG_WARN      0x00000002  // Warnings
#define DEBUG_LOAD      0x00000004  // Load events
#define DEBUG_FS        0x00000008  // EFI File system
#define DEBUG_POOL      0x00000010  // Alloc & Free (pool)
#define DEBUG_PAGE      0x00000020  // Alloc & Free (page)
#define DEBUG_INFO      0x00000040  // Informational debug messages
#define DEBUG_DISPATCH  0x00000080  // PEI/DXE/SMM Dispatchers
#define DEBUG_VARIABLE  0x00000100  // Variable
#define DEBUG_BM        0x00000400  // Boot Manager
#define DEBUG_BLKIO     0x00001000  // BlkIo Driver
#define DEBUG_NET       0x00004000  // Network Io Driver
#define DEBUG_UNDI      0x00010000  // UNDI Driver
#define DEBUG_LOADFILE  0x00020000  // LoadFile
#define DEBUG_EVENT     0x00080000  // Event messages
#define DEBUG_GCD       0x00100000  // Global Coherency Database changes
#define DEBUG_CACHE     0x00200000  // Memory range cachability changes
#define DEBUG_VERBOSE   0x00400000  // Detailed debug messages that may
                                    // significantly impact boot performance
#define DEBUG_ERROR     0x80000000  // Error

from uefi/edk2/MdePkg/Include/Library/DebugLib.h

Install FVP

Download the correct FVP version for the platform and version you selected.

For RD-N2 version RD-INFRA-2021.05.27 navigate to Neoverse N2 Reference Design FVP and click Download RD-N2.

Run the installer FVP_RD_N2.sh.

Modify FVP start script

Modify model-scripts/rdinfra/platforms/rdn2/run_model.sh:

remove -R parameter from PARAMS= section. This stops the model from running as soon as a debug server is connected.

PARAMS="--data css.scp.armcortexm7ct=$OUTDIR/scp_ramfw.bin@0x0BD80000 \
	--data css.mcp.armcortexm7ct=$OUTDIR/mcp_ramfw.bin@0x0BF80000 \
	-C css.mcp.ROMloader.fname=$OUTDIR/mcp_romfw.bin \
	-C css.scp.ROMloader.fname=$OUTDIR/scp_romfw.bin \
	-C css.trustedBootROMloader.fname=$OUTDIR/$BL1_IMAGE \
	-C board.flashloader0.fname=$OUTDIR/$FIP_IMAGE \
	-C board.flashloader1.fname=$PWD/nor1_flash.img \
	-C board.flashloader1.fnameWrite=$PWD/nor1_flash.img \
	-C board.flashloader2.fname=$PWD/nor2_flash.img \
	-C board.flashloader2.fnameWrite=$PWD/nor2_flash.img \
	-S \
	-C css.scp.pl011_uart_scp.out_file=${MODEL_TYPE,,}/${UART0_SCP_OUTPUT_FILE_NAME} \
	-C css.scp.pl011_uart_scp.unbuffered_output=1 \
	-C css.scp.pl011_uart_scp.uart_enable=true \

Rebuild:

~/rd-infra$ ./build-scripts/build-test-uefi.sh -p rdn2 all

Debug RDN2

Find the start address of UEFI module

TF-A loads BL33_AP_UEFI binary (the UEFI firmware) to the address specified by PLAT_ARM_NS_IMAGE_BASE in arm-tf/include/plat/arm/css/common/css_dev.h

~/rd-infra$ grep PLAT_ARM_NS_IMAGE_BASE arm-tf/include/plat/arm/css/common/css_def.h

#define PLAT_ARM_NS_IMAGE_BASE U(0xE0000000)

The start address is 0xE0000000

Start FVP

Update MODEL environment variable with your FVP_RD_N2 model executable.

export MODEL=~/FVP_RD_N2/models/Linux64_GCC-6.4/FVP_RD_N2

Go to model-scripts/rdinfra and run uefi.sh -p rdn2

~/rd-infra$ cd model-scripts/rdinfra/

~/rd-infra/model-scripts/rdinfra$ ./uefi.sh -p rdn2

The FVP will now start and wait for a debugger to connect.

Create a debug connection

Launch ArmDS Platinum 2021.a

Create a new debug connection

Select "Model Connection" and next

Enter a name for the connection.

Select "Add a new Model"

Select CADI as the model interface

Select "Browse for model running on local host"

Once your model terminal says “CADI Debug Server started for ARM Models...” the model should appear in arm DS.

Click Finish. The edit configuration window should then appear.

Under “Connection” select Imported/Infra5_RD_N2/Bare Metal Debug/ARM_Neoverse-N2x15 Multi-Cluster SMP

Under Debugger, check “Execute Debugger Commands” and enter a breakpoint for the UEFI entry point we found earlier.

Click Debug. The debugger should connect to the FVP.

Load Debug Symbols

If you have not connected to the FVP, click "Connect" to connect.

Currently it has stopped at EL3:0x00000000. This is the TF-A firmware. Click continue (F8) to start execution. It will then break at EL2N:0xE0000000.

Hit Continue (F8) again to let the FVP load into UEFI uninterrupted.

Enter this command in the commands tab

source "~/rd-infra/uefi/edk2/ArmPlatformPkg/Scripts/Ds5/cmd_load_symbols.py" -f (0xE0000000, 0x200000) -a -v

0xE0000000 is the UEFI entry point, 0x200000 is the size found in the build log:

Fd File Name:BL33_AP_UEFI (~/rd-infra/uefi/edk2/Build/RdN2/DEBUG_GCC5/FV/BL33_AP_UEFI.fd)

Generate Region at Offset 0x0

Region Size = 0x200000

Region Name = FV

Load symbols from UEFI console output

The number of loaded symbol files is smaller than the actual number of modules in the BK33_AP_UEFI binary. This is because the only region section in the FD section is the firmware volume FV=FVMAIN_COMPACT. Another FV region (FVMAIN) is not specified as part of FD, so we can’t provide the module using the –f flag.

To load symbols for FVMAIN we need to parse UEFI console output to extract addresses of its modules.

The UEFI log is located at ~/rd-infra/model-scripts/rdinfra/platforms/rdn2/rdn2

~/rd-infra$ cd model-scripts/rdinfra/platforms/rdn2/rdn2

The file name of the log is refinfra-XXXXX-uart-0-armtf_<date_time>

Use this script to generate a list of add-symbol-file commands for the debugger to load the symbols.

#!/bin/bash 

list=` sed 's/\r$//' $1 | grep "\.efi" |awk -F= '{ print $2 }'|awk '{ print  $1"/"$2 }' | sed 's/\.efi/\.dll/g' `

for item in $list ; 

do 

addr=`dirname $item` 

file=`basename $item` 

fullfile=`grep add-symbol-file $1 | grep "/$file" |uniq |awk '{ print $2 }'` 

for f in $fullfile ; do 

    if [ -f $f ]; then 

    entrypointaddr=`../../../../../tools/gcc/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-objdump -S $f |grep "<_ModuleEntryPoint>" |awk '{ print "0x"$1  }' ` 

    loadaddr=`echo 16o$(($addr - $entrypointaddr))p|dc` 

    echo "add-symbol-file $f EL2:0x$loadaddr" 

    fi 

done 

done ;

Make sure to run the script in model-scripts/rdinfra/platforms/rdn2/rdn2 so that the aarch64-none-linux-gnu-objdump path is correct.

~/rd-infra/model-scripts/rdinfra/platforms/rdn2/rdn2$ ./add_symbol.sh refinfra-27364-uart-0-armtf_2021-07-09_10.04.09

add-symbol-file /home/fremil01/rd-infra/uefi/edk2/Build/RdN2/DEBUG_GCC5/AARCH64/MdeModulePkg/Universal/PCD/Dxe/Pcd/DEBUG/PcdDxe.dll EL2:0xFE6E1000

add-symbol-file /home/fremil01/rd-infra/uefi/edk2/Build/RdN2/DEBUG_GCC5/AARCH64/EmbeddedPkg/MetronomeDxe/MetronomeDxe/DEBUG/MetronomeDxe.dll EL2:0xFE6D9000

...

Copy the output from the script into the arm DS command window.

All the commands should run successfully, and the debug symbols should've now loaded.

UEFI Boot Process

Now that the debug symbols are loaded, we can run through the UEFI boot process.

Security (SEC)

First we enter the Security (SEC) stage. This is located where the branch instruction at the UEFI entry point is pointing. (0xE0017FD8).

The module is located at uefi/edk2/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore.inf. It is called ArmPlatformPrePeiCore.

Hit "Step Source Line (F5)" to enter the module.

The SEC module:

Gets the address of NtFwConfigDtBlob
Checks CurrentEL (which will be EL2 for RDN2)
Branches to SetupExceptionLevel2
- Zeros SCTRL_EL2
- Enables EL2 FIQ, EL2 IRQ, EL2 Serror and Abort in HCR_EL2
- Zeros CPTR_EL2 (Disables trapping access to trace functionality, SVE, Advanced SIMD, and floating point)
- Disable traps for EL0 and EL1 to access timer events
Sets up a stack pointer (using PcdCPUCoresStackBase and PcdCPUCorePrimaryStackSize)
Calls ArmPlatformPrePeiCore:CEntryPoint() with the value stored in 0xE0000008 as a parameter. (The value is 0xE001BFE0) which:
- Disables data cache (bit C in SCTLR_EL2)
- Invalidates data cache (DC IVAC)
- Invalidates instruction cache (IC IALLU)
- Writes VBAR_EL2 with a pointer to the PeiVectorTable (VBAR_EL2 will be overwritten at DXE stage as every stage has it's own exception handlers)
- Enables Floating Point
- Initialize Debug Agent
- Enables interrupt debug timer
- Prepares SecCoreData
  - BootFirmwareVolumeBase and size
  - Temporary RAM base address and size
  - Stack base address and size
Jumps to PEI entry point, passing SecCoreData and PPI descriptor as parameters.

PEI

Pei Foundation (PeiCore) entry point is located in uefi/edk2/MdePkg/Library/PeiCoreEntryPoint/PeiCoreEntryPoint.c

DXE

The entrypoint of the DxeCore is DxeMain()

BDS

BDSDxe reads the boot order list from EFI variables.

Boot timeout EFI is evaluated and user input from keyboard and mouse is supported. UI applications are shown.