The Linux kernel within the Arm Total Compute platform includes subsystem-specific features that showcase the capabilities of the platform. However, Linux operates at different exception levels and addresses. Therefore, using Arm DS to debug Linux can be challenging.

This guide describes how to perform the following tasks:

• Set up the Linux Kernel debug connection.
• Disable the Linux Kernel Address Space Layout Randomization.
• Load the Linux Kernel debug symbol file.
• Start the Linux Kernel debug.

This guide assumes that you already set up the debugging environment as described in the Guide to Set Up Debugging Environment for Total Compute Software Stack.

Note: The guidance given in this guide is based on the Total Compute TC2-2023.10.04 code. Future code updates might introduce changes, so the guidance might not apply to all cases.

Set up Linux Kernel debugging connection

The Guide to Set Up Debugging Environment for Total Compute Software Stack describes how to set up the Total Compute debugging environment in the Arm DS.

There are some considerations to be aware of when setting up the Linux Kernel debugging environment.

When debugging the Linux Kernel, select ARM_Cortex-A520x4 SMP Cluster 0 as your target as shown in the following figure:

Disable the Linux Kernel Address Space Layout Randomization

Kernel Address-space layout randomization (KASLR) is a well-known technique to make exploits harder by placing various objects at random addresses instead of fixed addresses. To make debugging the code more convenient, you can keep the address of the code fixed every time it runs. This allows the symbol table to be easily loaded during debugging. To achieve this, you can disable KASLR during compilation or at run time:

1.To disable KASLR during compilation, you can modify the boot arguments. This can be done by editing the file build-scripts/files/u-boot/tc2/fvp.cfg, as shown follows:

diff --git a/files/u-boot/tc2/fvp.cfg b/files/u-boot/tc2/fvp.cfg
index 91b9e54..8e86577 100644
--- a/files/u-boot/tc2/fvp.cfg
+++ b/files/u-boot/tc2/fvp.cfg
@@ -1,5 +1,5 @@
 CONFIG_TARGET_TOTAL_COMPUTE_FVP=y
-CONFIG_BOOTARGS="console=ttyAMA0 debug user_debug=31 earlycon=pl011,0x2A400000 loglevel=9 androidboot.hardware=total_compute androidboot.boot_devices=1c050000.mmci ip=dhcp androidboot.selinux=permissive allow_mismatched_32bit_el0 systemd.log_level=info"
+CONFIG_BOOTARGS="console=ttyAMA0 debug user_debug=31 earlycon=pl011,0x2A400000 loglevel=9 androidboot.hardware=total_compute androidboot.boot_devices=1c050000.mmci ip=dhcp androidboot.selinux=permissive allow_mismatched_32bit_el0 systemd.log_level=info nokaslr"
 CONFIG_BOOTDELAY=1

2. To disable KASLR at runtime, you can change the boot arguments on the U-Boot command line, as shown follows:

Hit any key to stop autoboot:  0
TOTAL_COMPUTE# setenv bootargs  ${bootargs} nokaslr
TOTAL_COMPUTE#

Load the Linux Kernel debug symbol file

Starting from version 5.11 of Linux kernel, If the kernel is booted at EL2, it always begins with a nVHE init, and then drops to EL1 to initialize the kernel. Then upgrade the kernel EL to EL2 if VHE is capable, as shown in the following figure:

Depending on the debugging requirements, it might be necessary to focus on multiple stages of the code during the debugging process. To debug these stages, it is important to add the corresponding symbol tables for each stage:

1. Load the debug symbol for the booting process at EL2 with MMU disabled
2. Load the debug symbol for the booting process at EL1 with MMU disabled and MMU enabled with ID mapping
3. Load the debug symbol for EL1 with MMU enabled
4. Load the debug symbol for EL2 with MMU enabled 

1. Load the debug symbol for the booting process at EL2 with MMU disabled

A. Find the kernel load address.

On the Arm Total Compute platform, the Linux kernel is loaded by U-Boot. You can find the address where the kernel is loaded by checking the U-Boot logs. See the following log as an example:

## Checking Image at a0000000 ...
   FIT image found
   FIT description: Total Compute: fitImage
    Image 0 (kernel)
     Description:  Linux Kernel
     Type:         Kernel Image
     Compression:  uncompressed
     Data Start:   0xa00000c4
     Data Size:    38554112 Bytes = 36.8 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x80200000
     Entry Point:  0x80200000
     Hash algo:    sha1
     Hash value:   32e5e3d98fd89160e6379c7f5287c3ab5060347c

	. = KIMAGE_VADDR;

	.head.text : {
		_text = .;
		HEAD_TEXT
	}

grep "T _text$"  output/buildroot/tmp_build/linux/System.map
ffff800008000000 T _text

0x80200000 - 0xffff800008000000 = 0x800078200000 

add-symbol-file output/buildroot/tmp_build/linux/vmlinux  EL2N:0x800078200000

2. Load the debug symbol for the booting process at EL1 with MMU disabled and MMU enabled with ID mapping 

After the Linux kernel transitions from EL2 to EL1, if the MMU is disabled or enabled with identity mapping, the symbol table offset remains the same. It is at 0x800078200000, as in the previous step. Therefore, you can use the following command to load the symbol table:

add-symbol-file output/buildroot/tmp_build/linux/vmlinux  EL1N:0x800078200000

3. Load the debug symbol for EL1 with MMU enabled

After KASLR is disabled, the Linux kernel maps the physical address of the loaded Linux image to a virtual address. This virtual address matches the address in the symbol table. Therefore, after the MMU is enabled, you can use the following command to add the symbol table:

add-symbol-file output/buildroot/tmp_build/linux/vmlinux  EL1N:0

4. Load the debug symbol for EL2 with MMU enabled 

You can use the following command to load the symbol table after the MMU is enabled in EL2:

add-symbol-file output/buildroot/tmp_build/linux/vmlinux  EL2N:0

In the Debugger tab of the Linux Kernel debug configuration, include corresponding commands in Execute Debugger Commands to add symbol tables, as shown in the following figure:



Each time the debugging connection starts, these commands are automatically executed to load the symbol table files

You can also add other commands, such as setting breakpoints:

break _text
break start_kernel 

Start the Linux Kernel debugging

You can set any breakpoints at the Linux Kernel connection and click the Continue button as follows:





The Application Processor stops at the breakpoint and you can use the Continue button to continue debugging the Linux Kernel as follows: