Security is increasingly becoming more relevant, as the vulnerability surface grows along with the number of connected devices. Arm has prioritized security for many years, introducing TrustZone® as a security hardware architecture in 2003. One key TrustZone feature involves the system-level hardware isolation into two worlds: Secure world and Non-secure World (Normal). Processing elements (PEs) may be configured in one of two states through the Secure Configuration Register (SCR).
Traditional applications, general-purpose Operating Systems (OS) and Virtual Machine Monitors (VMM) run in a Rich Execution Environment (REE), which is hosted in the Non-secure World. By contrast, a Trusted Execution Environment (TEE) may be hosted in the Secure world. This provides services such as Secure Boot, credential authentication, Digital Rights Management (DRM) or encryption, which is managed by one or more Trusted OSs. It is important to remark that REE software failures and misbehaviors are isolated from impacting TEE.
A key component of TrustZone is the Secure Monitor (SM). This provides context switching between REE and TEE, while additionally handling Secure Monitor Calls (SMC), which constitute communication and service provisioning between worlds. In Armv8, a new architectural Exception level EL3 is designed for the SM to operate in. Arm provides a reference implementation of Secure world software known as Trusted Firmware (TF), including an SM implementation. This post briefly introduces TF, the gem5 simulator, motivations for TF support in gem5 and the future it enables.
Trusted Firmware-A (TF-A) is Arm’s reference implementation of Secure world software for A-profile architectures. It enables Secure Boot flow models, and provides implementations for the Secure Monitor executing at Exception level 3 (EL3) as well as for several Arm low-level software interface standards, including:
TF-A supports Secure Boot flow models. These are based on a hardware root-of-trust. The boot process is divided in multiple consecutive boot stages, in which each stage verifies the signature of the next one; the initial stage is usually hosted in ROM. In particular, the following stages are defined:
BL33 may be a traditional boot loader, including UEFI implementations such as Das U-boot or TianoCore EDK II, which are not part of TF-A. These stages may be packed in a single binary archive known as Firmware Image Package (FIP), which is later loaded in the platform.
Figure 1: Trusted Firmware boot stages
gem5 is an event-driven system simulator designed for computer architecture research. It is widely used in academia and industry, as well as within Arm; we have teams actively working to maintain and improve the upstream distribution of the simulator.
A wide spectrum of modelling solutions are available based on accuracy to simulation speed trade-offs: Loosely Timed (LT) solutions, such as QEMU or Arm Fast Models, provide high simulation speed environments to serve use cases such as software development, validation and integration. Cycle Accurate (CA) solutions, such as RTL simulations or Arm Cycle Models, cover fine-level IP tuning, benchmarking and selection. gem5 stands in the middle of the spectrum as an Approximately Timed (AT) solution, covering performance and power exploration of early system-level solutions.
Figure 2: Modelling solutions spectrum
gem5 can run existing real-world software, such as upstream Linux kernels, in what is known as Full System Mode. It also supports faster simulations through Syscall Emulation Mode, where the simulator itself provides kernel services, in a similar way to QEMU user-space emulation. Model parameters may be fine-tuned at simulation design time to cover different use cases based on specific micro-architectures or hardware generations.
Some of the motivations behind adding TF-A support to gem5 are:
TF-A includes support for multiple Platforms. Each Platform is characterized by the set of hardware components it hosts (such as PEs, peripherals, memory devices), as well as by the location of these in its memory map.
Arm provides a set of reference Fixed Virtual Platforms (FVP) developed on Fast Models. BasePlatform is the basis for FVPs and is offered free for customers to experiment and develop their Arm-targeted software. A minimal version of BasePlatform known as Foundation is provided, also free, as the simplest solution for an Arm-enabled platform. TF-A has official support for both BasePlatform and Foundation.
gem5 also provides a set of Arm reference platforms (the VExpress_GEM5_Base family). These are loosely based on a Versatile Express RS1 platform with a slightly modified memory map to accommodate for gem5-specific peripherals. There are two variants: VExpress_GEM5_V1 and VExpress_GEM5_V2, the latter provides a GICv3 model. Unfortunately, there is no TF-A support for VExpress_GEM5_Base.
Initially, three main approaches were considered:
The first approach involves establishing VExpress_GEM5_V2 as a reference platform to justify the contribution to TF-A. This means firmware developers would have to decide between Foundation and VExpress_GEM5_V2 as their target platform, while no compatibility would be provided.
The second approach involves restructuring existing models in VExpress_GEM5_V2, such as the GICv3, as well as adding missing ones, such as extra UARTs and SP805, to accommodate the Foundation memory map. Although feasible, this breaks backwards compatibility for existing VExpress_GEM5_V2 users.
On top of avoiding the previous drawbacks, the third approach offers a path for unifying gem5 and Fast Models reference platforms, which yield additional benefits stated in the conclusions.
From the decision above, the following contributions to gem5 have arisen:
To test these contributions, we have successfully brought up a system where TF-A loads upstream Das U-boot as BL33, and the latter loads upstream Linux kernel as Non-trusted EL1-Payload. Within this simulation, we have run the official TF-A tests to stress EL3 services. The fundamental Framework Validation tests are successful, along with the main PSCI services, including PE suspend and resume, PE hot plug and affinity info. We have also brought up OP-TEE OS as Secure EL1-Payload, however this has not been tested further.
The following are extra gem5 contributions resulting from the project work:
This section introduces a step-by-step guide on bringing up a TF-A-based simulation in gem5. Please, follow the suggested order, as some build stages take artifacts from previous ones.
Make sure the build requirements for each software artifact is satisfied before following this guide. Tested on Ubuntu 20.04.
We create a directory to store build outputs:
mkdir -p resources/binaries resources/disks resources/semihosting
git clone https://gem5.googlesource.com/public/gem5 -b v20.0.0.1 cd gem5/ scons -j $(nproc) build/ARM/gem5.opt cd - make -C gem5/util/term
git clone https://github.com/torvalds/linux.git -b v5.4 make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C linux defconfig make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C linux -j $(nproc) mv linux/arch/arm64/boot/Image resources/semihosting/
git clone https://github.com/u-boot/u-boot.git -b v2020.07-rc3 make CROSS_COMPILE=aarch64-linux-gnu- -C u-boot vexpress_aemv8a_semi_defconfig
We use the default FVP configuration and the following parameter values:
make CROSS_COMPILE=aarch64-linux-gnu- -C u-boot menuconfig
make CROSS_COMPILE=aarch64-linux-gnu- -C u-boot -j $(nproc) mv u-boot/u-boot.bin resources/binaries/
git clone https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git -b v2.3 make CROSS_COMPILE=aarch64-linux-gnu- \ PLAT=fvp \ BL33=../resources/binaries/u-boot.bin \ -C trusted-firmware-a all fip mv trusted-firmware-a/build/fvp/release/bl1.bin resources/binaries/ mv trusted-firmware-a/build/fvp/release/fip.bin resources/binaries/
Note: you may build a debug version of TF-A by passing DEBUG=1 and LOG_LEVEL=50 to the build command. You can then check gem5's terminal output to see step by step the boot process of TF-A.
git clone https://github.com/buildroot/buildroot.git -b 2020.05-rc3 make ARCH=aarch64 BR2_JLEVEL=$(nproc) CROSS_COMPILE=aarch64-linux-gnu- \ -C buildroot/ arm_foundationv8_defconfig
We use the default Foundation configuration and the following parameter values:
make ARCH=aarch64 BR2_JLEVEL=$(nproc) CROSS_COMPILE=aarch64-linux-gnu- \ -C buildroot/ menuconfig
make ARCH=aarch64 BR2_JLEVEL=$(nproc) CROSS_COMPILE=aarch64-linux-gnu- \ -C buildroot/ mv buildroot/output/images/rootfs.ext2 resources/disks/
We use gem5 auto-DTB generation:
export M5_PATH=resources/ && \ gem5/build/ARM/gem5.opt gem5/configs/example/arm/baremetal.py \ --workload ArmTrustedFirmware \ --disk-image resources/disks/rootfs.ext2 \ --num-cores 4 \ --machine-type VExpress_GEM5_Foundation \ --dtb-gen \ --semi-enable --semi-path resources/semihosting/ mv m5out/system.dtb resources/semihosting/devtree.dtb
Note: configurations from 1 to 4 cores are supported.
Before continuing, check that your resources directory looks as follows:
resources binaries bl1.bin fip.bin u-boot.bin disks rootfs.ext2 semihosting Image devtree.dtb
Now you may run the simulation:
export M5_PATH=resources/ && \ gem5/build/ARM/gem5.opt gem5/configs/example/arm/baremetal.py \ --workload ArmTrustedFirmware \ --disk-image resources/disks/rootfs.ext2 \ --num-cores 4 \ --machine-type VExpress_GEM5_Foundation \ --semi-enable --semi-path resources/semihosting/
And connect to it:
gem5/util/term/m5term 3456
You should see the different bootloader stages and finally the Linux kernel boot process. Once finished, you’ll be presented with the shell login provided by the root filesystem; enter “root” as username and you’ll gain access.
By providing a Foundation model in gem5, we not only inherit TF-A support, but also any other Fast Models Foundation software support, existing and future. This enables Arm customers to easily integrate Fast Models and gem5 in their development process, enabling reasoning about system-level performance and power of their solutions.
Initial TF-A support has been contributed, reviewed and was published along with the first major gem5 release of 2020, v20.0.0.0. The project has comprised development of the necessary features for TF-A support in gem5, however extensive testing of these features is still required for a full verification.
TF-A provides its own Secure EL1-Payload test, which would help stress dispatchers and SMC handling. EL1-Payloads other than OP-TEE OS, such as Android Trusty TEE should also be tested. Official TF-A tests should be exhausted as to verify other features, including firmware updates or SCMI. We look forward to the gem5 community improving its support as interest grows. Should you have any questions, reach out to myself or Giacomo Travaglini.
Contact Adrian Herrera Contact Giacomo Travaglini