Previously, I explained how to create a minimal, single-core ARM® Cortex™-A15 system running Linux®. In this article I will update the hardware design to use a dual-core ARM Cortex-A15 CPU and run SMP (Symmetric Multiprocessing) Linux. While the first system was interesting, I’m fairly certain no single-core A15 systems have ever been built, and most engineers will require multi-core systems for common design and verification tasks.
Two hardware design changes are needed to enable SMP Linux. First, the CPU must be updated to the dual-core A15. This is a matter of simply updating the CPU model to the A15x2 CPU. For those who have not build models using Carbon IP Exchange, the web-based portal that builds models directly from RTL code, there is simple configuration page used to select the A15 model parameters. The key value is to make sure the “Number of CPU Cores” is set to 2 as shown in the picture below.
Once this is configured, and the model is created, it can be instantiated on the SoC Designer canvas in place of the single-core A15 model and connected to the interrupt from the PL011 UART and to the CCI-400 in the same way.
The second change is the addition of an extra memory which is used to communicate the starting address for the secondary core. This memory will take the place of the Versatile Express System Registers. Recall in the previous article I explained that 1 line of source code had to be changed to avoid using a timer value from the System Registers. For SMP Linux there is another register (offset 0x30) which is used to pass the jump address to the secondary CPU. For the minimal system, the only behavior needed is to provide a simple memory at the base address of the System Registers, this is system address 0x1c010000. Only offsets 0x30 and 0x34 are used and the values must be initialized to 0 because the secondary code waits for a non-zero value. When the seconary CPU sees a non-zero value it will jump to the address contained in at 0x1c010030. If this address is not 0 at startup the system will not boot properly. SoC Designer simple memory models initialize to 0 so no special handling is needed.
The Linux image needs to be recompiled with SMP support. This is done using the normal kernel configuration process.
$ make ARCH=arm menuconfig
The option to enable SMP is under the Kernel Features menu. Make sure Symmetric Multi-Processing is selected as shown below.
After enabling SMP rebuild the kernel as before, adjusting the CROSS_COMPILE to match the prefix of your ARM cross compiler:
$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- -j 4
This will create a new zImage which is ready for the dual-core A15 design. The remainder of the steps to prepare the final software image is the same. For the dual-core case I named the final product a15x2-linux.axf and loaded this file into the simulator.
The main challenge in running SMP Linux is getting the secondary CPU started. After reset, both CPUs will start running the code in boot.S which is located at 0x80000000. The first step is to determine if the code is running CPU0 or CPU1. This is done by reading the CPU ID register located in co-processor 15 (CP15). This register is also referred to as the Multiprocessor Affinity Register, MPIDR. It provides information about which core of an MPCore processor the code is running on, and which cluster of a multi-cluster system the code is running on. In this case we have only a single cluster and two cores so the code simply identifies CPU ID 0 as the primary core and CPU ID 1 as the secondary.
The primary core finishes the boot loader and immediately starts running Linux, while the secondary core waits in the boot loader for a jump address to be provided at address 0x1c010030.
The picture below shows the code for the secondary CPU.
The primary CPU which is running Linux is responsible to release the secondary CPU by writing the jump address and sending an interrupt. In the Linux 3.13.1 kernel, this is found in arch/arm/mach-vexpress/platsmp.c at line 225. Putting a breakpoint on this line of code and single stepping will reveal the details. The underlying code will write the jump address and take care of all the details to start the secondary CPU. The well commented last line in the screenshot below gives the details.
Below is a screenshot of the memory contents for the System Registers address range. The primary CPU actually wrote 0xffffffff into address 0x1c010034 and then the 32-bit jump address into 0x1c010030. Because this is a memory view from the perspective of the CPU, I entered the virtual address into the memory viewer window, which is 0xf8010000.
If everything works correctly, there should be some messages in the boot log showing that 2 CPUs are running.
As a final check on the platform, I booted the SMP Linux and created a Swap & Play checkpoint. I restored the checkpoint into a cycle-accurate simulation and issued the command:
$ cat /proc/cpuinfo
The terminal output confirms the state of the simulation was successfully transferred to the cycle accurate simulation and both cores are reported to be running.
In summary, with the addition of an extra memory model in the location of the Versatile Express System Registers and modifying the Linux kernel configuration parameter to enable SMP we are up and running with a dual-core ARM Cortex-A15 minimal hardware design. This design supports all of the same Swap & Play and benchmarking features as the single-core design and can be extended to add more hardware detail for specific design tasks as needed.
As before, all of the work that I’ve done here to prepare Linux and the hardware design is available as a CPAK on The specified item was not found. IP Exchange in the Cortex-A15 CPAK area.
Jason Andrews
Very Good simple explanations for this efficient article. Good Job !