The Linux kernel provides an ARM PMU driver for counting events such as cycles, instructions, and cache metrics. My previous article covered how to access data from the PMU automatically within SoC Designer by enabling hardware profiling events. It also discussed how to enable access from a Linux application so the application can directly access the PMU information. This article covers how to use the ARM Linux PMU driver to gather performance information. In the previous article, the Linux application was accessing the PMU hardware directly using system control coprocessor instructions, but this time a device driver and a system call will be used. As before, I used a Carbon Performance Analysis Kit (CPAK) for a Cortex-A53 system running 64-bit Linux.
The steps covered are:
The first step is to enable profiling in the Linux kernel. It is not always easy to identify the minimal set of values to enable kernel features, but in this case I enabled “Kernel performance events and counters which is found under General setup" then under "Kernel Performance Events and Counters".
I also enabled Profiling support on the General setup menu.
Once these options are enabled recompile the kernel as usual by following the instructions provided in the CPAK.
Below is the device tree entry for the PMU driver. All Carbon Linux CPAKs for Cortex-A53 and Cortex-A57 include this entry so no modification is needed. If you are working with your own Linux configuration confirm the pmu entry is present in the device tree.
When the kernel boots the driver prints out a message:
hw perfevents: enabled with arm/armv8-pmuv3 PMU driver, 7 counters available
hw perfevents
If this message is not in the kernel boot log check both the PMU driver device tree entry and the kernel configuration parameters listed above. If any of them are not correct the driver message will not appear.
One way to get performance information from a Linux application is to use the perf_event_open system call. This system call does not have a glibc wrapper so it is called directly using syscall. Most of the available examples create a wrapper function, including the one shown in the manpage to make for easier usage.
The process is similar to many other Linux system call. First, get a file descriptor using open() and then use the file descriptor for other operations such as ioctl() and read(). The perf_event_open system call uses a number of parameters to configure the events to be counted. Sticking with the simple case of instruction count, the perf_event_attr data structure needs to be filled in with the desired details.
perf_event_open
perf_event_attr
It contains information about:
Other system call arguments include which event to trace (such as instructions), the process id to trace, and which CPUs to trace on.
A setup function to count instructions could look like this:
At the end of the test or interesting section of code it’s easy to disable the instruction count and read the current value. In this code example, get_totaltime() uses a Linux timer to time the interesting work and this is combined with the instruction count from the PMU driver to print some metrics at the end of the test.
get_totaltime()
One of the features of perf_event_open is the ability to use a group file descriptor to create groups of events with one group leader and other group members with all events being traced as a group. While all of this is possible it may be helpful to look at the perf command, which comes with the Linux kernel and provides the ability to control the counters for entire applications.