AMBA ATP Engine 3.1: programmable traffic generation

We are pleased to announce the release of version 3.1 of AMBA ATP Engine, an open-source implementation of the AMBA Adaptive Traffic Profiles (ATP) synthetic traffic framework specification.

This is a major release enabling users to program AMBA ATP traffic generation from Linux environments. Users can now model runtime dependencies on other software and system state to correctly capture the effects on the activation of traffic generation.

Performance exploration of complex SoCs

The pursuit of automation, distributed intelligence and richer user experiences, such as self-driving vehicles and AR and VR, is fostering embedded systems with increased connectivity and data processing capabilities.

Automotive, robotics, and personal device systems implement an increasing variety of sensors and actuators, and use specialized compute to efficiently execute machine learning, graphics processing and similar algorithms to process the sensed data. Systems-on-a-chip (SoCs) are thus becoming more heterogeneous, both in the components they integrate and in the traffic patterns originating from those components.

Simulation for early performance exploration of complex SoCs must balance accuracy and speed to produce meaningful results in a reasonable time. Usage of highly accurate micro-architectural models of components often results in long simulation times, unsuitable for the purpose. Moreover, the availability of detailed models and the software layers to drive them is often limited at early design stages.

This raises an opportunity to model the performance properties of these components through abstractions which are faster to simulate while retaining a reasonable degree of accuracy. This leads to an increase in productivity during early performance exploration.

AMBA ATP ecosystem

To address this opportunity, in April 2019 Arm introduced AMBA Adaptive Traffic Profiles (ATP), a standard specification for modeling traffic characteristics of hardware interfaces. A standard representation, portable across systems, allows users to concisely express these characteristics in traffic profiles, which can then be integrated with a system to either inject or serve traffic. Since its release, AMBA ATP has been adopted by our partners in both industry and academy.

In May 2020, Arm introduced AMBA ATP Engine, an open-source implementation of AMBA ATP. The Engine is aimed at fostering the adoption of AMBA ATP, and can be integrated into host platforms by using adapters. Arm provides an adapter implementation for platforms simulated through the widely popular gem5 simulator. 

Full-system simulators such as gem5 support booting and running unmodified software stacks from real systems, for example a Linux operating system with a set of hosted applications. Full-system simulations can better capture software interactions on access to system resources, for example constraints on access to shared memory, CPU scheduling priorities and ownership of devices to program from CPU code. 

The motivation for software control

Traffic generation in devices programmed from CPU code is subject to the time the CPU offloads the operation to the device, which depends on software interactions and system state. Statically scheduled traffic generation cannot capture the effect of these dependencies. Moreover, the address ranges accessed by the device traffic during the operation are also programmed by software.

The initial release of AMBA ATP Engine, namely AMBA ATP Engine 3.0, provides adapters with standard APIs to both configure and activate traffic profiles at simulation design time. The new AMBA ATP Engine 3.1 introduces a programmability layer for exposing these APIs to software. This enables runtime configuration of address ranges and traffic generation subject to the previously mentioned dependencies.

Introducing AMBA ATP Engine 3.1

We are pleased to announce the release of AMBA ATP Engine 3.1. There are the following highlights:

• Standard specification of an optional programmers model. Enables software compatibility across compliant adapter implementations, and removes the need for adapter-specific software stacks.
• Implementation of the programmers model for the gem5 adapter. Enables use and development of software for ATP-enabled gem5 simulations.
• Implementation of a Linux kernel module to drive the programmers model. The module can manage the gem5 adapter and any other adapters compliant with the specification. Enables development of user-level applications.
• Official distribution of a full-system environment through a new Yocto layer. Enables easy and fast bring up of an ATP-enabled gem5 simulation.
• Support for gem5-20.1.

The following sections summarize the highlights of the release. Please refer to the official documentation for details on design and usage.

A standard programmers model

The programmers model defines a set of standard registers and operations to program traffic streams in Engine Requester devices. Each Engine Requester is identified inside the Engine by a master_id, specified in .atp files. The adapter can implement the programmers model for each Engine Requester it exposes. These are some of the features from the programmers model:

• Independent configuration of memory regions for read and write traffic from a traffic stream.
• Programming of labels to identify particular trafficstreams across the system, similarly to Arm MPAM PARTID.
• Asynchronous notifications of traffic stream completion through interrupts.
• Discovery of exposed Engine Requesters through standard Device tree

ATPDevice: programmability for the gem5 adapter

A new ATPDevice gem5 model represents an exposed Engine Requester. ATPDevice implements the standard programmers model. Users can instantiate multiple ATPDevices within a simulation, and program them independently.

ATP and gem5 system diagram

Software stack to drive traffic generation

Two new Linux kernel modules are now available in the official Engine repository.

• The device moduleexposes the programmability layer of Engine Requesters to software, and is the main interface for programming traffic streams. This module can drive any device compliant with the standard programmers model, including the ATPDevice Each ATP device managed by the device module is available as /dev/<name>. 
• The buffer manager moduleprovides allocation of memory buffers shareable between Engine Requesters and CPU cores. This enables efficient implementation of producer-consumer software models. Examples of these models include a CPU process consuming processed images produced by an image signal processor (ISP), or a Display Processing Unit (DPU) consuming a rendered frame produced by a GPU. The buffer manage module is available as /dev/atpbuffer. 

A user API in the form of a C user-space library is provided along the kernel modules. This eases the development of applications against the Linux software stack by providing high-level APIs. It is also compatible with C++ applications.

The following is an example application with this behavior:

1. Obtains file descriptors for both the buffer manager and the device to program, that is DEV0.
2. Allocates a four KiB shareable buffer, andattaches DEV0 as a user of the buffer. 
3. Obtains a unique ID for the traffic stream STR0. This is unique within the Engine and can be used for further operations.
4. Starts traffic generation on DEV0 by playing the traffic stream STR0. It uses a label value of 1 to identify the flow within the system. The 4KiB buffer is used as a source for read traffic. Since there is no write traffic, -1 is passed as last argument.

This example deliberately omits closing calls to release the buffer and file descriptors.

// Other headers ... 
// ATP user API 
#include "atp.h" 
 
int main() { 
    int bm_fd = open("dev/atpbuffer", O_RDWR); 
    int dev_fd = open("dev/atpDEV0", O_RDWR); 
 
    int buf_fd = atp_buffer_get(bm_fd, 4096); 
    atp_device_attach(dev_fd, buf_fd); 
     
    uint64_t str_id; 
    atp_device_unique_stream(dev_fd, "STR0", &str_id); 
    uint32_t flow_id = 1; 
    atp_device_play_stream(dev_fd, str_id, flow_id, buf_fd, -1); 
 
    // ... 
} 

Finally, a C++ test suite based on CppUnit is provided to verify the functionality of kernel modules and user API.

meta-atp: a ready environment for programmable traffic generation

Yocto is an open-source project which helps users create consistent and ready to use environments based on several software layers. These layers may originate from different providers. Arm contributes its layers through the open source meta-arm repository.

To facilitate access to an environment with AMBA ATP programmable traffic generation, we are pleased to announce the release of the meta-atp layer as part of the meta-arm repository for Yocto stable version 3.3. The meta-atp layer provides a ready to use environment based on gem5 and AMBA ATP Engine integration, along with a Linux image provisioned with ATP kernel modules, user API and test suite.

Users can build and access the environment in as much as four commands. See the official documentation for details. 

Conclusion

With the release of AMBA ATP Engine 3.1, users can now program AMBA ATP traffic generation from Linux environments. This enables users to benefit from AMBA ATP models while subjecting traffic generation to software interactions and system state.

Moreover, the specification of a standard programmers model eases the development of new adapters with programmable traffic generation support. Kernel modules are compatible with any compliant adapter implementation.

The release of the meta-atp Yocto layer further facilitates access to a ready to use environment providing programmable AMBA ATP traffic generation. This can serve as a reference for more complex host platform configurations. 

Users can now download the new ATP Engine 3.1 from the official repository. For easy bring up of a minimal development platform, users can follow the Yocto Project quick build guide and use meta-atp on Customizing your build for specific hardware. See the official meta-atp documentation for more details.