Arm Community
Arm Community
  • Site
  • User
  • Site
  • Search
  • User
  • Groups
    • Research Collaboration and Enablement
    • DesignStart
    • Education Hub
    • Innovation
    • Open Source Software and Platforms
  • Forums
    • AI and ML forum
    • Architectures and Processors forum
    • Arm Development Platforms forum
    • Arm Development Studio forum
    • Arm Virtual Hardware forum
    • Automotive forum
    • Compilers and Libraries forum
    • Graphics, Gaming, and VR forum
    • High Performance Computing (HPC) forum
    • Infrastructure Solutions forum
    • Internet of Things (IoT) forum
    • Keil forum
    • Morello Forum
    • Operating Systems forum
    • SoC Design and Simulation forum
    • 中文社区论区
  • Blogs
    • AI and ML blog
    • Announcements
    • Architectures and Processors blog
    • Automotive blog
    • Graphics, Gaming, and VR blog
    • High Performance Computing (HPC) blog
    • Infrastructure Solutions blog
    • Innovation blog
    • Internet of Things (IoT) blog
    • Operating Systems blog
    • Research Articles
    • SoC Design and Simulation blog
    • Smart Homes
    • Tools, Software and IDEs blog
    • Works on Arm blog
    • 中文社区博客
  • Support
    • Arm Support Services
    • Documentation
    • Downloads
    • Training
    • Arm Approved program
    • Arm Design Reviews
  • Community Help
  • More
  • Cancel
Arm Community blogs
Arm Community blogs
Architectures and Processors blog Simplifying Embedded Linux with SystemReady IR
  • Blogs
  • Mentions
  • Sub-Groups
  • Tags
  • Jump...
  • Cancel
More blogs in Arm Community blogs
  • AI and ML blog

  • Announcements

  • Architectures and Processors blog

  • Automotive blog

  • Embedded blog

  • Graphics, Gaming, and VR blog

  • High Performance Computing (HPC) blog

  • Infrastructure Solutions blog

  • Internet of Things (IoT) blog

  • Operating Systems blog

  • SoC Design and Simulation blog

  • Tools, Software and IDEs blog

Tell us what you think
Tags
  • Embedded Linux
  • U-Boot
  • BSA
  • Software Standards
  • UEFI
  • EBBR
  • SystemReady IR
  • ArmDevSummit21
  • firmware
Actions
  • RSS
  • More
  • Cancel
Related blog posts
Related forum threads

Simplifying Embedded Linux with SystemReady IR

Grant Likely
Grant Likely
October 18, 2021
9 minute read time.

As someone who has been building embedded Linux systems for the past 16 years, I am proud of the amazing products built with Linux on Arm. Yet, I am also concerned about how much effort is required to bring up every platform. By any measure, the embedded Linux ecosystem is a healthy one. It is used in every industry. The flexibility and functionality of Linux is easy to adapt into new products, and the open-source ecosystem enables all of us to collaborate and make Linux even better.

On the other hand, the software stack on most embedded devices is entirely custom. The OS is tied to the platform, and even the smallest change means rebuilding the entire image. For example, over the lifetime of a product the hardware design may change several times to account for component availability, to improve reliability, or for cost reduction. While outward functionality of the product may not change, each iteration may require a new OS image which supports the vendor long term. Similarly, many products use the same base hardware design and only differ on the user-space application software. However, in typical embedded development each variant has a different OS image that is supported separately and without any shared binary components.

The problem here is not building embedded systems. The excellent tools we have in Yocto, Buildroot, OpenWRT, and other make it easy to build an embedded product. Rather, the problem is that each build configuration requires extra effort and cost to support and maintain. There is not a good way to amortize support across multiple products by using the same binary components. An example is security patches: with one or a few build configurations you may be able to keep up to date with security patches, but the effort required scales linearly with every additional build. If you have 100 products, then every time a security patch is applied, 100 different kernel configurations need to be rebuilt, tested, and then deployed. The effort quickly scales out of control.

By finding ways to share components between products, it becomes a lot more feasible to maintain multiple products long term. One way to do this is to bring in standards for how components interact, which ensures interoperability and allows custom products to pull in generic software components where appropriate.

Solving the embedded problem

At this point, it is tempting to say, “desktops and servers standards have already solved this problem. Let's use those.” While true, PC platform standards ensure any OS can boot on any hardware in that space. However, it is also simplistic and does not recognize either the hardware constraints or the level of customization required in embedded. We need to handle custom hardware interfaces, and often many components throughout the software stack need to be modified to work in the embedded device. It also does not recognize that the embedded ecosystem is heavily invested in mature technologies like U-Boot and Device tree, and there is a high resistance to migrate away from them. To be useful in embedded, any standards introduced must account for the needs of the embedded ecosystem.

It is also important to understand how the level of standardization impacts the way products are developed. If there is little standardization, like embedded Linux is now, it provides high flexibility on how embedded systems are built, but creates a high maintenance cost over time to support custom developments. On the other hand, strict standards would ensure a high degree of interoperability, but may not provide the flexibility needed by the embedded sector. If standards are too strict, then they are ignored, and they will not do anything to enable new and innovative products. There is a balance between standardization and flexibility, particularly with embedded, which enables innovation and creates better products.

Arm SystemReady program: The balance of standardization

Hitting the right level of standardization for embedded is exactly what the SystemReady IR certification program addresses. SystemReady IR extends the promise of “Just Works” to embedded Linux products by choosing standards that can be implemented using the technology already in use today. A SystemReady IR certification guarantees the platform implements a standard firmware interface that OSes can depend on without customization, and that support for the platform has been merged into mainline Linux.

Or in concrete terms, a SystemReady IR compliant platform can boot generic Linux distributions using the exact same image format and boot flow used by Arm servers and laptops. Using the same boot flow opens all kinds of possibilities for the embedded ecosystem. To list a few, it means that instead of maintaining everything themselves, device vendors can build products using well-supported distributions who have the resources to manage updates and security fixes. It means that custom Yocto images can be booted and tested using virtual machines in the cloud before being deployed out to real hardware. It means engineering effort does not need to be devoted to managing per-platform differences, and, crucially, it becomes easy to support many hardware devices even when building a completely custom OS stack. SystemReady IR means that you can focus on the features that make your product great, and not on individual device enablement problems.

SystemReady IR is the embedded and IoT band of the larger SystemReady program. SystemReady defines and certifies compliance with hardware and firmware standards for Arm systems, with bands for different parts of the Arm ecosystem. The detailed requirements of each band are defined in the SystemReady Requirements Specification (SRS) which is linked to at the end of this article. The list of SystemReady IR certified platforms can be found on the SystemReady web page. The first IR certification was issued in June 2021, and there will be at least 11 certifications published before Arm DevSummit in October 2021, with many more in the certification pipeline.

Standards that work for embedded

For standards to be relevant in embedded they need to be two things: They must foster interoperability, and they must work with existing software stacks. Most embedded Linux products are built using U-Boot firmware, a Device tree system description and a custom Linux kernel. U-Boot and Device tree are mature technologies that are widely deployed. However, the way they are used makes interoperability difficult. Deployment is different on every platform due to slight differences in the U-Boot boot scripts. The OS is tied to the platform because the Device tree is tightly coupled to the OS. Keeping the software up to date requires custom code because there is not a standard firmware update interface. So, while the components are capable and mature, lacking standards there is no way to ensure interoperability.

When designing SystemReady IR, we identified the major issues that can be resolved with reasonable design changes. We looked for where we could create a defined interface between the platform (hardware & firmware) and the operating system that would simplify the engineering required. To do that we focused on the specific problems faced by embedded developers. The following table is a helpful summary:

Problem Relevant standard Design change
Every platform has a different boot flow UEFI (existing industry standard) Enable U-Boot’s UEFI with CONFIG_EFILOADER
OS images are tied to specific hardware Device tree Embed DT in U-Boot instead of inside OS image
Every platform is updated in a different way UEFI UpdateCapsule() Enable U-Boot’s UpdateCapsule()
Generic Linux will not boot due to missing drivers for non-standard IP 1. Linux upstream-first policy 2. Arm BSA 1. Get SoC support into mainline Linux 2. Use BSA-compliant IP when possible

These changes; enabling UEFI, providing DT by default, supporting UpdateCapsule(), and support in mainline solves most of the OS interoperability problems. All the major OS vendors expect the UEFI boot flow and U-Boot already implements UEFI. Enabling it immediately makes U-Boot compatible. Configuring U-Boot to provide a Device tree to the OS by default means that the platform can describe itself to the OS. So, the OS does not need to carry around details about every hardware platform. For generic distributions this is a critical issue because they cannot support per-platform customizations at scale. The OS is generic and cannot contain machine-specific details. Similarly enabling UpdateCapsule() provides a standard interface for updating system components so that the OS does not need to understand a machine-specific update mechanism. Finally, vendor kernels are the biggest pain points in embedded Linux development because it means each product requires a different kernel tree and it makes the platform unsupportable by the Linux distributions. By requiring the hardware be supported in mainline Linux, the distros can ship support for the hardware in their generic images. In addition custom Linux builds can track Linux stable releases instead of depending on a vendor to keep their fork up to date.

Everything listed here is supported today by U-Boot, Trusted Firmware, Linux, the major distros, and Yocto. The configuration options are easy to turn on, and once enabled they open up a wide range of OS and hardware options when building a product. Best of all it does not require replacing any of the software you are already using.

SystemReady IR: Standards and certification

Where SystemReady IR comes in is to formalize the standards discussed previously into a specific set of requirements that silicon vendors and ODMs can implement. It sets up a certification testing program to verify compliance with the specifications. At a high level, the IR band requires that:

  • Firmware implements a subset of UEFI as defined in Embedded Base Boot Requirements (EBBR)
  • Firmware by default provides a device tree suitable for booting mainline Linux
  • Firmware can be updates using UEFI UpdateCapsule()
  • At least two Linux distros must be able to boot and install using the UEFI boot flow

Tools, test suites and resources are provided by the IR program to help step through the process of certification. The most important resource is the SystemReady IR: IoT integration, test, and certification guide which covers all the details of how to enable and test for SystemReady IR compliance. The tools are open-source and useful links to them are in the SystemReady IR guide.

Getting certified

Arm SystemReady: Getting started diagram

The certification process itself is straight-forward. The first step is to contact us and let us know about the platform you want to certify. We ask a few questions to confirm that you can run and pass the certification test procedure, and then put you in contact with one of the SystemReady IR test labs. You need to provide the test lab with a copy of your own test results as well as a sample of the hardware so that the lab can reproduce and confirm your results. Finally, the test lab provides the full results of their testing to Arm. If the platform meets all the requirements and passes testing, then Arm will provide a formal SystemReady certificate and list the platform on the SystemReady IR web page.

What next?

The SystemReady IR program is actively testing and certifying boards now. If you have any questions or if you have a product to be certified, then you can contact us at systemready@arm.com. You can read the SystemReady IR Integration, Test, and Certification guide and try running the IR Architecture Compliance Suite (ACS) on your platform. All the program details can be found on our website. Finally, you can help us define the next iteration of SystemReady IR by getting involved with the EBBR community project. Contact and meeting details can be found on the EBBR project page.

Resources

  • Website - SystemReady homepage
  • Video - The Value and Path to SystemReady IR Certification 
  • Video - Deploying Yocto Linux on SystemReady IR Compliant Hardware
  • Blog - Progress in the Arm SystemReady Program: Enabling systems where software ‘just works’
  • Blog - The Journey to Cloud Native Edge: Project Cassini
  • Contact us
  • SystemReady IR Integration, Test, and Certification guide
  • Deploying Yocto on SystemReady IR Compliant Hardware Guide
  • SystemReady IR ACS
  • https://gitlab.arm.com/systemready/systemready-template
  • https://gitlab.arm.com/systemready/systemready-scripts
Anonymous
  • Olivier Bernard
    Offline Olivier Bernard over 1 year ago

    Great work Grant!!!

    • Cancel
    • Up 0 Down
    • Reply
    • More
    • Cancel
Architectures and Processors blog
  • Optimizing TIFF image processing using AARCH64 (64-bit) Neon

    Ramin Zaghi
    Ramin Zaghi
    This guest blog shows how 64-bit Neon technology can be used to improve performance in image processing applications.
    • October 13, 2022
  • Arm A-Profile Architecture Developments 2022

    Martin Weidmann
    Martin Weidmann
    2022 additions to Arm A-Profile architecture covering Virtual Memory System Architecture, SME2 and mitigating some ROP attacks with Guarded Control Stack.
    • September 29, 2022
  • A closer look at Arm A-profile support for non-maskable interrupts

    Christoffer Dall
    Christoffer Dall
    Arm is adding support in both the CPU and Generic Interrupt Controller (GIC) architecture for NMIs. But what is an NMI? how does operating systems software use these features?
    • May 23, 2022