Accelerating ML Collaboration with Arm NN and Linaro

When ancient man first got round to putting wheels under things, it’s unlikely that it was a lone genius having a eureka moment that gave birth to the ultimate Neolithic must-have. Like all technologies, the wheel was almost certainly a collaborative effort, with multiple groups trying various materials and constructions, before hitting on the winning – and enduring – formula. Any new technology needs time to mature, and a collaborative approach usually hastens the time taken to achieve an optimal outcome.

As it was with the wheel, so it is with machine learning (ML) – which is why we recently announced that Arm is donating the Arm NN inference engine and software developer kit (SDK) to Linaro’s Machine Intelligence Initiative. As part of this initiative – which aims to be a focal point for collaborative engineering in the ML space – we’ll also be opening Arm NN to external contributions.

Arm NN enables ML workloads on power-efficient devices by bridging the gap between existing neural network frameworks and power-efficient Arm Cortex CPUs, Arm Mali GPUs, or the Arm ML processor. While it’s already free-of-charge and open-source, by opening it up to external contributions and collaboration, the project aims to reduce redundant engineering and fragmentation in the deep learning and neural network acceleration ecosystem, and drive development of new technology solutions.

“The TensorFlow team is excited to work with Arm and Linaro to expand support for edge devices, and we’re looking forward to integrating with the Arm NN library. We think this kind of standard, open source interface for neural computing will improve the experience for product developers across the Arm ecosystem.”

Pete Warden, Technical lead of the TensorFlow mobile and embedded team at Google

Accelerating ML Adoption

The Machine Intelligence Initiative aims to accelerate the adoption of ML across the widest number of applications. As a certain someone once said, the growth of ML represents the biggest inflection point in computing for more than a generation: its impact is already being felt in fields as diverse as healthcare, linguistics and logistics, and that impact is set to grow. I truly can’t think of a sector that will remain untouched by its transformative power – but that, in itself, creates a number of issues. You can become expert in machine learning, but not all the application domains. You can become an expert in a specific domain, but not all frameworks and platforms. You can build a machine learning System-on-Chip (SoC) but it won’t address all markets. In short, we need experts from a number of fields to come together to address these problems.

As a founding member of the initiative, we’re aiming to create a meeting ground for these experts to combine and share their knowledge in order to achieve the optimal solution for each use case, quickly and easily.

A common software interface, such as Arm NN, supporting industry-leading frameworks and tools, will enable developers to deploy machine learning more rapidly, across the widest number of applications.

Moving ML to the Edge

ML at the edge means increased responsiveness, with improved security. Advances in compute power mean that even highly constrained, battery-powered edge devices such as wearables and sensors have the ‘intelligence’ needed to keep the processing on-device. And once you stop sending your data back and forth to the cloud, you quite naturally reduce latency – not to mention cost – while the risk of exposure diminishes. In short, edge compute restricts access and guarantees control. The reasons for a shift towards edge compute are compelling.

Much of today’s edge ML runs on CPUs, and while they – along with GPUs – are getting better and better at the task, the variety and sophistication of the networks we’d like to run at the edge means that many new use cases will require accelerators. These dedicated ML accelerators will come in many different forms, so a software abstraction – such as ArmNN – will be required. This essentially removes the need for custom code targeting specific accelerators, allowing developers to continue using their preferred frameworks and tools, while Arm NN seamlessly converts the result to run on the underlying platform.

This represents a significant reduction in effort for developers. By providing them with a ready-made ‘building block’ that copes with incoming graphs and parses the related data, they no longer need to worry about creating a full, end-to-end software stack. They need only create a block of code to interface with ArmNN, leaving them free to focus on the key differentiators that make their product unique. In short, they don’t need to reinvent the wheel, they can concentrate their efforts on the whistles and bells.

Jem Davies discusses Arm’s donation of Arm NN to Linaro’s Machine Intelligence Initiative at Linaro Connect Vancouver 2018

Linaro’s Machine Intelligence Initiative

The Machine Intelligence Initiative will initially focus on inference for Arm Cortex-A SoCs and Arm Cortex-M MCUs running Linux and Android, both for edge compute and smart devices. The team will collaborate on defining an API and modular framework for an Arm runtime inference engine architecture based on plug-ins supporting dynamic modules and optimized shared Arm compute libraries. The work will rapidly develop to support a full range of processors, including CPUs, NPUs, GPUs, and DSPs and it is expected that Arm NN will be a crucial part of this.

Under the initiative, both companies and individuals can contribute code, so if you have expertise to share, why not help us to…

• Improve static and heterogeneous scheduling
• Support a greater range of devices
• Improve the device interface
• Streamline tool flows
• Work towards standards for all

If you’d like to join the Machine Intelligence Initiative, contact Linaro, or hit the button below to download Arm NN.

Download Arm NN