Arm Community
Arm Community
  • Site
  • User
  • Site
  • Search
  • User
  • Groups
    • Research Collaboration and Enablement
    • DesignStart
    • Education Hub
    • Innovation
    • Open Source Software and Platforms
  • Forums
    • AI and ML forum
    • Architectures and Processors forum
    • Arm Development Platforms forum
    • Arm Development Studio forum
    • Arm Virtual Hardware forum
    • Automotive forum
    • Compilers and Libraries forum
    • Graphics, Gaming, and VR forum
    • High Performance Computing (HPC) forum
    • Infrastructure Solutions forum
    • Internet of Things (IoT) forum
    • Keil forum
    • Morello Forum
    • Operating Systems forum
    • SoC Design and Simulation forum
    • 中文社区论区
  • Blogs
    • AI and ML blog
    • Announcements
    • Architectures and Processors blog
    • Automotive blog
    • Graphics, Gaming, and VR blog
    • High Performance Computing (HPC) blog
    • Infrastructure Solutions blog
    • Innovation blog
    • Internet of Things (IoT) blog
    • Operating Systems blog
    • Research Articles
    • SoC Design and Simulation blog
    • Tools, Software and IDEs blog
    • 中文社区博客
  • Support
    • Arm Support Services
    • Documentation
    • Downloads
    • Training
    • Arm Approved program
    • Arm Design Reviews
  • Community Help
  • More
  • Cancel
Arm Community blogs
Arm Community blogs
AI and ML blog Arm Ethos-U65: Powering innovation in a new world of AI devices
  • Blogs
  • Mentions
  • Sub-Groups
  • Tags
  • Jump...
  • Cancel
More blogs in Arm Community blogs
  • AI and ML blog

  • Announcements

  • Architectures and Processors blog

  • Automotive blog

  • Embedded blog

  • Graphics, Gaming, and VR blog

  • High Performance Computing (HPC) blog

  • Infrastructure Solutions blog

  • Internet of Things (IoT) blog

  • Operating Systems blog

  • SoC Design and Simulation blog

  • Tools, Software and IDEs blog

Tags
  • Endpoint AI
  • Machine Learning (ML)
  • Ethos-U65
Actions
  • RSS
  • More
  • Cancel
Related blog posts
Related forum threads

Arm Ethos-U65: Powering innovation in a new world of AI devices

Tanuj Arora
Tanuj Arora
October 19, 2020
4 minute read time.

There is an explosion of edge and endpoint artificial intelligence (AI) happening in the world today, having a significant effect on our everyday lives. AI technologies are being used in public safety, improving the retail experience, transportation and a large and rapidly growing list of other scenarios. Consumer applications from smart door locks to home electronics are revolutionized by AI. In infrastructure AI is being used in a growing number of areas such as data plane optimization and power management. AI truly is, going everywhere.

Earlier this year, we announced new additions to Arm’s Machine Learning (ML) portfolio to enable extremely low-power machine learning inference at the endpoint. The combination of the Arm Cortex-M55 CPU and the world’s first microNPU (Neural Processing Unit), the Arm Ethos-U55 NPU, provides an enormous 480x increase in ML performance over previous generation Cortex-M CPUs alone. This combination enables devices to run neural network inference on endpoint devices and not send massive amounts of data to the cloud. Keeping the data on device not only makes these systems more responsive but also more reliable, secure, and private. The success of the Ethos-U55 makes me confident that we are rapidly moving towards an exciting future with unprecedented AI developments on devices.

Expanding AI into new devices with the new Arm Ethos-U65

To enable even more innovation and expand AI applications into more devices, we have announced the latest addition to our Ethos product line. The Arm Ethos-U65 microNPU which provides neural network acceleration in high-performance embedded devices and subsystems. The Ethos-U65 maintains the power efficiency of the Ethos-U55, while extending its applicability to Arm Cortex-A, Cortex-R, and Arm Neoverse based systems. Although Ethos-U65 can be used alongside any Cortex-A or Neoverse CPU, it is particularly well suited to Arm CPUs with advanced vector capabilities, like the Arm Cortex-A55.

This is the chip diagram for Ethos-U65

Micro size, mega efficiency

The Ethos-U55 introduced our first microNPU architecture, which allows acceleration of neural networks in extremely low-area and with low-power consumption. Adding to the success of the Ethos-U55, and maintaining a focus on power efficiency, the Ethos-U65 extends its applicability to Arm Cortex-A and Neoverse based systems. The Ethos-U65 provides a new, higher performance point for more demanding applications, achieving 1 TOPs. This enables new capabilities in devices such as high-resolution smart cameras, smart home solutions, and even infrastructure applications such as bandwidth and power management subsystems.

Graph: Powering innovation in a new world of AI applications

Extended network support

Fundamental to the Ethos-U product line’s design is its native support for ML operators, giving it the ability to execute the most popular neural networks completely on the NPU with no operator fallback to the CPU. For Ethos-U65, this operator support has been further updated and expanded. Nonetheless, where fallback to a CPU is still necessary, these operators are typically still accelerated through highly optimized software from Arm such as Arm Compute Library or CMSIS-NN.

Network support in Ethos-U

Ethos-U65 provides two different configurations of 256 and 512 MACs/cycle. It includes a dual AXI which delivers better bandwidth for weight bound networks.

  • SRAM (128-bit AXI) and DRAM (128-bit AXI) support. A-class memory system support

For these more complex systems, this results in an average increase in network performance (infs/sec) of 150% over Ethos-U55.

Choose the right system for your ML requirements

Ethos-U65 is designed for use with DRAM based systems, which leads to higher bandwidth availability. This allows Ethos-U65 to be used with more classes of embedded systems:

  • Higher Performance Cortex-A and Neoverse based SoCs OR
  • Low power Cortex-M based SoCs for battery powered devices (with or without DRAM)

Different systems applicable to Ethos-U

While using Ethos-U65 in a Cortex-A or Neoverse based system, the software flow remains the same as Ethos-U55, using TFLmicro runtime. The TFLmicro stack runs on a companion Cortex-M processor next to the NPU and handles any offload operators that cannot be executed on the microNPU itself.

A significant advantage of Ethos-U65 is enabling the execution of networks of any size. With the Ethos-U65. it is possible to efficiently accelerate much larger networks, enabling applications like real time object detection, classification, and recognition, to name a few.

Additionally, with Ethos-U65, any investments made building ML applications on Arm processors is not lost and remains reusable, as Ethos-U65 uses the same software and tools as Ethos-U55. Along with unified software and tools, Arm Ethos-U microNPUs are supported by a rich and vibrant ecosystem of partners providing a wide variety of solutions across audio and vision use-cases including speech recognition, image classification and object detection.

Accelerating AI capability into edge and endpoint devices

Providing more AI capability into edge and endpoint devices opens the door to un-imaginable innovation, creativity and efficiency, leading to amazing future products. A recent article I read claimed, ‘AI will be as transformative as electricity’. For me, AI is a once in a generation change in computing that transforms everything from cloud servers, to smart home solutions through to the tiniest IoT devices. The opportunities are massive, the market is open to everyone and I cannot wait to see what applications are enabled by the Ethos-U65 NPUs.

Learn more about Ethos-U65

Visit the Arm newsroom blog to read more about today's announcement and our technology partnership with NXP on the Ethos-U65.

Anonymous
AI and ML blog
  • Reviewing different Neural Network Models for Multi-Agent games on Arm using Unity

    Sofia Jegnell
    Sofia Jegnell
    During the Game Developer Conference (GDC) in March 2023, we showcased our multi-agent demo called Candy Clash, which is a mobile game containing 100 intelligent agents.
    • September 11, 2023
  • Part 1: Build dynamic and engaging mobile games with multi-agent reinforcement learning

    Koki Mitsunami
    Koki Mitsunami
    Using the Unity ML-Agents Toolkit, we developed Candy Clash demo, the multi-agent system, where dozens of rabbit characters work as a team, aiming to crack their opponent's egg.
    • August 16, 2023
  • Benefit of pruning and clustering a neural network for before deploying on Arm Ethos-U NPU

    George Gekov
    George Gekov
    How to design a neural network so that the compiler is able to efficiently compress the weights of your ML model and benefit fully from the capabilities of the Ethos-U hardware?
    • July 22, 2023