Our everyday lives generate huge amounts of data and information – digital, biological, physical, and sensorial. With advances in AI, this data can be used to create incredible benefits for humankind. To realize this challenge and extract useful information, we need to be able to process this data when it is generated, where it generated. At Arm we are on a mission to enable machine learning (ML) on-device, allowing data to be processed, analyzed, and utilized in the real world. This helps provide several advantages for consumers – from enhanced security and privacy to more reliability and responsiveness.
With Arm, ML workloads will run on the Arm Cortex-A CPU – the world’s foremost ML processor, deployed in almost every smartphones in the world, and a vast array of other devices. However, ML can benefit from having a specialized Neural Processing Unit (NPU) which delivers an exponential improvement in performance and efficiency. An NPU’s processing capability enables the development of exciting new applications to enable true digital immersion. An example of this is a shopping application, which places virtual objects in a physical space or smart home-hubs with augmented reality (AR) story-times for kids. Beyond smartphones, NPUs can also enable a wide range of devices such as life-saving smart baby cameras which can monitor a baby’s breathing or temperature.
The Arm Ethos-N78 NPU is Arm’s highly scalable and efficient second-generation NPU, delivering ML on-device, and building on the success of the Arm Ethos-N77 NPU. The Ethos-N78 NPU is available in a range from 1 TOP/s to 10 TOP/s and supports a wide array of configurability.
Supporting over 90 unique configurations and allowing partners to configure the MACs, SRAM size and vector capability, Ethos-N78 provides unprecedented flexibility to our silicon partners. Our flexibility ensures partners can fine-tune their design to meet the optimum balance of performance, power, and area. In addition, the Ethos-N78 can be implemented across a wide range of devices with complete and transparent software compatibility and portability.
Unprecedented flexibility for SoC architects
The Ethos-N78 provides up to 30% more area efficiency than the previous generation, allowing partners to achieve more in less silicon area. While silicon area is an important cost metric, DRAM bandwidth is an equally precious resource in electronic systems. The Ethos-N78 has been specifically designed to use less DRAM bandwidth, consuming up to 40% less DRAM data per inference, allowing our partners to implement ML using less memory, further reducing system power and costs. The Ethos-N78 allows extensive use of ML in software applications while still ensuring long battery life.
Advances in performance and efficiency
Performant hardware is one part of the ML equation. It is equally important to have an efficient software stack for developers to deploy their chosen ML networks on the target hardware. The Ethos-N78 software stack provides a choice of two flows – an offline compilation flow based on the TVM compiler and an interpreted (on device or online) flow for use with Android NN API based on Arm NN. The offline and online flows work unified across all target Arm hardware IP (CPU, GPU, NPU) enabling a strategy of write once and deploy everywhere. Supporting all popular frameworks including TensorFlow, TensorFlow Lite, PyTorch and ONNX, among others, developers can continue working with their favorite framework.
The Ethos-N NPUs are supported by the Ethos-N Static Performance Analyzer tool ensuring developers can profile and tune their networks on the Ethos-N NPU well before availability of silicon and significantly reduce time-to-market.
Unified software stack for developers
The diverse range of applications and devices using ML today require NPUs to be highly flexible and adaptable solutions to satisfy the wide variety of requirements. Use of on-device ML has advanced a long way in the last few years. Initially, on-device ML was adopted in mobile phones for functions that include face unlock and voice user interface. However, today we find ML usage expanding to cover new innovative uses – from delivering stunning photographs to cool AR-based applications. Beyond mobile, we find ML being extensively used in applications including HD security cameras, smart home-hubs, and DTV to deliver new features and user experiences. The Ethos-N78 NPU with its unmatched flexibility and advancements in performance and power-efficiency enables our partners to unleash the potential of ML on-device.
Learn more about Ethos-N78Visit the Arm newsroom blog
Thank you for the opportunity to learn about Cortex+Ethos+N78. ~ Neil Sandage, Here. I would like to know of the 90 ASIC features. And what is possible, while being powered via a bicycle dyno, or cross trainer elliptical. = new engineering standard, elliptical cross trainers power envelops of 1 hours.