Arm launched a new class of machine learning (ML) processor, called a microNPU or micro Neural Processing Unit, designed to accelerate ML inference in area-constrained embedded and IoT devices. Arm Ethos-U55 is a first-generation microNPU and when combined with the Arm Cortex-M55 processor it provides a 480x uplift in ML performance over previous generation Cortex-M processors. This unlocks new cases and has the potential to bring Endpoint AI to billions of devices.
Arm launched a new range of Arm ecosystem Fixed Virtual platforms (FVPs) available free of cost and without license control. These FVPs enable execution of full software stacks, providing a widely available platform ahead of silicon.
Learn more about the Arm ecosystem Fixed Virtual Platforms (FVPs) refer to this blog:
An ecosystem FVP based on the Cortex-M55 and Ethos-U55 processors, known as Corstone-300 FVP targeting Endpoint AI applications was recently released. This FVP is a complete Arm system based on the Arm MPS3 platform including Fast Models of the processors, RAMs, peripherals, and debug support.
This blog presents new ML software examples that you can run on the Corstone-300 FVP to help you get started with your software development on the Ethos-U55 right away even before silicon is ready. Before we dive into specifics of the software examples ported to the Corstone-300 FVP, let us go over the software flow for the Ethos-U55 as this is relatively new to most developers.
Software developers can benefit from the processing capability of Ethos-U55 by using the TensorFlow Lite for microcontrollers ML framework, which runs on the application processor (Cortex-M55 in the case of the Corstone-300 FVP). The process starts by training or acquiring a TensorFlow model that is to be accelerated. The model is then quantized to 8-bit integer format and converted to the standard TensorFlow Lite flat file format (.tflite file). The TensorFlow Lite flat file is then input to the NN optimizer tool from Arm which is run offline. The output of the tool is an optimized TensorFlow Lite file which is now ready for deployment on your Ethos-U55 target device or in this case the Corstone-300 FVP.
The NN optimizer tool identifies which ML operators can be executed by the Ethos-U55 microNPU and substitutes these with a sequence of special operations. The few remaining ML operators can be executed on the Cortex-M55 processor either by optimized kernels from the CMSIS-NN library or by fallback support in TensorFlow Lite reference kernels. The Ethos-U55 driver manages the communication and workloads that execute the inferences on the Ethos-U55 microNPU.
We have uploaded a docker project on github with two ML software examples using the TensorFlow Lite micro framework – image classification and micro_speech. The software examples are compiled with Arm Compiler 6.15. Arm Compiler is available as a component of Arm Development Studio 2020.1. If you do not have access to this tool, a free 30-day license is available.
To ease development the docker project includes all the steps to compile and run these examples on the Corstone-300 FVP using the software flow previously described.
The image classification example uses the MobilenetV2-1.0 quantized uint8 neural network model openly available on TensorFlow hub. The pre-trained network model is trained on more than a million images from the ImageNet database and can classify images into 1000 object categories. There are 4 sample input images provided with this example (cat, dog, kimono, and tiger).
The second example is micro_speech. This audio detection example uses a quantized uint8 neural network model and can recognize 2 keywords “yes” and “no” from the input speech data. A sample input speech data file is provided with this example.
You can now run both these neural network models on the Corstone-300 FVP. You can evaluate the performance of the network on Ethos-U55 by viewing the inference cycle count estimation and classification results of the network execution. In this FVP, the timing and in turn performance of the Cortex-M55 and the rest of the system components is not modeled.
With the Corstone FVP, Arm Compiler 6.15 and the software examples already available, you can get started on your ML software development for Arm Ethos-U55 today. Trying out the latest Arm solution and tools for Endpoint AI has never been easier!"
Download ML Software examples