The world around us is becoming a whole lot smarter. We hear lots of predictions about how the Internet of Things (IoT), cloud computing, and other exciting technologies will change our lives in the next decades. IoT is enriching and transforming applications throughout our daily lives in industries from home and healthcare to smart cities, retail, and industrial. The application space is diverse with hundreds of sub-segments and thousands of applications. These applications generate a lot of data, but the data itself is not the important thing, it is the value we can extract from it. We cannot rely on the common approach of sending all the data back to servers in the cloud. As the data increases this cannot scale. We need a different solution.
Transferring data from endpoints to the cloud introduces costs including longer latency, data transmission energy, bandwidth, and server capacity which in the end can make or break the value of a use case. This occurs especially in IoT, where there are many applications that rely on data analytics and decision making in real-time and at the lowest latency possible. In fact, sometimes endpoints might only have limited (if any) connectivity. As a result, intelligence has to be distributed across the network to make the right processing capabilities available in the right place, all the way to the endpoint at the source of the data. Increasing the on-device compute capabilities combined with machine learning techniques, has the potential to unleash real-time data analytics in the IoT endpoints – this convergence is also known as “Endpoint AI”.
Endpoint AI is all about making endpoints – even the smallest – smarter and more intelligent, to perform the necessary real-time analytics in the endpoint itself while:
Let's take an example: Imagine a small endpoint device that monitors the behavior of water pipes and aims to detect anomalies, such as pipe leakage. This has enormous value for homeowners but also insurance companies if burst pipes can be detected early enough to avoid damage in a house. Such a device only needs cloud connectivity if a leakage is detected and uses on-device processing capabilities to create value.
However, as discussed previously, what we have seen so far is a “Cloud-first” approach where the intelligence resides in the cloud and the data is constantly transferring along the network. But this is changing to meet power, bandwidth, and cost requirements and thereby enable a wider set of use cases. Using the example of the lead detector above, moving the processing capabilities into the endpoint has transformed battery life from months to years.
For many use cases, the intelligence is best distributed across the network, including all the way to the endpoints. Arm offer solutions for each node in the whole network: from data centers, (edge) servers, and gateways all the way to endpoints.
Arm Enables AI Everywhere, On Any Device
Much of the sensor data generated today is discarded because it is simply too costly in terms of energy and bandwidth to transmit it. Imagine a battery-operated IoT endpoint which is using Bluetooth, NB-IoT, or other means to connect to a network. The endpoint collects data from various sensors and sends the information to a gateway or another device. However, the communication part of this use case consumes often the most amount of power of the whole system. As a result, system architects have to make trade-offs to work within the power constraints of a system. They need to juggle the data rate and processing of sensor data in the endpoint to keep battery life at an acceptable value.
What if we could use machine learning-based approaches to save or send data only if an interesting event occurs, thereby reducing the usage of the radio and optimizing battery life? While the computing resources on these endpoints have historically been too small to support signal processing and machine learning on the device, Endpoint AI now makes this possible.
In fact, there are many applications which use ML techniques in Arm Cortex-M-based devices already today. This is driven by a community effort called tinyML backed by many leading organizations, including Arm, with the goal to deploy machine learning on MCUs.
The tinyML foundation is helping to drive the intersection between the traditional embedded world and the new world of Endpoint AI. Thanks to this effort, it is now possible for more developers to run increasingly complex deep learning models directly on microcontrollers, accelerating Endpoint AI and unleashing new use cases. A quick glance under the hood shows that this is widely based on Arm IP. Cortex-M processors provide the right feature set to address a wide range of existing use cases.
Audio Analytic, for example, are using an Arm Cortex-M0+ processor to implement sound recognition. Other examples use the DSP compute capabilities of Cortex-M4, Cortex-M7, and Cortex-M33:
We are at the beginning of an exciting period in which new technology enables new capabilities and makes more demanding use cases in IoT endpoints possible. Endpoints are increasingly gathering data, and which could be analyzed energy-efficiently to find patterns and trigger the next step of processing. That is why IoT endpoints and microcontrollers (MCUs) must become even more capable to cope with the increasing requirements.
To support the vision of Endpoint AI, Arm specifically developed Cortex-M55 (Arm’s most AI capable Cortex-M processor) and Ethos-U55 (the industry's first microNPU for Cortex-M) which are the ideal combination to address many emerging use cases. Cortex-M55 paired with Ethos-U55 deliver a combined 480x increase in ML performance over existing Cortex-M processors.
And it is not just about IP. The evolution of Arm CMSIS-DSP and CMSIS-NN libraries and our collaboration with Google’s TensorFlow Lite Micro team will enable developers to easily port their work on previous Cortex-M platforms to Cortex-M55 and Ethos-U55.
Hear from Ian Nappier, Tensor Flow Lite Micro product manager about how Arm and Google are working together to make it easier for developers to deploy endpoint AI.
McKinsey has identified more than 100 use cases across 11 sectors which could create a value of $250 billion in hardware value for edge and endpoint processing by 2025. By analyzing these use cases specifically for Endpoint AI, we can identify that the main use cases align around three areas: Vision, Voice, and Vibration.
We can expect use cases and applications that target any one of the three categories or multiple categories. For example, anomaly detection can be performed using vibration monitoring but also sound recognition techniques or using images.
Endpoint AI now enables the support for signal processing and machine learning on the device. Many applications requiring ML techniques are already unlocked today with Arm Cortex-M based devices. Our new Arm Cortex-M55 and Ethos-U55 will satisfy the next generation of even more demanding endpoint AI-related use cases.
Visit our Endpoint AI website to learn more about the Arm solution.
[CTAToken URL = "https://pages.arm.com/cortex-M55-consultation.html" target="_blank" text="Talk to an expert" class ="green"]