In late July 2024, the Abdus Salam International Center for Theoretical Physics (ICTP) teamed up with IBM, Harvard’s John A. Paulson School of Engineering and Applied Sciences, Columbia University’s Barnard College, and UNIFEI Brazil to host the TinyML for Sustainable Development workshop at IBM’s São Paulo headquarters.
The event, supported by the Arm Developer Program, convened a diverse group of researchers, industry professionals, and sustainability experts from across Latin America, focusing on cutting-edge innovations at the intersection of TinyML, Edge AI, and environmental sustainability. Participants engaged in collaborative discussions and hands-on sessions exploring real-world applications of AI on the Edge to address pressing global challenges.
The event showcased cutting-edge research while also highlighting how TinyML is being used to address critical environmental and societal challenges in the region. Featuring a diverse range of presentations and a hands-on challenge. The workshop provided participants with the opportunity to engage directly with innovative solutions and contribute to the ongoing dialog on sustainable technology.
TinyML first pioneered the optimization and deployment of CNNs to the edge, laying the groundwork for running AI models on resource-constrained devices. As the field evolves, it's natural for TinyML to also evolve to encompass the broader spectrum of edge AI solutions and for concepts for example quantization and low bit precision pioneered in TinyML to extend to Transformer architectures which are at the heart of generative AI.
In this way, TinyML has contributed towards widening the focus of sustainable AI at the edge and this evolution is driving the adaptation of Transformers alongside CNNs at the edge. This will create in a new wave of AI capabilities.
Evegeni Gousev, the TinyML Foundations Board Chairman provided an update on the future of TinyML reflecting the exciting road ahead in line with these developments which will create a new era of TinyML 2.0 where we will see range of AI solutions from discriminative to generative living side by side at the edge. This will require compute that is able to provide real time model performance required at the edge while also consuming less power than traditional acceleration.
Arm CPUs, known for their balance of compute power and energy efficiency, have been central to tinyML's success from the beginning starting on Cortex M. With the introduction of the KleidiAI libraries, it is now possible to optimize a wider range of use cases, enabling more sophisticated AI applications including to run efficiently at the edge expanding AI inference to Cortex A and Cortex X CPU cores found at the edge.
Arm Kleidi refers to a set of specialized open-source libraries designed by Arm to optimize and accelerate certain types of computational tasks on Arm CPUs. The Kleidi family includes KleidiCV, which focuses on image processing, and KleidiAI, which enhances AI-related workloads.
There is increasing focus on the energy requirements of AI, and it makes sense to provide a way for Large Language Models to run more efficiently on lower power edge hardware.
Keeping with this philosophy of TinyML and Sustainability and growing span of TinyML and Edge AI, the audience was provided with an overview of KleidiAI for edge based MPU deployments of Transformers and the Cortex M85 as an advanced MCU target for deployments of CNNs. The Cortex M85 sits at the top end of the MCU portfolio provided by Arm superseding the Cortex M7 which has powered the most power MCU solutions for many years.
With a thorough coverage of Arm capabilities, it was encouraging to also see local development taking place on Arm specifically at the University of Sao Paulo where Marcelo Zuffo has built an impressive ecosystem that has culminated in the Caninos Loucos Program.
Further highlights included the unveiling of the Seeed SenseCap Watcher also powered by Arm Cortex-M and the Ethos-U NPU. The Seeed SenseCap watcher was presented by Eric Pan the CEO and founder of Seeed Studio and is a physical AI agent that is the first of its kind and will unlock many new use cases ideas in the space of Edge AI. The device runs local AI models on the low power Arm Cortex-M55 Ethos-U55 NPU and has a complementary cloud backend including LLM’s and a supporting Android App that has a low code conversational feature for creating end to end applications.
The workshop culminated in a hands-on set of exercises and workshops led by David Cuartielles the Co-Founder of Arduino and Prof Marcelo Rovai of Unifei. The Nicla Vision was used to demonstrate the power of computer vision and audio AI solutions at the edge on Arduino’s most compact and feature packed device to date. The Nicla Vision is powered a dual core MCU that both an Arm Cortex-M7 and Cortex-M4 high performance cores.
It was clear to see that Arm is ubiquitous in terms of compute in the Edge AI space especially where low power and performance is key.
Sustainability will become increasingly important as AI continues to expand its role in society. Arm is perfectly suited to this, offering a low-power solution that doesn’t compromise on performance, which will be essential in reducing carbon emissions.
Special thanks goes to Marcelo Rovai, Marco Zennaro, David Cuartielles, José Alberto Ferreira Filho, Rodrigo Neumann Barros Ferreira, Eric Pann, Evegeni Gousev, Vijay Janapa Reddi, Flavio du Pin Calmon, Pete Bernard Jeremy Ellis and Brian Plancher as well as sponsors IBM,Unifei, ICTP the tinyML Foundation, Harvard, Barnard College( Columbia) and tinyML4D for making this happen. Next year a follow up workshop will be held continuing.
Read about TinyML Education Read about TinyML Foundation Join the Arm Developer Program