Software Optimization: Using SIMD on modern architectures

On November 29th, 2024, VectorCamp, in collaboration with the ACM Student Chapter of the University of Patras, hosted an engaging event for undergraduate and graduate students, as well as the broader University community.

Hosted at the Computer Engineering & Informatics Department of the University of Patras, the event was divided into two main parts:

• Presentations: Topics included Software Optimization and an exploration of leveraging large language models (LLMs) for SIMD optimization
• Workshop: A hands-on session focused on SIMD in IoT technologies, using Raspberry Pi Zero 2 W devices

The event drew over 75 participants, including students and faculty members. This initiative was proudly supported by the Arm Developer Program.

Speeches:

The event started with a presentation on the Arm Developer Program, providing an overview of its benefits and tools, such as the Arm Developer Hub. The speaker emphasized the program’s value for attendees, encouraging them to sign up and join the community to take advantage of the free resources available.

After this introduction, the event continued with the first speech, focusing on Software Optimization and SIMD (Single Instruction, Multiple Data). The speaker discussed the importance of efficient software design in modern computing, explaining how SIMD can accelerate tasks by performing operations on multiple data points simultaneously. Key topics included:

• Code profiling: Understanding its importance and how it guides optimization efforts.
• Algorithm and Data Structure Selection: Recognizing when to revise these for better performance.
• Generic Code Optimizations: Techniques like loop unrolling and cache prefetching to improve execution speed and efficiency.

The second speech highlighted the use of large language models (LLMs) for SIMD optimization. The speaker showcased VectorCamp’s projects SIMD.info and SIMD.ai and discussed how developers can train LLMs to analyze, refactor, and create efficient code snippets. Key points covered included:

• Why LLMs: Their value for code generation, optimization, and problem-solving
• Data Engineering: The importance of curating high-quality datasets for training
• System Prompting: Crafting effective prompts to guide LLMs for precise outputs
• Approaching and Evaluating LLMs: Integrating LLMs into workflows and measuring performance

Workshop: SIMD in IoT technologies

The second part of the event was a hands-on workshop that is focused on applying SIMD in IoT technologies.

Participants were divided into teams of three, each provided a Raspberry Pi Zero 2 W as the primary hardware for the session. The Raspberry Pi Zero 2 W, chosen for its NEON engine support, allowed teams to experiment with SIMD optimization.

The challenge was to optimize a pre-provided scalar code using NEON intrinsics, requiring participants to leverage their understanding of SIMD principles in the context of low-power IoT devices. Throughout the session, teams engaged in collaborative problem-solving, debugging, and testing their optimized code.

The workshop concluded with a performance evaluation to identify the most efficient solutions. The top three teams received prizes, adding a competitive and motivational element to the workshop.

This practical session highlighted the importance of resource-efficient programming in IoT, a key skill in this rapidly growing field.

Event conclusion

VectorCamp, in collaboration with the ACM Student Chapter of the University of Patras, successfully hosted an engaging event that bridged the gap between theoretical knowledge and practical application in the fields of software optimization and SIMD.

The event provided participants with insights into modern software design techniques and the role of LLMs in optimization. It also offered hands-on experience through a workshop on SIMD in IoT technologies using Raspberry Pi Zero 2 W devices.

The support from the Arm Developer Program highlighted the importance of industry collaboration in fostering innovation. The event drew over 75 participants, including students and faculty members, making it a valuable opportunity for learning, networking, and advancing expertize in cutting-edge technologies.

Join the Arm Developer Program