In my role at Arm, I talk with a wide variety of partners across the Arm ecosystem. This is often about the key technology trends they are seeing and challenges they are facing day-to-day. From these discussions, I am pleased to hear about the value of Arm’s support to the ecosystem, often through our tools, insights and technical documents. However, I’m conscious that we cannot stand still. New technical challenges are faced by our ecosystem on a daily basis. Particularly as our partners push the boundaries of what’s possible through Arm’s technologies to create the best possible user experiences on consumer devices. This is something that is front-of-mind for Arm in the run-up to our first ever DevSummit event.
In this blog, I set out just some of the top trends that we are hearing about from our ecosystem and what Arm is doing to continue to support the ecosystem in their endeavours.
I’m always struck by the consistent need from our ecosystem to squeeze as much performance out of devices as possible. At Arm, we not only release our Mobile IP every year, which improves performance year-on-year. However, we also understand that analysis, tools and software, along with the silicon IP, are essential ingredients in the overall performance calculation. This is particularly applicable to our Mali GPUs where small optimizations can make a big difference to the quality of games.
A big trend we’re noticing is PC and console games being ported to mobile. The mobile device has evolved to a powerful machine capable of console-like gaming experiences. However, these console games need to have their assets – such as the graphics, images and content in a game which are described in our artist best practice guides – optimized for mobile. This is important because sometimes a single PC-quality asset which has been naively ported to mobile can use half the graphics budget in a scene, even if that asset was just an unimportant piece of background imagery. Optimizations help by enabling a similar level of fidelity, but staying within the budgets of mobile. This will be even more crucial when enabling cross-platform deployment, where users can move seamlessly between console and mobile while playing the same game.
Mobile gaming
As well as the need for asset optimization guidance, there can be a lack of insight and understanding around memory bandwidth, which affects battery use. On mobile we care about bandwidth a lot! Not only is the memory bandwidth on mobile devices more restricted than in PC and console, using that bandwidth means more battery consumption and less gaming time on the go. This brings me nicely onto my next point…
One of the key reasons why consumers stop playing games on mobile is that their battery eventually dies (often after four hours or less). Our ecosystem partners are keen to see greater insights around battery usage, so they can optimize their content (often gaming) effectively. The need for optimization is not restricted to the GPU or the CPU. For example, geolocation and multi-use play require intensive use of GPS and mobile data, which are significant consumers of battery power.
Looking to the future, AR wearables that sit on the user’s temple will provide thermal challenges due to the size and location of devices. The slightest overheat will be immediately felt by the user. Therefore, any power savings will be a big win for our ecosystem. Optimizations around memory usage will be important for these future devices.
The overwhelming feedback from our ecosystem is that they love metrics and insights – the more meaningful the better, in fact. Greater visibility around driver memory, battery consumption and how High Dynamic Range (HDR) impacts power consumption are all in demand. Metrics shown through on-screen displays appear to be most helpful, particularly for developers when they are building content. The overall idea of seeing metric deltas quickly and in real-time is a big positive for developers. It prevents them from rebuilding, resending and doing further permutations for testing. This enables a more efficient development process and provide a quicker time-to-market for applications. Tools are one way to provide these insights and optimizations….
Optimized gaming experience
The main use for tools is heavily optimizing assets and making this process as efficient as possible – working straight “out of the box” without any extra drivers. Going back to the first point of “every little helps”, being able to squeeze optimization with good debugging tools allows developers to see what they can do better to boost performance further. In addition, any tools providing better run-time, diagnostic and telemetry stats are helpful for developers when identifying performance bottlenecks.
This is something that we aim to support at Arm. For example, Performance Advisor is an Arm tool that generates easy-to-read performance analysis reports. This allows developers to understand how different workloads function across the different technologies. It also identifies where bottlenecks occur within the system based on the rich technical performance data gathered from platforms based on Arm’s Cortex CPUs and Mali GPUs. Performance Advisor is part of Arm Mobile Studio 2020, an Arm product that offers the deepest insights across all of Arm’s technologies on Android devices.
Moreover, we are involved with different partnerships for tools that are designed to boost performance. This includes Google’s Android GPU Inspector (AGI), a tool designed to help game studios squeeze every ounce of performance from GPUs. Also, as part of our ongoing partnership with Unity, we are working to ensure that Arm and Unity profiling tools work well together on a broad range of mobile devices. All these insights and tools will be important when designing the immersive experiences of the future….
It is likely that there will be future innovation challenges for the ecosystem. This is particularly true for Augmented Reality (AR), Virtual Reality (VR), Computer Vision (CV) and Machine Learning (ML), which all demand a lot more additional compute.
A future VR experience
End-to-end security is another consideration, particularly when you consider the AR wearables of the future that could have ‘always-on’ cameras and sensors. Privacy concerns mean that as much data as possible needs to be processed only on the device, and that this data is held securely. The more data that can be securely processed on the device, the less needs to be transmitted to the cloud. However, this will require yet more compute power. Therefore, we need to think about how we enable these future compute capabilities. This is where Total Compute comes into play….
Arm Total Compute is a paradigm shift in how Arm thinks about and designs its IP. It takes a holistic, solution-focused approach to SoC design, moving beyond individual IP elements to designing and optimizing the system as a whole. This will ensure that tomorrow’s devices will be fully capable of processing the advanced, complex and demanding multi-domain workloads of the future. Alongside performance improvements, it will enable more robust device security than ever before.
Most importantly, Total Compute will enable developers to access and use all of a device’s performance and security features through their familiar developer process and environment. This allows developers to design and build more sophisticated, efficient and cost-effective solutions that meet future market and device requirements. It ensures that the features in Arm’s technologies are supported and exposed throughout the software ecosystem. Finally, it provides developers with frameworks for programming, debugging, and analyzing across all of Arm’s IP.
As I mentioned at the beginning, our role at Arm is to help and support our ecosystem so they can continue to innovate and create exciting and compelling content. While pain-points and challenges are evident, we believe that our tools, insights and guides are helping the ecosystem reach their objectives. However, we are very aware that the ecosystem still demands more – more performance optimizations, more insights and more tools – so they can provide the best possible user experiences. This is something we are committed to providing at Arm, now and in the future. Our new Total Compute approach will help to accelerate this support even further, particularly for developers.
We recommend attending Arm’s first ever virtual DevSummit between 6th and 8th October for more information about our work with the Arm ecosystem and wider developer community.
[CTAToken URL = "https://devsummit.arm.com/" target="_blank" text="Register for DevSummit" class ="green"]