Today we are introducing “Arm Accuracy Super Resolution” (Arm ASR), which is a best-in-class open-source solution for upscaling on mobile devices. This article describes our approach and encourages you to join us on this journey.
Great games take the player on an emotional journey. There are many tools in the game developer kit that can be used to this end: evocative music, imaginative controls schemes, and graphical effects to name but a few. Focusing on graphics, it is breathtaking just how sophisticated they can be in mobile games. Games are rendered to screens that contain millions of pixels and refreshed at rates of 60Hz and higher. It is staggering.
Ambition is endless and modern mobile games push the envelope with complex lighting and shaders to deliver on the developer’s vision. It all comes at a cost, taxing GPUs and drawing larger amounts of power. Even on high-end PCs we recognize that something must give. That is where upscaling techniques come in.
Upscaling works by rendering some stages of the frame at lower resolution and then later applying the technique to scale from lower to higher resolution. It should not be used at every step in rendering a frame, as it can lead to unsightly defects when rendering full screen effects or user interfaces. However, earlier in the pipeline it can be used to great effect and provide anti-aliasing as a part of the process.
Integration of upscaling into the frame
Of the many solutions available, our attention was drawn to AMD’s FidelityFX Super Resolution 2 (FSR2). It is an element of GPUOpen, an open-source project that began in 2016. Working with the permissive MIT license, the intention is to provide portable solutions that ease the burden of addressing the disparities that exist between PC and consoles. These values captured our attention and provoked us to ask ourselves what a good solution for mobile should look like.
Upscalers come in two varieties: spatial and temporal. The former was the first to arrive and become available to developers.
Spatial upscalers work frame-by-frame to generate results, which make them simpler to understand as well as having lower requirements from the game engine that hosts them. Examples of spatial upscalers include FSR1 and Qualcomm’s Game Super Resolution (GSR). The drawback of this technique is that developers cannot be too ambitious in choosing the resolution to render at because this will lead to blurred final images. Conversely they are relatively cheap from a computational standpoint.
Temporal upscalers are more complex, as they combine information that is derived from multiple frames to generate the end result. However, they will, in general, produce higher quality images from lower resolution targets. This comes with additional input requirements from the engine. For example, depth and motion vector information must be provided. A reactive mask is desirable to deal with features that lack depth or motion information, such as particle effects.
We decided to move directly to temporal upscaling to address common graphical performance challenges, while offering benefits to mobile game developers. Our starting point was AMD’s FSR2 upscaler, which produces fine results but with an implementation that is only affordable on PC and high-end consoles. We derived our solution from FSR2, which means that developers will benefit from the familiar API and configuration options.
As part of our investigation, we took the well-known bistro scene. We added more local, overlapping lights and attenuation of the main light to create a realistic computing challenge, which was rendering approximately 2.8 million triangles.
Bistro scene courtesy of NVIDIA and released under Creative Commons CC-BY-4.0 license.
We analyzed results collected from a commercial mobile device that features an Arm Immortalis-G720 GPU and a display resolution of 2800x1260. We were able to demonstrate significant GPU performance improvements.
Framerate Analysis of Native Resolution and Upscaled Performance using Arm ASR, FSR2, FSR1, and GSR.
Equally importantly, using this technique rendered high-quality results at a stable, low temperature. Rendering at a native resolution inevitably led to undesirable thermal throttling, which in games can ruin the user experience and shorten engagement.
The excellent performance of Arm ASR is a combination of highly efficient shader code which reduced both GPU load and bandwidth requirements.
GPU Performance Analysis of FSR2 and Arm ASR
These performance improvements translate into power savings, meaning that users can enjoy longer battery life which is a huge benefit in everyday life. As part of our collaboration with MediaTek, it validated this assertion and provided us with the following results using a Dimensity 9300 handset.
Power Consumption for native full resolution (1080p), Arm ASR quality, balanced and performance upscaling from 540p to 1080p, plus native half resolution (540p).
The Arm game content team have been hard at work producing an Unreal Engine demo that is intended to challenge future mobile GPUs. We could not resist the opportunity to apply Arm ASR to this content.
“Mori” demo, courtesy of the Arm Content Team
As you can see, there are various fine details in this scene. Which is why we support Robust Contrast-Adaptive Sharpening (RCAS) in Arm ASR, just like AMD’s FSR1 and 2. The results speak for themselves:
Enlarged sections of the “Mori” scene.
Arm ASR is now available as an early-access plugin for Epic Games’ Unreal Engine. The video below highlights the benefits of Arm ASR ON, with it also using Unreal Engine’s desktop renderer to bring performance optimization opportunities for developers that would otherwise be beyond their reach.
We are very proud of the results of this work and want to share it with the developer community under an MIT open-source license. This will allow all developers to see the benefits of Arm ASR first-hand and experiment with their own projects. If you would like to become an early adopter, please sign up to the newsletter.
Sign up