We talked recently about the evolution of mobile gaming, from the earliest days of snake, through the first Mali-powered phone right up to where we are today with the growing popularity of complex, high fidelity games on mobile. It isn't just us seeing this trend either, Google recently agreed that 'a connected trend we’re seeing surfacing in 2017 on mobile ... is the rise of high fidelity games on Google Play'. Of course, these advancements in gaming wouldn’t be possible without the advancements in mobile graphics that have taken place over the last two decades as well. Whilst a high performance GPU should of course be able to support high end gaming, how do we know exactly what performance points and levels of complexity to target with our next generation products? Well, not only do we work closely with our amazing ecosystem of developers and partners to understand exactly what they need from us, but we also do our own homework too.
Our in-house demo team are working continuously to discover, develop and understand the next generation of gaming technology to ensure our GPUs, video and display processors are more than capable of providing an awesome gaming experience. In this blog I’d like to take you through the various demos we’ve produced over the years, considering why we did what we did and what it meant, right up to our most recent demo, CircuitVR, released at GDC 2017. To start with though, let’s head back to 2010 when Lost was finally finishing and Lady Gaga was rocking raw meat at the MTV awards…
Developed on OpenGL ES 2.0, Trueforce was a space shooter demo built in collaboration with our friends at Allegorithmic to show high definition 3D stereoscopic gaming on a mobile device. OpenGL ES 2.0 was developed to enable fully programmable graphics pipeline, adapted for embedded platforms. In terms of today’s quality and complexity, this was pretty simple, with resolution of 1280p x 720p at >30 Hz, 16000 vertices and around 50 draw calls per frame. Even then, the visuals were pretty impressive, especially for a mobile device, and especially when you consider that Angry Birds was not only the most popular mobile game of the time, but also one of the more visually compelling.
Fast-forward two years and you get to 2012 and OpenGL ES 3.0 had just been released. Featuring numerous enhancements to the rendering pipeline, it was able to support far newer and more complex visual effects than its predecessor. Our Timbuktu 2 demo was designed to demonstrate the performance and visual enhancements enabled by OpenGL ES 3.0 and advanced shader techniques like lighting, shadows, dust based on particles, depth of field (DoF) and HDR. The demo mode was interchangeable to show wireframe, colours, textures, lighting, shadow, specular mapping, environment, bloom and depth of field. Powered by the Mali-T604 GPU, it was significantly more complex than Trueforce, with a resolution of 1920 x 1080p at >30 Hz, over 25,000 vertices and around 70 draw calls per frame.
Hauntheim was a haunted house demo with which we wanted to show the ability to incorporate a range of dynamic light sources, which affect the objects in a scene in a natural way, more efficiently through Pixel Local Storage (PLS). Desktop and console games at the time used Multiple Render Targets (MRT) to facilitate deferred lighting but this was the first demo showing the ability to run this entirely on the chip. The removal of the need to constantly call out to external memory greatly reduced the power usage, thereby extending gameplay and enabling greater complexity in the scene. This technique became the reference for what is now possible with Vulkan renderpasses; where Mali GPUs can fuse subpasses together and keep expensive framebuffer bandwidth entirely on-chip. This was an important step in the process of designing a modern API that enables efficient utilization of the mobile GPUs that sit in almost every smartphone available on the market today.
Released at GDC 2015, the Moon Temple demo ran on a Mali-T760 GPU and showed the capability of high fidelity gaming on mobile. Like Hauntheim, Moon Temple also focused on PLS, this time integrated with Epic’s Unreal Engine 4. It demonstrated how this technology could enable advanced techniques like fog particles in a more natural manner, so that they appear to fade away rather than clipping, for example. This demo also showcased Adaptive Scalable Texture Compression (ASTC), covering a broad range of bitrates and component types not covered by other compression formats and achieving a 25% bandwidth saving.
Chances are you’ve heard of this one before. We’ve talked about the creation of the original demo, which is designed to show complex rendering techniques like shadows and reflections based on local cubemaps. but also our experience of porting to VR on Unity. This was a complex demo featuring 80,000 vertices and around 200 draw calls per frame and was ported at a time when Unity had just released native VR support for Oculus Rift and Samsung Gear VR. We were able to demonstrate that even the most complex content could be successfully mapped to VR with only some minor changes. There were some aspects specifically removed, such as animation mode which looks awesome in the original but can feel a little nauseating in a headset. Our close collaboration with Unity means we’re able to try out and optimize the latest techniques and make sure developers can reap the benefits in the world’s most popular game engine.
The Protostar demo was created in collaboration with Samsung and Epic Games for GDC 2016 and represents a major step forward for the industry. For one thing, it was highly complex, with around 400,000 vertices and 400 draw calls per frame. That might look like a typo when compared to the other demos we’ve mentioned but the number of draw calls is almost entirely attributable to the existence of Vulkan. The massive advancements of this latest API supported far greater complexity than previous, and Protostar was one of the very first pieces of advanced content created on it. Not only that, but for the first time, Vulkan’s integration with Unreal Engine 4 meant developers could simply tick a box to run their application on Vulkan. This shows the industry’s growing appreciation of the need to support the ecosystem in developing awesome gaming and applications for mobile.
This year, GDC was all about VR and our stand was no different. We launched our brand new Circuit VR demo, featuring our little robot friend taking you on an immersive tour of all the workings of a smartphone. Mali GPUs currently power around 50% of mobile VR devices and we take this responsibility very seriously, which is why we spent our time building Circuit VR to show some of the latest advances in this still growing industry. In collaboration with SMI’s incredible eye tracking technology, we were able to show the benefits of foveated rendering to reduce the workload on the system by rendering only the area strictly necessary in full resolution. The reduced resolution of the surrounding areas, which we never focus on clearly, saves a huge amount of bandwidth and processing power. We were also able to highlight the benefits of 8x MSAA, the anti-aliasing technique which applies blended pixels around an edge which should appear smooth in order to reduce the visibility of jagged, or aliased, lines.
Of course, in the ARM Mali team the work never stops (OK so sometimes we’re playing games, but it’s all in the name of research, honest…) and we are of course assessing the technologies and advancements ahead of time to ensure our next generation of products provide a better than ever mobile gaming experience. With our latest high performance GPUs like Mali-G71 outstripping even some mainstream laptops in terms of performance, the sky is the limit for mobile gamers.