Following the launch of Arm’s first ever Total Compute solutions in May 2021, we are exploring how the different Premium, Performance and Efficiency solutions can be applied to different consumer device segments. We have already explored the smartphone, laptop and home device segments, but in this blog we are outlining the different possible Total Compute solutions for the XR (eXtended Reality) wearables market.
The biggest recent development in relation to the XR market is the introduction of the metaverse. Described as the ‘next evolution of the Internet’, the metaverse will converge real, digital, and virtual worlds into new realities where people will be able to do almost anything: get-togethers with friends and family, learning, working, business meetings, shopping, creating, gaming and entirely new experiences yet to be imagined.
As described in this Blueprint article, Arm is the gateway to the metaverse. XR via Arm-powered gateway consumer devices like standalone XR wearables – be they VR headsets or future AR smartglasses – will facilitate the extraordinary new experiences in the metaverse.
XR encompasses augmented reality (AR), virtual reality (VR) and mixed reality (MR). While all three ‘realities’ share common overlapping features and requirements, each has different purposes and underlying technologies.
AR enhances a user’s view of the real world by overlaying what we see with computer-generated information. In the future, holistic AR experiences will be delivered through wearable smartglasses. These devices must combine an ultra-low-power processor with multiple sensors including depth perception and tracking, all within a form factor that is light and comfortable enough to wear for long periods. We have already seen several new smartglasses models arrive in 2021, including the Spectacle smartglasses from Snap, Meta Ray-Ban, Lenovo ThinkReality A3, and Vuzix’s Next Gen Smart Glass.
VR completely replaces a user’s view, immersing them within a computer-generated virtual environment. VR-headset wearable devices, like the Arm-powered Oculus Quest and Quest 2, have been on the market for the past five years. These devices are often used for entertainment experiences, such as gaming, concerts, films, or sports, but are also moving into the social domain, like virtual meet-ups, as shown through the RecRoom VR platform shown below.
MR sits somewhere between AR and VR, as it merges the real and virtual worlds through superimposing virtual objects, characters or instructions into real-world environments, or even vice versa. The most prominent MR wearable device is the Arm-powered Microsoft HoloLens 2. Virtual instruction sets and graphics are superimposed into the user’s view to assist enterprise and industrial applications.
The level of compute performance and efficiency in XR wearable devices will vary based on the type of XR wearable and the complexity of use cases it is designed to enable. High-performance compute will be needed in contemporary standalone VR headsets used primarily for high-end gaming experiences, such as the Oculus Quest devices, whereas high efficiency will be needed for smaller, lighter devices like AR smartglasses. At the same time, there are also devices that sit between these two spectrums of performance and efficiency, such as MR headsets like the HoloLens 2, and tethered VR devices for consumer markets like the HP Reverb G2. Therefore, system on chip (SoC) solutions will need to be able to scale to fit the different use cases, workloads, and form factors. This is where the Arm Total Compute solutions can help.
Arm’s Total Compute solutions offer a full suite of scalable hardware IP (including the latest Armv9 CPUs, Mali GPUs and System IP), physical IP, software, tools, and standards to build the best SoC across different consumer device markets. These solutions offer different configurations for the specialized compute requirements that the range of XR wearable devices demand, from the high-performance VR and MR headsets to the ultra-efficient AR smartglasses.
The configurations for the high-performance VR and MR headsets represent our ‘performance’ Total Compute solutions. There are two possible configurations here. Firstly, the 1+3+4 CPU configuration of 1x Arm Cortex-X2, 3x Arm Cortex-A710 and 4x Arm Cortex-A510 for high performance. For slightly lower performance but higher efficiency, there is the 4+4 CPU configuration of 4x Cortex-A710 and 4x Cortex-A510. Both solutions can utilise the premium Arm Mali-G710 GPU or mainstream Arm Mali-G510 GPU for high-quality graphics.
Meanwhile, the ‘efficiency’ Total Compute solutions for the lightweight AR smartglasses can either utilise the 2+6 CPU configuration of 2x Cortex-A710 and 6x Cortex-A510, or even 4x Cortex-A510 for ultra-efficiency. These solutions can be supported by the ultra-efficient Arm Mali-G310 GPU.
As with all Total Compute solutions, underpinning the CPUs and GPUs in our Total Compute solutions for XR wearables is our System IP – CoreLink Interconnect CI-700 and NI-700. Both interconnect technologies provide improved energy efficiency and system performance to add further improvements across any Total Compute solution. All of the CPU configurations in our Total Compute solutions are bound together by Arm’s DSU-110, which is the backbone of these Armv9 based Cortex CPU cluster. This enables our partners to address different types of Arm-based XR wearables with different trade-offs in performance and efficiency.
The high-performance VR and MR headsets can pack in at least one Cortex-X2 into the Total Compute solution. Our Cortex-X CPUs really push peak performance requirements, which can be important for compute-intensive XR workloads like gaming and other entertainment experiences on VR and MR headsets. This drive for peak performance is complemented by the energy efficiency improvements of Cortex-A710, which enables sustained performance to maximise battery life across all XR wearable devices. This is a very important consideration for XR wearable devices that are increasingly untethered route for greater user mobility. This push for energy efficiency is further supported by Cortex-A510 that enables yet longer playing experiences.
For XR wearables where efficiency is paramount – like AR smartglasses – Cortex-A710 and Cortex-A510 CPUs will only be utilized in more efficiency-based Total Compute solutions. Not only does Cortex-A510 boost power efficiency by up to 20 percent through the 3-wide in-order design, but it also provides industry-leading area efficiency. This is very important in small, lightweight XR wearables where area matters!
An innovation that makes this possible is merged core microarchitecture. This allows two Cortex-A510 CPUs to be grouped into a complex, with multiple complexes per CPU cluster. The result is increased area efficiency at a higher performance point. This push for performance combined with high area efficiency is perfect for lightweight AR smartglasses.
The different XR wearable devices are supported by different GPUs. The premium Mali-G710 can be incorporated into Total Compute solutions for high-performance XR headsets. It improves the feature coverage of index-driven vertex shading (IDVS), a feature commonly used in XR use cases, as it renders a per-eye view with subtly different object positions in each.
Mali-G510 can also be incorporated into high performing XR wearable devices, offering the perfect balance of performance and efficiency. It delivers a 100 percent performance improvement and 22 percent energy savings for longer battery life over the previous generation Arm Mali-G57 GPU.
Compared to Mali-G710’s 7 to 16 configurable shader cores, Mali-G510 is capable of 2 to 6. However, it still incorporates many of the Mali-G710’s premium features, like command stream frontend (CSF), the redesigned and additional executional engine, and redesigned texture unit, to deliver high-quality graphics experiences. Then, on top, it provides formats for better HDR support, Arm Frame Buffer Compression (AFBC) uncompressed buffers and the new Arm Fixed Rate Compression (AFRC) for bandwidth reductions. AFRC is a particularly neat feature for XR wearables, as it guarantees a bandwidth and memory footprint reduction at a minimum area cost. This translates into performance uplifts and energy savings.
Mali-G310 is designed to deliver the highest possible performance at the smallest area cost, making it perfect for Total Compute solutions targeting AR smartglasses. It is our highest performing ultra-efficient GPU ever, with huge performance uplifts compared to the previous generation Arm Mali-G31 GPU across three performance areas – texturing performance (6x), Vulkan performance (4.5x) and Android UI content (2x). Like Mali-G510, Mali-G310 also offers formats for better HDR support and AFBC uncompressed buffers, but AFRC is only optional. However, the GPU does offer foveated rendering – a rendering technique which uses an eye tracker integrated with XR wearable devices to reduce the rendering workload – for an AR and VR boost.
To learn more about our Total Compute solutions, watch this new video about how they achieve accelerated performance growth across key compute workloads and use cases, and all consumer devices.
You can also learn more about Arm’s XR solutions by visit the new AR, VR and XR solutions webpage on Arm.com here.
[CTAToken URL = "https://www.youtube.com/watch?v=hGXSmj2_g7g" target="_blank" text="Learn more about Total Compute" class ="green"]