In the past, Arm has talked about the enormous potential of augmented reality (AR) and the “limitless possibilities” of this new technology. Back in February 2020, we published a blog setting out the numerous use cases and experiences that will be enabled by AR, from navigation and translation through to ‘superpowers’, such as visual and audio enhancements. While all these experiences are incredibly exciting, not just for consumers but the wider industry as well, we need the technology in place to make it a reality.
Many have pointed to numerous different reasons why consumer AR has not yet kicked off. These reasons include the lack of content, lack of style or comfort of today’s wearable devices, the poor battery life or even the lack of a major brand name device. These are all important factors; however, one major consideration that is often over-looked is the display. I believe that the true potential of AR will not be realized until we unlock many of the challenges for the display technology, especially the need for realistic depth perception. This can be enabled by holographic display, achieved with Computer-Generated Holography (CGH).
The AR smartglasses of the future
Depth perception is crucial for the human vision system to interpret what we see in the world. There are around 18 depth cues that help you determine distances between objects and their relationship to you, and each other. Your brain makes sense of these depth cues in the real world to provide an understanding about your surroundings. Should the depth cues provide conflicting information, then it can lead to confusion or even nausea.
Most of the AR wearables available today, such as smartglasses or head-mounted displays (HMDs) display digital images at one or, at best, two depth planes. It means that AR projections can often be mismatched with the real, three-dimensional world. If you look at the following illustration, images on the left show what today’s experience of using an AR wearable might be like. The bowl of apples is supposed to be placed on the sofa in the background, but it appears sharp, even if the viewer (in this case the camera) is focused on the near field. Similarly, the green virtual apple remains sharp even when we focus on the back field. To render this virtual scene correctly, we need a display technology capable of projecting multiple depth planes. Just like in the images on the right where the virtual apple and the bowl appear correctly in or out of focus depending on where we look, adapting to what our visual system expects.
Holographic projections generated using VividQ software, simulating scenarios where all virtual images are generated on a) one depth-plane and b) multiple depth-planes to match the real environment.
Holographic displays can help to solve this issue. In fact, Arm is working with British technology start-up VividQ to enable holographic displays for richer and more comfortable AR experiences through future AR wearables, such as smartglasses.
Holographic display is the process of engineering light to project three-dimensional, virtual scenes that possess a natural depth of field (as illustrated in the right previous image). This means highly realistic AR content that integrates seamlessly with the real world, while eliminating user discomfort. Holographic display is also brighter and more power efficient than other AR display solutions, making it a viable technology for not only the next generation AR wearables, but also automotive Head-Up Displays.
CGH is the method of creating holograms digitally from various 3D data sources, from game engines to depth-sensing camera output. Each hologram is encoded with all necessary information required to display a three-dimensional scene, with depth, to the viewer. The challenge for the adoption of CGH used to be the high compute requirement, which meant that holographic display was not a viable technology for consumer electronics. However, VividQ’s software for holographic display overcomes the computational challenges traditionally associated with CGH and allows for real-time generation of holograms for AR applications on commercially available GPUs.
Arm is collaborating with VividQ to enable holographic display on a mobile device by making VividQ’s software compatible with Arm Mali GPUs. The collaboration will enable the integration of holographic display into future AR wearables. According to a report by ABI Research, shipments of smartglasses are expected to reach 20 million units by 2024. Due to the ubiquity of Arm IP among mass market consumer electronics, from smartphones to DTVs, this is seen as a vital collaboration to ensure the accelerated adoption and scalability of holographic display. Therefore, this partnership could help to bring improved AR experiences to millions of consumers worldwide.
Tom Durrant, Co-Founder and Chief Development Officer of VividQ says:
“Technologies, such as navigation assistance, messaging and entertainment, are now irreplaceable and becoming increasingly integrated into our lives. The migration to AR wearables and next-generation consumer electronics makes these applications more natural to use, and calls for a correspondingly natural, immersive viewing experience. VividQ provides this with CGH. We are delighted to be working with Arm towards this ambitious goal of integrating CGH in mobile devices. By combining VividQ’s software expertise with Arm’s solutions in high-performance integrated compute, we aim to jump-start the mass consumer adoption of AR with holographic display.”
Our IP technology is already powering today’s leading AR-based devices and experiences. Through valued partnerships with companies like VividQ, we can address current AR challenges, particularly around display, and help to enable the next generation of AR devices. The innovation of VividQ around holographic display and CGH is helping to enhance the AR experience and ultimately enable the widespread adoption and use of AR. With Arm’s IP and support, we want to move these AR technologies to everyday consumer use.
If you are interested to learn more about VividQ and its CGH technology, please visit its website here. If you are an AR or VR developer interested in learning more about developing on Arm you might want to look at our upcoming virtual DevSummit.
Learn more about Arm's AR and VR solutions