Total computing solutions for wearable devices in the XR (Extended Reality) market

With Arm’s first Total Compute solution, we are exploring how different advanced, performance and efficiency solutions can be applied to different consumer device segments. We have already explored the smartphone, laptop and home device segments, but in this blog, we will outline the different Total Compute solutions that could exist for the XR (Extended Reality) wearables market.

XR Market

The biggest recent development related to the XR market is the introduction of the Metaverse. Described as “the next evolution of the internet,” the Metaverse will merge the real, digital, and virtual worlds into a new reality where people can do almost anything: hang out with friends and family, study, work, business meetings, shop, create, play, and entirely new experiences are yet to be imagined.

As described in this roadmap article, Arm is the gateway to the Metaverse. XR through Arm-powered gateway consumer devices such as standalone XR wearables — whether VR headsets or future AR smart glasses — will facilitate extraordinary new experiences in the Metaverse.

XR includes augmented reality (AR), virtual reality (VR), and mixed reality (MR). While all three “realities” share common overlapping features and requirements, each has different purposes and underlying technologies.

AR enhances the user's view of the real world by overlaying what we see with computer-generated information. In the future, the overall AR experience will be delivered through wearable smart glasses. These devices must combine ultra-low-power processors with multiple sensors including depth perception and tracking, all in a form factor that is light and comfortable enough to wear for long periods of time. We've already seen several new smart glasses models coming out in 2021, including Spectacle smart glasses from Snap, Meta Ray-Ban, Lenovo ThinkReality A3, and Vuzix's next-generation smart glasses.

VR completely replaces the user's field of vision, immersing them in a computer-generated virtual environment. VR headsets, such as the Arm-powered Oculus Quest and Quest 2, have been available in the past five years. These devices are often used for entertainment experiences, such as games, concerts, movies, or sports, but are also moving into the social realm, such as virtual gatherings, as shown in the RecRoom VR platform shown below.

MR lies between AR and VR as it merges the real and virtual worlds by overlaying virtual objects, characters, or instructions onto real-world environments, and vice versa. The most notable MR wearable is the Arm-powered Microsoft HoloLens 2. Virtual instruction sets and graphics are overlaid onto the user’s view to assist in enterprise and industrial applications.

The level of compute performance and efficiency of XR wearables will vary depending on the type of XR wearable and the complexity of the use cases it is designed to support. Modern standalone VR headsets will require high-performance compute, primarily for high-end gaming experiences, such as the Oculus Quest device, while smaller, lighter devices such as AR smartglasses will require high efficiency. At the same time, there will be devices that fall somewhere in between these two levels of performance and efficiency, such as MR headsets like the HoloLens 2, and tethered VR devices for the consumer market like the HP Reverb G2. Therefore, system-on-chip (SoC) solutions need to be able to scale to accommodate different use cases, workloads, and form factors. This is where Arm Total Compute solutions can help.

Total computing solutions for XR wearables

Arm's Total Compute solutions provide a complete set of scalable hardware IP (including the latest Armv9 CPU, Mali GPU and system IP), physical IP, software, tools and standards to build the best SoC across different consumer device markets. These solutions provide different configurations for the special computing requirements required by various XR wearable devices, from high-performance VR and MR headsets to ultra-efficient AR smart glasses.

Configurations for high-performance VR and MR headsets represent our "high performance" comprehensive computing solutions. There are two possible configurations here. First, a 1+3+4 CPU configuration of 1x Arm Cortex -X2, 3x Arm Cortex-A710, and 4x Arm Cortex-A510 for high performance. For slightly lower performance but more efficient cases, there is a 4+4 CPU configuration of 4x Cortex-A710 and 4x Cortex-A510. Both solutions can take advantage of the advanced Arm Mali-G710 GPU or the mainstream Arm Mali-G510 GPU for high-quality graphics.

Meanwhile, the “efficiency” Total Compute solutions for lightweight AR smart glasses can take advantage of a 2+6 CPU configuration of 2x Cortex-A710 and 6x Cortex-A510, or even 4x Cortex-A510 for ultra-efficiency. These solutions can be powered by the ultra-efficient Arm Mali-G310 GPU.

As with all Total Compute solutions, the CPUs and GPUs in our Total Compute solutions for XR wearables are based on our system IP – CoreLink Interconnect CI-700 and NI-700. Both interconnect technologies provide increased energy efficiency and system performance, further improving any overall compute solution. All CPU configurations in our Total Compute solutions are tied together by Arm’s DSU-110, which is the backbone of these Armv9-based Cortex CPU clusters. This enables our partners to address different types of Arm-based XR wearables with different trade-offs in performance and efficiency.

CPU

High-performance VR and MR headsets can integrate at least one Cortex-X2 into the Total Compute solution. Our Cortex-X CPUs really push the peak performance requirements, which is very important for compute-intensive XR workloads such as gaming and other entertainment experiences on VR and MR headsets. This drive for peak performance is complemented by the power efficiency improvements of the Cortex-A710, which enables sustained performance to maximize battery life for all XR wearables. This is a very important consideration for XR wearables, which are increasingly untethered to enable greater user mobility. The Cortex-A510 further supports this push for energy efficiency, enabling longer playback experiences.

For XR wearables where efficiency is critical (such as AR smart glasses), the Cortex-A710 and Cortex-A510 CPUs will be used solely for overall efficiency-based computing solutions. The Cortex-A510 not only improves power efficiency by 20% through a 3-wide in-order design, but also provides industry-leading area efficiency. This is very important for small, lightweight XR wearables where area is important!

One innovation that makes this possible is the merged core microarchitecture. This allows two Cortex-A510 CPUs to be combined into a single complex, with multiple complexes per CPU cluster. The result is increased area efficiency at a higher performance point. This drive for performance combined with high area efficiency is ideal for lightweight AR smart glasses.

GPU

Different XR wearables are powered by different GPUs. The advanced Mali-G710 can be incorporated into the Total Compute solution for high-performance XR headsets. It improves feature coverage for index-driven vertex shading (IDVS), a feature commonly used in XR use cases because it can render each eye view with slightly different object positions for each view.

Mali-G510 can also be integrated into high-performance XR wearables, providing a perfect balance of performance and efficiency. Compared to the previous generation Arm Mali-G57 GPU, it provides 100% performance improvement and 22% power saving, thereby extending battery life.

The Mali-G510 is capable of supporting 2 to 6 configurable shader cores compared to the 7 to 16 of the Mali-G710. However, it still incorporates many of the advanced features of the Mali-G710, such as the Command Stream Front End (CSF), a redesigned additional execution engine, and a redesigned texture unit to provide a high-quality graphics experience. Then, on top of that, it offers better HDR support formats, Arm Frame Buffer Compression (AFBC) uncompressed buffers, and new Arm Fixed Rate Compression (AFRC) to reduce bandwidth. AFRC is a particularly neat feature for XR wearables as it promises to reduce bandwidth and memory footprint at minimal area cost. This translates into performance gains and power savings.

Designed to deliver the highest possible performance at the lowest area cost, Mali-G310 is ideally suited for comprehensive compute solutions targeting AR smartglasses. It’s our highest performing, ultra-efficient GPU ever, with huge performance gains in three performance areas – texture performance (6x), Vulkan performance (4.5x), and Android UI content (2x) – compared to the previous generation Arm Mali-G31 GPU. Like Mali-G510, Mali-G310 also offers better HDR support formats and AFBC uncompressed buffers, but AFRC is only optional. However, the GPU does offer foveated rendering – a rendering technique that uses eye trackers integrated with XR wearables to reduce rendering workload – for enhanced AR and VR.