Arm Community
Arm Community
  • Site
  • User
  • Site
  • Search
  • User
  • Groups
    • Research Collaboration and Enablement
    • DesignStart
    • Education Hub
    • Innovation
    • Open Source Software and Platforms
  • Forums
    • AI and ML forum
    • Architectures and Processors forum
    • Arm Development Platforms forum
    • Arm Development Studio forum
    • Arm Virtual Hardware forum
    • Automotive forum
    • Compilers and Libraries forum
    • Graphics, Gaming, and VR forum
    • High Performance Computing (HPC) forum
    • Infrastructure Solutions forum
    • Internet of Things (IoT) forum
    • Keil forum
    • Morello Forum
    • Operating Systems forum
    • SoC Design and Simulation forum
    • 中文社区论区
  • Blogs
    • AI and ML blog
    • Announcements
    • Architectures and Processors blog
    • Automotive blog
    • Graphics, Gaming, and VR blog
    • High Performance Computing (HPC) blog
    • Infrastructure Solutions blog
    • Innovation blog
    • Internet of Things (IoT) blog
    • Operating Systems blog
    • Research Articles
    • SoC Design and Simulation blog
    • Tools, Software and IDEs blog
    • 中文社区博客
  • Support
    • Arm Support Services
    • Documentation
    • Downloads
    • Training
    • Arm Approved program
    • Arm Design Reviews
  • Community Help
  • More
  • Cancel
Arm Community blogs
Arm Community blogs
Graphics, Gaming, and VR blog Vulkan Mobile Best Practice - Appropriate Use of Surface Rotation
  • Blogs
  • Mentions
  • Sub-Groups
  • Tags
  • Jump...
  • Cancel
More blogs in Arm Community blogs
  • AI and ML blog

  • Announcements

  • Architectures and Processors blog

  • Automotive blog

  • Embedded blog

  • Graphics, Gaming, and VR blog

  • High Performance Computing (HPC) blog

  • Infrastructure Solutions blog

  • Internet of Things (IoT) blog

  • Operating Systems blog

  • SoC Design and Simulation blog

  • Tools, Software and IDEs blog

Tell us what you think
Tags
  • vulkan
  • High Fidelity Mobile Gaming
  • Graphics Developers
  • Graphics APIs
  • gpu
  • Graphics Asset Creation
Actions
  • RSS
  • More
  • Cancel
Related blog posts
Related forum threads

Vulkan Mobile Best Practice - Appropriate Use of Surface Rotation

Attilio Provenzano
Attilio Provenzano
October 22, 2019
5 minute read time.

Surface rotation is an issue you have likely never heard before Vulkan – or even until now. On desktop it is clearly not a problem, on mobile it is totally transparent to the application in GLES and can be transparent in Vulkan as well.

However, when working with Vulkan you must take extra care to avoid performance implications. To understand why, we need to start by understanding how display processors work on mobile.

Surface rotation in practice

Surface rotation on phones
Figure 1: Surface rotation in phones

Figure 1 shows what we usually expect when we rotate a phone. It seems straightforward enough: the swapchain size and aspect ratio changes but otherwise we render the scene the same way.

This seems simple, but there is a detail that makes matters much more complex: as far as the display processor is concerned, nothing has changed. The display processor will still scan the screen from top to bottom, left to right, meaning the physical corners of the screen, which remain the same after the screen is rotated. Figure 2 illustrates this behavior.

Top to Bottom Rotation

Figure 2: The display processor scans from top to bottom, left to right

The question then is the following: if the corners have changed after the screen rotates, how is the display processor still able to display the scene correctly? There is no magic here: someone needs to rotate the image so that the display processor can use it.

Figure 3 shows the conceptual steps required: the resolution is adjusted to landscape mode, but the corners don’t match anymore with the ones the display processor expects. A rotation is necessary to make them match.

Computational rotation

Figure 3: Conceptual steps for proper surface rotation

New questions arise! Who takes care of that rotation? Does it have a cost? Fear not, the answers are in the next section.

Surface rotation in GLES and Vulkan

As we said at the beginning, surface rotation used to be a non-issue in GLES. This is because the driver would take care of still allocating the image with the correct dimensions and patching all necessary API calls.

Then Vulkan came, and in an effort to provide a transparent API, this sort of driver meddling is not allowed anymore. Even if it was allowed, the low-level access to GPU resources that Vulkan provides would make it impossible to build such a layer in the driver.

Still, you may not have seen the issue at all, and with good reason. The trick lies in the preTransform parameter when you create your swapchain. You will normally set it to IDENTITY on desktop, and if you keep doing the same for mobile, you won’t even notice the rotation issue. Somebody takes care of that for you, all is good. Or is it?

PreTransform surface rotation

Figure 4: Surface rotation with preTransform = IDENTITY

If the driver cannot take care of the rotation, somebody else must take this burden. In Vulkan, it is the presentation engine. Android will ensure that your images are still displayed correctly on screen and will try to do so with the least performance cost.

Here is the big deal: the rotation has a performance cost. The driver patching in GLES was a cheap solution, as it did not require any memory accesses and memory bandwidth is at a premium on mobile.

There are two possible cases with Vulkan:

  • If the phone has a display processor which supports rotation, it can take care of it at the very end, without affecting system bandwidth;
  • If the phone has a simpler display processor, a separate hardware block (or potentially the GPU) must perform the rotation.

The latter case is the costly one, as a rotation means a full screen read and write to main memory. This is a lot of memory accesses, which can seriously affect the power consumption of your app.

The catch is that it is impossible to know if the phone you are using has a smart display processor. It is a system integration concern, which is unrelated to the GPU model. A way to test it is to use our surface rotation sample (see below) as a benchmark – if you see savings with proper use of surface rotation, your phone might not have a smart display processor.

With pretransform surface rotation

Figure 5: Surface rotation with preTransform = currentTransform

The correct approach, as shown in Figure 5, is for the app to take care of the rotation, drawing the image already rotated. It is now a two-step process which does not require any extra memory accesses.

An app can tell the presentation engine that it is going to take care of the rotation by setting preTransform to be the same as currentTransform from surface properties.

Note that this only applies to the swapchain image, all other render targets can be allocated without rotating them. This means that apps that have a more complex rendering pipeline, such as postprocessing, have an easier time with the rotation, because they can embed it in the last blit to the swapchain image.

If your app is rendering directly to the swapchain image you will need to make more adjustments, such as rotating the MVP matrix to match the screen orientation. Our Github tutorial on Surface Rotation covers these steps more in detail.

The sample

An extra responsibility falling on the app’s shoulders must be justified with some concrete savings. Therefore, we built a sample in our Mobile Best Practice for Mobile Developers which shows the two approaches and performance implications.

Scene with preRotation = IDENTITYFigure 6: Scene with preRotation = IDENTITY

Scene with preRotation = currentRotation
Figure 7: Scene with preRotation = currentRotation

As Figures 6 and 7 show, using the correct approach for surface rotation can reduce memory stalls by an order of magnitude - up to 88% savings in external read stalls and up to 91% savings in internal read stalls. This is on a recent phone with one of our high-end GPUs, but whose display processor did not support rotation.

Memory stalls are the memory accesses for which the memory bus is busy with other transactions. The bus contention between the rotation block and the GPU can slow down GPU work. Furthermore, the extra memory accesses will cause some extra power consumption, leading to faster overheating and lower battery life. Hence the need to be conscious of of the approached and apply the right technique.

We would encourage you to check out the project on Vulkan Mobile Best Practice GitHub page and try the sample for yourself! The sample code gives developers on-screen control to demonstrate multiple ways of using the feature; it also shows the performance impact of the different approaches through real-time hardware counters on the display. You are also warmly invited to contribute to the project by providing feedback and fixes and creating additional samples.

Vulkan Best Practices 

Anonymous
Graphics, Gaming, and VR blog
  • Arm Immortalis-G715 Developer Overview

    Peter Harris
    Peter Harris
    The new Arm®︎ Immortalis™︎ -G715 GPU is now available in consumer devices. This blog explores what is new, and how developers can get the best performance out of it.
    • March 20, 2023
  • Success in mobile games with ray tracing

    arm-phodges
    arm-phodges
    Blog provides details on how to use ray tracing techniques successfully across all mobile games on Arm-powered smartphones.
    • March 6, 2023
  • Arm at Vulkanised 2023

    Peter Harris
    Peter Harris
    A summary of the Arm talks at Khronos' Vulkanised 2023 event.
    • March 1, 2023