Surface rotation is an issue you have likely never heard before Vulkan – or even until now. On desktop it is clearly not a problem, on mobile it is totally transparent to the application in GLES and can be transparent in Vulkan as well.
However, when working with Vulkan you must take extra care to avoid performance implications. To understand why, we need to start by understanding how display processors work on mobile.
Figure 1: Surface rotation in phones
Figure 1 shows what we usually expect when we rotate a phone. It seems straightforward enough: the swapchain size and aspect ratio changes but otherwise we render the scene the same way.
This seems simple, but there is a detail that makes matters much more complex: as far as the display processor is concerned, nothing has changed. The display processor will still scan the screen from top to bottom, left to right, meaning the physical corners of the screen, which remain the same after the screen is rotated. Figure 2 illustrates this behavior.
Figure 2: The display processor scans from top to bottom, left to right
The question then is the following: if the corners have changed after the screen rotates, how is the display processor still able to display the scene correctly? There is no magic here: someone needs to rotate the image so that the display processor can use it.
Figure 3 shows the conceptual steps required: the resolution is adjusted to landscape mode, but the corners don’t match anymore with the ones the display processor expects. A rotation is necessary to make them match.
Figure 3: Conceptual steps for proper surface rotation
New questions arise! Who takes care of that rotation? Does it have a cost? Fear not, the answers are in the next section.
As we said at the beginning, surface rotation used to be a non-issue in GLES. This is because the driver would take care of still allocating the image with the correct dimensions and patching all necessary API calls.
Then Vulkan came, and in an effort to provide a transparent API, this sort of driver meddling is not allowed anymore. Even if it was allowed, the low-level access to GPU resources that Vulkan provides would make it impossible to build such a layer in the driver.
Still, you may not have seen the issue at all, and with good reason. The trick lies in the preTransform parameter when you create your swapchain. You will normally set it to IDENTITY on desktop, and if you keep doing the same for mobile, you won’t even notice the rotation issue. Somebody takes care of that for you, all is good. Or is it?
Figure 4: Surface rotation with preTransform = IDENTITY
If the driver cannot take care of the rotation, somebody else must take this burden. In Vulkan, it is the presentation engine. Android will ensure that your images are still displayed correctly on screen and will try to do so with the least performance cost.
Here is the big deal: the rotation has a performance cost. The driver patching in GLES was a cheap solution, as it did not require any memory accesses and memory bandwidth is at a premium on mobile.
There are two possible cases with Vulkan:
The latter case is the costly one, as a rotation means a full screen read and write to main memory. This is a lot of memory accesses, which can seriously affect the power consumption of your app.
The catch is that it is impossible to know if the phone you are using has a smart display processor. It is a system integration concern, which is unrelated to the GPU model. A way to test it is to use our surface rotation sample (see below) as a benchmark – if you see savings with proper use of surface rotation, your phone might not have a smart display processor.
Figure 5: Surface rotation with preTransform = currentTransform
The correct approach, as shown in Figure 5, is for the app to take care of the rotation, drawing the image already rotated. It is now a two-step process which does not require any extra memory accesses.
An app can tell the presentation engine that it is going to take care of the rotation by setting preTransform to be the same as currentTransform from surface properties.
Note that this only applies to the swapchain image, all other render targets can be allocated without rotating them. This means that apps that have a more complex rendering pipeline, such as postprocessing, have an easier time with the rotation, because they can embed it in the last blit to the swapchain image.
If your app is rendering directly to the swapchain image you will need to make more adjustments, such as rotating the MVP matrix to match the screen orientation. Our Github tutorial on Surface Rotation covers these steps more in detail.
An extra responsibility falling on the app’s shoulders must be justified with some concrete savings. Therefore, we built a sample in our Mobile Best Practice for Mobile Developers which shows the two approaches and performance implications.
Figure 6: Scene with preRotation = IDENTITY
Figure 7: Scene with preRotation = currentRotation
As Figures 6 and 7 show, using the correct approach for surface rotation can reduce memory stalls by an order of magnitude - up to 88% savings in external read stalls and up to 91% savings in internal read stalls. This is on a recent phone with one of our high-end GPUs, but whose display processor did not support rotation.
Memory stalls are the memory accesses for which the memory bus is busy with other transactions. The bus contention between the rotation block and the GPU can slow down GPU work. Furthermore, the extra memory accesses will cause some extra power consumption, leading to faster overheating and lower battery life. Hence the need to be conscious of of the approached and apply the right technique.
We would encourage you to check out the project on Vulkan Mobile Best Practice GitHub page and try the sample for yourself! The sample code gives developers on-screen control to demonstrate multiple ways of using the feature; it also shows the performance impact of the different approaches through real-time hardware counters on the display. You are also warmly invited to contribute to the project by providing feedback and fixes and creating additional samples.
Vulkan Best Practices