When testing my applications on Android 15 with a Mali-G710 MP7 (Pixel 7), I noticed that any time I submit a secondary command buffer to a render pass I get the following validation error when presenting the swap chain:
VUID-VkPresentInfoKHR-pImageIndices-01430: Validation Error: [ VUID-VkPresentInfoKHR-pImageIndices-01430 ] Object 0: handle = 0xb400007be5014a90, type = VK_OBJECT_TYPE_QUEUE; | MessageID = 0x48ad24c6 | vkQueuePresentKHR(): pPresentInfo->pSwapchains[0] images passed to present must be in layout VK_IMAGE_LAYOUT_PRESENT_SRC_KHR or VK_IMAGE_LAYOUT_SHARED_PRESENT_KHR but is in VK_IMAGE_LAYOUT_UNDEFINED.The Vulkan spec states: Each element of pImageIndices must be the index of a presentable image acquired from the swapchain specified by the corresponding element of the pSwapchains array, and the presented image subresource must be in the VK_IMAGE_LAYOUT_PRESENT_SRC_KHR or VK_IMAGE_LAYOUT_SHARED_PRESENT_KHR layout at the time the operation is executed on a VkDevice (docs.vulkan.org/.../wsi.html
After a few frames, the device is lost. I haven't encountered either issue on any other platform when running the same tester going through the same Vulkan code paths.
Here is a screenshot from RenderDoc when running the same tester on Linux with the same configuration to draw with a secondary command buffer:
I have highlighted the barriers that perform the layout transitions. Here are the contents of the barriers themselves:
Some other things to note:
When debugging, I even tried waiting for the GPU to be idle before processing the submit, and I still had the same behavior. The following can avoid both the validation and device lost issues:
So far I haven't been able to find the source of the error for the image transitions, command buffer setup, or queue submission and swapchain presentation. I see that Mali-G710 was advertised as the first Mali GPU with native secondary command buffer support. Are there extra steps that I'm missing when the hardware support is used for Mali, such as extra barriers? Could this be a driver issue, perhaps android-specific with the swapchains?
The project is open source, I can share it and instructions to reproduce if need be.
After some more digging around, another difference I noticed was the image barriers may appear in different primary command buffers for the code path that uses secondary command buffers. This is because the multithreaded rendering code path may create command buffers for other operations such as copying data on separate threads, then they get submitted together with the rendering command buffers. The ordering of those command buffers for submission is determined on the main thread (i.e. no thread contention), and I verified with printouts that they are submitted in order.
Another thing worth mentioning is that I am able to run on a different Android device with different hardware (Qualcomm) without errors.
The validation layer error can't be related to an Android issue or Mali driver issue, as that check is implemented above the API, so I'd focus on solving that one before worrying about anything else.
I'm thinking it might be platform-specific differences in what is returned from various Vulkan functions (e.g. swapchain images and indices, perhaps which command buffers are returned) that changes what the validator sees. Along those lines, I put in a bunch of print statements and verified that the images are what I expect (4 images are in the swap chain, and they are acquired in index order) and that the images are been consistent with image views used in the framebuffer and swapchain operations. As described above, I also verified that the command buffers are submitted in order, and I use a ring buffer to avoid re-using any command buffer pools for 3 frames. The first queue submit appears to succeed without error, but then the second queue submit and on have validation errors. By the time I wait for a fence, the device is lost.
Thus far I haven't been able to find anything incorrect with how the swap chain images are being submitted. I've tried to get a frame trace, but the Android Graphics Inspector tool crashes for me when loading the trace and RenderDoc hasn't been able to gather a trace either. The fact that it crashes from device lost so quickly might also complicate matters.
The project exhibiting the issue is available here: https://github.com/akb825/DeepSea. The update.sh script may be used to checkout the submodules and download pre-built packages for the tools required for building (texture conversion, shader compilation, etc.) and pre-built external libraries. To build for Android, you can run it with the following arguments: update.sh -m -t -l android-all. If running on Windows, this may be run through git bash. Once you have the requirements, you can open the Android project under the android folder.
update.sh -m -t -l android-all
The different testers can be selected under the Build Variants section in Android Studio, with the GUI testers being under the "app" variants. There are two testers that exhibit the issue: TestLighting and TestScene. TestScene is the simpler of the two, though multithreaded rendering (which is what triggers the issue) is disabled by default. To enable multithreaded rendering in that tester, in the file "testers/TestScene/TestScene.c" add the following line at the top of the setup() function on line 224: testScene->multithreadedRendering = true;
testScene->multithreadedRendering = true;