Unexpected behaviour of VKN subpasses on G76 (Samsung S10), G77 (S20FE)

About the slowdown on S20 with the input attachment sample, now having looked at the code a theory for what happens is:

• In the original code, there is no DS attachment in the second subpass. If there's no fusion you'll have:
  • Fragment job 0 (subpass 0): Clears color+depth, render, writes out color+depth
  • Fragment job 1 (subpass 1): Samples color-or depth-buffer from previous pass, writes out color to swap-chain
• However, with your changes there is now a DS attachment in the second subpass. So we get:
  
  • Fragment job 0 (subpass 0): Clears color+depth, render, writes out color+depth. (Same as before)
  • Fragment job 1 (subpass 1): Reads back depth-buffer (since there is now a DS attachment present), samples color-or depth-buffer from previous pass, writes out color to swap-chain
• In the ideal case, with fusion, we'd expect this:
  • Fragment job 0 (subpass 0 and 1 combined): Clear color+depth, render, <NextSubpass>, read color/depth from tile-buffer and write result to swap-chain image, write out swap-chain image to memory

As you can see, in the first two cases (no fusion) we end up with quite a lot of off-chip/DRAM traffic in order to write out the 'intermediate' images. Whereas when fusing the subpasses we avoid this by being able to keep the data-on-chip, and the only thing we need to write to memory is the final swapchain image. The difference between the first two cases is that in the second case we need a readback of the depth-buffer in the second fragment job (because there is one specified) -- and this extra work / BW may explain the slowdown. 

With this in mind it could be the slowdown you see here is not related to fusion working / not working, and that fusion in fact never happens on these devices. If so it would explain the lack of any differences with/without subpasses on the S10 and S20, but that there is a difference on the Nova 5T.

If this sounds good so far, I guess the only question remaining is why you see a noticeable slowdown on the Note 8 in your original case. A theory is that fusion is in fact working here, explaining there is *a* difference -- but of course we'd expect an improvement, not a slowdown.

Are you able to do any profiling on the Note 8 to try to get some information out of it about what might be happening on this device?

About the slowdown on S20 with the input attachment sample, now having looked at the code a theory for what happens is:

• In the original code, there is no DS attachment in the second subpass. If there's no fusion you'll have:
  • Fragment job 0 (subpass 0): Clears color+depth, render, writes out color+depth
  • Fragment job 1 (subpass 1): Samples color-or depth-buffer from previous pass, writes out color to swap-chain
• However, with your changes there is now a DS attachment in the second subpass. So we get:
  
  • Fragment job 0 (subpass 0): Clears color+depth, render, writes out color+depth. (Same as before)
  • Fragment job 1 (subpass 1): Reads back depth-buffer (since there is now a DS attachment present), samples color-or depth-buffer from previous pass, writes out color to swap-chain
• In the ideal case, with fusion, we'd expect this:
  • Fragment job 0 (subpass 0 and 1 combined): Clear color+depth, render, <NextSubpass>, read color/depth from tile-buffer and write result to swap-chain image, write out swap-chain image to memory

As you can see, in the first two cases (no fusion) we end up with quite a lot of off-chip/DRAM traffic in order to write out the 'intermediate' images. Whereas when fusing the subpasses we avoid this by being able to keep the data-on-chip, and the only thing we need to write to memory is the final swapchain image. The difference between the first two cases is that in the second case we need a readback of the depth-buffer in the second fragment job (because there is one specified) -- and this extra work / BW may explain the slowdown. 

With this in mind it could be the slowdown you see here is not related to fusion working / not working, and that fusion in fact never happens on these devices. If so it would explain the lack of any differences with/without subpasses on the S10 and S20, but that there is a difference on the Nova 5T.

If this sounds good so far, I guess the only question remaining is why you see a noticeable slowdown on the Note 8 in your original case. A theory is that fusion is in fact working here, explaining there is *a* difference -- but of course we'd expect an improvement, not a slowdown.

Are you able to do any profiling on the Note 8 to try to get some information out of it about what might be happening on this device?

Regarding the Note 8 Pro, below are before (left) and after (right) relevant metrics captured from our bench device. It does look like memory traffic and amount of tiles modified are reduced greatly.

One thing to try here is to zoom in (use the 1ms option) and try to select 1 frame using the range selector. Usually there is a nice gap between frames, if vsync limited, so this should be fairly easy if so. If not look for patterns in the workload, like e.g. when the bulk of the VS shading happens (this is usually the start of a frame).

This way you can directly compare the workload for a single frame, as opposed to workload inside some specific period of time. I find this useful when comparing performance between two cases.