I have questions mostly about the relationship between ARM's shareability domain, cache maintenance and barrier. The statements starting with "Question \<number\>:" are the questions. Let me give you the example code first and add my questions as we go. Please leave comments if any part of the statements is not clear for any reason(ex. vague expression, potential misunderstanding, broken English, etc.). Thanks community for your help in advance!
# Example code and explanations
STR #1, [X1] DMB ISHST STR #1, [X3]
excerpt from https://developer.arm.com/documentation/102336/0100/Limiting-the-scope-of-memory-barriers
Let's assume that `X1` and `X3` point to the data region, not the instruction region.
If the memory locations `X1` and `X3` point to are marked as Inner Shareable, we will not need any cache maintenance instruction(invalidate or clean) to make the PEs in the same Inner Shareable domain synchronized about the stores to `X1` and `X3` although the PEs in a different Inner Shareable domain will not be synchronized. By "synchronized" here, it means the store to `X1` and the store to `X3` are ordered within the same Inner Shareable domain.
If the memory locations `X1` and `X3` point to are marked as Outer Shareable and we change `DMB ISHST` to `DMB OSHST`, we will also not need any cache maintenance instruction to make the PEs in the different Inner Shareable domain but in the same Outer Shareable domain synchronized.
The problem comes up to me when the memory locations `X1` and `X3` point to are marked as Inner Shareable but we want to make the PEs in the different Inner Shareable domain but in the same Outer Shareable domain synchronized about the stores.
# Problem Statements
Here is an assumption on the environment:- Let’s say that the memory locations `X1` and `X3` point to are in the data region and marked as Inner Shareable, Inner Write-Back Cacheable and Outer Write-Back Cacheable.
Here is a goal that we want to achieve:- We want to make the PEs in the different Inner Shareable domain but in the same Outer Shareable domain synchronized about the stores to `X1` and `X3`.
Question 1: Is the code below enough to achieve the goal without any cache maintenance instruction involved? The code only changed the DMB’s shareability domain.
STR #1, [X1] DMB OSHST STR #1, [X3]
The answer seems no since HW cache coherency mechanism(ex. MESI or MOESI) will not work to make inner shareable memory synchronized outside of that domain. In other words, other PEs which are in a different inner shareable domain will not see the store to `X1` or `X3` done by the source PE. Also, using the barrier instructions does not mean changing the shareability domain of the to-be-affected memory locations.
Question 2: I guess we should add cache maintenance instructions after each store and change DMB to DSB as follows so that other PEs can observe the updated values to `X1` and `X3` and the DSB barrier can enforce the cache maintenance instruction to complete before the barrier instruction completes. Is this code enough to achieve the goal? or should we do more?
STR #1, [X1] DC clean for [X1] up to PoC or PoU DSB OSHST STR #1, [X3] DC clean for [X1] up to PoC or PoU
Question 3: It seems that the barrier instructions guarantee what the barriers are supposed to guarantee only when the to-be-affected memory locations' shareability domain matches with the shareability domain given to those barrier instructions. Is this understanding correct?
STR #1, [X1] STR #1, [X2] DMB OSHST STR #1, [X3] STR #1, [X4]
In the above code, for example, if `X1` and `X3` point to Inner Shareable memory and `X2` and `X4` point to Outer Shareable memory, I think the use of the `DMB OSHST` barrier will mean that from the outer shareable domain's point of view it does not care about the visibility and ordering of `X1` and `X3`(out of interest) and it only cares about the ones of `X2` and `X4`; However, the ones of `X1` and `X3` will be guaranteed right within the same inner shareable domain. Is this understanding correct?