We are running a survey to help us improve the experience for all of our members. If you see the survey appear, please take the time to tell us about your experience if you can.
Hi Experts,
I'm developing some kind of Secure OS on A57/53 big.LITTLE SoC.
While multi-core testing, I'm facing some wired problem on my world shared memory mechanism.
When I run world shared memory test on a single core (using affinity), it works well for a hundred of thousand times.
But once multi-core enabled, it sometimes fails.
Meaning to say, in the secure there is no content on the memory which is shared by non-secure.
I'm now doubting everything.
I believe that the CCI-400 works well between the big and LITTLE cluster's for cache coherency in the non-secure world.
But when world change occurs from non-secure to secure, does CCI-400 still guarantee cache coherency between two worlds?
Or Should I use any cache maintenance instruction?
My world shared memory mechanism is like this;
It might be a silly question, please advice me.
Hi kaylee,
The first and easiest question to ask is how you're mapping the target address in the secure world. A cache line fetched while executing in the Non-Secure world will be tagged as Non-Secure. A Secure OS mapping the same physical address to virtual address with the NS bit not set (therefore it's "secure memory") in the translation table entry will be tagged as Secure. In this way the same numerical physical address can be in the cache twice - a secure copy and a non-secure copy. Consider that the Security extensions define the S and NS physical address spaces as entirely separate (even if the system is designed such that they are the same view of the memory map).
If your secure OS is mapping memory from the non-secure world then it needs to map it with the NS bit set in the appropriate translation table entries. Here's the trick - it seems you're implying that you're directly re-using the mappings passed by the Non-Secure OS by referencing that NS OS's tables. In this case Linux makes no assumptions about whether memory is secure or non-secure - if secure, it wants the NS bit to be unset, if non-secure then the NS bit is ignored anyway, so it just doesn't use the bit. If you're just copying non-secure descriptors and modifying the address, and the NS bit is 0, then it's explicitly secure memory while you're in the secure world. Note that you also need to be careful accessing the table entries from the secure world!
If it's not as simple as that, then I am sure we can dive into it a little more.
It is usually best to assume all addresses passed from the Non-Secure world are to be set NS (specifically), and pass physical (not virtual) addresses, and this is how most reliable SMC APIs are designed.
Ta,
Matt
I should have answered the title question clearly too - the CCI-400, and all cache-coherent interconnects by ARM, do not assure coherency between the Secure and Non-Secure worlds because they are two separate physical address spaces.
You can think of the NS bit (which is a conjunction of SCR.NS, translation table entry NS, and whether you're not in Monitor mode) as a 33rd/41st/49th address bit (Short, LPAE, AArch64 physical addressing). In this sense, the NS+32-bit address 0x1_0000_8000 (NS) is not the same as NS+32-bit address 0x0_0000_8000 (S) and, obviously, two 'different' addresses need not be coherent.
The CCI-400 will, however, assure that all NS transactions are coherent in and of themselves, and all Secure transactions are coherent in and of themselves - so if you stay mapped as Non-Secure from Linux to your Secure OS, then the coherency is well-managed. Having the interconnect assume that the Secure and Non-Secure worlds have the same underlying memory map would be a violation of the Security Extensions.
Hi Matt,
Thanks for your answer and it really helps me.
I mapped NS address into secure world as _secure_ for sharing.
I guess it caused the problem.
Now I changed the table attributes as non-secure in that case then it works well. :)
And your comments makes me understand how cache works berween two worlds clearly.