Can somebody please explain how barrier transactions in ACE work?
Thanks in advance.
I'm not sure which part of ACE barriers are unclear, but your primary reference should be Section D8 in the ACE Specification. Specifically, D8.4.1 contains a diagram and the various requirements that the barrier must obey, which should give you a good idea of how they work.
The gist of ACE barriers is that they issued as two halves, one on the AW channel and one on the AR channel. All the transactions before the barrier appears on the channel are considered to be before the barrier. These transactions before the barrier must be observable to all transactions after the barrier. Transactions are considered to be after the barrier once both barrier responses have been received.
In general, it's not recommended to broadcast barriers within a system, as they are harmful to system performance. A system should ensure that once the response to a transaction is seen, that the transaction is then observable to any future transactions from any other masters, which ensures that barriers can be terminated within the master. In AXI5/ACE5, barriers have been deprecated for these reasons, and so should not be used.
Thank you so much for the reply. The part that is unclear is that what happens after Read Barrier and Write Barrier are issued by the master on AR channel and AW channel. What is the interconnect connecting the multiple masters supposed to do when it receives those two halves of the Barrier. Its AxADDR will be all zero. so to which all slaves it should guide these barriers. Also, I understand that Interconnect should construct and send back a reply for the Barrier transactions. How this can be done. And more important part is who will ensure the ordering of transactions, the interconnect or the slaves? Thanks again.
Normally, the only reason for broadcasting barriers into the system is when you have a point in the interconnect or the slave that is returning early write responses that do not originate from the slave. This can cause a write to be observable by the issuing master and a subset of masters, but not visible to other masters. A response from the final destination would guarantee that the write is visible to all masters. Note, it's possible to return early write responses and not need barriers, if its guaranteed that all accesses to a location will pass through the same point, and that point maintains ordering between accesses. So bearing this in mind...
The AxADDR information has no meaning. The interconnect will have to propagate the barrier as far into system as is required to guarantee that the ordering associated with the barrier is maintained. This might require the barrier to be broadcast to multiple slaves.
If the interconnect can ensure that accesses to a particularly region will remain ordered with respect to the barrier, then the barrier can be terminated in the interconnect, and the response given from the interconnect. If this isn't the case, then the barrier will need to be propagated downstream, until a point that the ordering guarantee can be maintained. Note, a downstream point could be another interconnect, not necessarily a slave.
The Specification doesn't state who is responsible for the returning the response. Some guidance is given at the end of Section D8.3.
Any component that applies will need to determine which transactions are before the barrier, and which transactions are after the barrier and then ensure that those before the barrier are observable by all those after the barrier. Again, there's not a required method of doing this, but some examples are given at the end of Section D8.3
Thank you so much for the wonderful reply ( and extremely sorry for not showing up for almost 20 days. I was out of station.)
Your reply clears almost all my doubts but for clarification please be kind enough to answer few more doubts.
Qn 1. In your opinion which system will be give better performance.
System A - Interconnect gives the reply for the barrier transaction and maintains the order of transactions.( Suppose its topology allows it to perform this way)
System B - Interconnect just passes the barrier transactions to the slaves and the slaves do all kinds of replying and maintaining order.
Qn 2. Also another doubt is that in the second system, the barrier is fed to multiple slaves, so all those will reply. So interconnect will have to collate all the reply and frame the one for the master. Is the system working like that?
Qn 3. Do the barrier transactions from Master 1 affect the order of transactions from Master 2?
Thanks again for spending your valuable time for answering my queries.
Jose.
Josesmn said:
I think it might depend on the design of the interconnect whether A or B would be the most performant, but generally speaking you only want to propagate the barriers as far into the system as they're required.
Josesmn said:Qn 2. Also another doubt is that in the second system, the barrier is fed to multiple slaves, so all those will reply. So interconnect will have to collate all the reply and frame the one for the master. Is the system working like that?
Correct.
No, the barriers only order transactions from the master that issued the barrier.
Thank you so much again for your replies.
Can you please tell me the difference between memory barrier and synchronization barrier?
Are memory barriers expected to order transaction to memory such as L2 Cache or DMC only and not transactions to peripherals?
[AXI4 - C8.1 Page C8-248] " A synchronization barrier is issued by a master to guarantee that all transactions issued before the barrier are observable by every master in the appropriate domain when the barrier completes." Since barriers from one master are not fed to other masters, how can this be done?
Thanks in advance,
Josesmn said:Can you please tell me the difference between memory barrier and synchronization barrier?
The key difference between a memory barrier and a synchronisation barrier is what exact guarantee is given relative to the other instructions.
A memory barrier guarantees that transactions after the barrier can observe transactions before the barrier.
A synchronisation barrier guarantees that when the barrier completes, then all the transactions before the barrier will be visible.
This is similar to the difference between a DMB and DSB within the Arm Architecture. A DMB ensures accesses before the DMB are ordered ahead of accesses after the DMB. A DSB ensures memory accesses before the DSB have completed before the DSB completes. So with an example:
STR
DMB
Send Interrupt
In this case, the memory barrier is not adequate to ensure that the store is visible before the interrupt is sent as it doesn't require that STR has completed before the interrupt is sent. A synchronisation barrier instead of the memory would ensure this, as it wouldn't complete until after the store has completed.
However, if we only needed order between transactions, then a memory barrier would be appropriate.
Josesmn said:Are memory barriers expected to order transaction to memory such as L2 Cache or DMC only and not transactions to peripherals?
No, they would order both.
Josesmn said:[AXI4 - C8.1 Page C8-248] " A synchronization barrier is issued by a master to guarantee that all transactions issued before the barrier are observable by every master in the appropriate domain when the barrier completes." Since barriers from one master are not fed to other masters, how can this be done?
This would be the responsibility of the interconnect. It would need to ensure that the transactions have propagated through to a point in the system where it would be guaranteed that if another master made an access to the same location, then the master would observe this transaction. Only then can the interconnect return a completion response to the barrier.
Paramo
Are these statements true?
1. So in the case of a Memory Barrier, to ensure the ordering, the interconnect should pass all the transactions issued before the Barrier to the slaves before passing the Barrier itself to the slaves.
2. In the case of a Synchronization Barrier, the interconnect should wait until it receive responses to all transaction issued before the Barrier before issuing a response to the Barrier itself.
3. Is it necessary to pass the barrier to the slaves? Or will it be enough to maintain the order in the interconnect and provide apt reply to the issuing Master?
Josesmn said:1. So in the case of a Memory Barrier, to ensure the ordering, the interconnect should pass all the transactions issued before the Barrier to the slaves before passing the Barrier itself to the slaves.
It would depend which component is required to enforce this ordering. If the downstream slave is enforcing the barrier ordering then yes, the pre-barrier transactions and the barrier would need to be passed to the slave in the same order.
If the interconnect is performing the ordering, then barrier just needs to ensure that the pre barrier transactions are observable to post barrier transactions. If the interconnect had a Point of Serialisation through which all accesses to that location had to pass, then it could just stall subsequent accesses to the same location until it could guarantee that the pre-barrier accesses are observable.
Josesmn said:2. In the case of a Synchronization Barrier, the interconnect should wait until it receive responses to all transaction issued before the Barrier before issuing a response to the Barrier itself.
Not necessarily. The ACE definition is only that the transactions are observable when the barrier completes. Only system domain transactions must have reached the end point before the barrier completes.
Similarly in the Arm Architecture, 'completed' only means that the access is observable to all masters in that domain.
Josesmn said:3. Is it necessary to pass the barrier to the slaves? Or will it be enough to maintain the order in the interconnect and provide apt reply to the issuing Master?
It entirely depends on the system. The barrier needs to be propagated far enough into the system that it can provide the required ordering guarantee for a group of masters. For example, Figure D8-1 in the ACE Spec is an example of a system where the first interconnect may need to pass the barrier downstream.
Thank you so much for all your replies. I may get back to you if I need more help.