Barrier Transactions in ACE

Josesmn said:

Qn 1. In your opinion which system will be give better performance.

System A - Interconnect gives the reply for the barrier transaction and maintains the order of transactions.( Suppose its topology allows it to perform this way)

System B - Interconnect just passes the barrier transactions to the slaves and the slaves do all kinds of replying and maintaining order.

I think it might depend on the design of the interconnect whether A or B would be the most performant, but generally speaking you only want to propagate the barriers as far into the system as they're required.  

Josesmn said:

Qn 2. Also another doubt is that in the second system, the barrier is fed to multiple slaves, so all those will reply. So interconnect will have to collate all the reply and frame the one for the master. Is the system working like that?

Correct.

Josesmn said:

Qn 3. Do the barrier transactions from Master 1 affect the order of transactions from Master 2?

No, the barriers only order transactions from the master that issued the barrier.

Josesmn said:

Qn 1. In your opinion which system will be give better performance.

System A - Interconnect gives the reply for the barrier transaction and maintains the order of transactions.( Suppose its topology allows it to perform this way)

System B - Interconnect just passes the barrier transactions to the slaves and the slaves do all kinds of replying and maintaining order.

I think it might depend on the design of the interconnect whether A or B would be the most performant, but generally speaking you only want to propagate the barriers as far into the system as they're required.  

Josesmn said:

Qn 2. Also another doubt is that in the second system, the barrier is fed to multiple slaves, so all those will reply. So interconnect will have to collate all the reply and frame the one for the master. Is the system working like that?

Correct.

Josesmn said:

Qn 3. Do the barrier transactions from Master 1 affect the order of transactions from Master 2?

No, the barriers only order transactions from the master that issued the barrier.

Josesmn said:

Can you please tell me the difference between memory barrier and synchronization barrier?

The key difference between a memory barrier and a synchronisation barrier is what exact guarantee is given relative to the other instructions.

A memory barrier guarantees that transactions after the barrier can observe transactions before the barrier.

A synchronisation barrier guarantees that when the barrier completes, then all the transactions before the barrier will be visible.

This is similar to the difference between a DMB and DSB within the Arm Architecture.  A DMB ensures accesses before the DMB are ordered ahead of accesses after the DMB.  A DSB ensures memory accesses before the DSB have completed before the DSB completes.  So with an example:

STR

DMB

Send Interrupt

In this case, the memory barrier is not adequate to ensure that the store is visible before the interrupt is sent as it doesn't require that STR has completed before the interrupt is sent.  A synchronisation barrier instead of the memory would ensure this, as it wouldn't complete until after the store has completed.

However, if we only needed order between transactions, then a memory barrier would be appropriate.

Josesmn said:

Are memory barriers expected to order transaction to memory such as L2 Cache or DMC only  and not transactions to peripherals?

No, they would order both.  

Josesmn said:

[AXI4 - C8.1 Page C8-248] " A synchronization barrier is issued by a master to guarantee that all transactions issued before the barrier are observable by every master in the appropriate domain when the barrier completes." Since barriers from one master are not fed to other masters, how can this be done?

This would be the responsibility of the interconnect.  It would need to ensure that the transactions have propagated through to a point in the system where it would be guaranteed that if another master made an access to the same location, then the master would observe this transaction.  Only then can the interconnect return a completion response to the barrier.

Josesmn said:

1. So in the case of a Memory Barrier, to ensure the ordering, the interconnect should pass all the transactions issued before the Barrier to the slaves before passing the Barrier itself to the slaves.

It would depend which component is required to enforce this ordering.  If the downstream slave is enforcing the barrier ordering then yes, the pre-barrier transactions and the barrier would need to be passed to the slave in the same order. 

If the interconnect is performing the ordering, then barrier just needs to ensure that the pre barrier transactions are observable to post barrier transactions.  If the interconnect had a Point of Serialisation through which all accesses to that location had to pass, then it could just stall subsequent accesses to the same location until it could guarantee that the pre-barrier accesses are observable.

Josesmn said:

2. In the case of a Synchronization Barrier, the interconnect should wait until it receive responses to all transaction issued before the Barrier before issuing a response to the Barrier itself.

Not necessarily.  The ACE definition is only that the transactions are observable when the barrier completes.  Only system domain transactions must have reached the end point before the barrier completes.

Similarly in the Arm Architecture, 'completed' only means that the access is observable to all masters in that domain.

Josesmn said:

3. Is it necessary to pass the barrier to the slaves? Or will it be enough to maintain the order in the interconnect and provide apt reply to the issuing Master?

It entirely depends on the system.  The barrier needs to be propagated far enough into the system that it can provide the required ordering guarantee for a group of masters.  For example, Figure D8-1 in the ACE Spec is an example of a system where the first interconnect may need to pass the barrier downstream.