The "Armv8 Architecture Reference Manual" says the following about the non-Reordering attribute for Device memory:
"For all memory types with the non-Reordering attribute, the order of memory accesses arriving at a single peripheral of IMPLEMENTATION DEFINED size, as defined by the peripheral, must be the same order that occurs in a simple sequential execution of the program."
Who sets IMPLEMENTATION DEFINED SIZE? Is it set by the hardware engineer who designs the processor, or can IMPLEMENTATION DEFINED SIZE be configured by software, for example, when software sets up the page tables? Could someone point me to a section of the "Armv8 Architecture Reference Manual" that clarifies this?
jatron said:Who sets IMPLEMENTATION DEFINED SIZE?
The answer is in the statement you quoted from the manual - '... single peripheral of IMPLEMENTATION DEFINED size, as defined by the peripheral, ...'.
A peripheral, for instance, may not tolerate any IO, targetting its MMIO registers, of size other than 32-bits.
I think the statement says that if a programmer attempts an IO of an invalid size, the non-Reordering guarantee becomes void. That makes sense, because any IO of a larger size, if not out-right rejected, may be broken down into multiple pieces, and that causes ordering problems between the pieces (what is the correct sequence of considering the pieces?). Similarly any IOs of smaller sizes, if not rejected similarly, may have to be coalesced, and that destroys the original ordering in which the smaller IOs arrived.
As an example, the gicv2 manual says,
"All registers support 32-bit word accesses with the access type defined in Table 4-1 on page 4-75 and Table 4-2 on page 4-76. In addition, the GICD_IPRIORITYRn, GICD_ITARGETSRn, GICD_CPENDSGIRn, and GICD_SPENDSGIRn registers support byte accesses. Whether any halfword register accesses are permitted is IMPLEMENTATION DEFINED."
Thanks. From your answer I gather that IMPLEMENTATION DEFINED SIZE is set in hardware. It cannot be configured by software. Let me know if I misunderstood.
Could you elaborate on this sentence: "Similarly any IOs of smaller sizes, if not rejected similarly, may have to be coalesced, and that destroys the original ordering in which the smaller IOs arrived." Why would smaller IOs need to be coalesced?
jatron said:From your answer I gather that IMPLEMENTATION DEFINED SIZE is set in hardware. It cannot be configured by software.
True on both counts.
jatron said:Why would smaller IOs need to be coalesced?
Because it was assumed, for the sake of providing an example, that the IOs were of size smaller than the sizes supported by the peripheral, and that they were not outright rejected but were allowed to reach it. Under such conditions, someone (perhaps a buffer on the cpu or a bus) merges the IOs in order to present them to the peripheral in the sizes it supports.
I think we are talking about different IMPLEMENTATION DEFINED sizes. The "Armv8 Architecture Reference Manual" defines IMPLEMENTATION DEFINED size as the amount of memory reserved for a memory-mapped peripheral.
"A Memory-mapped peripheral occupies a memory region of IMPLEMENTATION DEFINED size and can be accessed using load and store instructions."
It seems like you are talking about the size of the data accesses (e.g. 8 bits, 16 bits, 32 bits, 64 bits, etc.).
In any case, I think the answer to this question can be summarized as follows:
1. IMPLEMENTATION DEFINED size is the amount of memory reserved for a memory-mapped peripheral. It is set in hardware (defined by the peripheral).
2. The statement says that the non-Reordering guarantee becomes void when you are reading/writing to different peripherals. In other words, the non-Reordering attribute is not required to have an impact on the order of memory accesses to different peripherals, even if both memory regions are nR.
Does this interpretation seem correct?
jatron said:The "Armv8 Architecture Reference Manual" defines IMPLEMENTATION DEFINED size as the amount of memory reserved for a memory-mapped peripheral. "A Memory-mapped peripheral occupies a memory region of IMPLEMENTATION DEFINED size and can be accessed using load and store instructions."
The "Armv8 Architecture Reference Manual" defines IMPLEMENTATION DEFINED size as the amount of memory reserved for a memory-mapped peripheral.
jatron said:1. IMPLEMENTATION DEFINED size is the amount of memory reserved for a memory-mapped peripheral. It is set in hardware (defined by the peripheral).
This is not the correct interpretation. There is no specific noun called the "IMPLEMENTATION DEFINED SIZE". The term "IMPLEMENTATION DEFINED" is used as an adjective and/or adverb to describe various parameters of an implementation of the architecutre, including the size of the data accesses, or the size of the MMIO region, etc. Please check the glossary section of the manual for its definition. Here, your quote refers to the MMIO size, but your original post asked a question about data access sizes.
jatron said:It seems like you are talking about the size of the data accesses
Yes, because that is what the original question referred to.
jatron said:2. The statement says that the non-Reordering guarantee becomes void when you are reading/writing to different peripherals. In other words, the non-Reordering attribute is not required to have an impact on the order of memory accesses to different peripherals, even if both memory regions are nR.
The statement says that the order of the accesses the peripheral sees is the same as the program order if (1) the memory type has nR attribute, (2) the accesses are to the same peripheral, and (3) the accesses are of the size supported by the peripheral. Violation of any of these conditions means the nR guarantee is void.
Thanks for the clarification. I agree that IMPLEMENTATION DEFINED size can mean different things in separate contexts. A couple follow up questions:
1. Suppose software writes to two memory-mapped peripherals that are contiguous in memory. Addresses 0x000-0x1000 are for peripheral A and addresses 0x1000-0x2000 are for peripheral B. Assume both memory regions are nR.
volatile uint32_t *peripheral_a = (uint32_t *)0x0FFC; volatile uint32_t *peripheral_b = (uint32_t *)0x1000; *peripheral_a = 1; /* Assume the memory type is Device-nGnRE */ *peripheral_b = 1; /* Assume the memory type is Device-nGnRE */
Does the non-Reordering attribute guarantee that the write to peripheral A happens before the write to peripheral B? This would be important if peripheral A stores the firmware for peripheral B. Otherwise peripheral B could be released from reset before its firmware is ready.
void *fw_dest_addr = (void *)0x0000; /* peripheral A */ volatile uint32_t *reset_addr = (uint32_t *)0x1000; /* peripheral B */ /* Copy peripheral B's firmware to peripheral A's MMIO region */ memcpy(fw_dest_addr, fw_src_addr, 0x1000); /* Release peripheral B from reset */ *reset_addr = 1;
2. How do you know the original quote refers to data access sizes and not MMIO size? I feel like the quote can be interpreted both ways. Below are some additional quotes from the "Armv8 Architecture Reference Manual" that suggest that the original quote refers to MMIO size.
"The non-Reordering property is only required by the architecture to apply the order of arrival of accesses to a single memory-mapped peripheral of an IMPLEMENTATION DEFINED size, and is not required to have an impact on the order of observation of memory accesses to SDRAM."
"The non-Reordering attribute does not require any additional ordering, other than that which applies to Normal memory, between:
...
- Accesses with the non-Reordering attribute and accesses to different peripherals of IMPLEMENTATION DEFINED size."
jatron said:Does the non-Reordering attribute guarantee that the write to peripheral A happens before the write to peripheral B?
No.
jatron said:How do you know the original quote refers to data access sizes and not MMIO size?
Because of the grammar. The original quote is replicated below:
We do not read the sentence as "a single peripheral of IMPLEMENTATION DEFINED size". That interpretation is absurd.
The phrase "of IMPLEMENTATION DEFINED size" describes "memory accesses", not "peripheral".
Below is the same quote with commas placed to clarify the sentence:
"For all memory types with the non-Reordering attribute, the order of memory accesses, arriving at a single peripheral, of IMPLEMENTATION DEFINED size as defined by the peripheral, must be the same order that occurs in a simple sequential execution of the program."
Thanks for your answer to #1.
I'm not convinced about #2 though. Reading section "B2.7.2 Device memory" of the DDI 0487G.a document, I think it's more likely that "IMPLEMENTATION DEFINED size" refers to the MMIO size. Can we contact the authors to understand what the original intent was?
Comments on the content can be sent to errata@arm.com. Or, better still is to open a support case with Arm.