Numerous articles and papers have been written on emerging non-volatile memory (NVM) over the last few decades. These emerging technologies have been hyped as either:
Figure 1: PCM Gartner Hype Curve
While NVM technologies are still an active research topic, most of the ‘emerging’ technologies in the forefront are not actually new. Using phase change memory (PCM) as one example, the development cycle over the last five decades is illustrated with a Gartner Hype Curve. The first article on PCM was written in 1970 but it took over 20 years for the technology to gain initial momentum. Around 2007, industry targeted PCM as a NOR Flash replacement. In the research community, several papers were also published on hybrid (DRAM + PCM) memory solutions, claiming that PCM would replace Flash in the near term and eventually replace DRAM due to the scaling, speed and endurance characteristics of the technology1. However, PCM was not mature enough to live up to the expectations and take over the market in 2007. Fast forward ten years later, PCM is back in the news, now as a predominant storage class memory (SCM) solution. With the current maturity of PCM and other emerging technologies, we are finally moving towards the Plateau of Productivity.
The move from HDD to SSD initiated an interface change, from SAS and SATA to NVMe PCIe. NVMe, layered on top of PCIe, is a storage protocol that was designed to take advantage of faster SSD (non-volatile) media with a streamlined command set and significantly lower overheads. Parallel IO is a key component that is possible due to 64,000 Command queues, each with 64k entries compared to single queue depths of 32 and 356 for SATA and SAS respectively2.
Just as SSD’s triggered innovation for faster storage protocols, emerging NVM has triggered innovation for new memory protocols and optimized storage solutions, redefining the memory sub-system.
Figure 2: Emerging NVM Landscape
In the emerging NVM landscape no two memory technologies are equal, and similar to DRAM, there are various optimizations possible within each technology. The technology characteristics will determine the attachment point, protocol requirements, and how the memory is used with both volatile and non-volatile main memory use cases viable. Volatile memory uses include transformative capacity for big data applications and lower total cost of ownership (TCO) for multi-tenant systems. With non-volatility, applications are expanded to incorporate; check-pointing, journaling, logging, metadata storage, and persistent caching (just to name a few).
For all these use cases, a dedicated interface is required. Focusing on the DRAM bus, the following sections outline the requirements, trade-offs, and current status of emerging NVM interfaces.
DRAM protocols are currently defined in JEDEC, the Joint Electron Device Engineering Council standards body. JEDEC was created in 1958, with the first DDR spec, JESD-79, published in 2000. The organization has more than 300 member companies, which develop open standards for both DRAM devices and DIMMs (Dual In-line Memory Modules).
In 2018, JEDEC released the initial NVDIMM-N design specification, JESD248A. A year prior, the BAEBI (Byte-Addressable, Energy Backed Interface), firmware specification was published to define the software interface for NVDIMM-N. Essentially, this DIMM is a non-volatile DIMM with both DRAM and NAND, in which the NAND simply mirrors and backs up the contents of the DRAM. The host directly interfaces with the DRAM for fast, byte addressable memory access. The NAND is managed on-DIMM and written on a power-fail event to persist the DRAM contents. When power is restored, the on-DIMM media controller transfers the contents back to the DRAM for normal operation. The module is pin compatible with DDR4 and supported by the Linux kernel. With BIOS changes (e.g. NFIT), a system can see and manage the DIMM memory as persistent memory. Natively interfacing with DRAM provides a low-latency, byte addressable interface, but given that both NAND and DRAM coexist on the DIMM, the capacity is limited. Currently, 16GB NVDIMM-N solutions are available compared to 64GB commodity DDR4 RDIMMs. More performant, byte addressable NVM changes the landscape and removes the need for equal DRAM and NVM capacities on a single DIMM.
In addition to NVDIMM-N, JEDEC working groups are actively defining new specifications for NVM interfaces: NVDIMM-P and the NVRAM addendum to DDR5. These specifications specifically target emerging technologies while reusing existing DRAM frameworks. The NVDIMM-P and NVRAM addendum both leverage the DDR bus and are DDR pin compatible for minimal host controller changes and easier adaption. With the potential for larger density media, support for a larger address space is a base requirement for both specifications. For reference, the current JEDEC DDR4 specification supports 16-Gigabit devices. A standard 2-rank, 64-bit DIMM comprised of x4 devices in an 8-high 3DS package can achieve a total capacity of 512-Gigabytes. Alternatively, emerging NVM, such as PCM or ReRAM, has the potential to supply multiple terabytes of capacity per DIMM.
Figure 3: Gaussian Distribution of Memory Latency
The key difference between the NVRAM addendum and NVDIMM-P is non-deterministic timing and agnostic media support. Memory latency distribution in one metric that establishes the required level of latency determinism as shown with the Gaussian Distribution graph. Non-volatile media with a tight tail latency distribution for both read and write bursts can be managed and accessed with a deterministic interface like the NVRAM addendum. Alternatively, the NVDIMM-P specification supports non-deterministic timing and is media agnostic to enable support of a breadth of media technologies with varied characteristics and wider tail latency distributions.
The NVRAM addendum is built on top of the DDR5 device specification (still in development). The goal is to use the standard RAS, CAS commands to read and write the media contents with deterministic latencies. To account for larger capacities, NVRAM plans to introduce a new command, REXT (Row Extension) to increase addressability by 12-bits. Depending on the media profile, DRAM specific functions like Precharge, Refresh, bank power management may not be required, simplifying the state diagram and minimizing media specific overheads. Simulation results for random DRAM read data traffic interleaved across resources roughly correlate with other claims3 demonstrating 10% performance lost just for periodic refresh operations. However, the actual performance potential is dependent on the media type.
Non-volatile media that requires block access or management for wear-leveling and/or error correction can exhibit variable latency, requiring support for non-deterministic delays. With NVDIMM-P, the non-determinism is managed simply with an additional asynchronous feedback channel from the NVM media, alerting the host SoC when requests are complete.
Figure 4: Moving from a Deterministic to non-Deterministic Interface with a Handshake Extension
NVDIMM-P specifications are in development for both DDR4 and DDR5 compatible interfaces. Similar to the NVRAM addendum, these modules will be electrically compatible with a DRAM module. However, some pins are re-purposed and/or defined for an asynchronous feedback used to signal ‘Read Data Ready’ or DIMM status updates (interrupts, warnings, urgent error conditions, etc.). The address is extended with a new command that coupled with read and write commands enable up to 8TB of capacity per physical rank for DDR4 solutions4. Larger capacities, up to 32TB per DIMM will be available with DDR5 solutions. These modules can share a channel with DRAM and consequently all data transmission must be deterministic so that the host can schedule data access without bus contention. Once the host receives notification that data is ready via the asynchronous feedback, a new Send command can be issued by the host. This will trigger data transmission from the NVDIMM-P with a fixed latency. The following figure illustrates the interplay between non-deterministic media delay and deterministic data transfer for a read data transfer.
Figure 5: Non-Deterministic Read Timing Diagram
The host is able to offload multiple commands to the NVDIMM-P and the on-DIMM media controller can optimize execution of these commands based on resource availability, resulting in out of order completion. For read requests, the host must be able to correlate the data with the command and address. Existing 'packetized' protocols have “Tag” fields, incorporated in the request and response packets for this purpose. Similarly, NVDIMM-P defines additional meta-data (a read ID) to correlate response and requests on the host SoC. The meta-data field is transferred in-line with the data similar to ECC on ECC DIMMs. For DDR4 NVDIMM-P 8-bytes of meta-data are transferred for every 64-bytes of data. As already described, the meta-data includes a read ID to manage out of order read responses. Additionally, this bus contains the following fields to validate transmission, provide user defined flexibility, and manage write buffers.
The point of persistence has been defined as:
"The point in a memory system, if it exists, at or beyond the Point of Coherency, where a write to memory is maintained when system power is removed, and reliably recovered when power is restored to the affected locations in memory5."
This implies that the point of persistence is not necessarily at the media. Current solutions like NVDIMM-N utilize tethered battery backup to provide the energy required to push data from volatile media to NVM in a power-fail event. In this case, the point of persistence is the volatile media in which the battery backup is sufficient to successfully push the data to NVM in a power-loss scenario. While this is adequate for most cases, critical data may require a smaller failure domain where potential failures in the backup energy and/or backup process are minimized or optimally eliminated. From a programming model perspective, SNIA has defined “Deep FLUSH” as a flush “to the most reliable persistent domain available to software.” For deterministic media, this may simply be realized via a defined delay, with persistence intrinsically achieved. For non-deterministic media, this requires explicit feedback when a flush to the most reliable persistent domain is complete. With NVDIMM-P, multiple mechanisms are included in the specification to persist data. These include NVDIMM-N compatible solutions to minimize system and software impact. For this case, the NVDIMM is energy backed and a final FLUSH is issued on a power-fail event to move any buffered or cached data to NVM. For “Deep FLUSH” support, NVDIMM-P also specifies persistent write and FLUSH operations that can be issued during normal operation with feedback from the DIMM when the reliable persistent domain is achieved. The NVDIMM-P solution can therefore support both shallow and deep persistence guarantees for varied use-cases and requirements.
Emerging technologies are driving the need for new standards, with NVDIMM-P and the NVRAM addendum as examples under development in JEDEC. To help drive these efforts, Arm is actively involved, co-chairing the Hybrid DIMM Task Group and chairing the Future Memory Task Group which are developing the NVDIMM-P and NVRAM addendum specifications respectively. The JEDEC workshops on Oct 10 and Oct 17 in Santa Clara and Hsinchu Taiwan respectively provided details of the NVDIMM-P specification with Arm presenting at both workshops. Other non DDR interfaces, like CXL, are also viable for new memory technologies providing more composable systems at the potential cost of longer latency. More information on these attachment points in future blogs!
From the programmer perspective, new programming models are being developed to guarantee failure atomicity. These include various techniques like checkpointing, logging and journaling. Libraries like PMDK have also been developed for persistent memory to ease the programmer burden. Within Arm Research, work is ongoing to evaluate architecture requirements while analyzing the trade offs of failure atomicity overheads and programming complexity. A previous blog, Persistency for Synchronization Free Regions, details one solution. Additionally, persistent atomics for durable lock free data structures has also been announced at SPAA.
1 Stephan J. Hudgens, et al; The future of phase-change semiconductor memory devices, Journal of Non-Crystalline Solids, Volume 354, Issues 19–25, 2008, Pages 2748-2752.
2 Carol Sliwa; Speeds of storage networking technologies rise as flash use spikes: NVMe PCIe SSDs boost speed, lower latency over SAS and SATA SSDs. TechTarget.
3 Bill Gervasi; Expanding the world of heterogenous memory hierarchies: The evolving non-volatile memory story. MSST Conference, 2019.
4 Rob Peglar; What you can do with NVDIMMS. Persistent Memory PM Summit, 2019.
5 Arm; ARM Architecture Reference Manual ARMv8, for ARMv8-A architecture profile. 2017.
Contact Wendy Elasser