Persistent Memory 101

I’ve written a few guides for Persistent Memory recently and slipped in bits and pieces of information here and there. I decided to consolidate the little things like nomenclature into a different place. So if things in another post aren’t clear just because you haven’t read extensively then the answers should be here. Moving those things here will make them easier to find and keep them consistent. Hopefully its also less distracting to the content in the other posts.

Persistent Memory is different than traditional non-volatile memory in that it is actual memory addressable memory. Were talking about DIMMs located in DIMMs sockets on a motherboard. This is actual DRAM that that has an added mechanism or mechanisms for making it persistent across reboots or power outages.

While being incredibly fast, RAM does not easily lend itself to being a storage media. Most applications function by accessing a block device, not a memory region and these constructs just aren’t available without some assistance. DIMMs are not inherently all that serviceable. You have a pool of RAM and one stick goes bad. How can this be tolerated? How can I perform a hot-memory replacement with minimal impact? To address all of these things, CPU and Motherboard manufacturers have had to extend some specifications in order to do things like partition and group the DIMMs in intelligent and/or configurable ways.

Operating Systems have had to add device types and features to support an array of different access methods. Do we want to access our Persistent Memory as a traditional block device, a PMEM aware block device, or a character device that applications can access natively? All of these are possible but require different configuration steps and have different performance profiles.

Persistent Memory Nomenclature

PMEM
PMEM refers to DRAM memory spaces which have been backed by a persistence mechanism such as a battery and/or NAND directly attached for de-staging.

NVDIMM
An NVDIMM is the physical component of Persistent Memory. NVDIMMs fit into a RAM socket on an NVDIMM compatible motherboard.

Region
A PMEM Region is a logical unit of PMEM. A region could be a single NVDIMM or a partition on a single NVMDIMM. It could be all of the space on all of your NVDIMMs collectively, or it could be a partition sliced across multiple NVDIMMs. As a gross generalization, Regions are configured in BIOS/EFI and are constructed before the OS boots. I like to think of NVDIMMs as physical disks and a Region as a Logical Drive.

LIBNVDIMM
The Linux Kernel PMEM driver. This library is required for initializing NVDIMM Regions, and constructing Namespaces.

Name Space
Much like NVMe, PMEM makes use of Name Spaces. A Namespace can be an entire Region or a piece of a Region. The Namespace is the basic construct the Operating System will work against.

Page Cache or Buffer Cache
Read-after write cache where blocks written to persistent media are tracked and cached in memory. This process speeds up IO for disk based media. This process is unnecessary for PMEM and can actually slow things down. When PMEM is used as a block device hosting a file system, the page cache is in use.

DAX
Capability that allows a file system to bypass the page cache and write directly to PMEM. Currently EXT4 and XFS have DAX support if mounted with the dax mount option.

vPMem
VMware nomenclature for passing NVDIMM directly to a VM. When using vPMEM, PMEM capacity is passed directly through to a VM as a virtual NVDIMM. The guest Operating System must support NVDIMMs.

vPMemDisk
VMware nomenclature for presenting PMEM capacity to a VM as a vmdk file connected to a virtual SCSI controller.

ndctl
User space cli tool for configuring PMEM namespaces

PMDK
The Persistent Memory Development Kit provides additional tools and libraries for managing PMEM.

Character Device
Devices where the driver communicates by sending and receiving a single character at a time rather than a whole block of data. This is the type of device used by applications with native PMEM support.

Advertisements

Comments are closed.

Blog at WordPress.com.

Up ↑

%d bloggers like this: