What is NVMe? A Guide to Non-Volatile Memory Express

Photo: An M.2 NVMe SSD, for used high-speed, PCIe-based reading and writing of data

Non-volatile storage media has long been one of the most restrictive bottlenecks in high-performance computing.

Humanity generates 2.5 quintillion bytes of data daily. That data has to be stored somewhere, and in the modern computing ecosystem, consumers expect to be able to access it instantaneously.

The speed increases afforded from the movement to flash storage from traditional HDDs in the last decade quickly saturated contemporary storage interfaces, such as SATA and SAS. Thus, the industry has defined new standards, such as Non-Volatile Memory Express (NVMe), to outline how new storage devices should communicate over the PCIe bus.

The PCI Express bus’ serial nature, high speed, and low latency make it an ideal candidate for superseding Serial Advanced Technology Attachment (SATA) or Serial Attached Small Computer Systems Interface (SAS) for storage interconnectivity.

Photo: A closeup of a Samsung 970 PRO M.2 NVMe SSD

What is NVMe?

NVMe, which stands for Non-Volatile Memory Express, is a computer storage interface that uses the Peripheral Component Interconnect Express (PCIe) bus to transfer data to and from solid-state drives (SSDs) at high speeds.

The NVMe specification is the industry standard for PCIe-based SSDs and is known for its lower latency, higher input/output per second (IOPS), lower power consumption, lower total cost of ownership, and improved scalability when compared to aging interfaces like SATA or SAS.

The NVMe specification is owned and maintained by the non-profit consortium NVM Express, which comprises dozens of members in the technology and high-performance computing sectors, including IBM, Intel, NVIDIA, Microsoft, Micron, NGD Systems, and many others.

The most recent version of the NVMe specification, NVMe 1.4b, was released on September 20, 2020.

Before NVMe, SSDs relied on SATA or SAS for data transfer, and both SATA and SAS are still widely used today. But NVMe revolutionized data storage by increasing the speeds at which information on SSDs is accessed. Its developers accomplished this goal by exploiting the concept of parallel computing.

Today, the NVMe interface is implemented in SSDs of various form factors, most notably the M.2 and U.2 form factors. Other NVMe form factors include the add-in card (AIC) and Enterprise and Data Center SSD Form Factor (EDSFF).

What is an NVMe SSD?

An NVMe solid-state drive, or SSD, is a high-speed storage device that implements the NVMe specification, which allows SSDs to use the PCIe bus to read and write data from and to the SSD.

NVMe SSDs are much faster than SSDs using older but still widely used storage interfaces like SATA and SAS.

The U.2 form factor is one of the most common SSD form factors and is typically found in high-performance workstations, servers, and storage systems. U.2 NVMe SSDs can usually be found connected to the PCIe bus via a miniSAS connector or an NVMe RAID controller.

The M.2 form factor is usually reserved for internally mounted NVMe SSDs often found in small-form-factor computers, such as mini PCs and laptops. M.2 NVMe SSDs are often placed directly onto the motherboard, but they can also be installed using an NVMe RAID controller.

One example of an NVMe RAID controller that supports both U.2 and M.2 NVMe SSDs is the HighPoint 7500 Series PCIe Gen 4 M.2 NVMe RAID controller.

Photo: NVMe takes full advantage of multi-core processors' parallel processing capabilities.

How does NVMe work?

Here's the simple explanation: NVMe allows SSDs to be connected directly to the CPU via the PCIe bus to read and write data at high speeds. The NVMe logic is executed within a given SSD using an NVMe controller chip.

Here's a more technical explanation: NVMe exploits the parallelism potential of today's SSDs and significantly improves command queueing, allowing SSDs to manage more requests for data at once. It breaks apart single tasks into smaller tasks and runs them in parallel.

Let's keep going: NVMe SSDs can support up to 64,000 queues and commands, respectively, compared to their SATA or SAS counterparts, which support up to 32 and 256 pending commands, respectively, in one queue. What does this mean? Basically, NVMe uses fewer CPU cycles to manage data requests than SATA and SAS.

You can fathom the performance improvements resulting from the support of up to 65,000 queues and 65,000 commands. Learn more about the performance benefits of NVMe queues in Western Digital's NVMe Queues Explained.

In a nutshell, NVMe SSDs can more efficiently handle more data compared to their SATA or SAS counterparts.

Photo: NVme SSDs are much faster than SATA/SAS HDDs and SSDs.

Are NVMe SSDs faster than HDDs and other SSDs?

NVMe SSDs are significantly faster than HDDs and SSDs using older SATA and SAS interfaces.

For example, a typical SATA III SSD can achieve up to 600MB/s read and write speeds, whereas an NVMe SSD can achieve up to 3,500MB/s and 3,300MBs read and write speeds, respectively, using PCIe Gen 3.

Using PCIe Gen 4, an NVMe SSD can achieve up to 7,000MB/s and 5,000MB/s read and write speeds, respectively.

As you can see, the performance gains provided by connecting an embedded system's non-volatile storage media directly to the CPU via the PCIe bus are nothing short of exponential.

Check out these performance benchmarks of a hard disk drive (HDD), a SATA solid-state drive (SSD), four SATA SSDs using RAID 0, and an M.2 NVMe SSD.

Transfer Speeds: SATA HDD, SATA SSD, SSD RAID, & NVMe M.2 SSD

Graph: SATA SSD vs. NVMe SSD performance benchmarks

So, according to some benchmarks, an M.2 NVMe SSD is up to 806 percent faster than a SATA HDD, up to 237 percent faster than a SATA SSD, and up to 79 percent faster than four SATA SSDs in a RAID 0 configuration.

Still not convinced that NVMe SSDs are faster? Let's look at some examples of popular NVMe and SATA III SSDs currently on the market.

Samsung's 980 PRO PCIe Gen 4 M.2 NVMe SSD boasts sequential read speeds of up to 7,000MB/s and sequential write speeds of up to 5,100MB/s. Contrastingly, Samsung's 860 PRO SATA III SSD boasts sequential read speeds of up to 560MB/s and sequential write speeds of up to 530MB/s.

Okay, so we've proven that NVMe SSDs are faster, but surely these ridiculously fast speeds come with a much heftier price tag, right?

Let's find out.

Photo: Are NVMe SSDs more expensive? Generally, yes, but there are other factors to consider.

Are NVMe SSDs more expensive than SATA and SAS SSDs?

Demand and manufacturing scales have quickly diminished the price of NVMe SSDs, but they're still generally more expensive than SATA and SAS SSDs.

Let's take a look at some examples of some popular NVMe and SATA III SSDs currently on the market.

A 512GB Samsung 860 Pro SATA III SSD retails on Newegg at $99. A 500GB Samsung 980 PRO PCIe Gen 4 M.2 NVMe SSD retails on Newegg at $119.99.

Naturally, choosing higher storage capacities will typically increase the prices of both SSDs, but interestingly, as of May 2021, a 1TB Samsung 860 PRO SATA III SSD and a 1TB 980 PRO PCIe Gen 4 M.2 NVMe SSD both retail at $199 on Newegg.

As you can see, price parity is between NVMe SSDs, SATA SSDs, and SAS SSDs is slowly being achieved.

Photo: Depending on your requirements, NVMe SSDs may be the right choice for you.

Do you really need an NVMe SSD?

Ah, yes, the ultimate question: Should I buy an NVMe SSD for my high-performance computer? Do I really need an NVMe SSD?

That depends on your program's or application's storage requirements and whether your server or workstation supports NVMe.

You can start by checking whether your motherboard has any M.2 slots since this is one of the most common NVMe SSD form factors. If not, don't worry, you can always buy a PCIe M.2 adapter and make use of any M.2 NVMe SSDs that way. If you're using a U.2 NVMe SSD, look for a miniSAS connector, since U.2 NVMe SSDs use this connector to interface with the PCIe bus.

We recommend consulting your motherboard manual to find out if this is a possibility for you. You can also ask your high-performance computer manufacturer.

If you're managing exorbitant amounts of data and need read and write speeds to be as fast as possible, an NVMe SSD may be right for you, but your motherboard needs to support the NVMe interface.

If you're managing moderate amounts of data and speed is less important than price or storage capacity, stick with a SATA or SAS SSD for now until your needs warrant an upgrade to something faster.

Photo: Trenton Systems' NVMe JBOF Enclosure

Conclusion: Trenton Systems can help you decide if NVMe SSDs are right for you.

Trenton Systems' host-agnostic NVMe JBOF Enclosure is a popular choice for the customer needing lots of high-capacity NVMe SSDs for their program or application.

Our JBOF can be populated with 24 of NGD Systems' 32TB U.2 NVMe computational storage drives (CSDs), totaling 768TB of NVMe storage. There are no adapters or switches that convert signals, which reduces latency and provides you with a much faster storage solution than traditional SATA or SAS drive setups.

Pair it with one of our servers, such as the 3U BAM Server, boasting a whopping 11 PCIe Gen 4 slots, and you're ready to take on all your data storage and management challenges.

In fact, the BAM/JBOF storage server solution has already been tested and discussed in a Trenton case study here.

We have other options for incorporating both U.2 and M.2 NVMe SSDs into your Trenton computing solution. Let's figure this out together.