Pure FlashArray //CPure FlashArray //XPure StoragePrimary StorageData Center
Article • • 6 minute read

Trust the NVMe Hype

Everything you need to know about the next-generation storage medium

In this article

 The next big thing: NVMe

Non-volatile memory express, or NVMe, is poised as the next big paradigm shift in enterprise storage, but what exactly is it and why is it important? What does choosing NVMe mean for your data center over the next five years?

How did we get here?

The term NVMe is thrown around a lot, and the various stages of NVMe can cause confusion. Some manufacturers say their arrays are NVMe ready, while others say they have an NVMe back-end or front-end. These terms mean different things, and it's important to define NVMe and its variants.

At a very basic level, NVMe is a storage protocol that replaces the Small Computer System Interface (SCSI) protocol. SCSI protocol was designed many decades ago to address communication to a magnetic media (spinning drives). Conversely, NVMe is designed to take advantage of solid state drives (SSD), or flash, media.

SCSI was originally implemented with a limited single command queue that had a maximum queue depth of 32 commands. This single "lane" queue has proven to be a major bottleneck as storage media has evolved from spinning disk drives to SSD.

Why NVMe matters

In contrast to SCSI's single queue with 32 commands, NVMe has 64,000 parallel queues, each capable of holding up to 64,000 commands. The parallel queues greatly reduce latency, resulting in dramatically better throughput and overall storage performance.

The result is NVMe-based arrays that deliver lower latencies, better throughput and higher IOPS over first generation all-flash arrays based on SCSI. For applications, this means increased I/O parallelism, which translates into better CPU utilization. In simple terms, real-time applications run much faster (performance) with much greater density in the datacenter (efficiency).

The increase in performance and efficiency is key for applications dealing with real-time anything, especially cyber security, fraud analytics, data analytics, image processing (autonomous vehicles and facial recognition), financials and database deployments like Oracle or SAP.

These speed-sensitive applications also enjoy more predictable performance in addition to better efficiency, resulting in applications that perform more consistently with less required infrastructure.

NVMe variations

With all new technologies, it's important to discuss in terms of the overall IT architecture supporting an application. NVMe makes storage much faster, but it moves contention from the storage array back to the connective storage fabric, computation and operating system layers of the architecture.

In layman's terms, a new engine (NVMe drives) makes a car much faster, but without the right transmission (storage controller), tires (host), driver (host operating system) and an open road (storage fabric), it's not optimized.

Back-end (drives) NVMe

This refers to a storage array that uses NVMe drives, or media. This replaces standard SCSI-based SAS SSDs with NVMe SSDs and NVMe shelf connectivity. This is the most common type of NVMe array on the market today. Using the car analogy, back-end NVMe upgrades the car's engine, but leaves the tires alone.

Back-end Nand SSD

Front-end / NVMe-OF (host/controller) NVMe

This is an array that uses the NVMe protocol on the fabric between the host (server) and the storage controller. This might leverage different types of connectivity, including fibre channel media (FC-NVMe) or ethernet media (iWARP or RoCE).

Although infrequently used in the commercial space, another implementation of NVMe-OF relies on infiniband as transport. This implementation provides extremely low fabric latency and is more commonly encountered in high performance computing environments.

To implement NVMe-OF between the host and the storage controller, a customer needs a fabric that supports FC-NVMe, iWARP or RoCE, as well as adaptors for the host, or server. NVMe-OF upgrades the road the car drives on, along with the tires.

Back-end NVMe SSD

Operating systems

Operating system support should also be considered when planning for NVMe-OF. At the time of this writing, Suse Linux is the only supported OS that can take advantage of NVMe-OF. This means Suse Linux can take full advantage of the entirety of the NVMe fabric and realize the tremendous performance improvement that comes with it. Windows, VMware, Red Hat and others are currently in the development phase and will be released soon.

Non-NVMe-OF supported operating systems can still take advantage of storage arrays that have implemented NVMe media. Implementing NVMe on the back-end of arrays can only provide additional IO and reduce latency but will not support the full protocol benefits for operating systems and arrays that support NVMe-OF.

NVMe end-to-end

NVMe or NVMe-OF ready

Many manufacturers use the term "NVMe or NVMe-OF ready" to describe an array that will either only require a software update to support NVMe-OF or an array that can be upgraded (I/O modules may need to be replaced and software upgraded) to deliver NVMe-OF connectivity. It is important to get the details surrounding this statement from the manufacturer you are evaluating.

Why look at NVMe now?

In the last year, we've seen SSD drives in enterprise all flash arrays increasing in capacity; 15 and even 30TB drives are already available for purchase. When considering that with only 48 of those 30TB drives, an all flash array can easily provide over 1PB of usable capacity. Factoring in data reduction technologies available today, this can easily translate into 2.5PB of usable storage on a single array.

Even if a storage array could provide 1 million IOPs, given the amount of data stored, this would only provide 400 IOPs for every TB of data stored. To really benefit from the density that all flash arrays can achieve, organizations need to start evaluating their NVMe options.

NVMe lab at WWT

We recognize that planning for NVMe is difficult and involves multiple considerations within a well thought out architecture to get it right. When considering a move from traditional storage to NVMe, we invite you into our Advanced Technology Center (ATC) for a comprehensive NVMe storage briefing. Our experts will cover every detail surrounding NVMe and take your specific needs into consideration.

The NVMe infrastructure currently hosted in our ATC includes:

NVMe Storage Arrays

  • Full NVMe Arrays
    • NetApp A800
    • Apeiron AES 1000 (Full Proprietary NVMe)
  • Back-end NVMe Arrays (These will be full NVMe by 1stquarter 2019)
    • Dell EMC PowerMax
    • Pure FlashArray //X70
    • Pure FlashBlade
    • Vexata VX100F (Back-end NVMe ESMs)
  • NVMe-OF Arrays
    • NetApp A700 (FC-NVMe)
    • NetApp EF570 (IB-NVMe)

NVMe Fabric

  • Current: Cisco 9706 MDS Directors (NVMe over Fiber Channel)
  • Future: RoCE/iWARP future state

NVMe Host/Compute

  • Dell R740XD Servers – 32Gb FC to Support FC-NVMe

Get started with testing today by requesting our Enterprise Storage Infrastructure Workshop.

Technologies