vSPhere 6.7 – NVMe Disk Controller

VMware has added a lot of new media and IO related enhancements to vSphere 6.7. One of those enhancements is the Virtual NVMe Disk Controller.
Also check out my blog on new Persistent Memory features in 6.7! This is exciting stuff. here

First of all, this disk controller does not require the underlying media to be NVMe. You can attach a VMDK file from any datastore to the NVMe controller.

I ran some basic IO tests comparing the performance of the NVMe controller vs the Paravirtualized SCSI controller. My initial results show that there isn’t a benefit to using the NVMe controller with VMDKs that aren’t backed by an NVMe controller. For virtual disks that are backed by an NVMe controller, I am seeing a significant advantage.

While neither my test system nor my benchmarks were not optimized for ultra high performance. There was a significant increase in IO with a significant reduction in disk utilization. I am going to dive deeper into this in another thread and see how far I can push things. I’m also curious what I can push out of an NVMe RDM attached to the virtual NVMe controller.

Adding the NVMe controller is not difficult, however, NVMe devices are not treated the same way as SCSI devices are so there are some new considerations.

I’m using Ubuntu 18.04 for my test VM so installing the OS level prerequisites may be different for your Operating System.

IMPORTANT NOTE: Claims about performance enhancements depend greatly on the configuration of the Hardware, Hypervisor, and Guest Operating System. I will have a high-performance blog entry coming out soon that will go into more detail.

Step 1: Upgrade VM Hardware to Version 14 (ESXi 6.7 Compatible)

After Clicking through the prompts, you can see that your VM is now HW Version 14 which can support NVMe (and you may notice some other cool things I will talk about soon)

Step 2: Install Virtual NVMe Controller for the VM

Step 3: Power on the VM

Step 4: Upgrade / Install VMware Tools to the latest version. (Either the VMware Guest Tools Installer or the open-vm-tools package work fine)

Step 5: Configure your VM for NVMe Support

Verify that your VM has an NVMe Controller

Install the nvme-cli utility

List NVMe Namespaces. None are listed here because I did not attach a drive to the controller yet (so I can show you hot add).

Step 6: Attach a Virtual Disk to the NVMe Controller

Step 7: Re-scan for new NameSpaces note that none are listed yet, verifying hot-add.


Identify the device ID of your NVMe Controller

Re-scan the Controller for new Namespaces

Step 8: Create a new file system. Although some applications can use a Namespace natively, most of us will need to use a file system.

Your Namespace now has a file system on it. You can mount it and use it like any other drive. If the Hardware Controller backing the VMDK file is NVMe then you should see a significant performance advantage.

More information about managing NVMe Namespaces in Linux can be found here:

https://manpages.debian.org/testing/nvme-cli/nvme.1.en.html

https://www.mankier.com/package/nvme-cli


Advertisements

One thought on “vSPhere 6.7 – NVMe Disk Controller

Add yours

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Blog at WordPress.com.

Up ↑

%d bloggers like this: