NVMe over Fabrics (NVMe-oF) Choose your Fabric if you dare
With NVMe being the new target storage for ultra-low latency workloads, storage vendors are grappling for ways to introduce NVMe to their portfolio. While some vendors seem to be ahead of the industry and not requiring engineering, others will need to re-engineer to retrofit NVMe or produce completely new products within their portfolio. Fortunately for those trailing the industry, the market generally gives the much-needed time to catch up, as who wants to be part of the possible version 1 teething problems roll on version 2. Potential NVMe-oF users will need time to digest the costs, performance and infrastructure requirements this new technology will bring to the table. Will the benefits of NVMe-oF be enough to justify the costs of implementation? And if so the important questions around the NVMe Fabric.
The goal of NVMe-oF is to provide distance connectivity to NVMe devices with no more than 10 microseconds (μs) of additional latency over a native NVMe device inside a server. The big question around NVMe-oF is that of what fabric to choose, this will highlight questions like “can I use my current SAN fabric whether FC or Ethernet?”, but first, let’s look at the current supported fabrics transports available.
Local Bus
This is internal to the server with the NVMe storage directly attached to the PCIe bus permitting RDMA
NVMe over Fabrics using Fibre Channel
Fibre Channel users with 16Gbs and 32Gbs fabric running the latest firmware will support NVMe.
NVMe over Fabrics using RDMA
Ethernet RoCE, Ethernet iWARP and InfiniBand will support NVMe this means that most Ethernet networks are supported.
This all sounds like good news, the bad news is the current infrastructure may support NVMe-oF, but may not provide the necessary performance to take advantage of the low latency benefits of NVMe.Current AFA (All Flash Arrays) are already moving the Storage performance bottlenecks from arrays and into the storage networks, NVMe puts an alarming amount of extra pressure on the storage network with results that 10/25/40Gbe and even 50Gbe may be inadequate. Performance testing has shown that 4 x NVMe devices can saturate a 100Gbe network. With this type of demand on the networks, dedicated NVMe fabrics could be a recommendation going forward removing all other traffic to support storage SLA on the network.
Anyone looking at deploying NVMe-oF should consider their up and coming storage network requirement, keeping in mind the high demand the NVMe-oF will place on their network.
Viadex are well positioned to provide recommendations around fabric types and alternatives to NMVe-oF until such time that the cost becomes more palatable and/or NMVe-oF performance is a critical business requirement.
For more on Viadex and how we can help you out go to www.viadex.com
