Every cloud environment is rooted in virtualization and is defined by three pillars: network virtualization, server virtualization, and storage virtualization. The VMware NSX Edge Node plays an essential role in virtualizing networking and security services. The throughput supported by the NSX Edge Node is critical for the entire ecosystem and network services running on it.
In this blog, we outline NSX Bare Metal Edge performance for customers implementing Bare Metal Edge for their virtual networking infrastructure. Using NSX Bare Metal Edge (with no services running) with 4x100Gbps interfaces, RFC2544 performance tests yielded a North-South throughput of up to 388 Gbps (97%-line rate) and up to 3 Tbps for the entire cluster (when using 8 Edge Nodes), providing significant throughput for North-South traffic in the virtual network infrastructure.
Please refer to VMware NSX Bare Metal Edge Performance white paper for more information on test and settings used to achieve these results.
Key Hardware Considerations for NSX Bare Metal Edge
The choice of hardware for the NSX Bare Metal Edge is driven by bandwidth requirements and the throughput you want to achieve. Key considerations include:
- Physical NIC: Consider the bandwidth, connectivity, and resiliency requirements for the Bare Metal Edge.
- CPU: NSX Bare Metal Edge supports a broad list of processors from Intel and AMD. The major consideration for the CPU is the number of cores which can impact the throughput the NSX Edge can support.
- Memory: Selecting memory compatible with the CPU and higher transfer speed (MT/s or mega transfers per second) will influence the maximum throughput.
- PCI slots: When selecting servers, select servers from a generation that supports the PCI-E generation of NIC cards.
- Storage: Allocate appropriate disk space for your Bare Metal Edge. Local storage on the Bare Metal Edge is preferred. Also, the storage controller should be supported by the Bare Metal Edge Ubuntu software.
Test case scenario and topology
The performance tests were executed with a Bare Metal Edge in the following scenario:
1. Logical topology:
2. Test Scenario:
3. Achieved Throughput:
4. Hardware specifications used for the above test:
NSX Version: 4.0.1.1.0.20598726
BM Edge version: 4.0.1.1.0.20598726
Ring-Buffer size is increased to 4096 for Tx and RX/Hyperthreading is disabled by default/Flow control is disabled.
For performance tuning, consider adjusting ring buffer size, disabling flow control and hyper threading, and enabling flow cache. More information can be found in the linked document.
Conclusion and analysis:
This blog post outlines some key hardware considerations for maximizing the performance of the NSX Bare Metal Edge, especially for low latency and high throughput environments.
The RFC tests produced the following results:
- A single Bare Metal Edge node can achieve speeds of up to 388Gbps for North-South traffic
- An NSX Bare Metal Edge cluster can achieve up to 3Tbps for North-South traffic
For detailed study please refer to VMware NSX Bare Metal Edge Performance white paper .