Things to Consider When Choosing a Network Card for Live Migration

Things to Consider When Choosing a Network Card for Live Migration


Live Migration is a technology that enables the seamless migration of virtual machines from one server to another. This feature is critical for high availability, ease of maintenance, and disaster recovery solutions in virtualization environments. In environments with high performance requirements, adequate network infrastructure and hardware are crucial.

Using a High-Performance NIC
  • 10 Gbps or Higher Speeds: Live Migration moves large amounts of data. A high-speed network card (e.g., 10 Gbps, 25 Gbps, or higher) is preferred.
  • SR-IOV Supported NIC: Single Root I/O Virtualization (SR-IOV) support improves network performance. SR-IOV allows VMs to directly access the network card and reduces CPU load.

SR-IOV (Single Root I/O Virtualization) is a technology that allows a single physical network card (NIC) to directly access multiple virtual machines (VMs). This technology enables network device hardware resources to be shared among virtual machines, resulting in high performance and low latency in virtualization environments . SR-IOV helps physical network cards communicate more efficiently with virtual machines because the virtual machines directly access the network card.


  • Sharing the Network Card to Virtual Machines:
    • SR-IOV partitions a physical network card into multiple virtualized network interfaces . Each virtual machine can directly access these virtual network interfaces (vNICs). This is similar to how VMs have their own dedicated virtual network cards instead of sharing a physical network card.
  • Hardware-Based Virtualization:
    • SR-IOV enables hardware virtualization of network cards. This reduces CPU overhead because network processing is handled directly by the network card, eliminating the need for an operating system or hypervisor (virtualization software).
  • Low Latency and High Performance:
    • Because SR-IOV provides direct hardware access, it offers high performance and low latency . This is especially important for high-bandwidth applications and systems that require low latency tolerance .
  • Efficient Sharing of Network Card Resources:
    • SR-IOV provides multiple virtual network connections per network card by efficiently distributing the physical network card. Each vNIC can utilize the capacity of the physical network card.


Benefits of SR-IOV:

  • Low CPU Usage:
    • SR-IOV reduces CPU overhead on network traffic. Network traffic is processed directly by the physical network card, eliminating any additional workload on the hypervisor or operating system.
  • High Bandwidth:
    • Because each virtual machine communicates directly with the physical network card, high bandwidth is provided. This is ideal for operations requiring large data transfers.
  • Lower Latency:
    • Because SR-IOV manages network traffic directly with the device, network latency is significantly lower. This is a critical advantage in high-performance scenarios such as Live Migration , data center applications , and cloud services .
  • Efficient Virtual Machine Management:
    • SR-IOV efficiently shares the network card among virtual machines so that each VM can operate as an independent network connection.

Using a Separate NIC
  • Use a separate physical NIC for Live Migration. This avoids conflict with other operations such as Hyper-V management traffic , VM traffic, and storage traffic.
RDMA Supported Cards

  • Network cards with RDMA (Remote Direct Memory Access) support reduce CPU load and improve Live Migration performance.
    • Cards that support RoCE (RDMA over Converged Ethernet) or iWARP protocols may be preferred.

RDMA (Remote Direct Memory Access) is a technology that allows two computers to directly access their memory over a network. This technology allows data to be transferred directly between the memory, bypassing the computer's CPU and operating system when being transferred over the network . This significantly reduces network latency and CPU overhead, resulting in much faster and more efficient data transfer.

Key Features of RDMA:
  • Low Latency and High Throughput:
    • RDMA allows data transmission with low latency because data is processed and transferred directly in memory, meaning there is no load on the processor.
    • In this way, data transfer becomes faster and congestion on the network is reduced.
  • Reducing CPU Load:
    • In traditional network transmissions, the CPU and operating system are involved in processing the data, but with RDMA, this processing is done directly by the hardware, thus drastically reducing the load on the CPU.
  • Memory Access:
    • RDMA provides direct access to the memory of remote machines, allowing both systems to read and write data to their memory during data transmission over the network.
  • High Bandwidth:
    • RDMA is particularly suitable for applications requiring high bandwidth, such as workloads requiring high-volume data transfer, such as financial transactions or big data analytics.

RDMA works through network hardware (NICs) . This hardware is provided by specialized network cards that support RDMA (e.g., Mellanox ConnectX series, Intel Omni-Path). These cards allow data to be written to and read from memory directly without touching the system's processor. Two main protocols are used for this:

  • RoCE (RDMA over Converged Ethernet):
    • RDMA is an RDMA protocol that operates over Ethernet networks. It is generally preferred in data centers with Ethernet infrastructure.
  • iWARP (Internet Wide Area RDMA Protocol):
    • RDMA technology operates over TCP/IP protocol. It enables the use of RDMA over the Internet.
NIC Fault Tolerance and Redundancy
  • Redundancy and high availability can be achieved by combining multiple network cards with NIC Teaming or Switch Independent Mode .
Network Configuration for Live Migration

Using a Dedicated Network

  • Use an isolated network for Live Migration traffic. This prevents other network traffic from affecting Live Migration performance.
    • For example: You can configure VLAN for Live Migration.

QoS (Quality of Service) Configuration
  • Configure QoS settings on the network to prioritize Live Migration traffic.
    • In Hyper-V, you can regulate traffic with the Weight or Minimum Bandwidth settings.
IP Addressing
  • Assign a different subnet or a different IP configuration to the network cards used for Live Migration. This will help you avoid interfering with other traffic.

NIC Selection and Configuration for Live Migration Step by Step

Step 1: NIC Selection

  • Choose a card with 10 Gbps or faster.
  • If you're using an RDMA-enabled card, enable RDMA. This should work with the SMB Direct protocol.
Step 2: Physical Network Separation
  • Assign a separate network card for Live Migration.
  • For example:
    • NIC 1: VM Traffic
    • NIC 2: Storage Traffic
    • NIC 3: Live Migration Traffic
Step 3: Configure Live Migration Settings
  • Open Hyper-V Settings .
  • Click on Live Migration in the left menu .
  • Check the box Enable incoming and outgoing live migrations .
  • Select the Use these IP addresses for live migration option and enter the IP address of the network card reserved for Live Migration.
Step 4: SMB Direct and RDMA Control (Optional)
  • You can use PowerShell to enable RDMA:

    Set-NetOffloadGlobalSetting -NetworkDirect Enabled

Step 5: Traffic Monitoring and Performance Testing
  • You can use performance counters (Performance Monitor) or network monitoring tools to monitor network traffic during the Live Migration process.

Controlling and Managing SR-IOV Feature

Checking SR-IOV Support

Get-NetAdapterAdvancedProperty -Name "<NetworkAdapterName>" | Where-Object {$_.DisplayName -like "SR-IOV*"}
#Or
Get-NetAdapterSriov -Name "*"
#Or
Get-NetAdapterSriov -Name "<NetworkAdapterName>"

Enabling SR-IOV Feature

Set-NetAdapterAdvancedProperty -Name "<NetworkAdapterName>" -DisplayName "SR-IOV" -DisplayValue "Enabled"
#Or
Enable-NetAdapterSriov -Name "<NetworkAdapterName>"

Disabling SR-IOV Feature

Set-NetAdapterAdvancedProperty -Name "<NetworkAdapterName>" -DisplayName "SR-IOV" -DisplayValue "Disabled"
#Or
Disable-NetAdapterSriov -Name "<NetworkAdapterName>"
Control and Manage RDMA Feature

Checking RDMA Support
To get information about RDMA support, you can use the following command:

Get-NetAdapterRdma -Name "<NetworkAdapterName>"

Enabling RDMA Feature
To enable RDMA, you can use the following command:

Enable-NetAdapterRdma -Name "<NetworkAdapterName>"

Disabling RDMA Feature
To disable RDMA, you can use the following command:

Disable-NetAdapterRdma -Name "<NetworkAdapterName>"
Extra Tips

  • Update NIC Firmware: Keep your network card drivers and firmware updated.
  • Test in Cluster Environment: Try the Live Migration process with a test VM in different scenarios (under load, idle).
  • Backup: Create NIC Teaming configuration to backup traffic.
  • Iperf: Iperf is a widely used tool for testing network bandwidth and latency. You can test network connections to measure the effects of SR-IOV and RDMA.

iperf -s # Server side
iperf -c <ServerIP> # Client side
Previous Post Next Post
Search
Recent Posts

Tags
  • Business
  • Digital
  • IT Solution
  • Technology
  • Cyber Security
  • Digital
  • Finance
  • Software