VXLAN_EVPN_Cumulus_Lab_Test

Sun, 19 Jul 2026 02:00:00 +0800

This lab builds a VXLAN EVPN fabric on Cumulus Linux 5.4: one BGP spine, an MLAG leaf pair (leaf1/leaf2) acting as a single logical VTEP, and a standalone leaf (leaf3). Linux1 is dual-homed to the MLAG pair over an LACP bond, while Linux21 and Linux22 share a stretched VLAN with a distributed anycast gateway.

Lab requirements

VLAN 100: Linux1 192.168.100.10/24, gateway 192.168.100.1
VLAN 121: Linux21 192.168.121.21/24, Linux22 192.168.121.22/24, anycast gateway 192.168.121.1 on all leaves

Lab environment

This lab runs in EVE-NG. The four switches (spine, leaf1, leaf2, leaf3) are Cumulus VX 5.4 nodes, and the three hosts (Linux1, Linux21, Linux22) are lightweight Alpine-style Linux nodes configured through /etc/network/interfaces and OpenRC’s rc-service. Any emulator that boots Cumulus VX 5.4 with the port mapping in section 2 will work; adjust the host commands if your Linux image uses a different init system or network configuration method.

RoCE_Cumulus_Linux

Fri, 17 Jul 2026 20:00:00 +0800

Part 1: RoCE and RDMA fundamentals

In the context of NVIDIA/Cumulus Linux and HPC networking, RoCE stands for:

RoCE = RDMA over Converged Ethernet

It is widely used in AI clusters, HPC, GPU clusters, storage networks, and low-latency data centers.

What is RoCE?

RoCE allows RDMA (Remote Direct Memory Access) to run over an Ethernet network.

Normally, when one server sends data to another:

Application
 |
 v
TCP/IP Stack
 |
 v
Kernel
 |
 v
NIC
 |
 v
Ethernet

The CPU has to copy memory, handle interrupts, process TCP/IP, and switch between user and kernel space. This consumes CPU cycles and adds latency.

Cumulus on Council of Elrond

VXLAN_EVPN_Cumulus_Lab_Test

Lab environment

RoCE_Cumulus_Linux

Part 1: RoCE and RDMA fundamentals

What is RoCE?