https://songkou.github.io/tags/vxlan/Recent content in VXLAN on Council of ElrondHugoenSun, 19 Jul 2026 12:00:00 +0800https://songkou.github.io/posts/vxlan-evpn-architecture/Sun, 19 Jul 2026 12:00:00 +0800https://songkou.github.io/posts/vxlan-evpn-architecture/<p>VXLAN EVPN combines a scalable Layer 2 data plane with a standards-based MP-BGP control plane. VXLAN carries Ethernet frames across a routed IP fabric; EVPN distributes endpoint, subnet, and tunnel-reachability information so the fabric does not have to discover everything by flooding.</p> <details class="post-toc"> <summary>Contents — 22 sections</summary> <nav id="TableOfContents"> <ul> <li><a href="#standards-and-implementation-scope">Standards and implementation scope</a></li> <li><a href="#1-why-overlays-exist">1. Why overlays exist</a> <ul> <li><a href="#11-layer-2-and-layer-3-overlays">1.1 Layer 2 and Layer 3 overlays</a></li> <li><a href="#12-problems-vxlan-addresses">1.2 Problems VXLAN addresses</a></li> </ul> </li> <li><a href="#2-vxlan-encapsulation-and-vteps">2. VXLAN encapsulation and VTEPs</a> <ul> <li><a href="#21-header-fields-and-encapsulation-overhead">2.1 Header fields and encapsulation overhead</a></li> <li><a href="#22-virtual-tunnel-end-point">2.2 Virtual Tunnel End Point</a></li> <li><a href="#23-vlan-bridge-domain-and-vni">2.3 VLAN, bridge domain, and VNI</a></li> </ul> </li> <li><a href="#3-underlay-design">3. Underlay design</a> <ul> <li><a href="#31-underlay-routing-choices">3.1 Underlay routing choices</a></li> <li><a href="#32-ecmp-behavior">3.2 ECMP behavior</a></li> <li><a href="#33-multicast-underlay-requirements">3.3 Multicast underlay requirements</a></li> </ul> </li> <li><a href="#4-design-considerations">4. Design Considerations</a> <ul> <li><a href="#41-ibgp-overlay-with-an-igp-underlay-versus-ebgp-everywhere">4.1 iBGP overlay with an IGP underlay versus eBGP everywhere</a></li> <li><a href="#42-bum-handling">4.2 BUM Handling</a></li> </ul> </li> <li><a href="#5-vxlan-data-plane-forwarding">5. VXLAN data-plane forwarding</a> <ul> <li><a href="#51-local-switching">5.1 Local switching</a></li> <li><a href="#52-known-remote-unicast">5.2 Known remote unicast</a></li> <li><a href="#53-bum-traffic">5.3 BUM traffic</a></li> </ul> </li> <li><a href="#6-what-evpn-adds">6. What EVPN adds</a> <ul> <li><a href="#61-route-reflectors">6.1 Route reflectors</a></li> <li><a href="#62-underlay-versus-overlay-bgp">6.2 Underlay versus overlay BGP</a></li> <li><a href="#63-evpn-address-family-and-route-installation">6.3 EVPN address family and route installation</a></li> </ul> </li> <li><a href="#7-evpn-building-blocks-rd-rt-and-route-types">7. EVPN building blocks: RD, RT, and route types</a> <ul> <li><a href="#71-route-distinguisher">7.1 Route Distinguisher</a></li> <li><a href="#72-route-target">7.2 Route Target</a></li> <li><a href="#73-important-evpn-route-types">7.3 Important EVPN route types</a></li> </ul> </li> <li><a href="#8-evpn-multihoming-in-detail">8. EVPN multihoming in detail</a> <ul> <li><a href="#81-ethernet-segment-identifier">8.1 Ethernet Segment Identifier</a></li> <li><a href="#82-how-route-types-1-and-4-work-together">8.2 How route types 1 and 4 work together</a></li> <li><a href="#83-aliasing-and-known-unicast-load-balancing">8.3 Aliasing and known-unicast load balancing</a></li> <li><a href="#84-designated-forwarder-election">8.4 Designated Forwarder election</a></li> <li><a href="#85-split-horizon">8.5 Split horizon</a></li> <li><a href="#86-failure-sequences">8.6 Failure sequences</a></li> </ul> </li> <li><a href="#9-endpoint-learning-mobility-and-arp-suppression">9. Endpoint learning, mobility, and ARP suppression</a> <ul> <li><a href="#91-local-and-remote-learning">9.1 Local and remote learning</a></li> <li><a href="#92-host-mobility">9.2 Host mobility</a></li> <li><a href="#93-arp-suppression">9.3 ARP suppression</a></li> </ul> </li> <li><a href="#10-distributed-anycast-gateway-and-irb">10. Distributed anycast gateway and IRB</a> <ul> <li><a href="#101-same-subnet-forwarding">10.1 Same-subnet forwarding</a></li> <li><a href="#102-inter-subnet-forwarding">10.2 Inter-subnet forwarding</a></li> </ul> </li> <li><a href="#11-head-end-replication-and-control-plane-suppression">11. Head-end replication and control-plane suppression</a></li> <li><a href="#12-design-choices-pod-multi-pod-fabric-and-site">12. Design choices: pod, multi-pod, fabric, and site</a> <ul> <li><a href="#121-pod">12.1 Pod</a></li> <li><a href="#122-multi-pod">12.2 Multi-Pod</a></li> <li><a href="#123-multi-fabric-and-multi-site">12.3 Multi-fabric and Multi-Site</a></li> </ul> </li> <li><a href="#13-vxlan-multi-site-architecture">13. VXLAN Multi-Site architecture</a> <ul> <li><a href="#131-border-gateway-roles">13.1 Border gateway roles</a></li> <li><a href="#132-underlay-isolation">13.2 Underlay isolation</a></li> <li><a href="#133-pip-and-multi-site-vip">13.3 PIP and Multi-Site VIP</a></li> <li><a href="#134-anycast-bgw-and-designated-forwarder">13.4 Anycast BGW and designated forwarder</a></li> <li><a href="#135-bum-replication-modes">13.5 BUM replication modes</a></li> <li><a href="#136-selective-advertisement">13.6 Selective advertisement</a></li> <li><a href="#137-multi-site-control-plane-boundaries">13.7 Multi-Site control-plane boundaries</a></li> <li><a href="#138-tracking-and-restoration">13.8 Tracking and restoration</a></li> </ul> </li> <li><a href="#14-multi-site-forwarding-walks">14. Multi-Site forwarding walks</a> <ul> <li><a href="#141-inter-site-bridging">14.1 Inter-site bridging</a></li> <li><a href="#142-inter-site-routing">14.2 Inter-site routing</a></li> <li><a href="#143-inter-site-bum">14.3 Inter-site BUM</a></li> </ul> </li> <li><a href="#15-representative-cisco-nx-os-configuration-model">15. Representative Cisco NX-OS configuration model</a> <ul> <li><a href="#151-enable-features">15.1 Enable features</a></li> <li><a href="#152-underlay-loopback-and-routed-link">15.2 Underlay loopback and routed link</a></li> <li><a href="#153-overlay-bgp">15.3 Overlay BGP</a></li> <li><a href="#154-vlan-to-vni-and-tenant-vrf">15.4 VLAN-to-VNI and tenant VRF</a></li> <li><a href="#155-anycast-gateway-svi">15.5 Anycast gateway SVI</a></li> <li><a href="#156-nve-interface">15.6 NVE interface</a></li> <li><a href="#157-evpn-vni-policy">15.7 EVPN VNI policy</a></li> <li><a href="#158-multi-site-bgw-pattern">15.8 Multi-Site BGW pattern</a></li> </ul> </li> <li><a href="#16-failure-handling-and-verification">16. Failure handling and verification</a></li> <li><a href="#17-multi-tenant-connectivity-with-firewall-insertion">17. Multi-tenant connectivity with firewall insertion</a></li> <li><a href="#18-centralized-route-leaking-and-shared-services">18. Centralized route leaking and shared services</a> <ul> <li><a href="#181-why-centralize-route-leaking">18.1 Why centralize route leaking</a></li> <li><a href="#182-shared-internet-model">18.2 Shared Internet model</a></li> <li><a href="#183-common-route-leaking-failures">18.3 Common route-leaking failures</a></li> </ul> </li> <li><a href="#19-practical-design-checklist">19. Practical design checklist</a> <ul> <li><a href="#underlay">Underlay</a></li> <li><a href="#overlay">Overlay</a></li> <li><a href="#multihoming">Multihoming</a></li> <li><a href="#gateways-and-services">Gateways and services</a></li> <li><a href="#multi-site">Multi-Site</a></li> </ul> </li> <li><a href="#20-security-policy-and-operational-hardening">20. Security, policy, and operational hardening</a> <ul> <li><a href="#201-underlay-and-vtep-protection">20.1 Underlay and VTEP protection</a></li> <li><a href="#202-endpoint-and-tenant-controls">20.2 Endpoint and tenant controls</a></li> <li><a href="#203-scale-budgets">20.3 Scale budgets</a></li> <li><a href="#204-observability-baseline">20.4 Observability baseline</a></li> </ul> </li> <li><a href="#21-final-mental-model">21. Final mental model</a></li> </ul> </nav> </details> <h2 id="standards-and-implementation-scope">Standards and implementation scope</h2> <p>VXLAN EVPN is not defined by one document. The architecture is assembled from a data-plane encapsulation, an EVPN control plane, a mapping between EVPN and network-virtualization overlays, and later IRB and prefix-route extensions:</p>https://songkou.github.io/posts/vxlan-evpn-cumulus-5.4-lab-guide/Sun, 19 Jul 2026 02:00:00 +0800https://songkou.github.io/posts/vxlan-evpn-cumulus-5.4-lab-guide/<p>This lab builds a VXLAN EVPN fabric on Cumulus Linux 5.4: one BGP spine, an MLAG leaf pair (leaf1/leaf2) acting as a single logical VTEP, and a standalone leaf (leaf3). Linux1 is dual-homed to the MLAG pair over an LACP bond, while Linux21 and Linux22 share a stretched VLAN with a distributed anycast gateway.</p> <p><strong>Lab requirements</strong></p> <ul> <li>VLAN 100: Linux1 <code>192.168.100.10/24</code>, gateway <code>192.168.100.1</code></li> <li>VLAN 121: Linux21 <code>192.168.121.21/24</code>, Linux22 <code>192.168.121.22/24</code>, anycast gateway <code>192.168.121.1</code> on all leaves</li> </ul> <h2 id="lab-environment">Lab environment</h2> <p>This lab runs in EVE-NG. The four switches (<code>spine</code>, <code>leaf1</code>, <code>leaf2</code>, <code>leaf3</code>) are Cumulus VX 5.4 nodes, and the three hosts (<code>Linux1</code>, <code>Linux21</code>, <code>Linux22</code>) are lightweight Alpine-style Linux nodes configured through <code>/etc/network/interfaces</code> and OpenRC&rsquo;s <code>rc-service</code>. Any emulator that boots Cumulus VX 5.4 with the port mapping in section 2 will work; adjust the host commands if your Linux image uses a different init system or network configuration method.</p>