VXLAN (Virtual eXensible Local Area Network) is a encapsulation overlay protocol created to address a few common datacenter challenges. These challenges have become a lot more acute with the advent of cloud computing and multi-tenancy resource consumption. VXLAN was developed by Cisco, VMware, Microsoft any many others. It was developed to address the following datacenter concerns:
- The need for more mac address tables on the physical infrastructure. As virtualization increases so do the mac address tables to a point where layer 2 switching has become very costly.
- The need for layer 2 seperation beyond the current 4096 VLAN’s. With only 4096 available some datacenters have run out of VLAN’s
- With multi-tennancy it’s possible that administrators will have duplicate mac addresses or IP addresses which can cause security issues when implemented over shared links.
- Layer 2 does not implement redundancy like layer 3 does. A network administrator may wish to use equal cost multipath to provide redundancy which is not possible on layer 2.
- STP (Spanning tree protocol) used to avoid loops on layer 2 can potentially shut down paths that could be used to provide more bandwidth.
- Layer 3 alone cannot provide separation due to the possibility that some tenants may use the same IP ranges.
For these reasons and more VXLAN was developed to carry layer 2 traffic in an encapsulated tunnel. This tunnel is implemented with an additional header placed over a layer two packed and sent out as a UDP multicast. Each VXLAN is separated into VXLAN segments (think VLAN’s but the delievery is done via UDP multicast addresses, each segement has it’s own address)
We then implement a VTEP (VXLAN Tunnel End Point) which can subscribe to the multicast traffic. When a message is received the VTEP translates it into the correct VXLAN segment and decodes the message into a normal layer 2 packet. Then end points have no knowledge of multicast or VXLAN they think all communication is done via standard layer 2. It is important that the VTEP is implemented as close as possible to the hypervisor.
This technology can also be used to stretch a layer 2 network using IP across datacenters: Making it possible for the same layer 2 network to exist in two different locations.
Since the VXLAN throws a new header on the top of standard ethernet it is critical that your network supports frame sizes of at least 1600 (which is the MTU of a VXLAN segment).