Recently I have been thinking about configuration maximums of the current virtual distributed switches.   In the configuration maximum’s document for 5.5 it states the following:

– Total virtual network switch ports per host (VDS and VSS ports) – 4096
– Maximum active ports per host (VDS and VSS) – 1016
– Hosts per distributed switch – 1000
– Static/Dynamic port groups per distributed switch – 6500
– Ephemeral port groups per distributed switch – 1016
– Distributed virtual network switch ports per vCenter – 60000

The first question is between these numbers:

– Total virtual network switch ports per host (VDS and VSS ports) – 4096
– Maximum active ports per host (VDS and VSS) – 1016

In order to explain these numbers you must have some context about how a vDS and VSS work and allocate ports:

  • virtual standard switch (VSS)- allocates ports statically when a port group is created on the local ESXi host – so if you allocate 24 ports to a port group then 24 ports are taken.
  • virtual distributed switch (dVS) – allocates ports to the port group in vCenter but each individual ESXi host only allocates ports based upon currently powered on machines (assuming Dynamic or static port binding).  so if you create a dVS port group with 24 ports but there is only one virtual machine in the port group it would only take one port on it’s assigned ESXi host.

Ephemeral ports on a dVS work just like a VSS, so each local ESXi host uses all ports in a port group.

 

What is a proxy switch?

Proxy or Ghost switch is a term that you may see around to reference the local copy of the vDS on each host.  The proxy switch only contains relavant information to its virtual machines.   When you vMotion a new virtual machine to the host, vCenter allocates a new port on the ESXi host and sync’s a new proxy configuration to that switch alone.

What is the difference between an active port and total ports?

An active port is defined different between the switches

  • VSS any port on a port group is considered active on each ESXi host
  • dVS static or dynamic port in use on the ESXi host
  • dVS Ephemeral any ports on the port group are allocated on all ESXi hosts.

 

So in order to hit the 4096 total ports you would need a combination of VSS and dVS ports.    When using a single dVS you will hit the 1016 total active ports and never hit the 4096 total ports.

Lets look at some dVS switch maximums:

– Static/Dynamic port groups per distributed switch – 6500
– Ephemeral port groups per distributed switch – 1016

These are software limits static and dynamic are enforced by the dVS at vCenter and have no relationship to the ESXi hosts.   Ephemeral port groups have the hard limit of 1016 which aligns with the maximum number of active ports, which assumes you have 1016 port groups each with a single port.

How about the last set of numbers:

– Hosts per distributed switch – 1000
– Distributed virtual network switch ports per vCenter – 60000

Not much to say here.  The 60,000 creates a boundary that may require you not to allocate 1,000 ports per port group, it is per vCenter not dVS.  So that limit can span multiple vDS’s.

Best practices and design considerations:

Given that only active ports take memory on a ESXi host there is no reason not to allocate larger port groups, then again since port groups can be grown dynamically there is no reason not to keep them small.  I vote for something in between.  It would provide the best manageability without getting close to the maximums.

© 2014, Joseph Griffiths. All rights reserved.

Leave a Reply