Let the wars begin. This article will discuss the current state of affairs between virtual switches in ESXi. I have not included any third party switches because I believe them to becoming quickly not part of the picture with NSX.
Whats the deal with these virtual switches?
Virtual switches are a lot like ethernet layer 2 switches. They have a lot of the same common features. Both switch types feature the following configurable items:
- Uplinks – connections from the virtual switch to the outside world – physical network cards
- Port Groups – groups of virtual ports with similar configuration
In addition both switch types support:
- Layer 2 traffic handling
- VLAN segmentation
- 801.1 Q tagging
- nic teaming
- Outbound traffic shaping
So the first question everyone ask’s is if two virtual machines are in the same vlan and on the same server does their communication leave the server?
No… communication between the two vm’s on the same ESXi host can communicate without leaving the switch.
Port Groups what are they?
Much like the name suggests port groups are groups of ports.. They can be best described as a number of virtual ports (think physical port 1-10) that are configured the same. Port groups can have a defined number of ports and expanded at will (like a 24 port switch or 48 port switch) There are two generic types of port groups:
- Virtual machine
- VMkernel
Virtual machine port groups is for guest virtual machines. VMkernel port groups are for ESXi management functions and storage. The follow are valid uses for VMkernel ports
- Management Traffic
- Fault Tolerance Traffic
- IP based storage
- vMotion traffic
You can have one or many port groups for VMkernel but each requires a valid IP address that can reach other VMkernel ports in the cluster.
At time of writing (5.5) the follow maximum’s apply
- Total switch ports per host: 4096
- Maximum active ports: 1016
- Port groups per standard switch:512
- Port groups per distributed switch: 6500
- VSS port groups per host: 1000
So as you can see vDS scales a lot higher.
Standard Virtual Switch
The standard switch has one real advantage. It does not require enterprise plus licensing to use. It has a lot less features and some draw backs including:
- No configuration sync – you have to create all port groups exactly the same on each host or lots of things will fail (even upper case vs lower case will cause it to fail)
Where do standard switches make sense? Small shops with a single port group they make a lot of sense. If you need to host 10 virtual machine on the same subnet then standard switches will work fine.
Advice
- Use scripts to deploy switches and keep them in sync to avoid manual errors
- Always try vMotions between all hosts before after each change to ensure nothing is broken
- Don’t go complex on your networking design – it will not pay off
Distributed Virtual Switch
Well the distributed virtual switch is a different animal. It is configured by vCenter and deployed to each ESXi host. The configuration is in sync. It has the following additional features:
- Inbound Traffic Shaping – Throttle incoming traffic to the switch – useful to slow down traffic to a bad neighbor
- VM network port block – Block the port
- Private VLAN’s – This feature requires switches that support PVLAN so you can create VLAN’s inbetween vlans
- Load – Based teaming – Best possible load balancing (another article on this topic later)
- Network vMotion – Because the dVS is owned by vCenter traffic stats and information can move between hosts when a virtual machine moves… on a standard switch that information is lost with a vMotion
- Per port policy – dVS allows you to define policy at the port level instead of port group level
- Link Layer Discoery Protocol – LLDP enables virtual to physical port discovery (your network admins can see info on your virtual switches and you can see network port info – great for troubleshooting and documentation)
- User defined network i/o control – you can shape outgoing traffic to help avoid starvation
- Netflow – dVS can output netflow traffic
- Port Mirroring – ports can be configured to mirror for diagnostic and security purposes
As you can see there are a lot of features on the vDS with two draw backs:
- Requires enterprise plus licensing
- Require vCenter to make any changes
The last draw back has provided a number of hybrid solutions over the years. At this point VMware has created a work around with the empherial port group type and the network recovery features of the console.
Advice in using:
- Backup your switch with PowerCli (a number of good scripts out there)
- Don’t go crazy just because you can if you don’t need the feature don’t use it
- Test your vCenter to confirm you can recover from a failure
So get to the point which one should I use?
Well to take the VCDX model here are the elements of design:
Availability
- VSS – deployed and defined on each ESXi host no external requirements + for availability
- dVS – deployed and defined by vCenter and requires it to provision new ports/ port groups – for availability
Manageability
- VSS – pain to manage in most environments and does not scale with lots of port groups or complex solutions – for manageability
- dVS – Central management can be deployed to multiple hosts or clusters at the same datacenter + for manageability
Performance
- VSS – performance is fine no effect on quality
- dVS – performance is fine no effect on quality other than it can scale up a lot larger
Recoverability
- VSS – is deployed to each host and stored on each host… if you loose it you have to rebuild from scratch and manually add vm’s to the new switch – for recoverability
- dVS – is deployed from vCenter and you always have it as long as you have vCenter. If you loose vCenter you have to start from scratch and cannot add new hosts. (don’t remove your vCenter it’s a very bad idea) + as long as you have a way to never loose your vCenter (does not exist yet)
Security
- VSS – Offers basic security features not much more
- dVS – Wider range of security features + for security
End Result:
dVS is better is most ways but costs more money. If you want to use dVS it might be best to host vCenter on another cluster or ensure it’s availability.
