Virtual switches are a fun topic in ESX, They are unique on each ESX node and not shared across the cluster. This problem was addressed in ESX 4.0 with distributed virtual switches (DVS) which allows you to create switches on vCenter and pass it to all nodes. Unfortunately DVS is available only in the plus licenses which cost about $1000 more per processor. For those of us without DVS are forced to script out vSwitches. The process is pretty simple but has to be done in the right order from the service console:
- Create the vSwitch
- Create port groups
- Assign VLAN tags to port groups if required
- Apply security policy
- Link a nic to the switch
- Create a service console if required
- Assign ip addresses if required
- Enable vmotion if required
1.
# Create New vSwitches # create a vSwitch with 56 ports for our service console esxcfg-vswitch -a vSwitch0 # create a vSwitch with 56 ports for the vmkernel network esxcfg-vswitch -a vSwitch1 # create a vSwitch with 1024 ports for VM's esxcfg-vswitch -a vSwitch2:1024 2.# Create Base port groups # Service console port group esxcfg-vswitch --add-pg="Service Console" vSwitch0 # Vmkernel port group esxcfg-vswitch --add-pg="Vmkernel" vSwitch1 # Port group for FT esxcfg-vswitch --add-pg="FT" vSwitch1 # Port group for VM's in VLAN 801 esxcfg-vswitch --add-pg="VM - 801" vSwitch2 # Port group for VM's in VLAN 802 esxcfg-vswitch --add-pg="VM - 802" vSwitch23. # Assign VLAN's to port groups esxcfg-vswitch -p "VM - 801" -v 801 vSwitch2 esxcfg-vswitch -p "VM - 802" -v 802 vSwitch24. # Default setting on ESX allow for mac changing and sniffing fix this via these commandsvmware-vim-cmd hostsvc/net/vswitch_setpolicy --securepolicy-macchange=false vSwitch0 vmware-vim-cmd hostsvc/net/vswitch_setpolicy --securepolicy-promisc=false vSwitch0 vmware-vim-cmd hostsvc/net/vswitch_setpolicy --securepolicy-forgedxmit=false vSwitch0 vmware-vim-cmd hostsvc/net/vswitch_setpolicy --securepolicy-macchange=false vSwitch1 vmware-vim-cmd hostsvc/net/vswitch_setpolicy --securepolicy-promisc=false vSwitch1 vmware-vim-cmd hostsvc/net/vswitch_setpolicy --securepolicy-forgedxmit=false vSwitch1 vmware-vim-cmd hostsvc/net/vswitch_setpolicy --securepolicy-macchange=false vSwitch2 vmware-vim-cmd hostsvc/net/vswitch_setpolicy --securepolicy-promisc=false vSwitch2 vmware-vim-cmd hostsvc/net/vswitch_setpolicy --securepolicy-forgedxmit=false vSwitch25.# Link primary nic to switch esxcfg-vswitch --link=vmnic0 vSwitch0 esxcfg-vswitch --link=vmnic6 vSwitch1 esxcfg-vswitch --link=vmnic2 vSwitch1#Link VMnetwork to vSwitch2 esxcfg-vswitch --link=vmnic1 vSwitch2 esxcfg-vswitch --link=vmnic3 vSwitch2 esxcfg-vswitch --link=vmnic5 vSwitch2 esxcfg-vswitch --link=vmnic7 vSwitch2 6.esxcfg-vswif -a vswif0 -i192.168.10.45 -n 255.255.255.0 -p "Service Console" 7.esxcfg-vmknic -a -i 192.168.20.10 -n 255.255.255.0 -p "Vmkernel" vmkernel esxcfg-vmknic -a -i 192.168.20.40 -n 255.255.255.0 -p "FT"8.
vmware-vim-cmd hostsvc/vmotion/vnic_set vmk0The only thing I missed was setting a default order on the nic’s if you have multiple nic’s: For example my vSwitch1 has two port groups with 2 vnics I can choose to force a vnic for each group:
# Force Vmkernel to use vmnic6 unless it's unavailable vmware-vim-cmd /hostsvc/net/portgroup_set -–nicorderpolicy-active=vmnic6 vSwitch1 “Vmkernel” vmware-vim-cmd /hostsvc/net/portgroup_set -–nicorderpolicy-standby=vmnic2 vSwitch1 “Vmkernel” vmware-vim-cmd /hostsvc/net/portgroup_set -–nicorderpolicy-active=vmnic2 vSwitch1 “FT” vmware-vim-cmd /hostsvc/net/portgroup_set -–nicorderpolicy-standby=vmnic6 vSwitch1 “FT”