Month: September 2013

Posted on

VXLAN failed to prepare cluster domain-xxx

I ran into this issue last week.  I was preparing a new cluster for VXLAN and failed to open the firewalls to the vib could be pushed out to the ESXi hosts.  This caused the prepare for network vitalization to fail.  When I tried to remove it everything looked ok until I prepared again… this threw the following error:

domain-cxx already has been configured with a mapping.

This is a big problem and threw me for a loop.  VMware support was able to help… and I suggest unless you don’t care about your cluster or vShield implementation that you call them to solve it… but here is the fix I have used many times to resolve this issue:

Login to a vCloud cell (or download curl to your pc)

Run the following command:

curl -i -k -H "Content-type: application/xml" -u admin:default -X DELETE https://<vsm-ip>/api/2.0/vdn/map/cluster/<domain-cXXX>/switches/dvs

Replace the admin:default with your username:password for vShield and the vsm-ip with your vShield IP address.  Replace domain-cXXX with the domain name from the error.  Remove all < and > they are just to show where I placed the variables.  Now your ready for a prepare again.

Posted on

PowerCLI upgrade vmware tools on next reboot

I ran across this command somewhere and it’s really useful for the vCloud hosted environment where your tenants never upgrade their tools.   Of course you have to be allowed upgrade the tools make sure your not breaking your SLA or service contract by doing this.  Here is the command:

Foreach ($v in (get-vm)) {
$vm = $v | Get-View
$vmConfigSpec = New-Object VMware.Vim.VirtualMachineConfigSpec
$vmConfigSpec.Tools = New-Object VMware.Vim.ToolsConfigInfo
$vmConfigSpec.Tools.ToolsUpgradePolicy = "UpgradeAtPowerCycle"
$vm.ReconfigVM($vmConfigSpec)
}
Posted on

vCloud Director ORG VM’s cannot contact the internet

I have been playing around a lot of late with vCloud director.  I tried it out in the 1.0 days and found it to be lacking.  I used it again in the 1.5 days and also found it to be a pain.  It looks like VMware has got it figured out with 5.1.  Don’t get me wrong it’s still a lot of components pasted together but when setup it works great.   I did run into a few issues with changes. made between versions.  In 1.5 your would deploy vApp’s to a routed ORG network and it just worked they could get to the internet.   This has changed in 5.1.  See this article for full information.  Here are the steps to get it working again.   This example assumes that you want all your ORG vApp’s to have the same ip address.

  1. Create the ORG Edge gateway (this should be done when you create the org if not do it now)
  2. Sub-allocated some external IP’s to the Edge gateway
  3. Create a SNAT rule on the edge gateways external interface (this is the part that got me… not the ORG network but the edge external) This should be a internal range like 192.168.10.0/24 to a single external IP like 8.8.8.4 (if I was a google DNS server)
  4. Add a firewall rule that allows all outgoing traffic from your internal (use the term internal) to external (term external) protocol any and action is allow.

That’s it and it works.  I will be writing a lot more on vCloud in the near future so keep tuned.

Posted on

VXLAN how does it work and what is it?

VXLAN (Virtual eXensible Local Area Network)  is a  encapsulation overlay protocol created to address a few common datacenter challenges.  These challenges have become a lot more acute with the advent of cloud computing and multi-tenancy resource consumption.  VXLAN was developed by Cisco, VMware, Microsoft any many others.  It was developed to address the following datacenter concerns:

  • The need for more mac address tables on the physical infrastructure.  As virtualization increases so do the mac address tables to a point where layer 2 switching has become very costly.
  • The need for layer 2 seperation beyond the current 4096 VLAN’s.  With only 4096 available some datacenters have run out of VLAN’s
  • With multi-tennancy it’s possible that administrators will have duplicate mac addresses or IP addresses which can cause security issues when implemented over shared links.
  • Layer 2 does not implement redundancy like layer 3 does. A network administrator may wish to use equal cost multipath to provide redundancy which is not possible on layer 2.
  • STP (Spanning tree protocol) used to avoid loops on layer 2 can potentially shut down paths that could be used to provide more bandwidth.
  • Layer 3 alone cannot provide separation due to the possibility that some tenants may use the same IP ranges.

For these reasons and more VXLAN was developed to carry layer 2 traffic in an encapsulated tunnel.   This tunnel is implemented with an additional header placed over a layer two packed and sent out as a UDP multicast.   Each VXLAN is separated into VXLAN segments (think VLAN’s but the delievery is done via UDP multicast addresses, each segement has it’s own address)

We then implement a VTEP (VXLAN Tunnel End Point) which can subscribe to the multicast traffic.  When a message is received the VTEP translates it into the correct VXLAN segment and decodes the message into a normal layer 2 packet.  Then end points have no knowledge of multicast or VXLAN they think all communication is done via standard layer 2.   It is important that the VTEP is implemented as close as possible to the hypervisor.

This technology can also be used to stretch a layer 2 network using IP across datacenters: Making it possible for the same layer 2 network to exist in two different locations.

Since the VXLAN throws a new header on the top of standard ethernet it is critical that your network supports frame sizes of at least 1600 (which is the MTU of a VXLAN segment).

Posted on

Comparing Virtual Switchs and Physical Switchs

As we get into software defined networking the lines will blur more and more but at this point normal virtual switches have not changed much over the years.

What does a vSwitch do?

  • Layer 2
    • has MAC address tables
    • forwards ethernet frames
    • supports VLAN configurations
    • able to trunk based on 802.1q VLAN tags
    • can establish port channels
  • Cannot support the following on layer 2
    • Dynamic negotiation protocols (like DTP or PAgP)
    • Cannot be connected to another vSwitch (making loops impossible and removing the need for STP)
  • vSwitches are unique in that
    • They authoritatively know the MAC addresses of all connected VM’s no need for learning
    • Does not need to perform IGMP snooping because it knows the multicast of attached vm’s

 

Posted on

PowerCLI CDROM’s disconnect and reconnect

In powershell it’s really easy to disconnect and reconnect cdrom drives:

To see all connected drives:

Get-VM | where {($_ | Get-CDDrive).ISOPath -ne $null}  |FT Name, @{Label="ISO file"; Expression = { ($_ | Get-CDDrive).ISOPath }}

To auto disconnect all:

Get-VM | Get-CDDrive | Where {$_.ConnectionState.Connected} | Set-CDDrive -Connected $false -Confirm:$false