How do I backup individual esxi hosts

Using the vMA you can backup and restore individual ESXi hosts using the vicfg-cfgbackup  command.   If you log into your vMA and type vicfg-cfg-backup –help you get some of the following information:

Synopsis: /usr/bin/vicfg-cfgbackup OPTIONS [<backupfile>]

Command-specific options:
 --force
 -f
 Force the restore of the configuration.
 --load
 -l
 Restore configuration onto the host
 --quiet
 -q
 Do not prompt for user confirmation.
 --reset
 -r
 Resets host, restore to factory settings.
 --save
 -s Backup the host configuration.

So in order to backup your ESXi host you just need to select a host and do a -s.  I like to do these sort of things automatically.   Here is my setup:

  • Create a location for backups
mkdir ~/backup
  • Create a script to do backups
#!/bin/bash
# Script created by Joseph Griffiths
# bloodygranola.com
# Date to create a unique file
date=`/bin/date '+%m-%d-%y-%H-%M'`;
# Directory to put the backup
backupdir="/home/vi-admin/backup"
# How many days we should keep backups
days="30"
# Function to do backup
function backup(){
/opt/vmware/vma/bin/vifptarget -s $1
/usr/bin/vicfg-cfgbackup -s $backupdir/$1-$date.bak
}
# Do the backup
backup "esxi01.vclass.local";
backup "esxi02.vclass.local";
# Clean up
find $backupdir -mtime +$days -exec rm -rf {} \;

Well we want to run this automatically each day correct?  We can do that with crontab lets set it for 2 am everyday

crontab -e

Enter the following:

0 2 * * * /home/vi-admin/backup.sh > /dev/null 2>&1

What about restoring?

It’s done with the same command except the -f for force and -l for load

vicfg-cfgbackup -f -l filename

How do vmware tools communicate with the host?

Great question.  Well with ESXi 5.1 they communicate with the host via a virtual chip that is part of the virtual hardware.  Vmware tools access this chip and the host accesses the chip allowing for communication without networking.   Vmware even provides a API to program against this chip to create socket communication.

How do I improve network performance between VM’s

Let’s create an example situation.  You have two machines a application server and a database.  These machines do a lot of network traffic to each other.  You want to ensure they have the best possible performance.  In vmware worlds there are a few tricks to help you out.  First make sure your using the latest vmknic available.  After you have upgraded the nic’s the only thing left is to take advantage of the vmware ring buffer.

What is the vmware ring buffer?

Since virtual switches are defined as software dumb switches they operate in ram.  Ram is really fast.  So if you can send a message without leaving your virtual switch it can be really fast.   The ring buffer exists on each esxi host and where possible allows guests to communicate with each other (layer 2 so they have to be in the same subnet)  without involving anything but RAM.  You can see a 2X performance gain by just using the ring buffer.

How do I use it?

Keep your talking vm’s on the same host using affinity rules and your all set.

Vcenter Install failed port 80 in use

I normally use a Oracle backend for my database to vcenter but the other day I setup a single box with SQL server for the database.  I ran into issues with port 80 already being in use.  After a few netstats I found it was the SQL Server Reporting Services.  Which unless you plan on doing reporting can be safely turned off and shutdown.

VXLAN failed to prepare cluster domain-xxx

I ran into this issue last week.  I was preparing a new cluster for VXLAN and failed to open the firewalls to the vib could be pushed out to the ESXi hosts.  This caused the prepare for network vitalization to fail.  When I tried to remove it everything looked ok until I prepared again… this threw the following error:

domain-cxx already has been configured with a mapping.

This is a big problem and threw me for a loop.  VMware support was able to help… and I suggest unless you don’t care about your cluster or vShield implementation that you call them to solve it… but here is the fix I have used many times to resolve this issue:

Login to a vCloud cell (or download curl to your pc)

Run the following command:

curl -i -k -H "Content-type: application/xml" -u admin:default -X DELETE https://<vsm-ip>/api/2.0/vdn/map/cluster/<domain-cXXX>/switches/dvs

Replace the admin:default with your username:password for vShield and the vsm-ip with your vShield IP address.  Replace domain-cXXX with the domain name from the error.  Remove all < and > they are just to show where I placed the variables.  Now your ready for a prepare again.

PowerCLI upgrade vmware tools on next reboot

I ran across this command somewhere and it’s really useful for the vCloud hosted environment where your tenants never upgrade their tools.   Of course you have to be allowed upgrade the tools make sure your not breaking your SLA or service contract by doing this.  Here is the command:

Foreach ($v in (get-vm)) {
$vm = $v | Get-View
$vmConfigSpec = New-Object VMware.Vim.VirtualMachineConfigSpec
$vmConfigSpec.Tools = New-Object VMware.Vim.ToolsConfigInfo
$vmConfigSpec.Tools.ToolsUpgradePolicy = "UpgradeAtPowerCycle"
$vm.ReconfigVM($vmConfigSpec)
}

VXLAN how does it work and what is it?

VXLAN (Virtual eXensible Local Area Network)  is a  encapsulation overlay protocol created to address a few common datacenter challenges.  These challenges have become a lot more acute with the advent of cloud computing and multi-tenancy resource consumption.  VXLAN was developed by Cisco, VMware, Microsoft any many others.  It was developed to address the following datacenter concerns:

  • The need for more mac address tables on the physical infrastructure.  As virtualization increases so do the mac address tables to a point where layer 2 switching has become very costly.
  • The need for layer 2 seperation beyond the current 4096 VLAN’s.  With only 4096 available some datacenters have run out of VLAN’s
  • With multi-tennancy it’s possible that administrators will have duplicate mac addresses or IP addresses which can cause security issues when implemented over shared links.
  • Layer 2 does not implement redundancy like layer 3 does. A network administrator may wish to use equal cost multipath to provide redundancy which is not possible on layer 2.
  • STP (Spanning tree protocol) used to avoid loops on layer 2 can potentially shut down paths that could be used to provide more bandwidth.
  • Layer 3 alone cannot provide separation due to the possibility that some tenants may use the same IP ranges.

For these reasons and more VXLAN was developed to carry layer 2 traffic in an encapsulated tunnel.   This tunnel is implemented with an additional header placed over a layer two packed and sent out as a UDP multicast.   Each VXLAN is separated into VXLAN segments (think VLAN’s but the delievery is done via UDP multicast addresses, each segement has it’s own address)

We then implement a VTEP (VXLAN Tunnel End Point) which can subscribe to the multicast traffic.  When a message is received the VTEP translates it into the correct VXLAN segment and decodes the message into a normal layer 2 packet.  Then end points have no knowledge of multicast or VXLAN they think all communication is done via standard layer 2.   It is important that the VTEP is implemented as close as possible to the hypervisor.

This technology can also be used to stretch a layer 2 network using IP across datacenters: Making it possible for the same layer 2 network to exist in two different locations.

Since the VXLAN throws a new header on the top of standard ethernet it is critical that your network supports frame sizes of at least 1600 (which is the MTU of a VXLAN segment).

Comparing Virtual Switchs and Physical Switchs

As we get into software defined networking the lines will blur more and more but at this point normal virtual switches have not changed much over the years.

What does a vSwitch do?

  • Layer 2
    • has MAC address tables
    • forwards ethernet frames
    • supports VLAN configurations
    • able to trunk based on 802.1q VLAN tags
    • can establish port channels
  • Cannot support the following on layer 2
    • Dynamic negotiation protocols (like DTP or PAgP)
    • Cannot be connected to another vSwitch (making loops impossible and removing the need for STP)
  • vSwitches are unique in that
    • They authoritatively know the MAC addresses of all connected VM’s no need for learning
    • Does not need to perform IGMP snooping because it knows the multicast of attached vm’s

 

PowerCLI CDROM’s disconnect and reconnect

In powershell it’s really easy to disconnect and reconnect cdrom drives:

To see all connected drives:

Get-VM | where {($_ | Get-CDDrive).ISOPath -ne $null}  |FT Name, @{Label="ISO file"; Expression = { ($_ | Get-CDDrive).ISOPath }}

To auto disconnect all:

Get-VM | Get-CDDrive | Where {$_.ConnectionState.Connected} | Set-CDDrive -Connected $false -Confirm:$false