Lots of new posts

Yes I have added a ton of posts in the last few minutes this is because I just moved everything from my other blog back here.  I am making a single home on the internet. Sorry to bomb your readers.

Thanks,

Joseph

PowerCLI List all VMDK disks and what datastore they are on

Once an a while I need to list all my vmdk’s and what datastore they are on… So I created this script with output to CSV:

$outputFile = "D:\tmp\output.csv" 

$VMsAdv = Get-VM | Sort-Object Name | % { Get-View $_.ID } 
$myCol = @() 
ForEach ($VMAdv in $VMsAdv) 
{ 
    ForEach ($Disk in $VMAdv.Layout.Disk) 
    { 
        $myObj = "" | Select-Object Name, Disk 
        $myObj.Name = $VMAdv.Name 
        $myObj.Disk = $Disk.DiskFile[0] 
        $myCol += $myObj 
    } 
} 
$myCol | Export-Csv $outputFile -NoTypeInformation

 

NFS for virtual machines Why?

A lot of shops are using NFS for VMFS datastores.  A lot of custom VMware storage arrays use it too like Tintri.  Why?  Well it brings some advantages to the tables.  In a lot of cases the storage vendor has been able to tweak the protocol to work faster with vm’s.  I won’t get into line speed questions but that is one to consider with 10GB NFS vs 8GB or 16GB fiber channel.

How is NFS different?

  • The file system itself is not managed by the ESXi host, ESXi just uses the protocol.
  • The burden of scaling is not placed upon the storage stack but instead the networking stack.  If networking fails you loose everything.
  • Since NFS uses ethernet multipathing is a complex situation that requires normally static link aggregation or dynamic LACP (802.3ad) (with multi-switch aggregation) You can also use multiple VMkernel networks on separate vSwitches and subnets.

Files that make up a virtual machine ESXi

For the longest time I always wondered what exactly all those files inside your directory do and their purpose so here is a handy guide.

Configuration File -> VM_name.vmx
Swap File -> VM_name.vswp or vmx-VM_NAME.vswp
BIOS File -> VM_name.nvram
Log files -> vmware.log
Disk descriptor file -> VM_name.vmdk
Disk data file -> VM_name-flat.vmdk
Suspended state file -> VM_name.vmss
Snapshot data file -> VM_name.vmsd
Snapshot state file -> VM_name.vmsn
Template file -> VM_name.vmtx
Snapshot disk file -> VM_name-delta.vmdk
Raw Device map file -> VM_name-rdm.vmdk

  • .vmx – Contains all the configuration information and hardware settings for the virtual machine, it is stored in text format.
  • .vswp – is a file that is always created for virtual machines during power on. It’s equal to the size of allocated ram minus any memory reservation at boot time. This swap file is used when the physical host exhausts all of its allocated memory and guest swap is used.
  • .nvram – is a binary formated file that contains BIOS information much like a BIOS chip. If deleted it is automatically recreated when the virtual machine is powered back on.
  • .log – Log files are created when the machine is power cycled the current log is always called vmware.log

Powercli – How to get Vmware tools version

Classic problem keeping vmware tools up to date.   Here is a set of power shell scripts to help you figure out what machines have old tools and need updates.  I run a version of this script as an email sent to me once a week.

Q: How do I get a list of the Vmware tools on each virtual machine?

get-vm | % { get-view $_.id } | select Name, @{ Name="ToolsVersion"; Expression={$_.config.tools.toolsVersion}},@{ Name="ToolStatus"; Expression={$_.Guest.ToolsVersionStatus}}

Q: How do I get a list of the Vmware tools on each powered on virtual machine?

get-vm | where {$_.powerstate -ne "PoweredOff" } | % { get-view $_.id } | select Name, @{ Name="ToolsVersion"; Expression={$_.config.tools.toolsVersion}},@{ Name="ToolStatus"; Expression={$_.Guest.ToolsVersionStatus}}

Q: How do I get a list of the Vmware tools that are not up to date on each powered on virtual machine?

get-vm | where {$_.powerstate -ne "PoweredOff" } | where {$_.Guest.ToolsVersionStatus -ne "guestToolsCurrent"} | % { get-view $_.id } | select Name, @{ Name="ToolsVersion"; Expression={$_.config.tools.toolsVersion}}, @{ Name="ToolStatus"; Expression={$_.Guest.ToolsVersionStatus}}

Q: How do I get a list of the Vmware tools that are not up to date on each powered on virtual machine and output it to csv?

get-vm | where {$_.powerstate -ne "PoweredOff" } | where {$_.Guest.ToolsVersionStatus -ne "guestToolsCurrent"} | % { get-view $_.id } | select Name, @{ Name="ToolsVersion"; Expression={$_.config.tools.toolsVersion}}, @{ Name="ToolStatus"; Expression={$_.Guest.ToolsVersionStatus}} | Export-Csv -NoTypeInformation -UseCulture -Path C:\Temp\VMHWandToolsInfo.csv

Powercli Basics Part 1

We all have to start somewhere so what better place than the basics. I kept jumping into complex scripts without working out the basics. For years I have written scripts in shell or perl but I have never spent much time in PowerShell. This is a few basic items to get your started with PowerCLI for Vmware.

What can you do with PowerCLI?

Just about anything you can do in the command line and a lot more. It allows full access to Vmware in every respect. It can be used to automate tasks, produce reports or resolve problems.

Why do we want to automate tasks and produce reports we can do that manually?

This has been suggested more than once. If you have to do it more than once it should be automated why? It reduces to human error risk factor and produces repeatable results everytime. It also helps you think about the process before you do it reducing missed steps. Automation is the key to more time as an administrator and less problems.

First thing first:
Your first step into the world of PowerCLI is installing it. You can download it from vmware’s site for free and you should update it everytime a new version of ESX comes out. Each updated version gets more and more functionality and commands. Make sure you run the installer as Administrator. The next step is to change the execution policy. Microsoft felt that it was a good idea to have all Powershell code signed off by a third party before allowing it to run. This eliminates the chance that someone’s virus could be run. It also makes it impossible for your to run your code so you need to change it.

Changing execution policy
1. Log into Powershell by right clicking on it and selecting run as Administrator
2. Run the following command:

set-executionpolicy remotesigned

3. Anwer yes to the question

Now you need to connect to your first vmware node with PowerCLI.   Right click on powercli on your desktop and select run as administrator.  Once it loads up you need to connect to either vsphere or an esx host with the following command:

connect-viserver IP_address

It has a series of questions which can be answered on the command line or via the prompt (like your login information) and your connected to your host / vcenter.

You can now issue your first powercli command to locate some information about your host or hosts:

get-vmhost

Great now I know how to pull information about my host but what if I want to see all the information it pulls from my host or from any command.  That can be done with the fl command.  Powershell uses a piping structure that will be familiar to all Linux / Unix shell coders.

get-vmhost | fl

How about displaying only selected fields.  For example lets display the used CPU and used memory on each host:

get-vm | select Name, CpuUsageMhz, MemoryUsageGB

Now we have the state of a beautiful reporting system.  Look for more in the next part.

 

Powercli Basics Part 2

Welcome to part 2 we are going to expand on our last code slice:

get-vm | select Name, CpuUsageMhz, MemoryUsageGB

Which produces a nice list of the CpuUsageMhz and MemoryUsageGB per node.   Now we are going to expand that command to produce a report that displays the percent of Memory used and Cpu used across your whole cluster.   First of all we need variables

Variables

Variables in powershell are very flexable.  They are all defined with a dollar $ sign in front of them.  An variable can be an array via $variable_name[number] or a multidementional  array via $variable_name[number].field_name for example

To create variable that holds the number 5 we do the following

$variable=5

To display we do the following:

$variable

In order to load multiple items into an variable making it an array we just add more elements on the command line separated by a comma

$variable=5,6
$variable

To address individual elements of the array you use [number]

$variable[0]

With Vmware you almost always have multiple fields as show by the following command from part 1.

get-vmhost | fl

So when you put the output from get-vmhost into a variable it creates a multidemential array which can be addressed like this:

$output=get-vmhost
$output

So we can choose a single field via the following pattern $variablename[num].fieldname for example

$output[0].NumCpu

What do we do if we have multiple hosts that we want to query .. lets use a for loop:

[sourcecode language="powershell"]
$hosts=get-vmhost

foreach($vmhost in $hosts){

  $vmhost.Name

  $vmhost.CpuTotalMhz

  $vmhost.CpuUsageMhz

  $vmhost.MemoryTotalGB

  $vmhost.MemoryUsageGB

}

[/sourcecode]

Pretty simple stuff… but we are missing a critical part… I think it would be better to understand the totals from my whole cluster not each individual host.  For that we need some math operators:

= Equals

+= Increases by value and appends variable

-= Decreases by value and appends to variable

*= Multiplies by value and appends to variable

/= Divides by value and appends to variable

%= Divides and assigns the remainder to variable (modulus)

So back to our code… lets assume we want to add up all our used CPU and used Ram and figure out what percentage we are using on our whole cluster:

 

[sourcecode language="powershell"]
$hosts=get-vmhost

foreach($vmhost in $hosts){

  $cputotal+=$vmhost.CpuTotalMhz

  $cpuusage+=$vmhost.CpuUsageMhz

  $memorytotal+=$vmhost.MemoryTotalGB

  $memoryused+=$vmhost.MemoryUsageGB

}

 $percpuused=(($cpuusage / $cputotal) * 100)

 Write-Host "You are currently using $percpuused% of your cpu."

 $permemused=(($memoryused / $memorytotal) * 100)

 Write-Host "You are currently using $permemused% of your memory."
[/sourcecode]

One problem with this output… no one cares how many decimal points of a percent you are using. We need to edit the numbers to drop the decimal points:

[sourcecode language="powershell"]
$hosts=get-vmhost

foreach($vmhost in $hosts){

  $cputotal+=$vmhost.CpuTotalMhz

  $cpuusage+=$vmhost.CpuUsageMhz

  $memorytotal+=$vmhost.MemoryTotalGB

  $memoryused+=$vmhost.MemoryUsageGB

}

 $percpuused="{0:N0}" -f (($cpuusage / $cputotal) * 100)

 Write-Host "You are currently using $percpuused% of your cpu."

 $permemused="{0:N0}" -f (($memoryused / $memorytotal) * 100)

 Write-Host "You are currently using $permemused% of your memory."
[/sourcecode]

The addition of “{0:N0}” -f knocks off the decimal points .. you can also keep to places after the decimal point by this command:
“{0:N2}” -f

That’s it for now. In part three I will show you how to make your report a nice HTML email.

Powercli – Virtual Switches

Q: How do I add a new port group to a virtual switch on each ESX host

A: Using powercli you could run the following:

[sourcecode language="powershell"]
$cluster_name = "Your_Cluster_Name"
$vSwitch = "vSwitch_you_want_to_add_vlan_to"
$PortGroupName = "Name_of_Your_Port_Group"
$PortGroupVLAN = "VLAN_of_Your_Port_Group"

foreach ($esx in get-VMhost -Location $cluster_name | sort Name) { $esx | Get-VirtualSwitch -Name $vSwitch | New-VirtualPortGroup -Name “$PortGroupName″ -VlanId $PortGroupVLAN }

[/sourcecode]

Q: How do I change the number of ports on my vSwitch using PowerCli

A: With the following commands:

[sourcecode language="powershell"]
$vSwitch = "Your_vSwitch_Name"
$NumberPorts = "Number_of_ports"

Get-VMHost | % {Get-VirtualSwitch -VMHost $_ -Name $vSwitch | % { Set-VirtualSwitch -VirtualSwitch $_ -NumPorts "$NumberPorts" }

[/sourcecode]

Q: How do I create a vSwitch from PowerCli
A: With this code:

[sourcecode language="powershell"]
# You can do multiple vmnic's on the command just use a , between
$vmnic = "Name_of_vmnic_you_want_assigned_to_switch"
$vSwitch = "Name_of_new_vSwitch"
$NumPorts = "Number_of_ports"

New-VirtualSwitch -Name $vSwitch -NumPorts $NumPorts -Nic $vmnic

[/sourcecode]

Enable Remote shell in ESXi (ssh)

So you like to ssh/have console into your ESXi host don’t you?  You enjoy the raw power.  We all do but here are some reasons why it’s a bad idea:

  • Security risk – one more way into all your virtual machines that is commonly hacked
  • Memory waste – having it enabled eats memory, disabled uses none

If you are stuck on enabling it you might want to consider locking it down with the ESXi firewall enabling ssh connections from only a few known source ports.  Like this (from esxi host or vMA):

List current firewall rules:

esxcli network firewall ruleset list

This will output something like this:

Name Enabled
------------------ -------
sshServer true
sshClient false
nfsClient false
dhcp true
dns true
snmp true
ntpClient true
CIMHttpServer true
CIMHttpsServer true
CIMSLP true
iSCSI true
vpxHeartbeats true
updateManager false
faultTolerance true
webAccess true
vMotion true
vSphereClient true
activeDirectoryAll false
NFC true
HBR true
ftpClient false
httpClient false
gdbserver false
DVFilter false
DHCPv6 false
DVSSync false
syslog false
IKED false
WOL true
vSPC false
remoteSerialPort false
vprobeServer false

To show the firewall default actions run the following command:

esxcli network firewall get
Default Action: DROP
 Enabled: true
 Loaded: true

To see the current firewall rules on sshServer use the following:

esxcli network firewall ruleset list --ruleset-id sshServer
Name Enabled
--------- -------
sshServer true

See current allowed ip list:

esxcli network firewall ruleset allowedip list --ruleset-id sshServer
Ruleset Allowed IP Addresses
--------- --------------------
sshServer All

Turn the allow all list to false:

esxcli network firewall ruleset set --ruleset-id sshServer --allowed-all false

Add the 10.10.101.0/24 subnet to the ssh allow list:

esxcli network firewall ruleset allowedip add --ruleset-id sshServer --ip-address 10.10.101.0/24

Check current list

esxcli network firewall ruleset allowedip list --ruleset-id sshServer

Now you have locked down your host access to ESXi ssh but I keep getting a warning on the host that ssh/console is enabled…. I love this error it really shows you vmware’s feelings about this feature.   Here is how you make it stop, well you can do it via the gui but let’s do it in the console:

esxcli system settings advanced set -o /UserVars/SuppressShellWarning -i 1

Lost vcenter with ESXi host and cannot power on with vDS

I have run into this issue a few times now.  My vcenter is a virtual machine.  Running on a vDS.  My ESXi host fails in a odd way: for example the host does not fail but the nic’s stop passing traffic.  My isolation response keeps the vm powered on and HA see’s that the host is still accessing storage so no fail over happens.   How do I get it back?  Well I can power it off and move it to another host but I cannot power it on… Why?

Because my vDS is setup with static ports.  Which means when vcenter is not available each ESXi host cannot start new virtual machines because they cannot allocate ports on the vDS.   For a long time I ran my vcenter on a standard switch to avoid this issue.  Now Vmware has provided a new option.  In order to use this option we need to understand vDS port group types:

  • Static Binding – The default setting, when you power on a virtual machine it is assigned a port and guarenteed that port going forward.  The only time it looses this connection is when a vm is removed from the port group.  The main issue with static binds port groups is they have a set number of ports available at creation time.  If you try to power on a vm without available ports it fails.  This has been fixed in 5.0 with a auto grow feature. Static assignments are made by virtual center not the host.  Virtual center then creates the port on the ESXi host.
  • Dynamic Binding- This is now deprecated and should not be used.
  • Ephemeral Binding – This is a type of port binding to vm that is created by the host (not virtual center as in Static) The assignment is released by the host when a virtual machine is powered off or deleted.  There are almost unlimited number of ports available (limited by host maximum number of ports).

So why don’t we just make all our port groups Ephemeral?

  • They don’t scale as well you are only allowed to have 256 total port groups with ephemeral.
  • They consume a lot more resources on each host.
  • Vmware does not want you to do it… as evident by the auto-expand static binding.

So when do I use ephemeral?

On my vcenter port group so I can move it and power on without having a virtual center powered on.

Solved that Catch 22 …