Category: VMWare

This has been a pain of mine for a while now… I have to use the customized HP image but I need some additional Storage drivers to support VAII.  So I have to install HP’s ESXi then install the drivers later… with this tool on a Windows box I can bundle them together without much pain:

ESXi-Customizer -> http://www.v-front.de/p/esxi-customizer.html

For the longest time I would not allow people to power on virtual machines with more than one CPU without proving to me they needed more than one virtual cpu.  This is due to strict co-scheduling in vmware.   Over the years they have relaxed this a lot.  Making multiple CPU virtual machines a lot more possible.  To understand the issue take a look at the following diagram:

The assumption of this diagram is rather silly but it helps me explain the problem.  It is important to take into account that this diagram assumes that both virtual processors require 100% of the physical CPU.  As such they have to share it.   This problem is not too bad.  This is known as co-scheduling on the virtual CPU.  In vsphere 4 you did co-scheduling at the virtual machine level; meaning this 2 vCPU vm could not run on a single core single socket physical system.   Because both vCPU’s had to be scheduled to run at the same time.   Since Vmware 5 now schedules per vCPU we have a lot more flexability. The co-stop allows us to get reasonable performance on each vCPU.

It is critical to understand that adding more vCPU’s may not always provide better performance.  If there is any contention for cpu’s then you will have the co-stoping practice enacted.  Causing slow downs.  You can monitor this in esxtop with the following settings:

• Press c for cpu
• Press upper case V to see per virtual machine

The key fields here are:

• %USED – (CPU Used time) % of cpu used at current time.  This number is represented by 100 X Number_of_vCPU’s so if you have 4 vCPU’s and your %USED shows 100 then you are using 100% of one cpu or 25% of four cpu’s.
• %RDY – (Ready) % of time a vCPU was ready to be scheduled on a physical processor but could not be due to contention.  You do not want this above 10% and should look into anything above 5%.
• %CSTP – (Co-Stop) % in time a vCPU is stopped waiting for access to physical CPU high numbers here represent problems.  You do not want this above 5%
• %MLMTD – (Max Limited) % of time vmware was ready to run but was not scheduled due to CPU Limit set (you have a limit setting)
• %SWPWT – (Swap Wait) – Current page is swaped out

Vmware Scenario’s are hypothetical situations and solutions.

This came from a post on the official Vmware forums:

– We have a ESXi server with 1 Socket 4 Core CPU with Hyper theading.  We currently have 2 VM’s with 2 vCPU’s and the performance is great.  We want to add a third VM and split up the cores among the 3 VM’s.  The only issue is the third vm will be running a really old application in Windows XP.  This application is very CPU sensitive lack of resources will cause it to crash and it’s single threaded.   How do I solve this issue?

This is fine situation that takes into account the issues discussed in this article .   There are a few questions left unanswered.

First are the first two vm’s using over 75% of their two CPU’s?

– Why do I ask this question?  Well most of the time servers are not using much of there cpu.  It seems to be the most under utilized resource in most ESXi setups.  Unless we have resource contention processor co-stop does not figure into the equation.

So we are going to assume that the processors are not 75% or more used all the time.  So co-stop is now not an issue.  Given that assumption the solution might just be as simple as reservation:

Take your new virtual machine and use CPU reservation to ensure it always get’s 100% of it’s single cpu. You single threaded application will never fail and since your not using 75% or more of the resources on your other vm’s you should never even notice.

What about hypertheading?

Well there are lots of theories on this matter an it really depends on your situation.  Unless you have over subscribed your CPU then none of the theories really come into play.

What is the vMA?

vMA stands for vSphere Management Assistant.  Is it a virtual appliance provided by vmware that is running SUSE Linux 11 and has some custom vmware commands for scripting and authentication that allows you to manage a vmware infrastructure.   You can download it from Vmware for free from this location:

http://www.vmware.com/support/developer/vima/

In order to deploy into your infrastructure you need to deploy it into your ESXi host like any other virtual appliance:

From the web client

• vCenter -> Hosts -> select your ESXi host
• Actions -> Deploy OVF Template
• Go through the prompts to deploy

Once the OVF has been deployed you can power it on.  It will ask you for ip information.  Once it’s setup with ip you can manage it via the web interface https://your_ip:5480.  From here you can setup common settings and update the machine.   You just need to login as vi-admin with the default password of vmware.  Make sure to change this default password in the web interface.

Command Line

vMA’s real power is in the command line you can access it via ssh logging in as vi-admin.    Vmware has included all their common commands but one command is very powerful it’s vifp (vi fast pass) this allows you to stored login information for ESXi hosts so you don’t have to type them on every script.   You can use it like this:

To add a server (you should do this for vcenter and each ESXi host):

vifp addserver vcenter_or_esxi_host

To see what servers you have in vifp type the following:

vifp listservers

There are times where you want to manage only one host at a time you can select your target server via this command

vifptarget -s server_to_manage_name_from_listservers

Once you have selected a target you will notice it as part of your command line prompt.

Once you have added hosts and selected a target any vmware commands you run will be against the target host.

You can also add the vMA to active directory so you don’t have to store the passwords in vifp.  This is done via :

sudo domainjoin-cli join domain_name domain_admin_user

Commands:

Since the vMA is running SUSE linux a full featured linux kernel is available to us.  Which means there are a lot of things we can do to make our life easier.   You can use aliases and variables.  For example if I wanted to execute the command:

esxcli hardware cpu list

I could use variables to help:

variable_name=”data for variable”
For example:

cmd="hardware cpu list"

Then I could create an alias for the esxcli command:

alias e="esxcli"

Then I could run the following command:

e $cmd

And that would execute

esxcli hardware cpu list

If you want to keep these aliases or variables then you can add them to your environment settings via these commands:

vi ~/.profile

Add your variables and aliases and they will be available to you when you login.  There is one word of caution these variables are not available to your scripts unless you manually run them.  Any variables you need in scripts should be included in the scripts.

Some common aliases I use are the following:

alias e='esxcli'
alias v='vmware-cmd'
conn="Your connection information for older commands"