Administering vSphere using Perl – Introducing VSPP

Yes, I know you can use the vSphere Perl SDK and it’s ‘OK’ , however, as I wanted to a) learn more perl and b) learn more about the vSphere SDK I started playing with the idea of creating an abstraction layer on top of the vSphere perl SDK that provided perl functions similar in operation to the PowerCLI cmdlets available.

Why?  Well there is a) and b) above, but additionally there is c) because I can 🙂

I figured some of the code might be interesting to some so I created a project over at sourceforge.net and i’m making the code available for anyone who wants it –  if no one does then that is OK as well.

The project is called VSPP (vsphere power perl) , yes you can tell I don’t work in marketing!  You can find the code via subversion at sourceforge.   It’s still pretty crude, no installable packages yet (or maybe ever).

Here’s an example piece of code (also in the svn repos) that Displays all the Hosts in a Datacenter using the VSPP api.

#!/usr/bin/perl -w

# DisplayHostsDatacenter: Display all the hosts in a specific datacenter
#
use strict;
use warnings;

use vspp;

my %opts = (
Datacenter => {
type => "=s",
help => "Datacenter name to display hosts for",
required => 1}
);
Opts::add_options(%opts);
Opts::parse();
Opts::validate();

my $dcname=Opts::get_option('Datacenter');

Util::connect();

my $dc = vspp::GetDatacenter( Name => $dcname ) ;
my $hosts = vspp::GetVMHost( Location => $dc );
print "Datacenter $dcname has " . scalar(@$hosts) . " hosts \nThey are :\n";
foreach (@$hosts) {
print $_->name . "\n";
}

Util::disconnect();

And the output looks like this :

> ./DisplayHostsDatacenter.pl --server  vc1.example.com --Datacenter HomeDC
Datacenter HomeDC has 5 hosts 
They are :
vesx3.example.com
vesx1.example.com
vesx2.example.com
pesx1.example.com
pesx2.example.com

This is ‘similar’ to what you would see in PowerCLI and considerably easier than using the vSphere SDK directly.

Anyway, enjoy.

Tidbit #2

  • Over at the Citrix Community Blog they continue their discussion about using powershell to mange XenDesktop.   To me, powershell is ok, and it’s really nice to see what is a dramatic improvement in windows scripting capability but when you’ve come from a Linux platform with the wealth of scripting languages available you struggle to be super impressed.  Don’t get me wrong powershell is a fantastic step forward for windows scripting.   That being said, a lot of virtualisation vendors are putting considerable effort into adding cmdlets for powershell to manage their hypervisors.  Those toolkits often aren’t available in equivalent form for Linux so even with the superior scripting possibilities you’re lacking the high level virtualisation constructs to wrap that wonderful scripting technology around.   There’s a definite trend to manage virtual infrastructure with powershell.  Personally, i’d prefer another option.
  • Oracle transforms SGE (Sun grid Engine) from free to 90 day evaluation.  It’s pretty sad, but Oracle is far better at making money than sun was so I can understand why they’re doing it.   I’m not sure the current user base is sufficiently cashed up to make the transition and I suspect most will look around for alternatives and only pay if they can’t find one.
  • Oracle released a white paper on Architectural Strategies for Cloud Computing
  • Citrix and HP produced an interesting whitepaper on Analyzing Citrix XenServer persistent performance metrics from Round Robin Database logs – it’s important to measure and monitor performance of your virtual infrastructure, you pay enough for it, make sure it’s performing properly 🙂
  • Something of interest to me is Google have finally released their chat client for Linux.  Am I still a second class google citizen because I run Linux?
  • Veeam released a lite (read as free) version of their reporter product
  • An excellent summary of The State of Open Source System Automation by Aleksey Tsalolikbin over at linux-mag is well worth a look if you want to understand the state of play for linux automation.
  • RHEV bug RHSA-2010:0627-01 DoS or possible privilege escalation on the host.  The geek in me wants to explore this a bit more – damn you free time, where are you.
  • Another excellent whitepaper on VMware vCenter Server Performance and Best Practices for vSphere 4.1 from vmware.
  • Great howto on Installing And Using OpenVZ On Ubuntu 10.04.   Openvz creates secure Linux containers.   If you’re only running a Linux workload and thinking about virtualising it then it’s worth a closer look.  Of course libvirt can manage openvz based linux virtualisation.

Tidbit#1 -Managing other peoples stuff with your tools.

Interesting things i’ve found this week and of course my comments.

vSphere OVF tool

Firstly, for those wondering what the OVF tool is then you can go here.

If you’re wondering what OVF is, well here is a good introduction. The short summary is the Open Virtualization Format (OVF) describes an open, secure, portable, efficient, and flexible format for the packaging and distribution of one or more virtual machines.

So how do you create an OVF file from a VM. It’s simple.

Select the VM you want to export in your vSphere client. Then using the File menu, select Export

Then it’s just a case of following your nose and saving the OVF export to somewhere with sufficient disk space.

At the end of the process you get this :

and the OVF export is complete. The exported contents look like this :

~/ovf$ ls
winxp-sp3-disk1.vmdk winxp-sp3.mf winxp-sp3.ovf

The mf file is a set of SHA1 hashes for the OVF and any of the VMDK files.
The ovf file is an xml file that attempts to describe the virtual machine in an independent/open format which is in theory importable into virtualisation products that fully support OVF files – I must try that 🙂

To display the OVF file you can run the ovftool in probe mode.

~/ovf$ ovftool winxp-sp3.ovf
Opening OVF source: winxp-sp3.ovf
OVF version: 1.0
Name: winxp-sp3

Download Size: 20.35 GB

Deployment Sizes:
Flat disks: 37.27 GB
Sparse disks: Unknown
Networks:
Name: VM Network
Description: The VM Network network

Virtual Hardware:
Family: vmx-07
Disk Types: SCSI-buslogic

Completed successfully

To import the OVF you use a command like (the syntax for the vSphere locator is a bit odd – I highly suggest you read the OVF Tool Guide)

$ ovftool --powerOn --datastore=NFS1 winxp-sp3.ovf vi://10.100.0.100/HomeLab/Host?ip=10.100.0.3
Opening OVF source: winxp-sp3.ovf
Please enter login information for target vi://10.100.0.100/
Username: geoff
Password: ********
Opening VI target: vi://geoff@10.100.0.100/HomeLab/Host
Deploying to VI: vi://geoff@10.100.0.100/HomeLab/Host
Disk progress: 1%

and on the VC you’ll see

Eventually you’ll get

Powering on VM: winxp-sp3
Completed successfully

Sure you can deploy from templates, but what if you have multiple environments in a variety of network locations and you’d like to have a common set of templates – enter the OVF. With a repository full of OVF’s accessible via http you can centrally store and distribute standard images out into all of your virtual environments.

Of course this is quite a simplistic example of creating/deploying from an OVF file. In the future maybe all application servers will be deployed from vApp appliances built with VMware Studio – do you really need systems administrators poking around on individually customized vm’s when in most cases they can be stateless appliances (well stateless apart from the configuration information used at deployment time). Something to ponder.

vsphere VM hot plug CPU script

I was teaching myself how to code scripts using the vSphere SDK for perl.

I was running all this on an Ubuntu 10.04 system

It’s not the fanciest script in the world – it was just to demonstrate the concept of modifying a VM configuration on the fly and seeing what happened in the VM (in this case the VM is a SLES 11 x86_64 system).

Note: Not all systems support hot plugging memory or cpu and they will need to have the option enabled to allow hot plugging – this of course has to be set when the VM is powered off. Once set you’re ok for the future.

The script is called vmhotplugcpu.pl

–server enter the vCenter server you want to connect to.
-vmname enter the name of the VM you want
–cpu Enter the amount of vCPU’s you want to add or remove from the VM either as a positive or negative number

There are plenty of other options as set by the SDK itself.

The best way to run this is after you’ve created a credstore so you don’t have to constantly re-enter the username / password of the VC account.

As you can see from this screenshot the VM in question a SLES11 system only has one CPU.

and this is confirmed by top on the system

If I run my hotplug script

the VC shows some activity

and if we look at the VM setting once the script has run – note – this VM was powered on when we did this.

What’s this, the system still shows 1 cpu !

If we look at /var/log/messages we can see the cpu being added

But to make it active we need to bring it online

and now we get

So what happens if we try to remove a cpu

If we check the VM it doesn’t support hot removal of cpu’s 😦

The best we can do is to mark the cpu offline in Linux

and we can see in /var/log/message the cpu has gone offline

The script is here in case you wondered and I haven’t cleaned it up – I was just trying to work out the API for myself so the code isn’t pretty. You can find plenty of examples supplied with the SDK, that’s how I got the start for this script.

#!/usr/bin/perl -w
#

use strict;
use warnings;

use FindBin;
use lib "$FindBin::Bin/../";

use VMware::VIRuntime;
use XML::LibXML;
use AppUtil::VMUtil;
use AppUtil::XMLInputUtil;

$Util::script_version = "1.0";

sub display;
sub customize;
sub validate { my $valid = 1; return $valid; };
sub check_missing_value;

my %opts = (
'vmname' => {
type => "=s",
help => "The name of the virtual machine",
required => 1,
},
'cpu' => {
type => "=s",
help => "The number opf cpus to add or remove",
required => 1,
},
);

Opts::add_options(%opts);
Opts::parse();
Opts::validate(\&validate);

my $cpucount = Opts::get_option('cpu');
my $vmname = Opts::get_option('vmname');

# connect to the server
Util::connect();
customize();
Util::disconnect();

sub customize() {
my $vm_views = Vim::find_entity_views(view_type => 'VirtualMachine',
filter => {"config.name" => $vmname});
if(defined @$vm_views) {
foreach(@$vm_views) {
if ($_->runtime->powerState->val eq 'poweredOff'){
Util::trace(0, "For hot(un)plugging cpus, VM '$vmname' should be powered on\n");
}
else {
my $num_cpu = $_->config->hardware->numCPU + $cpucount;
Util::trace(0, "VM '$vmname' CPUs =$num_cpu\n");
my $vmConfig =
VirtualMachineConfigSpec->new (numCPUs => $num_cpu);

eval {
Util::trace(0,"Updating cpu allocation...\n");
$_->ReconfigVM (spec => $vmConfig);
$_->update_view_data();
};

if ($@) {
if (ref($@) eq 'SoapFault') {
if (ref($@->detail) eq 'CustomizationFault') {
Util::trace(0, "\n Cannot Perfrom this operation"
." System Error" . "\n");
}
elsif (ref($@->detail) eq 'NotSupported') {
Util::trace(0, "\nThe operation is not supported"
." on the object" . "\n");
}
elsif (ref($@->detail) eq 'HostNotConnected') {
Util::trace(0, "\nUnable to communicate with the remote host, "
."since it is disconnected" . "\n");
}
elsif (ref($@->detail) eq 'InvalidState') {
Util::trace(0, "\nThe operation is not allowed in the"
." current state" . "\n");
}
elsif (ref($@->detail) eq 'InvalidPowerState') {
Util::trace(0, "\nThe attempted operation cannot be"
." performed in the current state" . "\n");
}
elsif (ref($@->detail) eq 'UncustomizableGuest') {
Util::trace(0, "\nCustomization is not supported for"
." the guest operating system" . "\n");
}
else {
Util::trace(0, "\n". $@ . "\n\n");
}
}
else {
Util::trace(0, "\n". $@ . "\n\n");
}
}
}
}
}
else {
Util::trace(0, "No Virtual Machine Found With Name '$vmname'\n");
}
}

__END__

rhev vs vmware – DPM

Well it seems there have been a few blog posts about the relative merits of RHEV powersave modes versus VMWARE DPM.

A couple of the better examples are here and supported by a blog post here.

If you read those articles then it seems that you’re far better off with DPM – but would you be?

Don’t get me wrong I’m a big fan of vSphere and i’m also a fan of RHEV. Competition is a good thing and ultimately the consumer wins – well hopefully 🙂

As things currently stand, vSphere DPM is certainly more efficient (power-wise) than RHEV – powering off servers has to be more power efficient than even the most aggressive cpu frequency scaling.

So what am I going on about here. If you look at the competitive pricing guide between RHEV and vSphere and actually do a quick dollar analysis of the RHEV/vSphere solutions then it can be quite revealing. I should point out I have no idea if the prices in the whitepaper are accurate – i’m just referring to them to demonstrate another way to look at the numbers.

In the windows scenario presented in the whitepaper there are 9 systems running 100 windows vm’s. Over a 3 year period the costs are given as $205,980 using RHEV and $284,382 for using vSphere. The difference being $78,402 in the favour of RHEV

How much of an impact could DPM have on this price difference?

In the 9 systems, i’m going to assume an aggressive 6 systems could be powered down (vSphere DPM) or put into idle state (RHEV) for 5 hrs in a 24 hr period.

Using the power consumption numbers from the above linked blogs (I don’t have my own numbers) then an example active server would run at approx 300 W and an idle server would run at 140 W.

If all 9 servers are on continuously we get 9 * 300 * 24 = 64.8 kWh

If 6 are idle for 5 hrs, then in the case of DPM they would be powered off saving

6 * 5 * 300 = 9 kWh

In the case of RHEV they would run at the lower power consumption, giving us a saving of

6 *5 * (300-140) = 4.8 kWh

Clearly DPM saves us 4.2 kWh in the above contrived case.

Over 3 years that would save us

3 * 365 * 4.2 = 4599 kWh

over the RHEV solution – certainly good for the environment.

If you see how much money that might save you it depends on how much you pay for power. If I use an expensive case of $0.50 / kWh then that would be

0.50 * 4599 = $2299.50 over 3 years – nothing to sneeze at.

However, vSphere is $78,402 more expensive over 3 years and i’ve only saved $2299.50 due to the more efficient DPM.

Hmm, $78,000 can buy me a whole lot of power!

Maybe I should buy RHEV and donate the difference to charity 🙂

Of course, everything above is contrived, but I just wanted to see how the numbers stacked up given the sales and marketing material going around. You have to look at the complete picture in either case as it applies to YOU. If the only differentiator for you is DPM and you’re interested in saving money they why wouldn’t you go RHEV. If there are features you *need* that only exist in vSphere then you’ll have to go that way until RHEV catches up (assuming it does).

Is any of the above data accurate – no idea – the costings come from Redhat and the power savings were just quoted example by people kind enough to measure the power and put their data on the net – the rest is up to you !

SUSE Linux Enterprise Server for VMware

According to this announcement you can now get

When you make a qualifying purchase of VMware vSphere, the industry’s leading virtualization platform, you will be entitled to receive SUSE Linux Enterprise Server (SLES) for VMware and a subscription to patches and updates at no additional cost (see terms and conditions below). By running SLES for VMware, you’ll also have the option to purchase technical support services for directly through VMware.

The terms and conditions are on the above web page. It will be very interesting to see how this pans out. This may be just what the Novell doctor ordered.