A. Mikkelsen

Just released the complete powershell script to control vRanger backup from vCenter.

Read about the script and it’s features here.

Download it here.

Since Vizioncore came out with vRanger 4.x, last year, I been working on porting the VBS script “vRanger Script version 2.0” to Powershell.
During this time I have found several bugs/missing functionality in the vAPI.
Vizioncore have corrected the bugs with new releases of vRanger.
The first vRanger version where all vAPI cmdlets worked in was version 4.2.3.

Porting the script from VBS presented me with a few challenges.

• A reported error in the Get-Date cmdlet in Powershel 2.0.
  Had to use a workaround.

A few words on the scrips functionality.

• Control vRanger backups from vCenter (simple – one interface).
• Perform a FULL backup of DiskO of a VM.
• Perform a FULL backup of all disks of a VM.
• Backup a VM on a specific weekday.
• Control what week (Even/Odd).
• Create a log file.
• Delete previous backup files from repository.
• Delete previous savepoints.
• Send an email when the jobs are completed – incl. succes/errors.
• Offload the files to tape – TSM
• Send mail when TSM is finished offloading backup files.
• And many more…..

When I started coding the script, I found, that if I used functions for all functionality the script was easier to read. Using functions also supplied me with the opportunity of reusing functions – less work .

Some of the functions was created using examples posted and mailed by Scott Harold from Quest and TheVesi./vEcoShell.

Prerequirements

• Powershell 1.0 or 2.0 form Microsoft.
• Vizioncore vRanger 4.x
• A VMware vCenter server, a host and a few VMs.

The script can be downloaded here.

A quick overview of the functions.
Include the right PSSnapins

function LoadSnapin{
    Add-PSSnapin vRanger.API.PowerShell -ErrorAction SilentlyContinue
    Add-PSSnapin vmware.VimAutomation.core -ErrorAction SilentlyContinue
}

LoadSnapin

Delete all previous savepoints – from the repository.

function RemoveAllSavePoints{
	param ($RASP_RepoName)
	$RASP_repoID = Get-Repository | where {$_.Name -eq $RASP_RepoName}
	$RASP_SPlist = Get-RepositorySavePoint -ID $RASP_repoID.Id

	foreach ($RASP_SP in $RASP_SPlist){
		#write $RASP_SP
		Remove-SavePoint -SavePointsToRemove $RASP_SP
	}
}

RemoveAllSavePoints "Remote-Location"

Is a backup job exist with the same name as the one to create – delete it.

function Del-vRangerBackupJobs {
	param ($delJobName)
	$delTemplateID = Get-JobTemplate | where {$_.JobName -eq $delJobName}
	# Verify that a BackupJob named XXX is present before deleting it.
	if ($delTemplateID){
		#write $delTemplateID.JobName
		Remove-JobTemplate -id $delTemplateID.Id
	}
}

Del-vRangerBackupJobs "$Wednesday-Even"

Create the list of VMs to exclude.

function Filter-VMbyCF {
    param ($customF, $val)
    foreach ($vm in $vmlist) {
        $vm.CustomFields | ForEach-Object {
            $cf = $_
            if ($_.Key -like $customF -and $_.Value -like $val){
                    return $vm.name
            }
        }
    }
}

function New-vRangerExcludeList {
    param ($exclude, $include)
    $excludeArray = @()
    $exclude | ForEach-Object {
        if ($include -notcontains $_.Name){
            $excludeArray += ($_.Config.Uuid)
        }
    }
    return $excludeArray
}

$vmlist = Get-VM
$tmplist = Get-Template
$vmlist= $vmlist + $tmplist
$vmFilter = Filter-VMbyCF $customField $CFValue
$vmlistview = $vmlist | Get-View
$excludeList = New-vRangerExcludeList -exclude $vmlistview -include $vmFilter

Create the new backup job.

function New-vRangerBackupJob {
	param ($customJobName, $customCFValue, $customFieldName, $customRepoName, $customExcludeList, $customEmail)
        $jobDesc = "This backup job is created by Logica for use in DR.  All VMs with a Custom Field labeled: $customFieldName with a value of: $customCFValue will be backed up"
        $jobEntity = Get-InventoryEntity -Type VirtualCenter
	$jobRepos = Get-Repository | where {$_.Name -eq $customRepoName}
        $jobFlag = New-BackupFlag -CheckDestinationFreeSpace:$true -UseCompression:$true -PerformNetworkBackupOnFailure:$true

	# Check if all disks or just disk 0
	if ($customJobName -like "*-All*"){
		#write "All - Disks"
		#Add-BackupJobTemplate -JobName $customJobName -JobDescription $jobDesc -JobEntity $jobEntity -ExcludedVMList $customExcludeList -NotificationList $email -TargetRepository $jobRepos -Flags $jobFlag -NumberOfSavePoints 1 -SpaceSavingTechnologyTypeFlag None
		Add-BackupJobTemplate -JobName $customJobName -JobDescription $jobDesc -JobEntity $jobEntity -ExcludedVMList $customExcludeList -NotificationList $customEmail -TargetRepository $jobRepos -Flags $jobFlag -NumberOfSavePoints 1 -SpaceSavingTechnologyTypeFlag None -SpaceSavingCountThreshold 6 -SpaceSavingPercentSizeThreshold 50
	}
	else{
		#write "Only Disk 0"
		#Add-BackupJobTemplate -JobName $customJobName -JobDescription $jobDesc -JobEntity $jobEntity -ExcludedVMList $customExcludeList -NotificationList $email -IncludedDiskList 0 -TargetRepository $jobRepos -Flags $jobFlag -NumberOfSavePoints 1 -SpaceSavingTechnologyTypeFlag None
		Add-BackupJobTemplate -JobName $customJobName -JobDescription $jobDesc -JobEntity $jobEntity -ExcludedVMList $customExcludeList -NotificationList $customEmail -IncludedDiskList 0 -TargetRepository $jobRepos -Flags $jobFlag -NumberOfSavePoints 1 -SpaceSavingTechnologyTypeFlag None -SpaceSavingCountThreshold 6 -SpaceSavingPercentSizeThreshold 50
	}
}

New-vRangerBackupJob "Wednesday-Even" "Wednesday-Even" "Backup" "Remote-Location" $excludeList "to@email.com"

Start the backup job.

function Run-BackupJob{
	param ($runJobName)
	$runTemplateID = Get-TemplateID $runJobName
	Run-JobsNow $runTemplateID
}

Run-BackupJob "Wednesday-Even"

Wait for the backup job to finish.

function Get-TemplateID{
	param($tmpJobName)
	$arrTemplate = get-jobtemplate | where {$_.JobName -eq $tmpJobName}
	return $arrTemplate.ID
}

function WaitForJobToFinish{
	param ($strWaitJob)
	$strCompleted = "NotStarted"
        Start-Sleep -Seconds 300	# 5 minutes
	$jobTemplates = get-JobTemplate | where {$_.JobName -eq $strWaitJob}
	foreach ($job in $jobTemplates){
		do{
			$tasksinfo = get-job |where {$_.ParentJobTemplateID -eq $job.TemplateVersionID}
			foreach ($task in $tasksinfo){
				Start-Sleep -Seconds 300	# 5 minutes
				$strCompleted = $task.JobState

                # Make sure job is running - if not start it...
                if ($task.JobState -eq $NULL -or $task.JobState -eq "NotStarted"){
                    Run-BackupJob $strWaitJob
                    Start-Sleep -Seconds 120	# 2 minutes
                }
			}
		}
		while ($strCompleted -notmatch "Completed")
	}
}

WaitForJobToFinish "Wednesday-Even"

Send TSM finished mail.

function FuncMail {
	#param($strTo, $strFrom, $strSubject, $strBody, $smtpServer)
	param($To, $From, $Subject, $Body, $smtpServer)

	$msg = new-object Net.Mail.MailMessage
	$smtp = new-object Net.Mail.SmtpClient($smtpServer)
	$msg.From = $From
	$msg.To.Add($To)
	$msg.Subject = $Subject
	$msg.IsBodyHtml = 1
	$msg.Body = $Body
	$smtp.Send($msg)
}

FuncMail -To "to@email.com" -From "from@email.com" -Subject "vRanger Pro: TSM backup finished" -Body "Your Body" -smtpServer "your.mailserver.com"

A friend on the Danish VMUG pointed me towards this simple backup script, from GestaltIT.com.
http://gestaltit.com/all/tech/virtualization/simon/vmware-backup-powercli-script/

The script creates a snapshot of a VM (from a CSV file) and creates a Thin Provisioned clone based on the snapshot, on a specified datastore (from the CSV file).

This a smart and simple way to create a backup of a list of VM’s.

It’s a must read for PowerCLI users.

A fellow member in the Danish Usergroup has created a very simple backup script for ESX and ESXi using PowerShell and VCB.

http://www.virtualize.dk

I like that it’s small and simple.
Keep up the good work….

Yesterday I by mistake added the VirtualCenter servers local Users group to a folder in VC with ReadOnly permissions.

This resulted in that I couldn’t delete or change any permissions on the folder.

To solve the problem do the following (On a MSSQL2000 – should be almost the same on MSSQL2005):

• Open the SQL Server Enterprice Manager and browse to the VC_DB (or what you have called the VC database)
• Locate the table VPX_ACCESS and right click it and choose Open Table -> Return all rows
• In the buttom of the table you should be able to locate the wrong permissions entery – make a note of the ID.
  If you can’t find the user/group, you can browse the VPX_ENTITY table to locate ENTITY_ID you need.
• Open the SQL Query Analyzer an choose the VC database
• To delete the row that contains the wrong permissions run the below SQL code or modify it for you liking.
  To only delete row 221 from the table

  DELETE FROM esx.VPX_ACCESS WHERE ID = 211

  To delete all rows containing a specific user/group from the table

  DELETE FROM esx.VPX_ACCESS WHERE PRINCIPAL = 'your_user or group'

After you have deleted or updated all the permissions you need, you have to restart the VC server service before the changes will take effect.

Disclamer…..
All changes to the VC database should be avoided at all time.
Always make a backup of the database before making ANY changes to it.
All changes to the VC database using examples on this website is at your own risk.

A few weeks ago we were conducting a vRanger DR test of a VM (new host and new LUN).

The VM was restored succesfull, but when we powered it on we discovered that it for some unexplaned reason had lost the SCSI0:1 (100GB) and SCSI0:2(300GB) VMDK’s.
It had created two new 20GB VMDK’s instead.
We searched the LUN and found the two orginal *-flatvmdk files but not the descripter files.

Use Putty to identify the size of the *-flatvmdk file. ex. 100GB

ls -lah

From a VM (not running) create a new disk with the same size as the one you are missing. The name is not important.

Locate the newly created *.vmdk and *-flat.vmdk file. Copy the new *.vmdk file to the folder that contains the orginal *-flat.vmdk file.

cp rescue_me.vmdk /vmfs/volumes/mysan/rescued/rescued.vmdk

Use VI or NANO to change the following line as below from:
RW 419426200 VMFS “rescue_me-flat.vmdk”
To:
RW 419430500 VMFS “rescued-flat.vmdk”

Make sure that the name of the .vmdk file correspond to the SCSI*:*.fileName in the vmx file.
Now just power on the VM and the orginal disks are intac.

UPDATE
Today I found out that esXpress has created a website that can create a VMDK descriptor file.
http://www.esxpress.com/tools/wrapgen.php

Below is an exampel for the rescure_me-flat.vmdk file with a size of 100GB (107374182400 bytes)

“This first VMware® VMbook focuses on Business Continuity and Disaster Recovery (BCDR) and is intended to guide the reader through the step-by-step process to set-up a multisite VMware Infrastructure that is capable of supporting BCDR services.

The VMbook will provide very valuable insight into the considerations and design principles for a multisite virtual infrastructure data center that leverages array based replication for the replication of VMFS datastores; which is one of the prerequisites for Site Recovery Manager 1.0. The VMbook authors suggest your customers leverage this VMbook as a reference even if they are planning to implement a Site Recovery Manager based BCDR solution day one, as the BCDR solution detailed in this VMbook provides design principle guidance for a successful multisite virtual infrastructure data center deployment which is the underpinnings for a successful SRM deployment.”

The VMbook is available for download from: http://www.vmware.com/resources/techresources/1063

I’m now done with the new script and it’s working perfectly.

Background for the script:

At work we use vRanger to take DR snapshots of all our VM’s (more than 400).

We decided that we would only snapshot drive 0 and use a TSM client to backup the data in each VM and that the snapshots were only to be taken outside working hours.
These choices gave us some problems when running vRanger because we couldn’t schedule the snapshots from vRanger.

Because we have more than 400 VM’s we had to divide the list of witch VM’s to backup, into 14 days.
This means that one specific VM will only be snapshotted every 2 weeks.

In Virtual Center we created a custom field (=Backup) to tell the script witch VM’s to snapshot witch days.
We use full weekdays together with Even / Odd for dividing the VM’s into 14 days.
Ex. “Monday-Even” or “Sunday-Odd”

The solution was to make our own script that would handle the logic and just use vRanger to do the actual snapshotting.
We created a VBS script to hold the logic.
1.
Create a log file
2.
Delete snapshots from the day before (due to lack of storage space on the server)
3.
Generate the .cmd file from information in a custom field (Backup) in VC
4.
Call the .cmd file that was created earlier and contains the information on which VM’s to snapshot (Based on weekday – one file for each day).
5.
Start TSM (send the VM’s snapshots to tape)
6.
Send a status mail

(You have to use vRanger GUI to choose which drives to snapshot – changed from 3.17 -> 3.20)

I know this solution isn’t very dynamic but i works.

You can download the new script here – version 2.0.

Duncan over at YellowBricks.com found a undocumented fearure in the VCB config.js file

One of my customers wanted to use the default VCB framework but did not want to quiesce the VM for several reasons. (Databases, Active Directory etc.) I could not find an option in the config.js file but noticed the following in the file glue.js:

// A fallback to be able to switch to non-quiesced snapshots
if (typeof(NO_QUIESCE) != "undefined") {
cmd +="-Q 0 ";
}

In other words, setting the option “NO_QUIESCE” with no value in config.js results in the VM not being quiesced, default it will quiesce the VM! I added the following line to the  config.js file to accomplish this:

NO_QUIESCE="";

Frane Borozan has created a automated script based on VMware Converter to take a physical server and make fresh replica of it on VMware Server.

It is worth a look.

http://www.p2vbackup.com

A. Mikkelsen

At work we use vRanger to take DR snapshots of all our VM’s (more than 370).

We decided that we would only snapshot drive 0 and use a TSM client to backup tha data in each VM and that the snapshots were only to be taken outside working hours.
These choices gave us some problems when running vRanger because we couldn’t schedule the snapshots from vRanger.

The solution was to make our own script that would handle the logic and just use vRanger to do the actual snapshotting.
We created a VBS script to hold the logic.
1.
Create a log file
2.
Delete snapshots from the day before (do to lack of storage space on the server)
3.
Call the .cmd file that holds the information on witch VM’s to snapshot (Based on weekday – one file for each day).
4.
Start TSM (send the VM’s snapshots to tape)
5.
Send a status mail

We then created 7 .cmd files (one for each weekday) in witch we add a line for each VM to snapshot that day.
(You have to use vRanger GUI to choose witch drives to snapshot – changed from 3.17 -> 3.20)

I know this solution isn’t very dynamic but i works.

I’m in the process of upgrading the script so that it dynamicly creates a list of witch VM’s to snapshot based on a custom field in VC.
Furthermore i also want to create a script to update the vRanger database with witch drives to snapshot based on a custom field in VC.

I will upload the updated script as soon as it is done.
You can download the current script here.