This tool is the follow up release of the ghettoVCB
backup utility which allows users to perform backups of virtual machines residing on ESX(i) 3.5+/4.x+
servers using methodology similar to VMware's VCB
tool. By incorporating highly constructive feedback from the VMware community and utilizing the existing VI API, ghettoVCB’s framework was completely rewritten to be harder, better, faster, stronger
.
The primary motivation for ghettoVCBg2 was to provide ESXi users with access to the utility without relying on unlocking and utilizing the unsupported console. To satisfy this requirement, the rebuilt framework takes advantage of the
VI Perl Toolkit/vSphere SDK for Perl which is present in the
VIMA/vMA virtual appliance provided by VMware. As a result, ghettoVCBg2 provides a more proper backup solution that administrators can utilize in their virtual infrastructure.
As it stands, it is a requirement that ghettoVCB be invoked directly on each of the ESX servers hosting virtual machines in need of a backup. By taking advantage of VIMA/vMA, the entire backup process becomes centralized in turn eliminating the abovementioned requirement. Additionally, by leveraging the vi-fastpass library, unattended authentication between VIMA/vMA and target ESX servers is made possible. This alleviates the need to continually pass in credentials for each ESX server.
By centralizing the backup process, individual virtual machine backup lists pertaining to their respective ESX(i) servers are no longer required. ghettoVCBg2 has the ability to identify virtual machines and their respective hosts, potentially eliminating any past confusion associated with maintaining multiple lists.
In its current configuration, the script will allow up to 3 unique backups of the Virtual Machine before it will overwrite the previous backups; this however, can be modified to fit procedures if need be. Please be diligent in running the script in a test or staging environment before using it on production live Virtual Machines; this script functions well within our environment but there is a chance that it may not fit well into other environments.
VMs running on ESX(i) 3.5u2+ or ESX(i) 4.x+
VMware VIMA 1.0 or VMware vMA 4.x
Version | Support | Alternative |
ESX 3.5u2 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 3.5u3 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 3.5u4 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESXi 3.5u2 | Works for free | ghettoVCB\ |
ESXi 3.5u3 | Works for free | ghettoVCB\ |
ESXi 3.5u4 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 4.0 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESXi 4.0 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 4.1 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESXi 4.1 | Requires at least Foundation/Essentials license | ghettoVCB\ |
1) Download ghettoVCBg2.tar.gz
to your VIMA 1.0/vMA 4.x system
2) Extract the contents of the tarball
[vi-admin@scofield ~]$ tar -zxvf ghettoVCBg2.tar.gz
ghettoVCBg2/
ghettoVCBg2/ghettoVCBg2-vm_backup_configuration_template
ghettoVCBg2/ghettoVCBg2.pl
3) The script should be in a directory called
ghettoVCBg2
Prior to starting, ensure that the VMware VIMA/vMA host is managing the appropriate ESX(i) 3.5u2/4.x+ host(s)
1. Add ESXi hosts to VIMA/vMA managment using vifp
A password prompt for the vi-admin account and root password to the pertinent ESXi host(s) will be presented.
[vi-admin@vima-primp-industries ~]$ sudo vifp addserver olga.resnet.ucsb.edu
root@olga.resnet.ucsb.edu's password:
[vi-admin@vima-primp-industries ~]$
2. Verify that the appropriate hosts are being managed by VIMA/vMA
[vi-admin@vima-primp-industries ~]$ sudo vifp listservers
olga.resnet.ucsb.edu
himalaya.primp-industries.com
The following variables (similar to that of ghettoVCB) need to be defined within the script prior to execution.
Defining the datastore in which the backups are stored:
Defining the name of folder that will contain backups (if folder does not exist, it will automatically be created):
Defining the backup rotation per VM:
Defining the backup disk format (zeroedthick, eagerzeroedthick, thin, and 2gbsparse are available):
Defining the adapter type for backed up VMDK (buslogic, and lsilogic are available):
Defining whether the VM is powered down or not prior to backup (1 = enable, 0 = disable):
Note: VM(s) that are powered off will not require snapshoting
Defining whether virtual machine memory is snapped and if quiescing is enabled (1 = enable, 0 = disable):
Note: By default both are disabled
Defining the detail of logging (verbose or normal):
*Defining VMDK(s) to backup from a particular VM either a list of vmdks or "all"
VM_VMDK_FILES="myvmdk.vmdk"
Experimental Support Configs:
Note: Port 25 by default
my $SEND_MAIL = "no";
my $EMAIL_HOST = "emailserver";
my $EMAIL_DOMAIN = "localhost.localdomain";
my $EMAIL_TO = 'William Lam <william@primp-industries.com.com>';
my $EMAIL_FROM = 'ghettoVCBg2 <ghettoVCBg2@primp-industries.com.com>';
Ensure that you do not edit past this section:
########################## DO NOT MODIFY PAST THIS LINE ##########################
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl
Required command option 'vmlist' not specified.
Synopsis: ./ghettoVCBg2.pl OPTIONS
Command-specific options:
--config_dir
Name of directory containing VM(s) backup configurations
--dryrun (default '0')
Set to 1 to enable dryrun mode (default 0)
--output (default '/tmp/ghettoVCBg2.log')
Full path to output log (default /tmp/ghettoVCBg2.log)
--vmlist (required)
A file containing a list of virtual machine(s) to be backed up on host
Common VI options:
--config (variable VI_CONFIG)
Location of the VI Perl configuration file
--credstore (variable VI_CREDSTORE)
Name of the credential store file defaults to <HOME>/.vmware/credstore/vicredentials.xml on Linux and <APPDATA>/VMware/credstore/vicredentials.xml on Windows
--encoding (variable VI_ENCODING, default 'utf8')
Encoding: utf8, cp936 (Simplified Chinese), iso-8859-1 (German), shiftjis (Japanese)
--help
Display usage information for the script
--passthroughauth (variable VI_PASSTHROUGHAUTH)
Attempt to use pass-through authentication
--passthroughauthpackage (variable VI_PASSTHROUGHAUTHPACKAGE, default 'Negotiate')
Pass-through authentication negotiation package
--password (variable VI_PASSWORD)
Password
--portnumber (variable VI_PORTNUMBER)
Port used to connect to server
--protocol (variable VI_PROTOCOL, default 'https')
Protocol used to connect to server
--savesessionfile (variable VI_SAVESESSIONFILE)
File to save session ID/cookie to utilize
--server (variable VI_SERVER, default 'localhost')
VI server to connect to. Required if url is not present
--servicepath (variable VI_SERVICEPATH, default '/sdk/webService')
Service path used to connect to server
--sessionfile (variable VI_SESSIONFILE)
File containing session ID/cookie to utilize
--url (variable VI_URL)
VI SDK URL to connect to. Required if server is not present
--username (variable VI_USERNAME)
Username
--verbose (variable VI_VERBOSE)
Display additional debugging information
--version
Display version information for the script
Note: There are some additional command line arguments that can be provided upon runtime: --debug, --output and --vmlist. --vmlist is the only parameter required. The rest will have defaults if not explicitly defined.
--dryrun
--output
--config_dir
The input to this script is a file that contains the display name of the virtual machine(s) separated by a newline. When creating this file on a non-Linux/UNIX system, you may introduce a ^M character which may cause the script to misbehave. To ensure that this does not occur, please create the file on the ESX/ESXi host.
Provided is a sample of what the file should contain:
[vi-admin@scofield ~]$ cat virtual_machine_backup_list
VIMA
vMA-2
DISK_FORMAT | 2gbsparse |
ADAPTER_FORMAT | buslogic |
LOG_LEVEL | info |
VM_SNAPSHOT_MEMORY | 0 |
VM_SNAPSHOT_QUIESCE | 0 |
VMS_TO_BACKUP | (VIMA,vMA-2) |
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --vmlist virtual_machine_backup_list
[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
11-13-2009 23:34:53 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-13-2009 23:34:53 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-13-2009 23:34:53 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-13-2009 23:34:53 -- info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
11-13-2009 23:34:53 -- info: CONFIG - DISK_BACKUP_FORMAT = 2gbsparse
11-13-2009 23:34:53 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-13-2009 23:34:53 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-13-2009 23:34:53 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-13-2009 23:34:53 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-13-2009 23:34:53 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-13
11-13-2009 23:34:53 -- info: CONFIG - VM_VMDK_FILES = all
11-13-2009 23:34:58 -- info: Initiate backup for VIMA found on himalaya.primp-industries.com
11-13-2009 23:35:18 -- info: VIMA has 1 VMDK(s)
11-13-2009 23:37:37 -- info: Backup completed for VIMA!
11-13-2009 23:37:37 -- info: Initiate backup for vMA-2 found on himalaya.primp-industries.com
11-13-2009 23:37:47 -- info: vMA-2 has 1 VMDK(s)
11-13-2009 23:40:07 -- info: Backup completed for vMA-2!
11-13-2009 23:40:10 -- info: ============================== ghettoVCBg2 LOG END ==============================
VIMA
VM_BACKUP_DATASTORE | dlgCore-NFS-bigboi.VM-Backups |
VM_BACKUP_DIRECTORY | WILLIAM_BACKUPS |
DISK_FORMAT | zeroedthick |
ADAPTER_FORMAT | buslogic |
LOG_LEVEL | debug |
VM_SNAPSHOT_MEMORY | 1 |
VM_SNAPSHOT_QUIESCE | 1 |
VM_VMDK_FILES | VIMA_1.vmdk |
vMA-2
VM_BACKUP_DATASTORE | dlgCore-NFS-bigboi.VM-Backups |
VM_BACKUP_DIRECTORY | vGHETTO_BACKUPS |
DISK_FORMAT | thin |
ADAPTER_FORMAT | lsilogic |
LOG_LEVEL | info |
VM_SNAPSHOT_MEMORY | 0 |
VM_SNAPSHOT_QUIESCE | 0 |
VM_VMDK_FILES | all |
1. Create folder to hold individual VM backup policies (can be named anything):
[vi-admin@scofield ~]$ mkdir backup_config
2. Create individual VM backup policies for each VM and ensure each file is named exactly as the display name of the VM being backed up (use provided template to create duplicates):
[vi-admin@scofield backup_config]$ cp ghettoVCBg2-vm_backup_configuration_template VIMA
[vi-admin@scofield backup_config]$ cp ghettoVCBg2-vm_backup_configuration_template vMA-2
4. Using a preferred editor (nano or vi), edit each VM backup policy file
Listing of VM backup policy within backup configuration directory
[vi-admin@scofield backup_config]$ ls
ghettoVCBg2-vm_backup_configuration_template VIMA vMA-2
5. Pass in the value, backup_config, into the --config_dir parameter
(e.g.)
--config_dir backupConfigs
Example of two distinct VM backup policies will be applied to VIMA and vMA-2
Backup policy for "VIMA" (backup only 1 specific VMDKs)
[vi-admin@scofield backup_config]$ cat VIMA
VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups"
VM_BACKUP_DIRECTORY = "WILLIAM_BACKUPS"
VM_BACKUP_ROTATION_COUNT = "3"
DISK_BACKUP_FORMAT = "zeroedthick"
ADAPTER_FORMAT = "buslogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
VM_SNAPSHOT_MEMORY = "1"
VM_SNAPSHOT_QUIESCE = "1"
LOG_LEVEL = "debug"
VM_VMDK_FILES = "VIMA_1.vmdk"
Backup policy for VM "vMA-2" (backup all VMDKs found)
VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups"
VM_BACKUP_DIRECTORY = "vGHETTO_BACKUPS"
VM_BACKUP_ROTATION_COUNT = "3"
DISK_BACKUP_FORMAT = "thin"
ADAPTER_FORMAT = "lsilogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
VM_SNAPSHOT_MEMORY = "0"
VM_SNAPSHOT_QUIESCE = "0"
LOG_LEVEL = "info"
VM_VMDK_FILES = "all"
The following is an output of the script when utilizing the new --config_dir parameter with debug mode enabled.
Note: --vmlist is still a required parameter with the exception that backup policy files must exist for each VM in the list else the VM will be ignored in the backup process.
Execution of backup
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --config_dir backup_config --vmlist virtual_machine_backup_list
[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
11-14-2009 00:18:41 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-14-2009 00:18:45 -- info: CONFIG - USING CONFIGURATION FILE = VIMA
11-14-2009 00:18:45 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:18:45 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:18:45 -- info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
11-14-2009 00:18:45 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
11-14-2009 00:18:45 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-14-2009 00:18:45 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:18:45 -- info: CONFIG - VM_SNAPSHOT_MEMORY = YES
11-14-2009 00:18:45 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
11-14-2009 00:18:45 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:18:45 -- info: CONFIG - VM_VMDK_FILES = VIMA_1.vmdk
11-14-2009 00:18:45 -- info: Initiate backup for VIMA found on himalaya.primp-industries.com
11-14-2009 00:18:55 -- debug: VIMA original powerState: poweredOn
11-14-2009 00:18:55 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2009-11-14" for VIMA
11-14-2009 00:19:45 -- info: VIMA has 2 VMDK(s)
11-14-2009 00:19:45 -- debug: findVMDKFile: Found VMDK File: VIMA_1.vmdk
11-14-2009 00:19:45 -- debug: backupVMDK: Backing up "[http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk" to "[http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14/VIMA_1.vmdk"
11-14-2009 00:19:45 -- debug: backupVMDK: Signal copyThread to start
11-14-2009 00:19:45 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
11-14-2009 00:19:45 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
11-14-2009 00:19:45 -- debug: CopyThread: Start backing up VMDK(s) ...
11-14-2009 00:19:52 -- debug: copyTask: send copySuccess message ...
11-14-2009 00:19:52 -- debug: copyTask: waiting for next job and sleep ...
11-14-2009 00:20:46 -- debug: backupVMDK: Successfully completed backup for [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk Elapsed time: 1 min
11-14-2009 00:20:49 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2009-11-14" for VIMA
11-14-2009 00:20:51 -- debug: checkVMBackupRotation: Starting ...
11-14-2009 00:20:57 -- debug: Purging [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-13--3 due to rotation max
11-14-2009 00:21:05 -- info: Backup completed for VIMA!
11-14-2009 00:21:05 -- debug: reConfigureBackupParams: VM - vMA-2
11-14-2009 00:21:05 -- info: CONFIG - USING CONFIGURATION FILE = vMA-2
11-14-2009 00:21:05 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:21:05 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:21:05 -- info: CONFIG - VM_BACKUP_DIRECTORY = vGHETTO_BACKUPS
11-14-2009 00:21:05 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
11-14-2009 00:21:05 -- info: CONFIG - ADAPTER_FORMAT = lsilogic
11-14-2009 00:21:05 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:21:05 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-14-2009 00:21:05 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-14-2009 00:21:05 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:21:05 -- info: CONFIG - VM_VMDK_FILES = all
11-14-2009 00:21:05 -- info: Initiate backup for vMA-2 found on himalaya.primp-industries.com
11-14-2009 00:21:15 -- info: vMA-2 has 1 VMDK(s)
11-14-2009 00:23:22 -- info: Backup completed for vMA-2!
11-14-2009 00:23:25 -- info: ============================== ghettoVCBg2 LOG END ==============================
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --config_dir backup_config --vmlist virtual_machine_backup_list --dryrun 1
[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
11-14-2009 00:26:29 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-14-2009 00:26:34 -- info: CONFIG - USING CONFIGURATION FILE = VIMA
11-14-2009 00:26:34 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
11-14-2009 00:26:34 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
11-14-2009 00:26:34 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-14-2009 00:26:34 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_MEMORY = YES
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:26:34 -- info: CONFIG - VM_VMDK_FILES = VIMA_1.vmdk
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO VIMA ----------
11-14-2009 00:26:34 -- info: DEBUG - Host Build: VMware ESX 4.0.0 build-164009
11-14-2009 00:26:34 -- info: DEBUG - Host: himalaya.primp-industries.com
11-14-2009 00:26:34 -- info: DEBUG - Virtual Machine: VIMA
11-14-2009 00:26:34 -- info: DEBUG - VM ConfigPath: [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA.vmx
11-14-2009 00:26:34 -- info: DEBUG - VMX File: VIMA.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupConfigPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14/VIMA.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14
11-14-2009 00:26:34 -- info: DEBUG - VM Datastore: himalaya-local-SAS.Savvio
11-14-2009 00:26:34 -- info: DEBUG - VMDK(s):
11-14-2009 00:26:34 -- info: DEBUG - [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA.vmdk
11-14-2009 00:26:34 -- info: DEBUG - [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO VIMA ----------
11-14-2009 00:26:34 -- debug: reConfigureBackupParams: VM - vMA-2
11-14-2009 00:26:34 -- info: CONFIG - USING CONFIGURATION FILE = vMA-2
11-14-2009 00:26:34 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIRECTORY = vGHETTO_BACKUPS
11-14-2009 00:26:34 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
11-14-2009 00:26:34 -- info: CONFIG - ADAPTER_FORMAT = lsilogic
11-14-2009 00:26:34 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:26:34 -- info: CONFIG - VM_VMDK_FILES = all
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO vMA-2 ----------
11-14-2009 00:26:34 -- info: DEBUG - Host Build: VMware ESX 4.0.0 build-164009
11-14-2009 00:26:34 -- info: DEBUG - Host: himalaya.primp-industries.com
11-14-2009 00:26:34 -- info: DEBUG - Virtual Machine: vMA-2
11-14-2009 00:26:34 -- info: DEBUG - VM ConfigPath: [http://himalaya-local-SAS.Constellation|http://himalaya-local-SAS.Constellation] vMA-2/vMA-2.vmx
11-14-2009 00:26:34 -- info: DEBUG - VMX File: vMA-2.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupConfigPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] vGHETTO_BACKUPS/vMA-2/vMA-2-2009-11-14/vMA-2.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] vGHETTO_BACKUPS/vMA-2/vMA-2-2009-11-14
11-14-2009 00:26:34 -- info: DEBUG - VM Datastore: himalaya-local-SAS.Constellation
11-14-2009 00:26:34 -- info: DEBUG - VMDK(s):
11-14-2009 00:26:34 -- info: DEBUG - [http://himalaya-local-SAS.Constellation|http://himalaya-local-SAS.Constellation] vMA-2/vMA-2.vmdk
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO vMA-2 ----------
11-14-2009 00:26:37 -- info: ============================== ghettoVCBg2 LOG END ==============================
0Q: I'm having some issues or would like to provide feedback
0A: Please join the ghettoVCBg2 Group to post your comments/questions
1Q: I'm using ESXi and free licensed version ( I did not pay for anyting ), will this script still work?
1A: No, you need to have a licensed version of ESXi to use this script.
2Q: I have a VM listed in my backup file but no backup of that VM took place. The log stated that the VM could not be found. How could this be? I know that it’s running on host X…
2A: Ensure the ESX/ESXi host is being managed by VMware VIMA/vMA (sudo vifp listservers to verify)
3Q: Can I schedule backups to take place hourly, daily, monthly, yearly?
3A: Yes, do a search online for crontab.
4Q: I want to schedule my backup on Windows, how do I do this?
4A: Do a search for
plink. Make sure you have paired SSH keys setup between your Windows system and VIMA/vMA.
5Q: Do I have to use VMware VIMA/vMA to initiate the backup?
5A: Yes, vi-fastpass is being used and only VIMA/vMA provides this capability. However, you can kickoff the process from another host (*nix/win)
4Q: I only have a single ESXi host. I want to take backups and store them somewhere else. The problem is: I don't have NFS, iSCSI nor FC SAN. What can I do?
4A: You can use local storage to store your backups assuming that you have enough space on the destination datastore. Afterwards, you can use scp (WinSCP/FastSCP) or vifs (vCLI) to transfer the backups from the ESXi host to your local desktop.
5Q: I’m pissed; the backup is taking too long. My datastore is of type X?
5A: YMMV, take a look at your storage configuration and make sure it is optimized.
6Q: I noticed that the backup rotation is occurring after a backup. I don't have enough local storage space, can the process be changed?
6A: This is primarily done to ensure that you have at least one good backup in case the new backup fails. If you would like to modify the script, you're more than welcome to do so.
7Q: What is the best storage configuration for datastore type X?
7A: Search the VMTN forums; there are various configurations for the different type of storage/etc.
8Q: I want to setup NFS Server to run my backups. Which is the best and should it be virtual or physical?7
8A: Please refer to answer 7A. From experience, we’ve seen physical instances of NFS servers to be faster than their virtual counterparts. As always, YMMV.
9Q: When I try to use the script I get an error stating: "restricted version". What the heck is this?
9A: Please take a look at the ESX/ESXi Version Support Table in this document. The reason is explained here.
10Q: I'm getting error X when using the script or I'm not getting any errors, the backup didn’t even take place. Oh what could be wrong?
10A: First, you'll want to check the log (default /tmp/ghettoVCBg2) and see if there are any obvious errors. If not, please provide the entire output of the log, a short description of your environment and the type of VM(s) you're trying to backup—I will try to provide some assistance.
11Q: I have VMs that have snapshots. I want to back these things up but the script doesn’t let me do it. How do I fix that?
11A: VM snapshots are not meant to be kept for long durations. When backing up a VM that contains a snapshot, you should ensure all snapshots have been committed prior to running a backup. No exceptions will be made…ever.
12Q: I would like to restore from backup, what is the best method?
12A: The restore process will be unique for each environment and should be determined by your backup/recovery plans. At a high level you have the option of mounting the backup datastore and registering the VM in question or copy the VM from the backup datastore to the ESX/ESXi host. The latter is recommended so that you're not running a VM living on the backup datastore or inadvertently modifying your backup VM(s).
13Q: When using the experimental feature: Individual VM backup policy, if I don't have a policy will it use the default configurations defined in the script?
13A: No, this use case will not be supported, ever. If you decide to use the per VM policy, then the global configuration in the script is ignored. If a VM policy config is not found, the VM will not be backed up even if it's in the –-vmlist.
14Q: In vMA 4.0, vCenter target will be supported, can I just add vCenter target and assume this will work?
14A: At the moment, this is not a supported feature. You will need to add each ESX/ESXi host for the vi-fastpass library to be able to locate the host(s).
15Q: When I try to run the script I get: "-bash: ./ghettoVCBg2.pl: Permission denied", what is wrong?
15A: You need to change the permission on the script to be executable, chmod +x ghettoVCBg2.pl
16Q: Where can I download the latest version of the script?
16A: The latest version is available on this page at the bottom. To get the version of the script run
./ghettoVCBg2.pl --version
17Q: I'm running into an issue when I backup my VMs using NFS on Linux, I'm able to either get 1 successful backup and all subsequent backups fail OR I'm having issues where snapshots are not being removed upon completition and future backups fail.
17A: This problem/issue has been reported by quite a few users in the community and only affects a very small percentage of users overall under a specific configuration. The problem as I understand it today which is still an on-going investigation by myself and the community is as follows:
Setup: NFS on Linux where the underlying volume is EXT3 filesystem
and backups are failing primarily around larger sized VMs ( > 150GB).
The issue is during the final phase of a VM being backed-up in the rotation function where a request is sent out to the NFS server to either A) rotate/rename a directory B) delete an old backup. There is either a disconnect or delay that severely affects the backup process that may lead to a whole slew of "unexpected" behavior from having only 1 backup complete successful and all subsquent backups failing/etc.
First off, there is nothing wrong with the script from what we've found it and has to do with NFS server and specifically the volume that the backup datastore is hosted on. Thanks to community user
ralfoertner who had a simliar problem using
ghettoVCBg2 using Debian with EXT3 base filesystem for his NFS volume. He decided to do some additional troubleshooting to further isolate the problem and actually found by changing the underlying filesystem from EXT3 to XFS that the issue went away and resolve all issues he had ran into regarding an NFS on Linux backup solution.
Now, we're still 100% sure why the above solution has resolved the problem or what the actual problem is but I'll explain the current theory:
EXT3 is a journaled filesystem, you can do a search online for more information. There's a few advantages with EXT3 from it's predcessor EXT2, but one difference is in the approach it takes with journaling as explained here in greater detail:
http://www.ibm.com/developerworks/library/l-fs7.html
Basically EXT2 does only "metadata journling" whereas EXT3 does both data and metadata journaling, this would ensure that all changes to the filesystem where it's data or metadata is consistent in the event of an unexpected system crash. With EXT3, there are 3 journaling options: (
data=writeback mode,data=ordered mode,data=journal mode ) and by default it uses
data=ordered mode to ensure the consistency as described from above.
One easy method to check what mount options are configured, on a Linux system you can check /proc/mounts:
[vi-admin@scofield ~]$ cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw,data=ordered 0 0
/dev /dev tmpfs rw 0 0
/proc /proc proc rw 0 0
/sys /sys sysfs rw 0 0
/proc/bus/usb /proc/bus/usb usbfs rw 0 0
devpts /dev/pts devpts rw 0 0
/dev/VolGroup00/var /var/log ext3 rw,data=ordered 0 0
/dev/sda1 /boot ext3 rw,data=ordered 0 0
tmpfs /dev/shm tmpfs rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
So now what does this have to do with XFS and EXT3? With the defaults of EXT3, it may not always give you the best performance due to the extra overhead of the way it does its journaling, if you were to change the options to
data=writeback mode, you'll notice that this is simliar journaling method found in
XFS,JFS and ResierFS as described here:
http://www.ibm.com/developerworks/library/l-fs8.html
I believe that there are some additional overheads that maybe caused by using NFS on EXT3, which may explain the issue that users are seeing and that moving from EXT3 to XFS has "seemed" to solve the issues for those who have implemented the change. Though, before you go out and make this change, you need to understand your environment and what this change would exactly mean, please refer to differences in EXT2/XFS and EXT3, especially around a power interruption or system crash.
In conclusion, I'm not recommending one way or the other, it really depends on your environment. Though this has shown to resolve the issue as users have implemented to get around this problem, but it's definitely worth while to really understand the problem in your environment and figure out why and decide on the best solution. I would also recommend users perform their own tests to further isolate the problem in their environment and try local storage, using iSCSI to see if they can pin-point the problem.
If you have found this script or other scripts to be useful and would like to contribute back, please click
here to donate! Thank You
07/20/10 - build 4.3
Enhancements:
Bug Fixes:
03/27/10 - build 4.2
Enhancements:
03/27/10 - build 4.1
Enhancements:
02/24/09 - build 4.1
Bug Fixes:
05/26/09 - build 2.5.1
Enhancements:
Bug Fixes:
04/25/09 - build 2.5
Enhancements:
Bug Fixes:
02/21/09 - build 2.0
Initial release
I have a VM with a virtual RDM. When I run the script, it immediately says 'WARN - RDM was found for (vmname), backup will not take place.' The comments in the code seem to indicate that this is normal, though the 'features list' above says they're supported. Any advice?
Hi smyle,
I'm sorry, I think when I originally started writing the script I was implementing the features in ghettoVCB which did not support RDM period...I'm looking at the code right now and it is in fact blocking RDM's all together, let me jot that down as a note and I'll get back to you.
Thanks
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Everyone,
There is a known bug at the moment with vMA 4.0 and vSphere ESX(i) 4.0 that will not allow you to backup more than 1 VM, the logs will state that VM X can not be found after the first VM on the list is completed. I initially saw this with vMA 4.0 GA with vSphere ESX 4.0 RC but was hoping this might be due to the miss-match of the build. Something was dramatically alter/changed during RC release of vMA 4.0 and the GA release of vMA 4.0 that's causing this problem. I'll be investigating the root cause, but with VIMA 1.0 & ESX(i) 3.5u2+ will work and has been tested/verified.
Please let me know if you run into any other issues.
Thanks
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
The bug has been identified and the issue actually lies with the vSphere CLI's version of vifs, I'm still awaiting feedback from VMware to confirm the issue.
Some details can be found here: http://engr.ucsb.edu/~duonglt/vmware/vifs_bug.html
Once the issue has been identified, I'll push out a temporarily fix to remediate the issue.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Just updated the script to fix both the virtual RDM issue & a bug with VMware's vifs, let me know if anyone runs into any other issues
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hi,
It does not work for me with ESXi 4. It exits in 10-15 seconds. Looks like it fails to take a snapshot. I tried different types of the snapshots - same result. This is the latest version (same problem with the previous one).
Here is a typical log.
05-27-2009 18:03:53 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
05-27-2009 18:03:53 -- CONFIG - VM_BACKUP_DATASTORE = backup
05-27-2009 18:03:53 -- CONFIG - VM_BACKUP_DIRECTORY = backups
05-27-2009 18:03:53 -- CONFIG - DISK_BACKUP_FORMAT = eagerzeroedthick
05-27-2009 18:03:53 -- CONFIG - ADAPTER_FORMAT = lsilogic
05-27-2009 18:03:53 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
05-27-2009 18:03:53 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-27-2009 18:03:53 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-27-2009 18:03:53 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-05-27
05-27-2009 18:03:58 -- Initiate backup for "winxp" found on aaa.bbb.ccc.ddd!
05-27-2009 18:04:11 -- winxp original powerState: poweredOff
05-27-2009 18:04:11 -- winxp has 1 VMDK(s)
05-27-2009 18:04:11 -- Backing up "[nexenta_nfs] winxp/winxp.vmdk" to "[backup] backups/winxp/winxp-2009-05-27/winxp.vmdk"
05-27-2009 18:04:15 -- Backup completed for winxp/winxp.vmdk
05-27-2009 18:04:20 -- Backup completed for "winxp"!
Are you using the free version of ESXi unlicensed? Licensed ESXi 4.0 or eval ESXi 4.0? I assume you're using vMA 4.0?
Can you please re-run the script and set the loglevel to "verbose" and attach the entire log as an attachment. Also run the script passing in the commandline param: --debug 1 and provide the entire log output as an attachment.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Sorry, I could not figure out how to attach a file, so have to add the log inline.
It is Free ESXi 4 with a registered license (not eval).
Verbose mode run, followed by debug:
============================== 05-28-2009 18:16:52 -- ghettoVCBg2 LOG START ==============================
05-28-2009 18:16:52 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
05-28-2009 18:16:52 -- CONFIG - VM_BACKUP_DATASTORE = backup
05-28-2009 18:16:52 -- CONFIG - VM_BACKUP_DIRECTORY = backups
05-28-2009 18:16:52 -- CONFIG - DISK_BACKUP_FORMAT = thin
05-28-2009 18:16:52 -- CONFIG - ADAPTER_FORMAT = lsilogic
05-28-2009 18:16:52 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
05-28-2009 18:16:52 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-28-2009 18:16:52 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-28-2009 18:16:52 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-05-28
05-28-2009 18:17:49 -- Initiate backup for "winxp" found on aaa.bbb.ccc.ddd!
05-28-2009 18:18:28 -- winxp original powerState: poweredOff
05-28-2009 18:18:28 -- winxp has 1 VMDK(s)
05-28-2009 18:18:28 -- Backing up "[nexenta_nfs] winxp/winxp.vmdk" to "[backup] backups/winxp/winxp-2009-05-28/winxp.vmdk"
05-28-2009 18:18:33 -- Backup completed for winxp/winxp.vmdk
05-28-2009 18:18:39 -- Backup completed for "winxp"!
============================== 05-28-2009 18:18:39 -- ghettoVCBg2 LOG END ==============================
============================== 05-28-2009 18:27:08 -- ghettoVCBg2 LOG START ==============================
05-28-2009 18:27:08 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
05-28-2009 18:27:08 -- CONFIG - VM_BACKUP_DATASTORE = backup
05-28-2009 18:27:08 -- CONFIG - VM_BACKUP_DIRECTORY = backups
05-28-2009 18:27:08 -- CONFIG - DISK_BACKUP_FORMAT = thin
05-28-2009 18:27:08 -- CONFIG - ADAPTER_FORMAT = lsilogic
05-28-2009 18:27:08 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
05-28-2009 18:27:08 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-28-2009 18:27:08 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
05-28-2009 18:27:08 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-05-28
05-28-2009 18:27:12 -- -
DEBUG INFO winxp -
05-28-2009 18:27:12 -- DEBUG - Host Build: VMware ESXi 4.0.0 build-164009
05-28-2009 18:27:12 -- DEBUG - Host: aaa.bbb.ccc.ddd
05-28-2009 18:27:12 -- DEBUG - Virtual Machine: winxp
05-28-2009 18:27:12 -- DEBUG - VM ConfigPath: winxp/winxp.vmx
05-28-2009 18:27:12 -- DEBUG - VMX File: winxp.vmx
05-28-2009 18:27:12 -- DEBUG - BackupConfigPath: backups/winxp/winxp-2009-05-28/winxp.vmx
05-28-2009 18:27:12 -- DEBUG - BackupPath: backups/winxp/winxp-2009-05-28
05-28-2009 18:27:12 -- DEBUG - VM Datastore: nexenta_nfs
05-28-2009 18:27:12 -- DEBUG - VMDK(s):
05-28-2009 18:27:12 -- DEBUG - winxp/winxp.vmdk
05-28-2009 18:27:12 -- -
DEBUG INFO winxp -
============================== 05-28-2009 18:27:12 -- ghettoVCBg2 LOG END ==============================
Is your license either Foundation, Standard or Enterprise?
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
It is a Free license (I just downloaded and requested the license)
Product: ESXi 4 Single Server Licensed for 1 physical CPUs (1-6 cores per CPU)
Please take a look at the FAQ section, specifically 9A for your answer.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hi William
Good work on the scripts, not had a chance to try out this new script yet.
Love the centralized feature, will save on having to setup the NFS share on each ESXi host.
just wondering if you plan on adding in a feature where we don't have to keep lists of VM's at all, basically assuming that the user will always want to backup every VM they have. I know you now only have to keep the single list but I find I have so many VM's which come and go I often forget to add them to the backup list.
Cheers
Hi William,
thanks for this great script, this is exactly what I looked for.
But I have a small problem: The cloning doesn't work
I'm on ESXi 3.5u3 with vMA 4.0
When I'm on the ESXi via ssh and do the cloning with vmkfstools -i this works.
Your script displays "Unable to clone virtual disk : destName"
This is what it says:
$ ./ghettoVCBg2.pl --config_dir backupConfigs --vmlist vmlist --debug 0
Unable to clone virtual disk : destName
This is the logfile:
============================== 06-17-2009 12:15:10 -- ghettoVCBg2 LOG START ==============================
06-17-2009 12:15:13 -- CONFIG - USING CONFIGURATION FILE = pcbae04
06-17-2009 12:15:13 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
06-17-2009 12:15:13 -- CONFIG - VM_BACKUP_DATASTORE = nas_7_backup
06-17-2009 12:15:13 -- CONFIG - VM_BACKUP_DIRECTORY = sicherung
06-17-2009 12:15:13 -- CONFIG - DISK_BACKUP_FORMAT = thin
06-17-2009 12:15:13 -- CONFIG - ADAPTER_FORMAT = buslogic;
06-17-2009 12:15:13 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = YES
06-17-2009 12:15:13 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-17-2009 12:15:13 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-17-2009 12:15:13 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-06-17
06-17-2009 12:15:13 -- Initiate backup for "pcbae04" found on 5.20.33.2!
06-17-2009 12:15:28 -- pcbae04 original powerState: poweredOff
06-17-2009 12:15:28 -- pcbae04 has 1 VMDK(s)
06-17-2009 12:15:28 -- Backing up "[datastore1] pcbae04/pcbae04.vmdk" to "[nas_7_backup] sicherung/pcbae04/pcbae04-2009-06-17/pcbae04.vmdk"
06-17-2009 12:15:32 -- Backup completed for pcbae04/pcbae04.vmdk
06-17-2009 12:15:38 -- Purging "[nas_7_backup] sicherung/pcbae04/pcbae04-2009-06-17--2" due to rotation max
06-17-2009 12:15:46 -- Backup completed for "pcbae04"!
============================== 06-17-2009 12:15:47 -- ghettoVCBg2 LOG END ==============================
And this is the backupconfig for pcbae04:
=====================================
VM_BACKUP_DATASTORE = "nas_7_backup"
VM_BACKUP_DIRECTORY = "sicherung"
VM_BACKUP_ROTATION_COUNT = "2"
DISK_BACKUP_FORMAT = "thin"
ADAPTER_FORMAT = "buslogic";
POWER_VM_DOWN_BEFORE_BACKUP = "1"
LOG_LEVEL = "verbose"
VM_SNAPSHOT_MEMORY = "0"
VM_SNAPSHOT_QUIESCE = "0"
=====================================
The only file copied to the NAS is the .vmx file.
It seems the whole backup process works, even snapshots are generated.
Just the clone command itself fails.
Do you have an idea what happens here?
thanks in advance,
Martin
Thanks for the comments.
There will be no plans on changing the script to automatically assume all VM(s) will be backed up. If you need this feature you can modify the script accordlying, it's not that hard
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I know I ran into some oddities with VIMA 1.0's version of vmkfstools.pl but since you're using vMA 4.0, there should not be any issues. I've tested vMA with ESX 3.5u3 but not with ESXi and it should not matter in either case.
Looking at the logs, everything looks okay to me and I can't see anything that could be off. Can you provide the debugging output by running the following:
./ghettoVCBg2.pl --config_dir backupConfigs --vmlist vmlist --debug 1
Also can you modify the script and see what the actual command for trying to execute the backup is, please find line 623 which should look like the following:
$vmkfstools_cpy = `$vmkfstools_cmd --server "$host" --username "$host_username" --password "$host_password" - i "$_" -a $ADAPTER_FORMAT -d $DISK_BACKUP_FORMAT "$vmdk_backup_destination" 2>&1`;
and replace it with:
$vmkfstools_cpy = `$vmkfstools_cmd --server "$host" --username "$host_username" --password "$host_password" - i "$_" -a $ADAPTER_FORMAT -d $DISK_BACKUP_FORMAT "$vmdk_backup_destination"`; print "Copy command: ", $vmkfstools_cpy,"\n";
Re-run the script as the following:
./ghettoVCBg2.pl --config_dir backupConfigs --vmlist vmlist
FYI - You don't need to specify "--debug 0" as it defaults to 0.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Mea culpa!
Got it...
After an hour of debugging and digging in my great Perl knowledge I found the problem in the backupconfig:
ADAPTER_FORMAT = "buslogic";
That small semicolon at the end of the line leads to the ADAPTER_FORMAT variable being empty
So the vmkfstools command fails because -d needs a parameter.
Everything works now, thanks for the good work.
regards,
Martin
Tried the script, great work. If anyone else is backing up to NFS datastores you can mount the same store in the VMA appliance using the following:
sudo service portmap start
sudo service nfs start
sudo chkconfig portmap on
sudo chkconfig nfs on
sudo vi /etc/fstab (Add your nfs entry here e.g.)
10.192.4.99:/vmware /backup_vmware nfs defaults 0 0
sudo mount -a
Hope this helps
Thanks for this great script and documentation.
Some small errors:
In "vm-backup-config-template" there is a semicolon at the of of ADAPTER_FORMAT which breaks the script when using config_dir
And some text in the log output is wrong QUIESCE where it should be MEMORY:
diff -f ghettoVCBg2.pl.bak ghettoVCBg2.pl
c195
print LOG timeStamp('MDYHMS'), "CONFIG - VM_SNAPSHOT_MEMORY = $quitext\n";
.
>c390
print LOG timeStamp('MDYHMS'), "CONFIG - VM_SNAPSHOT_MEMORY = $quitext\n";
.
Thanks, a new vm-backup-config-template has been uploaded.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Is there any reason you would mount the backup datastore to vMA? In general, I would recommend locking down the NFS datastore to only ESX(i) hosts that way no accidental mounting from another host is allowed.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Thanks, a new vm-backup-config-template has been uploaded.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
THANK YOU for this script. It's a real life saver. However, GhettoVCBg2 will be the deciding factor of whether or not people upgrade from ESXi 3.5u3. You could remedy that by adding thin provisioning support and a logging fix to GhettoVCB4i.
There are many people just getting in to virtualization who are not going to buy a foundation license. They will grow with the first virtualization vendor that has a solution that makes businesses sense for their current need, which includes a backup. Those who buy a foundation license, automatically receive VMware's VCB. For one or two servers, I would rather use GhettoVCB4i and a Windows machine to schedule it. People will need custom oem.tgz files with ESXi to address their needs for the foreseeable future anyway.
Please consider adding thin provisioning support and a logging fix to GhettoVCB4i, with the thin provisioning being the most important.
Thanks!
Hello lamw,
thanks a lot for your work but i got problems to run it.
Just after the snapshot, i got an error to copy virtual disk. It say: "general fault cause by file. One of the parameters supplied is invalid."
I use ESXi 3.5U3 with vMA4.0.
I tried with 2 differents VMs, with the local datastore and one NFS store and it still...
in /var/log/messages:
Jun 26 13:46:52 Hostd: Invalid desination disk format 'preallocated'
Jun 26 13:46:52 Hostd: DISKLIB-LIB : Opened "/vmfs/volumes/4a4470d9-522186bf-68b6-0016355d8d4b/w2k3oem/w2k3oem.vmdk" (flags 0x6). A821FA4
Jun 26 13:46:52 Hostd: DISKLIB-LIB : Invalid type (0)
Jun 26 13:46:52 Hostd: DISKLIB-LIB : Failed to clone : One of the parameters supplied is invalid (1).
Jun 26 13:46:52 Hostd: Failed to clone disk : One of the parameters supplied is invalid (1).
Ok, i found the solution. It was an error from DISK_BACKUP_FORMAT definition
sorry
Hi lamw,
Thanks for the script. I think it will be exactly what we are looking for but I am having some issues getting it to work. I am running it from VMA 4.0 and connecting to 2 ESXi 3.5 hosts. I have attached the log with a trial run and a debug run. Looks like it is having troubles doing a snapshot. Would really appreciate if you could take a look and give me some ideas. Thanks.
============================== 06-27-2009 22:09:00 -- ghettoVCBg2 LOG START ==============================
06-27-2009 22:09:00 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
06-27-2009 22:09:00 -- CONFIG - VM_BACKUP_DATASTORE = /vmfs/volumes/SANDisk2
06-27-2009 22:09:00 -- CONFIG - VM_BACKUP_DIRECTORY = Backups
06-27-2009 22:09:00 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
06-27-2009 22:09:00 -- CONFIG - ADAPTER_FORMAT = buslogic
06-27-2009 22:09:00 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
06-27-2009 22:09:00 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-27-2009 22:09:00 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-27-2009 22:09:00 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-06-27
06-27-2009 22:09:02 -- Initiate backup for "AntivirusServer" found on esx2.brwmsc!
06-27-2009 22:09:06 -- AntivirusServer original powerState: poweredOn
06-27-2009 22:09:06 -- AntivirusServer has 1 VMDK(s)
06-27-2009 22:09:06 -- Backing up "[SANDisk2] Antivirus Server/Antivirus Server.vmdk" to "[/vmfs/volumes/SANDisk2] Backups/AntivirusServer/AntivirusServer-2009-06-27/Antivirus Server.vmdk"
06-27-2009 22:09:07 -- Backup completed for Antivirus Server/Antivirus Server.vmdk
06-27-2009 22:09:07 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-06-27" not found
06-27-2009 22:09:08 -- Unable to purge "[/vmfs/volumes/SANDisk2] Backups/AntivirusServer/GET https:esx2.brwmscfolderBackupsAntivirusServer?dsName=vmfsvolumesSANDisk2 unsuccessful : 404 Not Found" due to rotation max
06-27-2009 22:09:08 -- Backup completed for "AntivirusServer"!
============================== 06-27-2009 22:09:08 -- ghettoVCBg2 LOG END ==============================
============================== 06-28-2009 14:30:24 -- ghettoVCBg2 LOG START ==============================
06-28-2009 14:30:24 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
06-28-2009 14:30:24 -- CONFIG - VM_BACKUP_DATASTORE = /vmfs/volumes/SANDisk2
06-28-2009 14:30:24 -- CONFIG - VM_BACKUP_DIRECTORY = Backups
06-28-2009 14:30:24 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
06-28-2009 14:30:24 -- CONFIG - ADAPTER_FORMAT = buslogic
06-28-2009 14:30:24 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
06-28-2009 14:30:24 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-28-2009 14:30:24 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-28-2009 14:30:24 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-06-28
06-28-2009 14:30:25 -- -
DEBUG INFO AntivirusServer -
06-28-2009 14:30:25 -- DEBUG - Host Build: VMware ESX Server 3i 3.5.0 build-163429
06-28-2009 14:30:25 -- DEBUG - Host: esx2.brwmsc
06-28-2009 14:30:25 -- DEBUG - Virtual Machine: AntivirusServer
06-28-2009 14:30:25 -- DEBUG - VM ConfigPath: Antivirus Server/Antivirus Server.vmx
06-28-2009 14:30:25 -- DEBUG - VMX File: Antivirus Server.vmx
06-28-2009 14:30:25 -- DEBUG - BackupConfigPath: Backups/AntivirusServer/AntivirusServer-2009-06-28/Antivirus Server.vmx
06-28-2009 14:30:25 -- DEBUG - BackupPath: Backups/AntivirusServer/AntivirusServer-2009-06-28
06-28-2009 14:30:25 -- DEBUG - VM Datastore: SANDisk2
06-28-2009 14:30:25 -- DEBUG - VMDK(s):
06-28-2009 14:30:25 -- DEBUG - Antivirus Server/Antivirus Server.vmdk
06-28-2009 14:30:25 -- -
DEBUG INFO AntivirusServer -
============================== 06-28-2009 14:30:26 -- ghettoVCBg2 LOG END ==============================
Hello thanks for this script, I am still using your older script! Trying your new script gives me some errors
$ ./ghettoVCBg2.pl --config_dir backupConfigs --vmlist vm_backup_list
Use of uninitialized value in pattern match (m//) at /usr/lib/perl5/5.8.8/VMware/VIExt.pm line 124.
Use of uninitialized value in pattern match (m//) at /usr/lib/perl5/5.8.8/VMware/VIExt.pm line 124.
Use of uninitialized value in pattern match (m//) at /usr/lib/perl5/5.8.8/VMware/VIExt.pm line 124.
Use of uninitialized value in pattern match (m//) at /usr/lib/perl5/5.8.8/VMware/VIExt.pm line 124.
Use of uninitialized value in concatenation (.) or string at /usr/bin/vifs.pl line 294.
Unable to copy BBIKOL-ZW-DC01/BBIKOL-ZW-DC01.vmx to :
destinationName
backupConfigs: 1 file named BBIKOL-ZW-DC01
##################################
VM BACKUP CONFIGURATIONS ###
##################################
VM_BACKUP_DATASTORE = "vmbackup"
VM_BACKUP_DIRECTORY = "esx"
VM_BACKUP_ROTATION_COUNT ="7"
DISK_BACKUP_FORMAT = "zeroedthick"
ADAPTER_FORMAT = "lsilogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
LOG_LEVEL = "normal"
VM_SNAPSHOT_MEMORY = "0"
VM_SNAPSHOT_QUIESCE = "0"
Ideas?
I have some problem to backup a VM in my esxi 4.0 server. I always receive this error:
/vmfs/volumes/48d3c7c5-9c6dddaa-59ff-001ec9b5cb21 # sh ghettoVCB4i.sh test
Taking backup snapshot for ANTHEAPRO ...
DiskLib_Check() failed for source disk The file specified is not a virtual disk (15).
Destination disk format: VMFS zeroedthick
Cloning disk '/vmfs/volumes/datastore1/antheapro.GruppoAnthea.local/antheapro.GruppoAnthea.local.vmdk'...
Clone: 100% done.
Removing snapshot from ANTHEAPRO ...
sh: /vmfs/volumes/VMBACKUP/ESX15/ANTHEAPRO/ANTHEAPRO-2009-06-27: bad number
ghettoVCB4i.sh: line 349: syntax error: /vmfs/volumes/VMBACKUP/ESX15/ANTHEAPRO/ANTHEAPRO-2009-06-27+1
/vmfs/volumes/48d3c7c5-9c6dddaa-59ff-001ec9b5cb21 #
I can't see anything out of the ordinary, just make sure you don't have any "semi-colon" in your configuration file and that you're abiding by the "ESX/ESXi Version Support Table" on this page.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Looks like it's complaining about a param that's not correct or unable to parse, what version of ESX(i) and VIMA/vMA are you using? I would suggest keeping the two consistent between the old VI 3.5 and vSphere 4.0
Also make sure you don't have any "semi-colon" in your configuration file
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Not sure why you're getting this issue, are you using the latest ghettoVCB4i.sh script? Is it just failing for this single VM?
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Yes I'm using the last ghettoVCB4i.sh script.
The script is used to backup 20 vms and this is the only one that has some problems.
Hi William,
I still have problems here.
I have 2 ESXi 3.5u3 with one VMA4 each.
And on both systems the script can't find the snapshot it created prior to copying the disks.
This is what I get every night on both systems:
============================== 07-03-2009 02:00:01 -- ghettoVCBg2 LOG START ==============================
07-03-2009 02:00:04 -- CONFIG - USING CONFIGURATION FILE = stobae03
07-03-2009 02:00:04 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
07-03-2009 02:00:04 -- CONFIG - VM_BACKUP_DATASTORE = nas_9_backup
07-03-2009 02:00:04 -- CONFIG - VM_BACKUP_DIRECTORY = sicherung
07-03-2009 02:00:04 -- CONFIG - DISK_BACKUP_FORMAT = thin
07-03-2009 02:00:04 -- CONFIG - ADAPTER_FORMAT = lsilogic
07-03-2009 02:00:04 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-03-2009 02:00:04 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-03-2009 02:00:04 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-03-2009 02:00:04 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-03
07-03-2009 02:00:04 -- Initiate backup for "stobae03" found on 5.20.33.17!
07-03-2009 02:00:18 -- stobae03 original powerState: poweredOn
07-03-2009 02:00:20 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-03" for stobae03
07-03-2009 02:00:20 -- stobae03 has 2 VMDK(s)
07-03-2009 02:00:20 -- Backing up "[datastore1] stobae03/stobae03.vmdk" to "[nas_9_backup] sicherung/stobae03/stobae03-2009-07-03/stobae03.vmdk"
07-03-2009 02:05:01 -- Backup completed for stobae03/stobae03.vmdk
07-03-2009 02:05:01 -- Backing up "[datastore1] stobae03/stobae03_1.vmdk" to "[nas_9_backup] sicherung/stobae03/stobae03-2009-07-03/stobae03_1.vmdk"
07-03-2009 04:52:35 -- Backup completed for stobae03/stobae03_1.vmdk
07-03-2009 04:52:35 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-03" not found
07-03-2009 04:52:42 -- Backup completed for "stobae03"!
============================== 07-03-2009 04:52:42 -- ghettoVCBg2 LOG END ==============================
The snapshot is still there and works.
I then have to shut down the machine and then delete all snapshots in the virtualcenter snapshotmanager.
"removeallsnapshots" somehow doesn't work from console.
Another interesting point is, that after errors like this one, all other VMs in my vmlist won't be backed up.
I have 5 vms in my vmlist.
The first is backed up without problems, the second might have an error like the one above and the other three will not be found later on.
If the first backup has an error the following four VMs won't be found.
The problem lies imho somewhere in the VMA or the script, I can put the last one of my VMs on top of the vmlist and it will be found and backed up.
The hostd.log on the ESXi says the following:
http://www.myangelone.de/files/hostd.log-esxi.txt (the ESXi is 2h behind in time, sry)
And has anyone seen this error before?
This pops up on my ESXi every 20min all day long.
Regardless if it's used or not.
Even if no vm is running and no Infrastructure client connected.
I get this on both ESXi, too.
Failed to send response to the client: Broken pipe
Throw vmodl.fault.RequestCanceled
(vmodl.fault.RequestCanceled) {
dynamicType = ,
msg = ""
}
Failed to send response to the client: Broken pipe
Throw vmodl.fault.RequestCanceled
(vmodl.fault.RequestCanceled) {
dynamicType = ,
msg = ""
}
Failed to send response to the client: Broken pipe
Any help is appreciated, maybe I should switch to the old script.
Is there a way to modify the script, so that it deletes all snapshots after the copy process?
Maybe the problem is the named snapshot.
Great job on the script, I've got it backing up 3 Virtual Machine guests to a Windows XP 64 bit target.
One of the VMs works perfectly as expected, but two of them fail to close the Snapshots and move on to the next virtual machine.
I've checked the names and the scripts (and the logs) and the script tries to close snapshots with exactly the same name as it opens, but for some reason fails. Any ideas?
I think this has to do with the NFS mount (OpenFiler 2.3) that both ZEROEDTHICK and EAGERZEROEDTHICKas the disk type result in 'General fault caused by file. One of the parameters supplied is invalid' in the VI client.
When using disk type 'THIN' the backup completes, but the snapshot if left open and the log says it failed to find the snap - even though it's exactly the same name of the snap it created!
Hi, angelone and William.
We have the same problem with 'angelone' with backup more than 2 vm`s (ESXi 4, vMA 4).
We try to separate backups for Unix and Windows machines. Backup for Unix (Linux and FreeBSD) machines working perfectlty (3 vm`s), but for Windows machines backup work only for one, that located on top of backup list. Log:
============================== 07-10-2009 08:57:50 -- ghettoVCBg2 LOG START ==============================
07-10-2009 08:57:50 -- CONFIG - BACKUP_LOG_OUTPUT = /var/log/ghettoVCBg2.log
07-10-2009 08:57:50 -- CONFIG - VM_BACKUP_DATASTORE = Backup
07-10-2009 08:57:50 -- CONFIG - VM_BACKUP_DIRECTORY = BACKUPS
07-10-2009 08:57:50 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
07-10-2009 08:57:50 -- CONFIG - ADAPTER_FORMAT = lsilogic
07-10-2009 08:57:50 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-10-2009 08:57:50 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-10-2009 08:57:50 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-10-2009 08:57:50 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-10
07-10-2009 08:57:54 -- Initiate backup for "maildat.xxx" found on c3000-box1-vmware2.xxx!
07-10-2009 08:58:03 -- maildat.xxx original powerState: poweredOn
07-10-2009 08:58:05 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-10" for maildat.xxx
07-10-2009 08:58:05 -- maildat.xxx has 1 VMDK(s)
07-10-2009 08:58:05 -- Backing up "[Array10-1] maildat.xxx/maildat.xxx.vmdk" to "[Backup] BACKUPS/maildat.xxx/maildat.xxx-2009-07-10/maildat.xxx.vmdk"
07-10-2009 09:13:56 -- Backup completed for maildat.xxx/maildat.xxx.vmdk
07-10-2009 09:13:59 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-07-10" for maildat.xxx
07-10-2009 09:14:30 -- Backup completed for "maildat.xxx"!
07-10-2009 09:14:33 -- ERROR - Unable to locate VM: FC.xxx
07-10-2009 09:14:33 -- ERROR - Unable to locate VM: R2D2.xxx
07-10-2009 09:14:33 -- ERROR - Unable to locate VM: ts.xxx
07-10-2009 09:14:33 -- ERROR - Unable to locate VM: n2000-bms.xxx
Sometimes backup completed successfully, but error for 'top' machine persist:
07-09-2009 12:31:29 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-09" not found
And we can delete snapshot with Snapshot Manager 'by hand'.
I can e-mail verbose logging for this situations.
Thanks for future help.
now, does anyone know why removing all snapshots with the GUI manager works and "removeallsnapshots" from CLI tells me that there are no snapshots?
Sorry for the delay of response but I've been extremely busy over the last month and I currently do not have the cycles to look into the current issues. I'm aware there are some interesting problems being faced that may all be related. I will provide some more details when I get a chance to take a look at the issue hopefully in a few weeks after my VCP4 exam. Thanks for your patience
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hm, I don't think this is the problem here since ghettoVCB doesn't set these flags.
Our problem is that the snapshots aren't found and therefore can't be deleted.
Above KB article says, that you can't delete your snapshots because the left over flag phobits it.
I've read through this thread and I am really interested in using this script to backup my VMs. I am still learning how to use ESXi 4, so this might be a repeat question.
Can I use this script on my ESXi 4 host if I am using the "free" version? I saw something about it not working if you have not paid for a license due to some CLI thingy missing. If this is the case, then what to do for backup?
Thanks
Please re-read the document/requirements, specifically the section labeled "ESX/ESXi Version Support Table".
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Correct, that has nothing to do with the issues being seen now. The KB relates to using vDR (VMware Data Recovery) where snapshots are not being removed and there has been a fix for that which is NOT related to this script at all, unless you're using vDR.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
For those that are running into issues where backups are occurring but snapshots are not being removed or other issues that are similarly related, please take a look here http://tinyurl.com/layekn
The drawing starts today and you have ~1week to enter. Thanks everyone
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Is there any way to prevent the script from backing up all vmdks attached to the VM?
I just want to backup the primary vmdk, not vmdks across multiple datastores.
Right now when I backup a VM, the primary vmdk processes fine, but when it hits the secondardy vmdk on another datastore, it gives an error that the file is locked or in use.
Thanks in advance!
No, that is not a supported feature of the script. You're more than welcome to add logic to exclude certain VMDK(s), no support will be provided for this.
The script should work if you have multiple VMDK(s) across multiple datastores. Is the script taking snapshots prior to backup? Please provide more details and take a look at FAQ #10
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Yes I am doing snapshots prior to the backup.
The second disk is configured as an Independant Disk which is why snapshots will not run, I assume.
I was hoping by using it as an Independant disk, the backup would skip it.
Bingo, Independant Disk can not be snapshotted hence the error that its locked since w/o the snapshot and I assume the VM was powered on, it would be locked for write access.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hello,
I'm trying to use the script on two similar configuration: ESXi 3.5u3 free licensied and vMA 4.0
I got always same problems.
Snapshot is not removed.
VMs turned off are not turned on
The backup rotation isn't working either.
============================== 07-29-2009 22:01:01 -- ghettoVCBg2 LOG START ==============================
07-29-2009 22:01:04 -- CONFIG - USING CONFIGURATION FILE = SRVLINUX
07-29-2009 22:01:04 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
07-29-2009 22:01:04 -- CONFIG - VM_BACKUP_DATASTORE = NFS
07-29-2009 22:01:04 -- CONFIG - VM_BACKUP_DIRECTORY = DAILY
07-29-2009 22:01:04 -- CONFIG - DISK_BACKUP_FORMAT = thin
07-29-2009 22:01:04 -- CONFIG - ADAPTER_FORMAT = buslogic
07-29-2009 22:01:04 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-29-2009 22:01:04 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-29-2009 22:01:04 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
07-29-2009 22:01:04 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-29
07-29-2009 22:01:04 -- Initiate backup for "SRVLINUX" found on 192.168.0.110!
07-29-2009 22:01:15 -- SRVLINUX original powerState: poweredOn
07-29-2009 22:01:17 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-29" for SRVLINUX
07-29-2009 22:01:17 -- SRVLINUX has 1 VMDK(s)
07-29-2009 22:01:17 -- Backing up "[datastore1] SRVLINUX/SRVLINUX_2.vmdk" to "[NFS] DAILY/SRVLINUX/SRVLINUX-2009-07-29/SRVLINUX_2.vmdk"
07-29-2009 22:33:21 -- Backup completed for SRVLINUX/SRVLINUX_2.vmdk
07-29-2009 22:33:21 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-29" not found
07-29-2009 22:33:45 -- Unable to purge "[NFS] DAILY/SRVLINUX/SRVLINUX-2009-07-21--5" due to rotation max
07-29-2009 22:33:49 -- Backup completed for "SRVLINUX"!
============================== 07-29-2009 22:33:49 -- ghettoVCBg2 LOG END ==============================
============================== 07-29-2009 22:34:00 -- ghettoVCBg2 LOG START ==============================
07-29-2009 22:34:03 -- CONFIG - USING CONFIGURATION FILE = SRVHP
07-29-2009 22:34:03 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
07-29-2009 22:34:03 -- CONFIG - VM_BACKUP_DATASTORE = NFS
07-29-2009 22:34:03 -- CONFIG - VM_BACKUP_DIRECTORY = DAILY
07-29-2009 22:34:03 -- CONFIG - DISK_BACKUP_FORMAT = thin
07-29-2009 22:34:03 -- CONFIG - ADAPTER_FORMAT = lsilogic
07-29-2009 22:34:03 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-29-2009 22:34:03 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
07-29-2009 22:34:03 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
07-29-2009 22:34:03 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-29
07-29-2009 22:34:03 -- Initiate backup for "SRVHP" found on 192.168.0.110!
07-29-2009 22:34:14 -- SRVHP original powerState: poweredOn
07-29-2009 22:36:05 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-29" for SRVHP
07-29-2009 22:36:05 -- SRVHP has 1 VMDK(s)
07-29-2009 22:36:05 -- Backing up "[datastore1] SRVHP/SRVHP.vmdk" to "[NFS] DAILY/SRVHP/SRVHP-2009-07-29/SRVHP.vmdk"
07-30-2009 01:39:17 -- Backup completed for SRVHP/SRVHP.vmdk
07-30-2009 01:39:17 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-29" not found
07-30-2009 01:39:40 -- Unable to purge "[NFS] DAILY/SRVHP/SRVHP-2009-07-21--5" due to rotation max
07-30-2009 01:39:40 -- Backup completed for "SRVHP"!
============================== 07-30-2009 01:39:40 -- ghettoVCBg2 LOG END ==============================
============================== 07-30-2009 01:39:54 -- ghettoVCBg2 LOG START ==============================
07-30-2009 01:39:57 -- CONFIG - USING CONFIGURATION FILE = SRVAPPLI
07-30-2009 01:39:57 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
07-30-2009 01:39:57 -- CONFIG - VM_BACKUP_DATASTORE = NFS
07-30-2009 01:39:57 -- CONFIG - VM_BACKUP_DIRECTORY = DAILY
07-30-2009 01:39:57 -- CONFIG - DISK_BACKUP_FORMAT = thin
07-30-2009 01:39:57 -- CONFIG - ADAPTER_FORMAT = lsilogic
07-30-2009 01:39:57 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-30-2009 01:39:57 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
07-30-2009 01:39:57 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
07-30-2009 01:39:57 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-07-30
07-30-2009 01:39:57 -- Initiate backup for "SRVAPPLI" found on 192.168.0.110!
07-30-2009 01:40:08 -- SRVAPPLI original powerState: poweredOn
07-30-2009 01:41:44 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-07-30" for SRVAPPLI
07-30-2009 01:41:44 -- SRVAPPLI has 1 VMDK(s)
07-30-2009 01:41:44 -- Backing up "[datastore1] SRVAPPLI/SRVAPPLI.vmdk" to "[NFS] DAILY/SRVAPPLI/SRVAPPLI-2009-07-30/SRVAPPLI.vmdk"
07-30-2009 05:10:32 -- Backup completed for SRVAPPLI/SRVAPPLI.vmdk
07-30-2009 05:10:32 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-07-30" not found
07-30-2009 05:10:55 -- Unable to purge "[NFS] DAILY/SRVAPPLI/SRVAPPLI-2009-07-21--5" due to rotation max
07-30-2009 05:10:59 -- Backup completed for "SRVAPPLI"!
============================== 07-30-2009 05:10:59 -- ghettoVCBg2 LOG END ==============================
Curious, have you tried to backup the 3 VMs but instead of using the configuration file just a flat file containing the VMs that need to be backed up. I know few others have ran into similar situations and so far everyone has been using the configurations to execute their backups. Just wondering if there is a bug in the configuration backup method or if this is a general issue.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I made a test on saturday on a similar architecture.
I launched manually the script for each VM (so I modified each time vmlist file) without configuration file.I didn't try the backup rotation and power off VM.
All is ok for the first VM but for the two others, snapshot still not found.
It's really strange because, like i told you, i launched each backup process independently.
============================== 08-01-2009 16:19:48 -- ghettoVCBg2 LOG START ==============================
08-01-2009 16:19:48 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-01-2009 16:19:48 -- CONFIG - VM_BACKUP_DATASTORE = NFS
08-01-2009 16:19:48 -- CONFIG - VM_BACKUP_DIRECTORY = WEEKLY
08-01-2009 16:19:48 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-01-2009 16:19:48 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-01-2009 16:19:48 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-01-2009 16:19:48 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 16:19:48 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 16:19:48 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-01
08-01-2009 16:19:51 -- Initiate backup for "SRVAD" found on 192.128.10.250!
08-01-2009 16:20:01 -- SRVAD original powerState: poweredOn
08-01-2009 16:28:24 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-01" for SRVAD
08-01-2009 16:28:24 -- SRVAD has 1 VMDK(s)
08-01-2009 16:28:24 -- Backing up "[datastore1] SRVAD/SRVAD.vmdk" to "[NFS] WEEKLY/SRVAD/SRVAD-2009-08-01/SRVAD.vmdk"
08-01-2009 16:55:08 -- Backup completed for SRVAD/SRVAD.vmdk
08-01-2009 16:55:47 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-08-01" for SRVAD
08-01-2009 16:55:53 -- Backup completed for "SRVAD"!
============================== 08-01-2009 16:55:53 -- ghettoVCBg2 LOG END ==============================
============================== 08-01-2009 17:08:40 -- ghettoVCBg2 LOG START ==============================
08-01-2009 17:08:40 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-01-2009 17:08:40 -- CONFIG - VM_BACKUP_DATASTORE = NFS
08-01-2009 17:08:40 -- CONFIG - VM_BACKUP_DIRECTORY = WEEKLY
08-01-2009 17:08:40 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-01-2009 17:08:40 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-01-2009 17:08:40 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-01-2009 17:08:40 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 17:08:40 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 17:08:40 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-01
08-01-2009 17:08:43 -- Initiate backup for "SRVEXCHANGE" found on 192.128.10.250!
08-01-2009 17:08:52 -- SRVEXCHANGE original powerState: poweredOn
08-01-2009 17:11:37 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-01" for SRVEXCHANGE
08-01-2009 17:11:37 -- SRVEXCHANGE has 1 VMDK(s)
08-01-2009 17:11:37 -- Backing up "[datastore1] SRVEXCHANGE/SRVEXCHANGE.vmdk" to "[NFS] WEEKLY/SRVEXCHANGE/SRVEXCHANGE-2009-08-01/SRVEXCHANGE.vmdk"
08-01-2009 19:15:12 -- Backup completed for SRVEXCHANGE/SRVEXCHANGE.vmdk
08-01-2009 19:15:13 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-01" not found
08-01-2009 19:15:18 -- Backup completed for "SRVEXCHANGE"!
============================== 08-01-2009 19:15:18 -- ghettoVCBg2 LOG END ==============================
============================== 08-01-2009 19:20:59 -- ghettoVCBg2 LOG START ==============================
08-01-2009 19:20:59 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-01-2009 19:20:59 -- CONFIG - VM_BACKUP_DATASTORE = NFS
08-01-2009 19:20:59 -- CONFIG - VM_BACKUP_DIRECTORY = WEEKLY
08-01-2009 19:20:59 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-01-2009 19:20:59 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-01-2009 19:20:59 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-01-2009 19:20:59 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 19:20:59 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-01-2009 19:20:59 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-01
08-01-2009 19:21:02 -- Initiate backup for "SRVTSE" found on 192.128.10.250!
08-01-2009 19:21:11 -- SRVTSE original powerState: poweredOn
08-01-2009 19:45:45 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-01" for SRVTSE
08-01-2009 19:45:45 -- SRVTSE has 2 VMDK(s)
08-01-2009 19:45:45 -- Backing up "[datastore1] SRVTSE/SRVTSE.vmdk" to "[NFS] WEEKLY/SRVTSE/SRVTSE-2009-08-01/SRVTSE.vmdk"
08-01-2009 20:53:10 -- Backup completed for SRVTSE/SRVTSE.vmdk
08-01-2009 20:53:10 -- Backing up "[datastore1] SRVTSE/SRVTSE_1.vmdk" to "[NFS] WEEKLY/SRVTSE/SRVTSE-2009-08-01/SRVTSE_1.vmdk"
08-01-2009 22:48:32 -- Backup completed for SRVTSE/SRVTSE_1.vmdk
08-01-2009 22:48:32 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-01" not found
08-01-2009 22:48:38 -- Backup completed for "SRVTSE"!
============================== 08-01-2009 22:48:38 -- ghettoVCBg2 LOG END ==============================
Can you do the same test but with all 3 VMs on 1 backup vmlist and post your results.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hello,
i did the same test with 3 VMs on 1 backup vmlist. I post the result.
============================== 08-08-2009 11:28:24 -- ghettoVCBg2 LOG START ==============================
08-08-2009 11:28:24 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-08-2009 11:28:24 -- CONFIG - VM_BACKUP_DATASTORE = NFS
08-08-2009 11:28:24 -- CONFIG - VM_BACKUP_DIRECTORY = WEEKLY
08-08-2009 11:28:24 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-08-2009 11:28:24 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-08-2009 11:28:24 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-08-2009 11:28:24 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-08-2009 11:28:24 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-08-2009 11:28:24 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-08
08-08-2009 11:28:28 -- Initiate backup for "SRVAD" found on 192.128.10.250!
08-08-2009 11:28:38 -- SRVAD original powerState: poweredOn
08-08-2009 11:36:17 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-08" for SRVAD
08-08-2009 11:36:17 -- SRVAD has 1 VMDK(s)
08-08-2009 11:36:17 -- Backing up "[datastore1] SRVAD/SRVAD.vmdk" to "[NFS] WEEKLY/SRVAD/SRVAD-2009-08-08/SRVAD.vmdk"
08-08-2009 12:04:06 -- Backup completed for SRVAD/SRVAD.vmdk
08-08-2009 12:04:41 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-08-08" for SRVAD
08-08-2009 12:04:50 -- Backup completed for "SRVAD"!
08-08-2009 12:04:50 -- Initiate backup for "SRVTSE" found on 192.128.10.250!
08-08-2009 12:05:00 -- SRVTSE original powerState: poweredOn
08-08-2009 12:13:45 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-08" for SRVTSE
08-08-2009 12:13:45 -- SRVTSE has 2 VMDK(s)
08-08-2009 12:13:45 -- Backing up "[datastore1] SRVTSE/SRVTSE.vmdk" to "[NFS] WEEKLY/SRVTSE/SRVTSE-2009-08-08/SRVTSE.vmdk"
08-08-2009 12:55:26 -- Backup completed for SRVTSE/SRVTSE.vmdk
08-08-2009 12:55:26 -- Backing up "[datastore1] SRVTSE/SRVTSE_1.vmdk" to "[NFS] WEEKLY/SRVTSE/SRVTSE-2009-08-08/SRVTSE_1.vmdk"
08-08-2009 14:39:29 -- Backup completed for SRVTSE/SRVTSE_1.vmdk
08-08-2009 14:39:30 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-08" not found
08-08-2009 14:39:39 -- Backup completed for "SRVTSE"!
08-08-2009 14:39:39 -- ERROR - Unable to locate VM: SRVEXCHANGE
============================== 08-08-2009 14:39:39 -- ghettoVCBg2 LOG END ==============================
1st: OK.
2nd: Snapshot not removed.
3rd: VM not found.