This tool is the follow up release of the ghettoVCB
backup utility which allows users to perform backups of virtual machines residing on ESX(i) 3.5+/4.x+
servers using methodology similar to VMware's VCB
tool. By incorporating highly constructive feedback from the VMware community and utilizing the existing VI API, ghettoVCB’s framework was completely rewritten to be harder, better, faster, stronger
.
The primary motivation for ghettoVCBg2 was to provide ESXi users with access to the utility without relying on unlocking and utilizing the unsupported console. To satisfy this requirement, the rebuilt framework takes advantage of the
VI Perl Toolkit/vSphere SDK for Perl which is present in the
VIMA/vMA virtual appliance provided by VMware. As a result, ghettoVCBg2 provides a more proper backup solution that administrators can utilize in their virtual infrastructure.
As it stands, it is a requirement that ghettoVCB be invoked directly on each of the ESX servers hosting virtual machines in need of a backup. By taking advantage of VIMA/vMA, the entire backup process becomes centralized in turn eliminating the abovementioned requirement. Additionally, by leveraging the vi-fastpass library, unattended authentication between VIMA/vMA and target ESX servers is made possible. This alleviates the need to continually pass in credentials for each ESX server.
By centralizing the backup process, individual virtual machine backup lists pertaining to their respective ESX(i) servers are no longer required. ghettoVCBg2 has the ability to identify virtual machines and their respective hosts, potentially eliminating any past confusion associated with maintaining multiple lists.
In its current configuration, the script will allow up to 3 unique backups of the Virtual Machine before it will overwrite the previous backups; this however, can be modified to fit procedures if need be. Please be diligent in running the script in a test or staging environment before using it on production live Virtual Machines; this script functions well within our environment but there is a chance that it may not fit well into other environments.
VMs running on ESX(i) 3.5u2+ or ESX(i) 4.x+
VMware VIMA 1.0 or VMware vMA 4.x
Version | Support | Alternative |
ESX 3.5u2 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 3.5u3 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 3.5u4 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESXi 3.5u2 | Works for free | ghettoVCB\ |
ESXi 3.5u3 | Works for free | ghettoVCB\ |
ESXi 3.5u4 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 4.0 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESXi 4.0 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 4.1 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESXi 4.1 | Requires at least Foundation/Essentials license | ghettoVCB\ |
1) Download ghettoVCBg2.tar.gz
to your VIMA 1.0/vMA 4.x system
2) Extract the contents of the tarball
[vi-admin@scofield ~]$ tar -zxvf ghettoVCBg2.tar.gz
ghettoVCBg2/
ghettoVCBg2/ghettoVCBg2-vm_backup_configuration_template
ghettoVCBg2/ghettoVCBg2.pl
3) The script should be in a directory called
ghettoVCBg2
Prior to starting, ensure that the VMware VIMA/vMA host is managing the appropriate ESX(i) 3.5u2/4.x+ host(s)
1. Add ESXi hosts to VIMA/vMA managment using vifp
A password prompt for the vi-admin account and root password to the pertinent ESXi host(s) will be presented.
[vi-admin@vima-primp-industries ~]$ sudo vifp addserver olga.resnet.ucsb.edu
root@olga.resnet.ucsb.edu's password:
[vi-admin@vima-primp-industries ~]$
2. Verify that the appropriate hosts are being managed by VIMA/vMA
[vi-admin@vima-primp-industries ~]$ sudo vifp listservers
olga.resnet.ucsb.edu
himalaya.primp-industries.com
The following variables (similar to that of ghettoVCB) need to be defined within the script prior to execution.
Defining the datastore in which the backups are stored:
Defining the name of folder that will contain backups (if folder does not exist, it will automatically be created):
Defining the backup rotation per VM:
Defining the backup disk format (zeroedthick, eagerzeroedthick, thin, and 2gbsparse are available):
Defining the adapter type for backed up VMDK (buslogic, and lsilogic are available):
Defining whether the VM is powered down or not prior to backup (1 = enable, 0 = disable):
Note: VM(s) that are powered off will not require snapshoting
Defining whether virtual machine memory is snapped and if quiescing is enabled (1 = enable, 0 = disable):
Note: By default both are disabled
Defining the detail of logging (verbose or normal):
*Defining VMDK(s) to backup from a particular VM either a list of vmdks or "all"
VM_VMDK_FILES="myvmdk.vmdk"
Experimental Support Configs:
Note: Port 25 by default
my $SEND_MAIL = "no";
my $EMAIL_HOST = "emailserver";
my $EMAIL_DOMAIN = "localhost.localdomain";
my $EMAIL_TO = 'William Lam <william@primp-industries.com.com>';
my $EMAIL_FROM = 'ghettoVCBg2 <ghettoVCBg2@primp-industries.com.com>';
Ensure that you do not edit past this section:
########################## DO NOT MODIFY PAST THIS LINE ##########################
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl
Required command option 'vmlist' not specified.
Synopsis: ./ghettoVCBg2.pl OPTIONS
Command-specific options:
--config_dir
Name of directory containing VM(s) backup configurations
--dryrun (default '0')
Set to 1 to enable dryrun mode (default 0)
--output (default '/tmp/ghettoVCBg2.log')
Full path to output log (default /tmp/ghettoVCBg2.log)
--vmlist (required)
A file containing a list of virtual machine(s) to be backed up on host
Common VI options:
--config (variable VI_CONFIG)
Location of the VI Perl configuration file
--credstore (variable VI_CREDSTORE)
Name of the credential store file defaults to <HOME>/.vmware/credstore/vicredentials.xml on Linux and <APPDATA>/VMware/credstore/vicredentials.xml on Windows
--encoding (variable VI_ENCODING, default 'utf8')
Encoding: utf8, cp936 (Simplified Chinese), iso-8859-1 (German), shiftjis (Japanese)
--help
Display usage information for the script
--passthroughauth (variable VI_PASSTHROUGHAUTH)
Attempt to use pass-through authentication
--passthroughauthpackage (variable VI_PASSTHROUGHAUTHPACKAGE, default 'Negotiate')
Pass-through authentication negotiation package
--password (variable VI_PASSWORD)
Password
--portnumber (variable VI_PORTNUMBER)
Port used to connect to server
--protocol (variable VI_PROTOCOL, default 'https')
Protocol used to connect to server
--savesessionfile (variable VI_SAVESESSIONFILE)
File to save session ID/cookie to utilize
--server (variable VI_SERVER, default 'localhost')
VI server to connect to. Required if url is not present
--servicepath (variable VI_SERVICEPATH, default '/sdk/webService')
Service path used to connect to server
--sessionfile (variable VI_SESSIONFILE)
File containing session ID/cookie to utilize
--url (variable VI_URL)
VI SDK URL to connect to. Required if server is not present
--username (variable VI_USERNAME)
Username
--verbose (variable VI_VERBOSE)
Display additional debugging information
--version
Display version information for the script
Note: There are some additional command line arguments that can be provided upon runtime: --debug, --output and --vmlist. --vmlist is the only parameter required. The rest will have defaults if not explicitly defined.
--dryrun
--output
--config_dir
The input to this script is a file that contains the display name of the virtual machine(s) separated by a newline. When creating this file on a non-Linux/UNIX system, you may introduce a ^M character which may cause the script to misbehave. To ensure that this does not occur, please create the file on the ESX/ESXi host.
Provided is a sample of what the file should contain:
[vi-admin@scofield ~]$ cat virtual_machine_backup_list
VIMA
vMA-2
DISK_FORMAT | 2gbsparse |
ADAPTER_FORMAT | buslogic |
LOG_LEVEL | info |
VM_SNAPSHOT_MEMORY | 0 |
VM_SNAPSHOT_QUIESCE | 0 |
VMS_TO_BACKUP | (VIMA,vMA-2) |
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --vmlist virtual_machine_backup_list
[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
11-13-2009 23:34:53 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-13-2009 23:34:53 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-13-2009 23:34:53 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-13-2009 23:34:53 -- info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
11-13-2009 23:34:53 -- info: CONFIG - DISK_BACKUP_FORMAT = 2gbsparse
11-13-2009 23:34:53 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-13-2009 23:34:53 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-13-2009 23:34:53 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-13-2009 23:34:53 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-13-2009 23:34:53 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-13
11-13-2009 23:34:53 -- info: CONFIG - VM_VMDK_FILES = all
11-13-2009 23:34:58 -- info: Initiate backup for VIMA found on himalaya.primp-industries.com
11-13-2009 23:35:18 -- info: VIMA has 1 VMDK(s)
11-13-2009 23:37:37 -- info: Backup completed for VIMA!
11-13-2009 23:37:37 -- info: Initiate backup for vMA-2 found on himalaya.primp-industries.com
11-13-2009 23:37:47 -- info: vMA-2 has 1 VMDK(s)
11-13-2009 23:40:07 -- info: Backup completed for vMA-2!
11-13-2009 23:40:10 -- info: ============================== ghettoVCBg2 LOG END ==============================
VIMA
VM_BACKUP_DATASTORE | dlgCore-NFS-bigboi.VM-Backups |
VM_BACKUP_DIRECTORY | WILLIAM_BACKUPS |
DISK_FORMAT | zeroedthick |
ADAPTER_FORMAT | buslogic |
LOG_LEVEL | debug |
VM_SNAPSHOT_MEMORY | 1 |
VM_SNAPSHOT_QUIESCE | 1 |
VM_VMDK_FILES | VIMA_1.vmdk |
vMA-2
VM_BACKUP_DATASTORE | dlgCore-NFS-bigboi.VM-Backups |
VM_BACKUP_DIRECTORY | vGHETTO_BACKUPS |
DISK_FORMAT | thin |
ADAPTER_FORMAT | lsilogic |
LOG_LEVEL | info |
VM_SNAPSHOT_MEMORY | 0 |
VM_SNAPSHOT_QUIESCE | 0 |
VM_VMDK_FILES | all |
1. Create folder to hold individual VM backup policies (can be named anything):
[vi-admin@scofield ~]$ mkdir backup_config
2. Create individual VM backup policies for each VM and ensure each file is named exactly as the display name of the VM being backed up (use provided template to create duplicates):
[vi-admin@scofield backup_config]$ cp ghettoVCBg2-vm_backup_configuration_template VIMA
[vi-admin@scofield backup_config]$ cp ghettoVCBg2-vm_backup_configuration_template vMA-2
4. Using a preferred editor (nano or vi), edit each VM backup policy file
Listing of VM backup policy within backup configuration directory
[vi-admin@scofield backup_config]$ ls
ghettoVCBg2-vm_backup_configuration_template VIMA vMA-2
5. Pass in the value, backup_config, into the --config_dir parameter
(e.g.)
--config_dir backupConfigs
Example of two distinct VM backup policies will be applied to VIMA and vMA-2
Backup policy for "VIMA" (backup only 1 specific VMDKs)
[vi-admin@scofield backup_config]$ cat VIMA
VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups"
VM_BACKUP_DIRECTORY = "WILLIAM_BACKUPS"
VM_BACKUP_ROTATION_COUNT = "3"
DISK_BACKUP_FORMAT = "zeroedthick"
ADAPTER_FORMAT = "buslogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
VM_SNAPSHOT_MEMORY = "1"
VM_SNAPSHOT_QUIESCE = "1"
LOG_LEVEL = "debug"
VM_VMDK_FILES = "VIMA_1.vmdk"
Backup policy for VM "vMA-2" (backup all VMDKs found)
VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups"
VM_BACKUP_DIRECTORY = "vGHETTO_BACKUPS"
VM_BACKUP_ROTATION_COUNT = "3"
DISK_BACKUP_FORMAT = "thin"
ADAPTER_FORMAT = "lsilogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
VM_SNAPSHOT_MEMORY = "0"
VM_SNAPSHOT_QUIESCE = "0"
LOG_LEVEL = "info"
VM_VMDK_FILES = "all"
The following is an output of the script when utilizing the new --config_dir parameter with debug mode enabled.
Note: --vmlist is still a required parameter with the exception that backup policy files must exist for each VM in the list else the VM will be ignored in the backup process.
Execution of backup
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --config_dir backup_config --vmlist virtual_machine_backup_list
[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
11-14-2009 00:18:41 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-14-2009 00:18:45 -- info: CONFIG - USING CONFIGURATION FILE = VIMA
11-14-2009 00:18:45 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:18:45 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:18:45 -- info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
11-14-2009 00:18:45 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
11-14-2009 00:18:45 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-14-2009 00:18:45 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:18:45 -- info: CONFIG - VM_SNAPSHOT_MEMORY = YES
11-14-2009 00:18:45 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
11-14-2009 00:18:45 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:18:45 -- info: CONFIG - VM_VMDK_FILES = VIMA_1.vmdk
11-14-2009 00:18:45 -- info: Initiate backup for VIMA found on himalaya.primp-industries.com
11-14-2009 00:18:55 -- debug: VIMA original powerState: poweredOn
11-14-2009 00:18:55 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2009-11-14" for VIMA
11-14-2009 00:19:45 -- info: VIMA has 2 VMDK(s)
11-14-2009 00:19:45 -- debug: findVMDKFile: Found VMDK File: VIMA_1.vmdk
11-14-2009 00:19:45 -- debug: backupVMDK: Backing up "[http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk" to "[http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14/VIMA_1.vmdk"
11-14-2009 00:19:45 -- debug: backupVMDK: Signal copyThread to start
11-14-2009 00:19:45 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
11-14-2009 00:19:45 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
11-14-2009 00:19:45 -- debug: CopyThread: Start backing up VMDK(s) ...
11-14-2009 00:19:52 -- debug: copyTask: send copySuccess message ...
11-14-2009 00:19:52 -- debug: copyTask: waiting for next job and sleep ...
11-14-2009 00:20:46 -- debug: backupVMDK: Successfully completed backup for [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk Elapsed time: 1 min
11-14-2009 00:20:49 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2009-11-14" for VIMA
11-14-2009 00:20:51 -- debug: checkVMBackupRotation: Starting ...
11-14-2009 00:20:57 -- debug: Purging [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-13--3 due to rotation max
11-14-2009 00:21:05 -- info: Backup completed for VIMA!
11-14-2009 00:21:05 -- debug: reConfigureBackupParams: VM - vMA-2
11-14-2009 00:21:05 -- info: CONFIG - USING CONFIGURATION FILE = vMA-2
11-14-2009 00:21:05 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:21:05 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:21:05 -- info: CONFIG - VM_BACKUP_DIRECTORY = vGHETTO_BACKUPS
11-14-2009 00:21:05 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
11-14-2009 00:21:05 -- info: CONFIG - ADAPTER_FORMAT = lsilogic
11-14-2009 00:21:05 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:21:05 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-14-2009 00:21:05 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-14-2009 00:21:05 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:21:05 -- info: CONFIG - VM_VMDK_FILES = all
11-14-2009 00:21:05 -- info: Initiate backup for vMA-2 found on himalaya.primp-industries.com
11-14-2009 00:21:15 -- info: vMA-2 has 1 VMDK(s)
11-14-2009 00:23:22 -- info: Backup completed for vMA-2!
11-14-2009 00:23:25 -- info: ============================== ghettoVCBg2 LOG END ==============================
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --config_dir backup_config --vmlist virtual_machine_backup_list --dryrun 1
[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
11-14-2009 00:26:29 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-14-2009 00:26:34 -- info: CONFIG - USING CONFIGURATION FILE = VIMA
11-14-2009 00:26:34 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
11-14-2009 00:26:34 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
11-14-2009 00:26:34 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-14-2009 00:26:34 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_MEMORY = YES
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:26:34 -- info: CONFIG - VM_VMDK_FILES = VIMA_1.vmdk
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO VIMA ----------
11-14-2009 00:26:34 -- info: DEBUG - Host Build: VMware ESX 4.0.0 build-164009
11-14-2009 00:26:34 -- info: DEBUG - Host: himalaya.primp-industries.com
11-14-2009 00:26:34 -- info: DEBUG - Virtual Machine: VIMA
11-14-2009 00:26:34 -- info: DEBUG - VM ConfigPath: [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA.vmx
11-14-2009 00:26:34 -- info: DEBUG - VMX File: VIMA.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupConfigPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14/VIMA.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14
11-14-2009 00:26:34 -- info: DEBUG - VM Datastore: himalaya-local-SAS.Savvio
11-14-2009 00:26:34 -- info: DEBUG - VMDK(s):
11-14-2009 00:26:34 -- info: DEBUG - [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA.vmdk
11-14-2009 00:26:34 -- info: DEBUG - [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO VIMA ----------
11-14-2009 00:26:34 -- debug: reConfigureBackupParams: VM - vMA-2
11-14-2009 00:26:34 -- info: CONFIG - USING CONFIGURATION FILE = vMA-2
11-14-2009 00:26:34 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIRECTORY = vGHETTO_BACKUPS
11-14-2009 00:26:34 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
11-14-2009 00:26:34 -- info: CONFIG - ADAPTER_FORMAT = lsilogic
11-14-2009 00:26:34 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:26:34 -- info: CONFIG - VM_VMDK_FILES = all
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO vMA-2 ----------
11-14-2009 00:26:34 -- info: DEBUG - Host Build: VMware ESX 4.0.0 build-164009
11-14-2009 00:26:34 -- info: DEBUG - Host: himalaya.primp-industries.com
11-14-2009 00:26:34 -- info: DEBUG - Virtual Machine: vMA-2
11-14-2009 00:26:34 -- info: DEBUG - VM ConfigPath: [http://himalaya-local-SAS.Constellation|http://himalaya-local-SAS.Constellation] vMA-2/vMA-2.vmx
11-14-2009 00:26:34 -- info: DEBUG - VMX File: vMA-2.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupConfigPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] vGHETTO_BACKUPS/vMA-2/vMA-2-2009-11-14/vMA-2.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] vGHETTO_BACKUPS/vMA-2/vMA-2-2009-11-14
11-14-2009 00:26:34 -- info: DEBUG - VM Datastore: himalaya-local-SAS.Constellation
11-14-2009 00:26:34 -- info: DEBUG - VMDK(s):
11-14-2009 00:26:34 -- info: DEBUG - [http://himalaya-local-SAS.Constellation|http://himalaya-local-SAS.Constellation] vMA-2/vMA-2.vmdk
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO vMA-2 ----------
11-14-2009 00:26:37 -- info: ============================== ghettoVCBg2 LOG END ==============================
0Q: I'm having some issues or would like to provide feedback
0A: Please join the ghettoVCBg2 Group to post your comments/questions
1Q: I'm using ESXi and free licensed version ( I did not pay for anyting ), will this script still work?
1A: No, you need to have a licensed version of ESXi to use this script.
2Q: I have a VM listed in my backup file but no backup of that VM took place. The log stated that the VM could not be found. How could this be? I know that it’s running on host X…
2A: Ensure the ESX/ESXi host is being managed by VMware VIMA/vMA (sudo vifp listservers to verify)
3Q: Can I schedule backups to take place hourly, daily, monthly, yearly?
3A: Yes, do a search online for crontab.
4Q: I want to schedule my backup on Windows, how do I do this?
4A: Do a search for
plink. Make sure you have paired SSH keys setup between your Windows system and VIMA/vMA.
5Q: Do I have to use VMware VIMA/vMA to initiate the backup?
5A: Yes, vi-fastpass is being used and only VIMA/vMA provides this capability. However, you can kickoff the process from another host (*nix/win)
4Q: I only have a single ESXi host. I want to take backups and store them somewhere else. The problem is: I don't have NFS, iSCSI nor FC SAN. What can I do?
4A: You can use local storage to store your backups assuming that you have enough space on the destination datastore. Afterwards, you can use scp (WinSCP/FastSCP) or vifs (vCLI) to transfer the backups from the ESXi host to your local desktop.
5Q: I’m pissed; the backup is taking too long. My datastore is of type X?
5A: YMMV, take a look at your storage configuration and make sure it is optimized.
6Q: I noticed that the backup rotation is occurring after a backup. I don't have enough local storage space, can the process be changed?
6A: This is primarily done to ensure that you have at least one good backup in case the new backup fails. If you would like to modify the script, you're more than welcome to do so.
7Q: What is the best storage configuration for datastore type X?
7A: Search the VMTN forums; there are various configurations for the different type of storage/etc.
8Q: I want to setup NFS Server to run my backups. Which is the best and should it be virtual or physical?7
8A: Please refer to answer 7A. From experience, we’ve seen physical instances of NFS servers to be faster than their virtual counterparts. As always, YMMV.
9Q: When I try to use the script I get an error stating: "restricted version". What the heck is this?
9A: Please take a look at the ESX/ESXi Version Support Table in this document. The reason is explained here.
10Q: I'm getting error X when using the script or I'm not getting any errors, the backup didn’t even take place. Oh what could be wrong?
10A: First, you'll want to check the log (default /tmp/ghettoVCBg2) and see if there are any obvious errors. If not, please provide the entire output of the log, a short description of your environment and the type of VM(s) you're trying to backup—I will try to provide some assistance.
11Q: I have VMs that have snapshots. I want to back these things up but the script doesn’t let me do it. How do I fix that?
11A: VM snapshots are not meant to be kept for long durations. When backing up a VM that contains a snapshot, you should ensure all snapshots have been committed prior to running a backup. No exceptions will be made…ever.
12Q: I would like to restore from backup, what is the best method?
12A: The restore process will be unique for each environment and should be determined by your backup/recovery plans. At a high level you have the option of mounting the backup datastore and registering the VM in question or copy the VM from the backup datastore to the ESX/ESXi host. The latter is recommended so that you're not running a VM living on the backup datastore or inadvertently modifying your backup VM(s).
13Q: When using the experimental feature: Individual VM backup policy, if I don't have a policy will it use the default configurations defined in the script?
13A: No, this use case will not be supported, ever. If you decide to use the per VM policy, then the global configuration in the script is ignored. If a VM policy config is not found, the VM will not be backed up even if it's in the –-vmlist.
14Q: In vMA 4.0, vCenter target will be supported, can I just add vCenter target and assume this will work?
14A: At the moment, this is not a supported feature. You will need to add each ESX/ESXi host for the vi-fastpass library to be able to locate the host(s).
15Q: When I try to run the script I get: "-bash: ./ghettoVCBg2.pl: Permission denied", what is wrong?
15A: You need to change the permission on the script to be executable, chmod +x ghettoVCBg2.pl
16Q: Where can I download the latest version of the script?
16A: The latest version is available on this page at the bottom. To get the version of the script run
./ghettoVCBg2.pl --version
17Q: I'm running into an issue when I backup my VMs using NFS on Linux, I'm able to either get 1 successful backup and all subsequent backups fail OR I'm having issues where snapshots are not being removed upon completition and future backups fail.
17A: This problem/issue has been reported by quite a few users in the community and only affects a very small percentage of users overall under a specific configuration. The problem as I understand it today which is still an on-going investigation by myself and the community is as follows:
Setup: NFS on Linux where the underlying volume is EXT3 filesystem
and backups are failing primarily around larger sized VMs ( > 150GB).
The issue is during the final phase of a VM being backed-up in the rotation function where a request is sent out to the NFS server to either A) rotate/rename a directory B) delete an old backup. There is either a disconnect or delay that severely affects the backup process that may lead to a whole slew of "unexpected" behavior from having only 1 backup complete successful and all subsquent backups failing/etc.
First off, there is nothing wrong with the script from what we've found it and has to do with NFS server and specifically the volume that the backup datastore is hosted on. Thanks to community user
ralfoertner who had a simliar problem using
ghettoVCBg2 using Debian with EXT3 base filesystem for his NFS volume. He decided to do some additional troubleshooting to further isolate the problem and actually found by changing the underlying filesystem from EXT3 to XFS that the issue went away and resolve all issues he had ran into regarding an NFS on Linux backup solution.
Now, we're still 100% sure why the above solution has resolved the problem or what the actual problem is but I'll explain the current theory:
EXT3 is a journaled filesystem, you can do a search online for more information. There's a few advantages with EXT3 from it's predcessor EXT2, but one difference is in the approach it takes with journaling as explained here in greater detail:
http://www.ibm.com/developerworks/library/l-fs7.html
Basically EXT2 does only "metadata journling" whereas EXT3 does both data and metadata journaling, this would ensure that all changes to the filesystem where it's data or metadata is consistent in the event of an unexpected system crash. With EXT3, there are 3 journaling options: (
data=writeback mode,data=ordered mode,data=journal mode ) and by default it uses
data=ordered mode to ensure the consistency as described from above.
One easy method to check what mount options are configured, on a Linux system you can check /proc/mounts:
[vi-admin@scofield ~]$ cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw,data=ordered 0 0
/dev /dev tmpfs rw 0 0
/proc /proc proc rw 0 0
/sys /sys sysfs rw 0 0
/proc/bus/usb /proc/bus/usb usbfs rw 0 0
devpts /dev/pts devpts rw 0 0
/dev/VolGroup00/var /var/log ext3 rw,data=ordered 0 0
/dev/sda1 /boot ext3 rw,data=ordered 0 0
tmpfs /dev/shm tmpfs rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
So now what does this have to do with XFS and EXT3? With the defaults of EXT3, it may not always give you the best performance due to the extra overhead of the way it does its journaling, if you were to change the options to
data=writeback mode, you'll notice that this is simliar journaling method found in
XFS,JFS and ResierFS as described here:
http://www.ibm.com/developerworks/library/l-fs8.html
I believe that there are some additional overheads that maybe caused by using NFS on EXT3, which may explain the issue that users are seeing and that moving from EXT3 to XFS has "seemed" to solve the issues for those who have implemented the change. Though, before you go out and make this change, you need to understand your environment and what this change would exactly mean, please refer to differences in EXT2/XFS and EXT3, especially around a power interruption or system crash.
In conclusion, I'm not recommending one way or the other, it really depends on your environment. Though this has shown to resolve the issue as users have implemented to get around this problem, but it's definitely worth while to really understand the problem in your environment and figure out why and decide on the best solution. I would also recommend users perform their own tests to further isolate the problem in their environment and try local storage, using iSCSI to see if they can pin-point the problem.
If you have found this script or other scripts to be useful and would like to contribute back, please click
here to donate! Thank You
07/20/10 - build 4.3
Enhancements:
Bug Fixes:
03/27/10 - build 4.2
Enhancements:
03/27/10 - build 4.1
Enhancements:
02/24/09 - build 4.1
Bug Fixes:
05/26/09 - build 2.5.1
Enhancements:
Bug Fixes:
04/25/09 - build 2.5
Enhancements:
Bug Fixes:
02/21/09 - build 2.0
Initial release
Could this be a time stamp issue?
I was backing up my domain controller, and it usually only takes 2-3 minutes with no issues. I was testing a new NFS share and it took almost an hour, and had the error that it could not find the snap shot. I usually only have snap shot issues with my larger VM's that take at least an hour or more to backup.
I'm just lending my voice to state that I have experienced the same phenomenon as Al3xandr3. I haven't yet tried to determine if the order of running individual backups affects the result or if it's a specific fault of one of the VM's. I have experienced it with multiple VMs in one list and also individually listed VMs with manual starts. With all of my VM's in a single list, in order:
(1st) Win2k TS (not the assigned names, there are no spaces): OK
(2nd) Win2k3 App: Snapshot not removed. Once I found that the snapshot not only wasn't removed but also didn't show up in the VI Client to be deleted and required manual creation of a snapshot to then show 2 snapshots and deleting them then.
(etc) The rest: Backup doesn't begin
When I find the logs I'll review and post them up, hopefully correlating the same errors.
Environment is vSphere Essentials (1 ESX, 1 vCentre VM, vMA4. My VMs ranged from 15GB to 45GB at the time and backup time was not excessive. Brand new setup, overprovisioned atm and plenty of resources and speed. No activity during the time of backups which are stored on an iSCSI NAS.
Are there any plans to add compression to this version of ghettoVCB like there is in generation 1? Compression is key to our off-site backup regimen.
ghettoVCB compression is currently at an experimental support, no plans for g2 yet. There are still some outstanding issues that I've yet had the cycles to investigate for ghettoVCBg2 and I'm hoping I'll have some time before VMworld else it'll have to be after.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Seems I am getting the same errors in the logs as Al3xandr3.
Backup log:
============================== 08-15-2009 05:27:09 -- ghettoVCBg2 LOG START ==============================
08-15-2009 05:27:09 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-15-2009 05:27:09 -- CONFIG - VM_BACKUP_DATASTORE = Thecus_iSCSI_BkUp
08-15-2009 05:27:09 -- CONFIG - VM_BACKUP_DIRECTORY = vMA_Backups
08-15-2009 05:27:09 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-15-2009 05:27:09 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-15-2009 05:27:09 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-15-2009 05:27:09 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-15-2009 05:27:09 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-15-2009 05:27:09 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-15
08-15-2009 05:27:13 -- Initiate backup for "CapServer_New" found on myESX.domain.dc!
08-15-2009 05:27:24 -- CapServer_New original powerState: poweredOn
08-15-2009 05:28:14 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-15" for CapServer_New
08-15-2009 05:28:14 -- CapServer_New has 1 VMDK(s)
08-15-2009 05:28:14 -- Backing up "[COS_Datastore] CapServer_New/CapServer_New.vmdk" to "[Thecus_iSCSI_BkUp] vMA_Backups/CapServer_New/CapServer_New-2009-08-15/CapServer_New.vmdk"
08-15-2009 05:53:48 -- Backup completed for CapServer_New/CapServer_New.vmdk
08-15-2009 05:53:50 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-08-15" for CapServer_New
08-15-2009 05:54:00 -- Backup completed for "CapServer_New"!
08-15-2009 05:54:01 -- Initiate backup for "BrisSteelADS01" found on myESX.domain.dc!
08-15-2009 05:54:11 -- BrisSteelADS01 original powerState: poweredOn
08-15-2009 05:58:10 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-15" for BrisSteelADS01
08-15-2009 05:58:10 -- BrisSteelADS01 has 2 VMDK(s)
08-15-2009 05:58:10 -- Backing up "[COS_Datastore] BrisSteelADS01/BrisSteelADS01.vmdk" to "[Thecus_iSCSI_BkUp] vMA_Backups/BrisSteelADS01/BrisSteelADS01-2009-08-15/BrisSteelADS01.vmdk"
08-15-2009 06:14:03 -- Backup completed for BrisSteelADS01/BrisSteelADS01.vmdk
08-15-2009 06:14:03 -- Backing up "[COS_Datastore] BrisSteelADS01/BrisSteelADS01_2.vmdk" to "[Thecus_iSCSI_BkUp] vMA_Backups/BrisSteelADS01/BrisSteelADS01-2009-08-15/BrisSteelADS01_2.vmdk"
08-15-2009 06:31:22 -- Backup completed for BrisSteelADS01/BrisSteelADS01_2.vmdk
08-15-2009 06:31:22 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-15" not found
08-15-2009 06:31:37 -- Backup completed for "BrisSteelADS01"!
08-15-2009 06:31:37 -- ERROR - Unable to locate VM: CapServer2_New
============================== 08-15-2009 06:31:37 -- ghettoVCBg2 LOG END ==============================
.
.
.
.
.
.
Debug log (after the fact, snapshot was not there prior to backup above):
============================== 08-15-2009 07:25:08 -- ghettoVCBg2 LOG START ==============================
08-15-2009 07:25:08 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-15-2009 07:25:08 -- CONFIG - VM_BACKUP_DATASTORE = Thecus_iSCSI_BkUp
08-15-2009 07:25:08 -- CONFIG - VM_BACKUP_DIRECTORY = vMA_Backups
08-15-2009 07:25:08 -- CONFIG - DISK_BACKUP_FORMAT = thin
08-15-2009 07:25:08 -- CONFIG - ADAPTER_FORMAT = lsilogic
08-15-2009 07:25:08 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-15-2009 07:25:08 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-15-2009 07:25:08 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
08-15-2009 07:25:08 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-15
08-15-2009 07:25:12 -- -
DEBUG INFO CapServer_New -
08-15-2009 07:25:12 -- DEBUG - Host Build: VMware ESX 4.0.0 build-171294
08-15-2009 07:25:12 -- DEBUG - Host: myESX.domain.dc
08-15-2009 07:25:12 -- DEBUG - Virtual Machine: CapServer_New
08-15-2009 07:25:12 -- DEBUG - VM ConfigPath: CapServer_New/CapServer_New.vmx
08-15-2009 07:25:12 -- DEBUG - VMX File: CapServer_New.vmx
08-15-2009 07:25:12 -- DEBUG - BackupConfigPath: vMA_Backups/CapServer_New/CapServer_New-2009-08-15/CapServer_New.vmx
08-15-2009 07:25:12 -- DEBUG - BackupPath: vMA_Backups/CapServer_New/CapServer_New-2009-08-15
08-15-2009 07:25:12 -- DEBUG - VM Datastore: COS_Datastore
08-15-2009 07:25:12 -- DEBUG - VMDK(s):
08-15-2009 07:25:12 -- DEBUG - CapServer_New/CapServer_New.vmdk
08-15-2009 07:25:12 -- -
DEBUG INFO CapServer_New -
08-15-2009 07:25:12 -- WARN - Snapshot found for BrisSteelADS01, backup will not take place
08-15-2009 07:25:12 -- -
DEBUG INFO CapServer2_New -
08-15-2009 07:25:12 -- DEBUG - Host Build: VMware ESX 4.0.0 build-171294
08-15-2009 07:25:12 -- DEBUG - Host: myESX.domain.dc
08-15-2009 07:25:12 -- DEBUG - Virtual Machine: CapServer2_New
08-15-2009 07:25:12 -- DEBUG - VM ConfigPath: CapServer2_New/CapServer2_New.vmx
08-15-2009 07:25:12 -- DEBUG - VMX File: CapServer2_New.vmx
08-15-2009 07:25:12 -- DEBUG - BackupConfigPath: vMA_Backups/CapServer2_New/CapServer2_New-2009-08-15/CapServer2_New.vmx
08-15-2009 07:25:12 -- DEBUG - BackupPath: vMA_Backups/CapServer2_New/CapServer2_New-2009-08-15
08-15-2009 07:25:12 -- DEBUG - VM Datastore: COS_Datastore
08-15-2009 07:25:12 -- DEBUG - VMDK(s):
08-15-2009 07:25:12 -- DEBUG - CapServer2_New/CapServer2_New.vmdk
08-15-2009 07:25:12 -- -
DEBUG INFO CapServer2_New -
============================== 08-15-2009 07:25:12 -- ghettoVCBg2 LOG END ==============================
I get "Hard power off, VMware Tools is not installed" when try to use power off before backup feature.
The tools are there and VIC says "OK" for the tools status.
Has the feature ever been tested?
Another bug?
I thought to use the power off option it as a workaround since I had never succeeded backing up "hot" due to a known bug with non-deleted snapshots, but it does not seem to work properly either.
This feature has been tested and has worked. Please provide the following pieces of information on your environment:
1) What version of ESX(i)?
2) Are you using vCenter? What version? Are you using vMotion?
3) What version of VIMA/vMA are you running?
4) Please execute the following script to verify your VMware Tools status on your VMs:
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hello,
I would like to use this nice script but I also have the problem that the first VM gets backuped, the snapshot does not get deleted and the following VMs cannot be found.
I am using VIMA 1.0.
Here are the logfiles:
============================== 08-16-2009 15:37:44 -- ghettoVCBg2 LOG START ==============================
08-16-2009 15:37:44 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-16-2009 15:37:44 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
08-16-2009 15:37:44 -- CONFIG - VM_BACKUP_DIRECTORY = VM_BACKUPS
08-16-2009 15:37:44 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
08-16-2009 15:37:44 -- CONFIG - ADAPTER_FORMAT = buslogic
08-16-2009 15:37:44 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-16-2009 15:37:44 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
08-16-2009 15:37:44 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
08-16-2009 15:37:44 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-16
08-16-2009 15:37:50 -- -
DEBUG INFO vps006 -
08-16-2009 15:37:50 -- DEBUG - Host Build: VMware ESX Server 3i 3.5.0 build-123629
08-16-2009 15:37:50 -- DEBUG - Host: xx.xx.xx.xx
08-16-2009 15:37:50 -- DEBUG - Virtual Machine: vps006
08-16-2009 15:37:50 -- DEBUG - VM ConfigPath: vps006/vps006.vmx
08-16-2009 15:37:50 -- DEBUG - VMX File: vps006.vmx
08-16-2009 15:37:50 -- DEBUG - BackupConfigPath: VM_BACKUPS/vps006/vps006-2009-08-16/vps006.vmx
08-16-2009 15:37:50 -- DEBUG - BackupPath: VM_BACKUPS/vps006/vps006-2009-08-16
08-16-2009 15:37:50 -- DEBUG - VM Datastore: datastore1
08-16-2009 15:37:50 -- DEBUG - VMDK(s):
08-16-2009 15:37:50 -- DEBUG - vps006/vps006.vmdk
08-16-2009 15:37:50 -- -
DEBUG INFO vps006 -
08-16-2009 15:37:50 -- -
DEBUG INFO vps003 -
08-16-2009 15:37:50 -- DEBUG - Host Build: VMware ESX Server 3i 3.5.0 build-123629
08-16-2009 15:37:50 -- DEBUG - Host: xx.xx.xx.xx
08-16-2009 15:37:50 -- DEBUG - Virtual Machine: vps003
08-16-2009 15:37:50 -- DEBUG - VM ConfigPath: vps003/vps003.vmx
08-16-2009 15:37:50 -- DEBUG - VMX File: vps003.vmx
08-16-2009 15:37:50 -- DEBUG - BackupConfigPath: VM_BACKUPS/vps003/vps003-2009-08-16/vps003.vmx
08-16-2009 15:37:50 -- DEBUG - BackupPath: VM_BACKUPS/vps003/vps003-2009-08-16
08-16-2009 15:37:50 -- DEBUG - VM Datastore: datastore1
08-16-2009 15:37:50 -- DEBUG - VMDK(s):
08-16-2009 15:37:50 -- DEBUG - vps003/vps003.vmdk
08-16-2009 15:37:50 -- -
DEBUG INFO vps003 -
08-16-2009 15:37:50 -- -
DEBUG INFO vps001 -
08-16-2009 15:37:50 -- DEBUG - Host Build: VMware ESX Server 3i 3.5.0 build-123629
08-16-2009 15:37:50 -- DEBUG - Host: xx.xx.xx.xx
08-16-2009 15:37:50 -- DEBUG - Virtual Machine: vps001
08-16-2009 15:37:50 -- DEBUG - VM ConfigPath: vps001/vps001.vmx
08-16-2009 15:37:50 -- DEBUG - VMX File: vps001.vmx
08-16-2009 15:37:50 -- DEBUG - BackupConfigPath: VM_BACKUPS/vps001/vps001-2009-08-16/vps001.vmx
08-16-2009 15:37:50 -- DEBUG - BackupPath: VM_BACKUPS/vps001/vps001-2009-08-16
08-16-2009 15:37:50 -- DEBUG - VM Datastore: datastore1
08-16-2009 15:37:50 -- DEBUG - VMDK(s):
08-16-2009 15:37:50 -- DEBUG - vps001/vps001.vmdk
08-16-2009 15:37:50 -- -
DEBUG INFO vps001 -
============================== 08-16-2009 15:37:50 -- ghettoVCBg2 LOG END ==============================
============================== 08-16-2009 15:41:25 -- ghettoVCBg2 LOG START ==============================
08-16-2009 15:41:25 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-16-2009 15:41:25 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
08-16-2009 15:41:25 -- CONFIG - VM_BACKUP_DIRECTORY = VM_BACKUPS
08-16-2009 15:41:25 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
08-16-2009 15:41:25 -- CONFIG - ADAPTER_FORMAT = buslogic
08-16-2009 15:41:25 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-16-2009 15:41:25 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
08-16-2009 15:41:25 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
08-16-2009 15:41:25 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-08-16
08-16-2009 15:41:29 -- Initiate backup for "vps006" found on xx.xx.xx.xx!
08-16-2009 15:41:43 -- vps006 original powerState: poweredOn
08-16-2009 15:41:45 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-08-16" for vps006
08-16-2009 15:41:45 -- vps006 has 1 VMDK(s)
08-16-2009 15:41:45 -- Backing up "[datastore1] vps006/vps006.vmdk" to "[datastore1] VM_BACKUPS/vps006/vps006-2009-08-16/vps006.vmdk"
08-16-2009 16:26:19 -- Backup completed for vps006/vps006.vmdk
08-16-2009 16:26:19 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-08-16" not found
08-16-2009 16:26:27 -- Backup completed for "vps006"!
08-16-2009 16:26:27 -- ERROR - Unable to locate VM: vps001
08-16-2009 16:26:27 -- ERROR - Unable to locate VM: vps003
============================== 08-16-2009 16:26:27 -- ghettoVCBg2 LOG END ==============================
Is this a valid perl statement, as I have not found an example of using it anywhere else.
line 699 - $ref = $subRef if ($subCount);
I have changed it to the following, and it look good so far.
if ($subCount) {
$ref = $subRef;
}
It's a valid Perl statement, test it out for yourself.
Here's a quick example:
[vi-admin@scofield]$ cat test.pl #!/usr/bin/perl -w use strict; use warnings; my $subCount = 1; my $subRef = "hello world"; my $ref = "not hellow world"; $ref = $subRef if($subCount); print $ref . "\n";
Run it and you tell me what it prints
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I figured the problem with VM not being powered down was in the Tools being out of date, so sorry for the false alarm.
However, now the VM does not being powered back up after the backup. Even though I can see the code is there.
The script just quits after the last VMDK backup by saying "Backup completed", but without the "LOG END".
What could be wrong?
Hey William,
First off, thanks so much for making these scripts. Between ghettoShutdown.pl and ghettoVCBg2.pl (if I ever get it working) I'll have everything I need to run ESXi without worry!
That being said... I'm having some issues with getting ghettoVCBg2.pl working
Environment Setup:
- vmbox1.domain.com ESXi 3.5u3 host (free license) which has one guest
-- "vmguest1.domain.com" - a winXP "workstation"
- vmbox2.domain.com ESXi 3.5u3 host (free license) which has one guest
-- vMA-vmbox2.domain.com - a vMA 1.0 virtual appliance
This is the same setup I was using with my questions over in the ghettoShutdown.pl thread, and that's working very well now thanks!
Anyway, it -appears- to work, but when it's all done the only thing that appears in the backup folder is a single *.vmx file that's 2.03kb and that's it. The whole process takes no more than two minutes as well, which is quite a short time for something that's 10gb to go from one datastore to another (which are on separate physical hard drives in the same box) Here's a dump of the log
============================== 09-03-2009 09:49:48 -- ghettoVCBg2 LOG START ==============================
09-03-2009 09:49:48 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
09-03-2009 09:49:48 -- CONFIG - VM_BACKUP_DATASTORE = datastore2
09-03-2009 09:49:48 -- CONFIG - VM_BACKUP_DIRECTORY = VMBackups
09-03-2009 09:49:48 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
09-03-2009 09:49:48 -- CONFIG - ADAPTER_FORMAT = buslogic
09-03-2009 09:49:48 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
09-03-2009 09:49:48 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
09-03-2009 09:49:48 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
09-03-2009 09:49:48 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-09-03
09-03-2009 09:50:26 -- Initiate backup for "vmguest1" found on vmbox1.domain.com!
09-03-2009 09:50:45 -- vmguest1 original powerState: poweredOn
09-03-2009 09:50:47 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-09-03" for vmguest1
09-03-2009 09:50:47 -- vmguest1 has 1 VMDK(s)
09-03-2009 09:50:47 -- Backing up "[datastore1] vmguest1/vmguest1.vmdk" to "[datastore2] VMBackups/vmguest1/vmguest1-2009-09-03/vmguest1.vmdk"
09-03-2009 09:50:53 -- Backup completed for vmguest1/vmguest1.vmdk
09-03-2009 09:50:56 -- Removing Snapshot "ghettoVCBg2-snapshot-2009-09-03" for vmguest1
09-03-2009 09:51:06 -- Backup completed for "vmguest1"!
============================== 09-03-2009 09:51:06 -- ghettoVCBg2 LOG END ==============================
As you can see the log looks fine. There's plenty of space on datastore2 (like 100gb and the VM I'm trying to backup is 10GB).
Any ideas why it's not creating/copying anything to the backup folder other than a single tiny *.vmx file?
Jon
Hi William,
First of all, works great. Easy to configure and very fast free alternative.
I have a question, is it possible to have the status mailed to me.
Case of a backup error, I like to have a email.
Menno
Hi
I will try as described but i am hanging by creating a NFS-datastore in my ESXi/vMA
If I understand correctly the NFS-datastore have to be mounted on vMA (central)
But how to do ?
regards
Martin
Hi Menno,
you could pipe the cron output to mail.
Just append something like
| mail -s "ghettoVCBg2 " someone@mydomain.com
at the end of the cron entry.
But this way you will get the mail everytime the cronjob runs, not just in case of errors.
@martin
You have to mount the datastore on your ESXi to backup to it.
You can do this via the vCenter.
The only case in which you have to mount the datastore in your vMA is when you want to store the vmlist files in some central location.
Following Log, whats going wrong here, NFS_datastore is mounted on esxi03 !
============================== 09-07-2009 07:28:01 -- ghettoVCBg2 LOG START ==============================
09-07-2009 07:28:05 -- CONFIG - USING CONFIGURATION FILE = vMA
09-07-2009 07:28:05 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
09-07-2009 07:28:05 -- CONFIG - VM_BACKUP_DATASTORE = NFS_datastore
09-07-2009 07:28:05 -- CONFIG - VM_BACKUP_DIRECTORY = esxi03_backups
09-07-2009 07:28:05 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
09-07-2009 07:28:05 -- CONFIG - ADAPTER_FORMAT = lsilogic
09-07-2009 07:28:05 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
09-07-2009 07:28:05 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
09-07-2009 07:28:05 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
09-07-2009 07:28:05 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-09-07
09-07-2009 07:28:05 -- Initiate backup for "vMA" found on esxi03.softline.at!
09-07-2009 07:28:14 -- vMA original powerState: poweredOn
09-07-2009 07:28:14 -- vMA has 1 VMDK(s)
09-07-2009 07:28:14 -- Backing up "[datastore1] vMA/vMA.vmdk" to "[NFS_datastore] esxi03_backups/vMA/vMA-2009-09-07/vMA.vmdk"
09-07-2009 07:28:18 -- Backup completed for vMA/vMA.vmdk
09-07-2009 07:28:18 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-09-07" not found
09-07-2009 07:28:21 -- Unable to purge "[NFS_datastore] esxi03_backups/vMA/GET https:esxi03.softline.atfolderesxi03_backupsvMA?dsName=NFS_datastore unsuccessful : 404 Not Found" due to rotation max
09-07-2009 07:28:21 -- Backup completed for "vMA"!
============================== 09-07-2009 07:28:21 -- ghettoVCBg2 LOG END ==============================
William,
Phenomenal work and effort on this project, and thanks for all your support for the community! I have just executed a few initial trials, and like Shawn and martingreiner I am seeing a "404 Not Found due to rotation max" error. I have vMA and ESXi 3.5 build 184236 along with ghettoVCBg2.pl version 2.5.1. I was wondering if you had an idea what might be causing this error?
(debug output below):
============================== 09-17-2009 08:21:03 -- ghettoVCBg2 LOG START ==============================
09-17-2009 08:21:03 -- CONFIG - BACKUP_LOG_OUTPUT = ghettoVCBg2.log
09-17-2009 08:21:03 -- CONFIG - VM_BACKUP_DATASTORE = DSS1-bkupvm-eth4
09-17-2009 08:21:03 -- CONFIG - VM_BACKUP_DIRECTORY = ghettoVCB
09-17-2009 08:21:03 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
09-17-2009 08:21:03 -- CONFIG - ADAPTER_FORMAT = buslogic
09-17-2009 08:21:03 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
09-17-2009 08:21:03 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
09-17-2009 08:21:03 -- CONFIG - VM_SNAPSHOT_QUIESCE = YES
09-17-2009 08:21:03 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-09-17
09-17-2009 08:21:08 -- Initiate backup for "actitime" found on 192.168.50.7!
09-17-2009 08:21:24 -- actitime original powerState: poweredOn
09-17-2009 08:21:24 -- actitime has 1 VMDK(s)
09-17-2009 08:21:24 -- Backing up "[DSS1-bond0] actitime/actitime.vmdk" to "[DSS1-bkupvm-eth4] ghettoVCB/actitime/actitime-2009-09-17/actitime.vmdk"
09-17-2009 08:21:30 -- Backup completed for actitime/actitime.vmdk
09-17-2009 08:21:30 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-09-17" not found
09-17-2009 08:21:34 -- Unable to purge "[DSS1-bkupvm-eth4] ghettoVCB/actitime/GET https:192.168.50.7folderghettoVCBactitime?dsName=DSS1-bkupvm-eth4 unsuccessful : 404 Not Found" due to rotation max
09-17-2009 08:21:34 -- Backup completed for "actitime"!
09-17-2009 08:21:34 -- ERROR - Unable to locate VM: operator
============================== 09-17-2009 08:21:34 -- ghettoVCBg2 LOG END ==============================
Hi All,
I have been hacking this script to find out what the problem is with the snap shots not being deleted and have changed the call from the internal function to the external script snapshotmanager.pl.
#&remove_snapshot($vm_view, $snapshot_name, $vm_name);
my $res = `/home/vi-admin/bin/snapshotmanager.pl -server vc1 --operation remove --vmname $vm_name --username ******* --password ******** --snapshotname $snapshot_name --children 0`;
print LOG timeStamp('MDYHMS'), "Remove Snapshot $snapshot_name from VM $vm_name with results: $res";
It seems like you need to close and reopen the connections to get a refreshed listing of the snap shots? I tried using delays etc, but did not have any consistent results till I tried the external call. This new call has been working the past 2 weeks on the VM's that failed previously.
This has been on my to-do list, I'm wondering if the connection eventually times out while it's executing the backup ... I have few others that are running into the same issue with removing the snapshot, but I haven't had a chance to replicate the issue. Drop me your email via VMTN PM and I'll send out an email this weekend regarding the next steps and perhaps a live support chat.
Thanks for your patience
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I had this error and traced it (via perl debugger) to the 'restricted version' error. I had the script working perfectly and all of a sudden it stopped with a message like yours. Turns out that while I was evaluating the product it allows writeable CLI. I never used the script for a while, but sometime later I grabbed free licenses and applied it to all of my ESX4i servers. I'm guessing that this is what has changed the CLI to be read-only and therefor ghetto no longer works ! Can anyone confirm this ?
Take a look at this article for your answer: http://vmetc.com/2009/03/31/esxi-u4-ends-free-version-read-and-write-access-from-the-rcli/
The API was never meant for full read/write, this was a bug in VMware's code between U2 and U3 which was remediated in U4 and subsequent release in vSphere. I can understand where VMware's coming from, though I may not agree ... but you pay for you what you get. You still have the ability to use the unsupported SSH console for automating various set of tasks and licensed version of ESXi gives you more than just vCLI access and that is what makes them a leader in this space.
Good luck.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Thanks William,
I had read the article previously and it did not specifically mention ESX4i. I suppose I am disappointed because your excellent script worked during the evaluation and now it doesn't.
I have enabled 'tech support' mode on all 3 servers but I fear that it too one day soon will disappear.
The article was published well before vSphere came out, but again this was an internal bug that VMware had and the API was never intended to be free (both read/write), so once this was fixed it would be the case for ESXi 4.0 as well.
Take a look at this comparison chart for the various licensing versions: http://www.vmware.com/products/vsphere/buy/editions_comparison.html as you can see even the simplest licensed version provides much value added than just full r/w access to the API.
Regarding the unsupported console in ESXi, it's VMware's direction to go to the ESXi platform and eventually the classic ESX will no longer be around ( I wouldn't hold my breath anytime soon, but eventually). They may eventually remove the Busybox console as well but again, it'll take time but enjoy it for now.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hi William,
when i run the script, i get the following output:
print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 150
print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 300
print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 300
print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 300
print() on closed filehandle LOG at ./ghettoVCBg2.pl at line 198
ESXi 4 Enterprise, vMA 4.0, backup: nfs
Thank you in advance.
hi, I suppose the script can't write to the directoy or to the logfile you specified.
Try /tmp/ghettoVCB2.log that should work.
regards,
Martin
I wanted to give an update to everyone that may be using this script.
The issues that have been mentioned are being actively worked on, as you know I do support many many other scripts and there's only so many hours in each day I do assure you, I've not forgotten about you guys/gals and the issues being reported. I do ask for you patience and I'll provide an update as soon as I have one.
Stay tuned ...
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hi all here!
I/We have 3 ESX4 machines with a shared SAN datastore (all machines can see all SAN datastores). One machine has a "big" Local Storage and I want to backup remote VMs to the Local Storage using SAN, not over a network. Is it possible to configure ghettoVCBg2 to backup VMs this way?
Thanks a lot
Hi everyone,
I'm currently in development (as time permits) on some new features for the ghettoVCBg2 script and I'm looking for some individuals that may be interested in taking part in a BETA. The requirement is pretty simple, willing to help test and provide feedback on the next version of the script (remember to test on non-production VMs)
If you're interested, please PM on VTMN with your email address and the script you would like to take part in and I'll be selecting a group users and send out email notification in the coming weeks.
Thanks
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
When i make a backup from a big VM with 300-500 GB, the VM does not get started automatically after backup prozess.
What could be the matter of this? With smaller VMs it works.
This is a known issue with larger VMs and will be fixed in a future release.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
OK, thank you for the fast answer.
Do you already know what causes this issue? Maybe i can fix it byself.
Nope the solution is actually pretty complicated, we're pretty much there but requires further testing which will be part of the BETA. I won't go into further details, if you're already emailed me then you'll know in the coming weeks if you're one of the few few to be on the BETA.
Thanks for your patience.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
All right, i´ve sent you already an email last week for taking part in the BETA. I would help you willingly.
Hi,
I'm using ESXi 4.0, have a few virtual machines running, and trying to get the backup done. The thing is that I created all virtual machines with Thin disks. However, there are in all those VM's two vmdk's, one called .vmdk the backup seems to work, although I cannot find the backupfiles somewhere on my system. See the attached logs, can anybody help me out?
============================== 10-21-2009 15:44:02 -- ghettoVCBg2 LOG START ==============================
10-21-2009 15:44:02 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DIRECTORY = backup
10-21-2009 15:44:02 -- CONFIG - DISK_BACKUP_FORMAT = thin
10-21-2009 15:44:02 -- CONFIG - ADAPTER_FORMAT = lsilogic
10-21-2009 15:44:02 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
10-21-2009 15:44:02 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 15:44:02 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-10-21
10-21-2009 15:44:07 -- Initiate backup for "TS3" found on 192.168.72.113!
10-21-2009 15:44:14 -- TS3 original powerState: poweredOn
10-21-2009 15:44:14 -- TS3 has 1 VMDK(s)
10-21-2009 15:44:14 -- Backing up "[datastore1] TS3/TS3.vmdk" to "[datastore1] backup/TS3/TS3-2009-10-21/TS3.vmdk"
10-21-2009 15:44:18 -- Backup completed for TS3/TS3.vmdk
10-21-2009 15:44:18 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-10-21" not found
10-21-2009 15:44:19 -- Unable to purge "[datastore1] backup/TS3/GET https:192.168.72.113folderbackupTS3?dsName=datastore1 unsuccessful : 404 Not Found" due to rotation max
10-21-2009 15:44:19 -- Backup completed for "TS3"!
============================== 10-21-2009 15:44:19 -- ghettoVCBg2 LOG END ==============================
============================== 10-21-2009 16:00:29 -- ghettoVCBg2 LOG START ==============================
10-21-2009 16:00:29 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DIRECTORY = backup
10-21-2009 16:00:29 -- CONFIG - DISK_BACKUP_FORMAT = thin
10-21-2009 16:00:29 -- CONFIG - ADAPTER_FORMAT = lsilogic
10-21-2009 16:00:29 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
10-21-2009 16:00:29 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 16:00:29 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-10-21
10-21-2009 16:00:35 -- Initiate backup for "TS3" found on 192.168.72.113!
10-21-2009 16:00:42 -- TS3 original powerState: poweredOff
10-21-2009 16:00:42 -- TS3 has 1 VMDK(s)
10-21-2009 16:00:42 -- Backing up "[datastore1] TS3/TS3.vmdk" to "[datastore1] backup/TS3/TS3-2009-10-21/TS3.vmdk"
10-21-2009 16:00:45 -- Backup completed for TS3/TS3.vmdk
10-21-2009 16:00:47 -- Backup completed for "TS3"!
============================== 10-21-2009 16:00:47 -- ghettoVCBg2 LOG END ==============================
I'm not exactly sure what you're trying to do? Was the backup successful? Are you saying that you executed a backup of 1 VM (TS3) and after the backup, you manually renamed the files on the ESX(i) host? From the looks of the log, you should have your backup VM stored in /vmfs/volumes/backup/TS3/TS3-2009-10-21/ and note that if you browse your datastore using the vSphere Client that the *-flat.vmdk will not display, only the descriptor.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hi,
First I tried to do a backup, just of the VM TS3. That resulted in the following:
============================== 10-21-2009 15:44:02 -- ghettoVCBg2 LOG START ==============================
10-21-2009 15:44:02 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DIRECTORY = backup
10-21-2009 15:44:02 -- CONFIG - DISK_BACKUP_FORMAT = thin
10-21-2009 15:44:02 -- CONFIG - ADAPTER_FORMAT = lsilogic
10-21-2009 15:44:02 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
10-21-2009 15:44:02 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 15:44:02 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 15:44:02 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-10-21
10-21-2009 15:44:07 -- Initiate backup for "TS3" found on 192.168.72.113!
10-21-2009 15:44:14 -- TS3 original powerState: poweredOn
10-21-2009 15:44:14 -- TS3 has 1 VMDK(s)
10-21-2009 15:44:14 -- Backing up "datastore1 TS3/TS3.vmdk" to "datastore1 backup/TS3/TS3-2009-10-21/TS3.vmdk"
10-21-2009 15:44:18 -- Backup completed for datastore1 TS3/TS3.vmdk
10-21-2009 15:44:18 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-10-21" not found
10-21-2009 15:44:19 -- Unable to purge "datastore1 backup/TS3/GET https:192.168.72.113folderbackupTS3?dsName=datastore1 unsuccessful : 404 Not Found" due to rotation max
10-21-2009 15:44:19 -- Backup completed for "TS3"!
============================== 10-21-2009 15:44:19 -- ghettoVCBg2 LOG END ==============================
The backup directory was still empty. So, i tried to get info via Google, and saw something about two VMDK-files. So, I thought probably I can rename the TS3-flat.vmdk to TS3.vmdk. I did (made a saveset first) and then the second run:
============================== 10-21-2009 16:00:29 -- ghettoVCBg2 LOG START ==============================
10-21-2009 16:00:29 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DATASTORE = datastore1
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DIRECTORY = backup
10-21-2009 16:00:29 -- CONFIG - DISK_BACKUP_FORMAT = thin
10-21-2009 16:00:29 -- CONFIG - ADAPTER_FORMAT = lsilogic
10-21-2009 16:00:29 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
10-21-2009 16:00:29 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 16:00:29 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
10-21-2009 16:00:29 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-10-21
10-21-2009 16:00:35 -- Initiate backup for "TS3" found on 192.168.72.113!
10-21-2009 16:00:42 -- TS3 original powerState: poweredOff
10-21-2009 16:00:42 -- TS3 has 1 VMDK(s)
10-21-2009 16:00:42 -- Backing up "datastore1 TS3/TS3.vmdk" to "datastore1 backup/TS3/TS3-2009-10-21/TS3.vmdk"
10-21-2009 16:00:45 -- Backup completed for datastore1 TS3/TS3.vmdk
10-21-2009 16:00:47 -- Backup completed for "TS3"!
============================== 10-21-2009 16:00:47 -- ghettoVCBg2 LOG END ==============================
Still the backup directory was empty, but the log seems successfull. Can you point me in the right direction?
Regards,
Bas
BETA email invitations will be going out this weekend ... keep an eye out
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I have installed this wonderfull script. Keep up the good work!
I am having trouble with crontab. My crontab (vi-admin) looks like:
vi-admin@vma ~$ crontab -l
0 0 * * * /home/vi-admin/ghettoVCBg2.pl --config_dir /home/vi-admin/backupConfigs --vmlist /home/vi-admin/virtual_machne_back_list_belangrijk
0 10 * * 7 /home/vi-admin/ghettoVCBg2.pl --config_dir /home/vi-admin/backupConfigs --vmlist /home/vi-admin/virtual_machne_back_list
0 5 * * 1-5 /home/vi-admin/ghettoVCBg2.pl --config_dir backupConfigs --vmlist /home/vi-admin/virtual_machne_back_list_branderweg
But cron doesn't run it at all. Did I missed something? When I use the lines at the shell the commands make great backups.
a tail of /var/log/cron gives:
Password:
Oct 23 00:01:01 localhost crond[14745]: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 01:01:01 localhost crond[14987]: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 02:01:01 localhost crond[15229]: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 03:01:01 localhost crond[15471]: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 04:01:01 localhost crond[15713]: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 05:01:01 localhost crond[15955]: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 06:01:01 localhost crond[16224]: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 07:01:01 localhost crond[16466]: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 08:01:01 localhost crond[16708]: (root) CMD (run-parts /etc/cron.hourly)
Oct 23 08:26:49 localhost crond[2074]: (CRON) STARTUP (V5.0)
Why cron isn't running my backups?
I am trying to backup vms in an ESX4 host that has mostly SAN based storage to an ESXi server with local storage over the network to a remote location. It's not clear to me whether I need to mount an NFS store on the ESX4 host or if the store still needs to be visible to the ESX4 host or if the vMA should take care of this as long as it is listed as being managed by the vMA. Do I still need to mount the NFS store manually on each ESX4 host so that $VM_BACKUP_DATASTORE that I'm backing up to is visible to that individual ESX4 host?
I think I'm just confused on the syntax for the datastore or how the script works with vMA, I verified that both hosts are being managed by vMA with sudo vifp listservers and I have tried several different names for the $VM_BACKUP_DATASTORE value. Everything in the logs says the backup completes but on the ESX4 host it keeps saying invalid datastore specified so it is clear it doesn't know what datastore I am trying to backup to. I have also tried specifying the ESXi server that I am trying to backup to with the --server command option but doesn't seem to make a difference.
I’ve tried several different values and no luck.
my $VM_BACKUP_DATASTORE = "/vmfs/volumes/backups";
my $VM_BACKUP_DATASTORE = "https://172.16.x.x/folder?dcPath=ha-datacenter&dsName=backups";
my $VM_BACKUP_DATASTORE = "172.16.x.x/folder?dcPath=ha-datacenter&dsName=backups";
my $VM_BACKUP_DATASTORE = "172.16.x.x";
Thanks again for all your work any help is apprieciated!
The NFS datastore MUST be mounted on the ESX(i)host and must be visible to the host, vMA will instruct the various operations using the vSphere API to perform the backup to the NFS datastore or to any valid datastore for that matter.
Your syntax is for VM_BACKUP_DATASTORE is incorrect, you apparently did not go through the documentation throughly. The entry should just be the name of your datastore (e.g. datastore1)
If your NFS datastore is mounted on the ESX(i) host as backups then variable VM_BACKUP_DATASTORE should be set to backups
I would recommend going over the documentation and read through it carefully as much of your questions can be answered.
Thanks
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Thank you that helps alot!
Hi William,
Every night I am backing up my vm's with the script. with 3 of them the snapshot is not found and not deleted. For your information these Vm's are greater dan 30GB..
See the log file.
11-08-2009 00:52:06 -- CONFIG - USING CONFIGURATION FILE = UPDATES
11-08-2009 00:52:06 -- CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-08-2009 00:52:06 -- CONFIG - VM_BACKUP_DATASTORE = vmbackup
11-08-2009 00:52:06 -- CONFIG - VM_BACKUP_DIRECTORY = vmware
11-08-2009 00:52:06 -- CONFIG - DISK_BACKUP_FORMAT = zeroedthick
11-08-2009 00:52:06 -- CONFIG - ADAPTER_FORMAT = lsilogic
11-08-2009 00:52:06 -- CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-08-2009 00:52:06 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-08-2009 00:52:06 -- CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-08-2009 00:52:06 -- CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-08
11-08-2009 00:52:06 -- Initiate backup for "UPDATES" found on 10.3.1.25!
11-08-2009 00:52:16 -- UPDATES original powerState: poweredOn
11-08-2009 00:52:18 -- Creating Snapshot "ghettoVCBg2-snapshot-2009-11-08" for UPDATES
11-08-2009 00:52:18 -- UPDATES has 1 VMDK(s)
11-08-2009 00:52:18 -- Backing up "[NFS] updates/UPDATES_ESXi1.vmdk" to "[vmbackup] vmware/UPDATES/UPDATES-2009-11-08/UPDATES_ESXi1.vmdk"
11-08-2009 02:03:40 -- Backup completed for updates/UPDATES_ESXi1.vmdk
11-08-2009 02:03:40 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-11-08" not found
11-08-2009 02:03:55 -- Backup completed for "UPDATES"!
And a other:
============================== 11-09-2009 12:29:04 -- ghettoVCBg2 LOG START ==============================
11-09-2009 12:29:09 -- Initiate backup for "Helpdesk" found on 10.3.1.24!
11-09-2009 12:29:21 -- Helpdesk has 1 VMDK(s)
11-09-2009 13:07:02 -- WARNING: Snapshot "ghettoVCBg2-snapshot-2009-11-09" not found
11-09-2009 13:07:18 -- Backup completed for "Helpdesk"!
============================== 11-09-2009 13:07:22 -- ghettoVCBg2 LOG END ==============================
How is the Beta going? Can I help?
Regards,
Rob
Hi Rob,
As mentioned in previous replies, the issue with backing up large VMs has been resolved in the next release of the script. I can't say much more but keep an eye out for the next version of the script soon
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hi,
to get the email to work I had to change line # 302
my $smtp = Net::SMTP->new($EMAIL_HOST ,Hello => 'vMA - ghettoVCBg2',Timeout => 30,);
to
my $smtp = Net::SMTP->new($EMAIL_HOST,Hello => 'mycompanydomain.com',Timeout => 30,);
or just do this
my $smtp = Net::SMTP->new($EMAIL_HOST);
I was always getting the error that it could not connect if I did not put my local company domain in.
Thanks for the info! I thought I had it working without specifying the entry and using a dummy string but I guess not .... well I've added an additional variable that'll specify the email domain and if you don't want to specify it, it'll default to localhost.localdomain as defined here: http://perldoc.perl.org/Net/SMTP.html
Glad to hear everything else is good
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Great! Thanks for the quick fix.
Everything else is working great, no more snapshot problems! I also really like the selective backups!
Hello William,
Thanks fore the new script! I have downloaded the new script and give the backup a GO.
Running bt hand gives me:
$ ./ghettoVCBg2.pl --config_dir /home/vi-admin/backupConfigs --vmlist helpdesk
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
Use of uninitialized value in numeric le (<=) at ./ghettoVCBg2.pl line 996.
the /tmp/ghettoVCBg2.log:
$ tail -f /tmp/ghettoVCBg2.log
11-19-2009 10:28:34 -- info: Initiate backup for Helpdesk found on 10.3.1.24
11-19-2009 10:28:44 -- debug: Helpdesk original powerState: poweredOn
11-19-2009 10:28:44 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2009-11-19" for Helpdesk
11-19-2009 10:28:54 -- info: Helpdesk has 1 VMDK(s)
11-19-2009 10:28:54 -- debug: backupVMDK: Backing up "[NFS] Helpdesk/Helpdesk.vmdk" to "[vmbackup] vmware/Helpdesk/Helpdesk-2009-11-19/Helpdesk.vmdk"
11-19-2009 10:28:54 -- debug: backupVMDK: Signal copyThread to start
11-19-2009 10:28:54 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
11-19-2009 10:28:54 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
11-19-2009 10:28:54 -- debug: CopyThread: Start backing up VMDK(s) ...
11-19-2009 10:43:57 -- debug: backupVMDK: Backup progress: Elapsed time 15 min
11-19-2009 10:59:00 -- debug: backupVMDK: Backup progress: Elapsed time 30 min
11-19-2009 11:04:22 -- debug: copyTask: send copySuccess message ...
11-19-2009 11:04:22 -- debug: copyTask: waiting for next job and sleep ...
11-19-2009 11:05:02 -- debug: backupVMDK: Successfully completed backup for Helpdesk/Helpdesk.vmdk Elapsed time: 36 min
11-19-2009 11:05:18 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2009-11-19" for Helpdesk
11-19-2009 11:05:21 -- debug: checkVMBackupRotation: Starting ...
11-19-2009 11:05:40 -- debug: Purging vmware/Helpdesk/Helpdesk-2009-11-13--7 due to rotation max
11-19-2009 11:05:40 -- info: Backup completed for Helpdesk!
11-19-2009 11:05:40 -- debug: Main: Disconnect from: 10.3.1.24
11-19-2009 11:05:40 -- debug: Main: Login by vi-fastpass to: 10.3.1.25
11-19-2009 11:05:43 -- debug: Main: Disconnect from: 10.3.1.25
11-19-2009 11:05:43 -- debug: Main: Login by vi-fastpass to: 10.3.1.26
11-19-2009 11:05:45 -- debug: Main: Disconnect from: 10.3.1.26
11-19-2009 11:05:45 -- debug: Main: Calling final clean up
11-19-2009 11:05:45 -- debug: cleanUP: Thread clean up starting ...
11-19-2009 11:05:45 -- debug: cleanUp: Send exit to copyThread
11-19-2009 11:05:45 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit
11-19-2009 11:05:45 -- debug: copyTask: die ...
11-19-2009 11:05:45 -- debug: cleanUp: Join passed
11-19-2009 11:05:45 -- info: ============================== ghettoVCBg2 LOG END ==============================
Do you have a idea? The backup is running fine, with a large disk
Thanks!
kind regards,
Rob
It sounds like one of the logging variables is not configured correctly, can you please output one of the VM backup policy? The only supported logging option is either "info" or "debug" and hopefully that's defined properly in all your configs.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
William,
Thanks, you where right. I still used "verbose".
I'll change that right away.
Regards,
Rob