This tool is the follow up release of the ghettoVCB
backup utility which allows users to perform backups of virtual machines residing on ESX(i) 3.5+/4.x+
servers using methodology similar to VMware's VCB
tool. By incorporating highly constructive feedback from the VMware community and utilizing the existing VI API, ghettoVCB’s framework was completely rewritten to be harder, better, faster, stronger
.
The primary motivation for ghettoVCBg2 was to provide ESXi users with access to the utility without relying on unlocking and utilizing the unsupported console. To satisfy this requirement, the rebuilt framework takes advantage of the
VI Perl Toolkit/vSphere SDK for Perl which is present in the
VIMA/vMA virtual appliance provided by VMware. As a result, ghettoVCBg2 provides a more proper backup solution that administrators can utilize in their virtual infrastructure.
As it stands, it is a requirement that ghettoVCB be invoked directly on each of the ESX servers hosting virtual machines in need of a backup. By taking advantage of VIMA/vMA, the entire backup process becomes centralized in turn eliminating the abovementioned requirement. Additionally, by leveraging the vi-fastpass library, unattended authentication between VIMA/vMA and target ESX servers is made possible. This alleviates the need to continually pass in credentials for each ESX server.
By centralizing the backup process, individual virtual machine backup lists pertaining to their respective ESX(i) servers are no longer required. ghettoVCBg2 has the ability to identify virtual machines and their respective hosts, potentially eliminating any past confusion associated with maintaining multiple lists.
In its current configuration, the script will allow up to 3 unique backups of the Virtual Machine before it will overwrite the previous backups; this however, can be modified to fit procedures if need be. Please be diligent in running the script in a test or staging environment before using it on production live Virtual Machines; this script functions well within our environment but there is a chance that it may not fit well into other environments.
VMs running on ESX(i) 3.5u2+ or ESX(i) 4.x+
VMware VIMA 1.0 or VMware vMA 4.x
Version | Support | Alternative |
ESX 3.5u2 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 3.5u3 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 3.5u4 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESXi 3.5u2 | Works for free | ghettoVCB\ |
ESXi 3.5u3 | Works for free | ghettoVCB\ |
ESXi 3.5u4 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 4.0 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESXi 4.0 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 4.1 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESXi 4.1 | Requires at least Foundation/Essentials license | ghettoVCB\ |
1) Download ghettoVCBg2.tar.gz
to your VIMA 1.0/vMA 4.x system
2) Extract the contents of the tarball
[vi-admin@scofield ~]$ tar -zxvf ghettoVCBg2.tar.gz
ghettoVCBg2/
ghettoVCBg2/ghettoVCBg2-vm_backup_configuration_template
ghettoVCBg2/ghettoVCBg2.pl
3) The script should be in a directory called
ghettoVCBg2
Prior to starting, ensure that the VMware VIMA/vMA host is managing the appropriate ESX(i) 3.5u2/4.x+ host(s)
1. Add ESXi hosts to VIMA/vMA managment using vifp
A password prompt for the vi-admin account and root password to the pertinent ESXi host(s) will be presented.
[vi-admin@vima-primp-industries ~]$ sudo vifp addserver olga.resnet.ucsb.edu
root@olga.resnet.ucsb.edu's password:
[vi-admin@vima-primp-industries ~]$
2. Verify that the appropriate hosts are being managed by VIMA/vMA
[vi-admin@vima-primp-industries ~]$ sudo vifp listservers
olga.resnet.ucsb.edu
himalaya.primp-industries.com
The following variables (similar to that of ghettoVCB) need to be defined within the script prior to execution.
Defining the datastore in which the backups are stored:
Defining the name of folder that will contain backups (if folder does not exist, it will automatically be created):
Defining the backup rotation per VM:
Defining the backup disk format (zeroedthick, eagerzeroedthick, thin, and 2gbsparse are available):
Defining the adapter type for backed up VMDK (buslogic, and lsilogic are available):
Defining whether the VM is powered down or not prior to backup (1 = enable, 0 = disable):
Note: VM(s) that are powered off will not require snapshoting
Defining whether virtual machine memory is snapped and if quiescing is enabled (1 = enable, 0 = disable):
Note: By default both are disabled
Defining the detail of logging (verbose or normal):
*Defining VMDK(s) to backup from a particular VM either a list of vmdks or "all"
VM_VMDK_FILES="myvmdk.vmdk"
Experimental Support Configs:
Note: Port 25 by default
my $SEND_MAIL = "no";
my $EMAIL_HOST = "emailserver";
my $EMAIL_DOMAIN = "localhost.localdomain";
my $EMAIL_TO = 'William Lam <william@primp-industries.com.com>';
my $EMAIL_FROM = 'ghettoVCBg2 <ghettoVCBg2@primp-industries.com.com>';
Ensure that you do not edit past this section:
########################## DO NOT MODIFY PAST THIS LINE ##########################
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl
Required command option 'vmlist' not specified.
Synopsis: ./ghettoVCBg2.pl OPTIONS
Command-specific options:
--config_dir
Name of directory containing VM(s) backup configurations
--dryrun (default '0')
Set to 1 to enable dryrun mode (default 0)
--output (default '/tmp/ghettoVCBg2.log')
Full path to output log (default /tmp/ghettoVCBg2.log)
--vmlist (required)
A file containing a list of virtual machine(s) to be backed up on host
Common VI options:
--config (variable VI_CONFIG)
Location of the VI Perl configuration file
--credstore (variable VI_CREDSTORE)
Name of the credential store file defaults to <HOME>/.vmware/credstore/vicredentials.xml on Linux and <APPDATA>/VMware/credstore/vicredentials.xml on Windows
--encoding (variable VI_ENCODING, default 'utf8')
Encoding: utf8, cp936 (Simplified Chinese), iso-8859-1 (German), shiftjis (Japanese)
--help
Display usage information for the script
--passthroughauth (variable VI_PASSTHROUGHAUTH)
Attempt to use pass-through authentication
--passthroughauthpackage (variable VI_PASSTHROUGHAUTHPACKAGE, default 'Negotiate')
Pass-through authentication negotiation package
--password (variable VI_PASSWORD)
Password
--portnumber (variable VI_PORTNUMBER)
Port used to connect to server
--protocol (variable VI_PROTOCOL, default 'https')
Protocol used to connect to server
--savesessionfile (variable VI_SAVESESSIONFILE)
File to save session ID/cookie to utilize
--server (variable VI_SERVER, default 'localhost')
VI server to connect to. Required if url is not present
--servicepath (variable VI_SERVICEPATH, default '/sdk/webService')
Service path used to connect to server
--sessionfile (variable VI_SESSIONFILE)
File containing session ID/cookie to utilize
--url (variable VI_URL)
VI SDK URL to connect to. Required if server is not present
--username (variable VI_USERNAME)
Username
--verbose (variable VI_VERBOSE)
Display additional debugging information
--version
Display version information for the script
Note: There are some additional command line arguments that can be provided upon runtime: --debug, --output and --vmlist. --vmlist is the only parameter required. The rest will have defaults if not explicitly defined.
--dryrun
--output
--config_dir
The input to this script is a file that contains the display name of the virtual machine(s) separated by a newline. When creating this file on a non-Linux/UNIX system, you may introduce a ^M character which may cause the script to misbehave. To ensure that this does not occur, please create the file on the ESX/ESXi host.
Provided is a sample of what the file should contain:
[vi-admin@scofield ~]$ cat virtual_machine_backup_list
VIMA
vMA-2
DISK_FORMAT | 2gbsparse |
ADAPTER_FORMAT | buslogic |
LOG_LEVEL | info |
VM_SNAPSHOT_MEMORY | 0 |
VM_SNAPSHOT_QUIESCE | 0 |
VMS_TO_BACKUP | (VIMA,vMA-2) |
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --vmlist virtual_machine_backup_list
[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
11-13-2009 23:34:53 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-13-2009 23:34:53 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-13-2009 23:34:53 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-13-2009 23:34:53 -- info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
11-13-2009 23:34:53 -- info: CONFIG - DISK_BACKUP_FORMAT = 2gbsparse
11-13-2009 23:34:53 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-13-2009 23:34:53 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-13-2009 23:34:53 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-13-2009 23:34:53 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-13-2009 23:34:53 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-13
11-13-2009 23:34:53 -- info: CONFIG - VM_VMDK_FILES = all
11-13-2009 23:34:58 -- info: Initiate backup for VIMA found on himalaya.primp-industries.com
11-13-2009 23:35:18 -- info: VIMA has 1 VMDK(s)
11-13-2009 23:37:37 -- info: Backup completed for VIMA!
11-13-2009 23:37:37 -- info: Initiate backup for vMA-2 found on himalaya.primp-industries.com
11-13-2009 23:37:47 -- info: vMA-2 has 1 VMDK(s)
11-13-2009 23:40:07 -- info: Backup completed for vMA-2!
11-13-2009 23:40:10 -- info: ============================== ghettoVCBg2 LOG END ==============================
VIMA
VM_BACKUP_DATASTORE | dlgCore-NFS-bigboi.VM-Backups |
VM_BACKUP_DIRECTORY | WILLIAM_BACKUPS |
DISK_FORMAT | zeroedthick |
ADAPTER_FORMAT | buslogic |
LOG_LEVEL | debug |
VM_SNAPSHOT_MEMORY | 1 |
VM_SNAPSHOT_QUIESCE | 1 |
VM_VMDK_FILES | VIMA_1.vmdk |
vMA-2
VM_BACKUP_DATASTORE | dlgCore-NFS-bigboi.VM-Backups |
VM_BACKUP_DIRECTORY | vGHETTO_BACKUPS |
DISK_FORMAT | thin |
ADAPTER_FORMAT | lsilogic |
LOG_LEVEL | info |
VM_SNAPSHOT_MEMORY | 0 |
VM_SNAPSHOT_QUIESCE | 0 |
VM_VMDK_FILES | all |
1. Create folder to hold individual VM backup policies (can be named anything):
[vi-admin@scofield ~]$ mkdir backup_config
2. Create individual VM backup policies for each VM and ensure each file is named exactly as the display name of the VM being backed up (use provided template to create duplicates):
[vi-admin@scofield backup_config]$ cp ghettoVCBg2-vm_backup_configuration_template VIMA
[vi-admin@scofield backup_config]$ cp ghettoVCBg2-vm_backup_configuration_template vMA-2
4. Using a preferred editor (nano or vi), edit each VM backup policy file
Listing of VM backup policy within backup configuration directory
[vi-admin@scofield backup_config]$ ls
ghettoVCBg2-vm_backup_configuration_template VIMA vMA-2
5. Pass in the value, backup_config, into the --config_dir parameter
(e.g.)
--config_dir backupConfigs
Example of two distinct VM backup policies will be applied to VIMA and vMA-2
Backup policy for "VIMA" (backup only 1 specific VMDKs)
[vi-admin@scofield backup_config]$ cat VIMA
VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups"
VM_BACKUP_DIRECTORY = "WILLIAM_BACKUPS"
VM_BACKUP_ROTATION_COUNT = "3"
DISK_BACKUP_FORMAT = "zeroedthick"
ADAPTER_FORMAT = "buslogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
VM_SNAPSHOT_MEMORY = "1"
VM_SNAPSHOT_QUIESCE = "1"
LOG_LEVEL = "debug"
VM_VMDK_FILES = "VIMA_1.vmdk"
Backup policy for VM "vMA-2" (backup all VMDKs found)
VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups"
VM_BACKUP_DIRECTORY = "vGHETTO_BACKUPS"
VM_BACKUP_ROTATION_COUNT = "3"
DISK_BACKUP_FORMAT = "thin"
ADAPTER_FORMAT = "lsilogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
VM_SNAPSHOT_MEMORY = "0"
VM_SNAPSHOT_QUIESCE = "0"
LOG_LEVEL = "info"
VM_VMDK_FILES = "all"
The following is an output of the script when utilizing the new --config_dir parameter with debug mode enabled.
Note: --vmlist is still a required parameter with the exception that backup policy files must exist for each VM in the list else the VM will be ignored in the backup process.
Execution of backup
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --config_dir backup_config --vmlist virtual_machine_backup_list
[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
11-14-2009 00:18:41 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-14-2009 00:18:45 -- info: CONFIG - USING CONFIGURATION FILE = VIMA
11-14-2009 00:18:45 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:18:45 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:18:45 -- info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
11-14-2009 00:18:45 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
11-14-2009 00:18:45 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-14-2009 00:18:45 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:18:45 -- info: CONFIG - VM_SNAPSHOT_MEMORY = YES
11-14-2009 00:18:45 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
11-14-2009 00:18:45 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:18:45 -- info: CONFIG - VM_VMDK_FILES = VIMA_1.vmdk
11-14-2009 00:18:45 -- info: Initiate backup for VIMA found on himalaya.primp-industries.com
11-14-2009 00:18:55 -- debug: VIMA original powerState: poweredOn
11-14-2009 00:18:55 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2009-11-14" for VIMA
11-14-2009 00:19:45 -- info: VIMA has 2 VMDK(s)
11-14-2009 00:19:45 -- debug: findVMDKFile: Found VMDK File: VIMA_1.vmdk
11-14-2009 00:19:45 -- debug: backupVMDK: Backing up "[http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk" to "[http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14/VIMA_1.vmdk"
11-14-2009 00:19:45 -- debug: backupVMDK: Signal copyThread to start
11-14-2009 00:19:45 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
11-14-2009 00:19:45 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
11-14-2009 00:19:45 -- debug: CopyThread: Start backing up VMDK(s) ...
11-14-2009 00:19:52 -- debug: copyTask: send copySuccess message ...
11-14-2009 00:19:52 -- debug: copyTask: waiting for next job and sleep ...
11-14-2009 00:20:46 -- debug: backupVMDK: Successfully completed backup for [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk Elapsed time: 1 min
11-14-2009 00:20:49 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2009-11-14" for VIMA
11-14-2009 00:20:51 -- debug: checkVMBackupRotation: Starting ...
11-14-2009 00:20:57 -- debug: Purging [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-13--3 due to rotation max
11-14-2009 00:21:05 -- info: Backup completed for VIMA!
11-14-2009 00:21:05 -- debug: reConfigureBackupParams: VM - vMA-2
11-14-2009 00:21:05 -- info: CONFIG - USING CONFIGURATION FILE = vMA-2
11-14-2009 00:21:05 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:21:05 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:21:05 -- info: CONFIG - VM_BACKUP_DIRECTORY = vGHETTO_BACKUPS
11-14-2009 00:21:05 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
11-14-2009 00:21:05 -- info: CONFIG - ADAPTER_FORMAT = lsilogic
11-14-2009 00:21:05 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:21:05 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-14-2009 00:21:05 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-14-2009 00:21:05 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:21:05 -- info: CONFIG - VM_VMDK_FILES = all
11-14-2009 00:21:05 -- info: Initiate backup for vMA-2 found on himalaya.primp-industries.com
11-14-2009 00:21:15 -- info: vMA-2 has 1 VMDK(s)
11-14-2009 00:23:22 -- info: Backup completed for vMA-2!
11-14-2009 00:23:25 -- info: ============================== ghettoVCBg2 LOG END ==============================
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --config_dir backup_config --vmlist virtual_machine_backup_list --dryrun 1
[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
11-14-2009 00:26:29 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-14-2009 00:26:34 -- info: CONFIG - USING CONFIGURATION FILE = VIMA
11-14-2009 00:26:34 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
11-14-2009 00:26:34 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
11-14-2009 00:26:34 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-14-2009 00:26:34 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_MEMORY = YES
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:26:34 -- info: CONFIG - VM_VMDK_FILES = VIMA_1.vmdk
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO VIMA ----------
11-14-2009 00:26:34 -- info: DEBUG - Host Build: VMware ESX 4.0.0 build-164009
11-14-2009 00:26:34 -- info: DEBUG - Host: himalaya.primp-industries.com
11-14-2009 00:26:34 -- info: DEBUG - Virtual Machine: VIMA
11-14-2009 00:26:34 -- info: DEBUG - VM ConfigPath: [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA.vmx
11-14-2009 00:26:34 -- info: DEBUG - VMX File: VIMA.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupConfigPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14/VIMA.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14
11-14-2009 00:26:34 -- info: DEBUG - VM Datastore: himalaya-local-SAS.Savvio
11-14-2009 00:26:34 -- info: DEBUG - VMDK(s):
11-14-2009 00:26:34 -- info: DEBUG - [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA.vmdk
11-14-2009 00:26:34 -- info: DEBUG - [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO VIMA ----------
11-14-2009 00:26:34 -- debug: reConfigureBackupParams: VM - vMA-2
11-14-2009 00:26:34 -- info: CONFIG - USING CONFIGURATION FILE = vMA-2
11-14-2009 00:26:34 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIRECTORY = vGHETTO_BACKUPS
11-14-2009 00:26:34 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
11-14-2009 00:26:34 -- info: CONFIG - ADAPTER_FORMAT = lsilogic
11-14-2009 00:26:34 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:26:34 -- info: CONFIG - VM_VMDK_FILES = all
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO vMA-2 ----------
11-14-2009 00:26:34 -- info: DEBUG - Host Build: VMware ESX 4.0.0 build-164009
11-14-2009 00:26:34 -- info: DEBUG - Host: himalaya.primp-industries.com
11-14-2009 00:26:34 -- info: DEBUG - Virtual Machine: vMA-2
11-14-2009 00:26:34 -- info: DEBUG - VM ConfigPath: [http://himalaya-local-SAS.Constellation|http://himalaya-local-SAS.Constellation] vMA-2/vMA-2.vmx
11-14-2009 00:26:34 -- info: DEBUG - VMX File: vMA-2.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupConfigPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] vGHETTO_BACKUPS/vMA-2/vMA-2-2009-11-14/vMA-2.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] vGHETTO_BACKUPS/vMA-2/vMA-2-2009-11-14
11-14-2009 00:26:34 -- info: DEBUG - VM Datastore: himalaya-local-SAS.Constellation
11-14-2009 00:26:34 -- info: DEBUG - VMDK(s):
11-14-2009 00:26:34 -- info: DEBUG - [http://himalaya-local-SAS.Constellation|http://himalaya-local-SAS.Constellation] vMA-2/vMA-2.vmdk
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO vMA-2 ----------
11-14-2009 00:26:37 -- info: ============================== ghettoVCBg2 LOG END ==============================
0Q: I'm having some issues or would like to provide feedback
0A: Please join the ghettoVCBg2 Group to post your comments/questions
1Q: I'm using ESXi and free licensed version ( I did not pay for anyting ), will this script still work?
1A: No, you need to have a licensed version of ESXi to use this script.
2Q: I have a VM listed in my backup file but no backup of that VM took place. The log stated that the VM could not be found. How could this be? I know that it’s running on host X…
2A: Ensure the ESX/ESXi host is being managed by VMware VIMA/vMA (sudo vifp listservers to verify)
3Q: Can I schedule backups to take place hourly, daily, monthly, yearly?
3A: Yes, do a search online for crontab.
4Q: I want to schedule my backup on Windows, how do I do this?
4A: Do a search for
plink. Make sure you have paired SSH keys setup between your Windows system and VIMA/vMA.
5Q: Do I have to use VMware VIMA/vMA to initiate the backup?
5A: Yes, vi-fastpass is being used and only VIMA/vMA provides this capability. However, you can kickoff the process from another host (*nix/win)
4Q: I only have a single ESXi host. I want to take backups and store them somewhere else. The problem is: I don't have NFS, iSCSI nor FC SAN. What can I do?
4A: You can use local storage to store your backups assuming that you have enough space on the destination datastore. Afterwards, you can use scp (WinSCP/FastSCP) or vifs (vCLI) to transfer the backups from the ESXi host to your local desktop.
5Q: I’m pissed; the backup is taking too long. My datastore is of type X?
5A: YMMV, take a look at your storage configuration and make sure it is optimized.
6Q: I noticed that the backup rotation is occurring after a backup. I don't have enough local storage space, can the process be changed?
6A: This is primarily done to ensure that you have at least one good backup in case the new backup fails. If you would like to modify the script, you're more than welcome to do so.
7Q: What is the best storage configuration for datastore type X?
7A: Search the VMTN forums; there are various configurations for the different type of storage/etc.
8Q: I want to setup NFS Server to run my backups. Which is the best and should it be virtual or physical?7
8A: Please refer to answer 7A. From experience, we’ve seen physical instances of NFS servers to be faster than their virtual counterparts. As always, YMMV.
9Q: When I try to use the script I get an error stating: "restricted version". What the heck is this?
9A: Please take a look at the ESX/ESXi Version Support Table in this document. The reason is explained here.
10Q: I'm getting error X when using the script or I'm not getting any errors, the backup didn’t even take place. Oh what could be wrong?
10A: First, you'll want to check the log (default /tmp/ghettoVCBg2) and see if there are any obvious errors. If not, please provide the entire output of the log, a short description of your environment and the type of VM(s) you're trying to backup—I will try to provide some assistance.
11Q: I have VMs that have snapshots. I want to back these things up but the script doesn’t let me do it. How do I fix that?
11A: VM snapshots are not meant to be kept for long durations. When backing up a VM that contains a snapshot, you should ensure all snapshots have been committed prior to running a backup. No exceptions will be made…ever.
12Q: I would like to restore from backup, what is the best method?
12A: The restore process will be unique for each environment and should be determined by your backup/recovery plans. At a high level you have the option of mounting the backup datastore and registering the VM in question or copy the VM from the backup datastore to the ESX/ESXi host. The latter is recommended so that you're not running a VM living on the backup datastore or inadvertently modifying your backup VM(s).
13Q: When using the experimental feature: Individual VM backup policy, if I don't have a policy will it use the default configurations defined in the script?
13A: No, this use case will not be supported, ever. If you decide to use the per VM policy, then the global configuration in the script is ignored. If a VM policy config is not found, the VM will not be backed up even if it's in the –-vmlist.
14Q: In vMA 4.0, vCenter target will be supported, can I just add vCenter target and assume this will work?
14A: At the moment, this is not a supported feature. You will need to add each ESX/ESXi host for the vi-fastpass library to be able to locate the host(s).
15Q: When I try to run the script I get: "-bash: ./ghettoVCBg2.pl: Permission denied", what is wrong?
15A: You need to change the permission on the script to be executable, chmod +x ghettoVCBg2.pl
16Q: Where can I download the latest version of the script?
16A: The latest version is available on this page at the bottom. To get the version of the script run
./ghettoVCBg2.pl --version
17Q: I'm running into an issue when I backup my VMs using NFS on Linux, I'm able to either get 1 successful backup and all subsequent backups fail OR I'm having issues where snapshots are not being removed upon completition and future backups fail.
17A: This problem/issue has been reported by quite a few users in the community and only affects a very small percentage of users overall under a specific configuration. The problem as I understand it today which is still an on-going investigation by myself and the community is as follows:
Setup: NFS on Linux where the underlying volume is EXT3 filesystem
and backups are failing primarily around larger sized VMs ( > 150GB).
The issue is during the final phase of a VM being backed-up in the rotation function where a request is sent out to the NFS server to either A) rotate/rename a directory B) delete an old backup. There is either a disconnect or delay that severely affects the backup process that may lead to a whole slew of "unexpected" behavior from having only 1 backup complete successful and all subsquent backups failing/etc.
First off, there is nothing wrong with the script from what we've found it and has to do with NFS server and specifically the volume that the backup datastore is hosted on. Thanks to community user
ralfoertner who had a simliar problem using
ghettoVCBg2 using Debian with EXT3 base filesystem for his NFS volume. He decided to do some additional troubleshooting to further isolate the problem and actually found by changing the underlying filesystem from EXT3 to XFS that the issue went away and resolve all issues he had ran into regarding an NFS on Linux backup solution.
Now, we're still 100% sure why the above solution has resolved the problem or what the actual problem is but I'll explain the current theory:
EXT3 is a journaled filesystem, you can do a search online for more information. There's a few advantages with EXT3 from it's predcessor EXT2, but one difference is in the approach it takes with journaling as explained here in greater detail:
http://www.ibm.com/developerworks/library/l-fs7.html
Basically EXT2 does only "metadata journling" whereas EXT3 does both data and metadata journaling, this would ensure that all changes to the filesystem where it's data or metadata is consistent in the event of an unexpected system crash. With EXT3, there are 3 journaling options: (
data=writeback mode,data=ordered mode,data=journal mode ) and by default it uses
data=ordered mode to ensure the consistency as described from above.
One easy method to check what mount options are configured, on a Linux system you can check /proc/mounts:
[vi-admin@scofield ~]$ cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw,data=ordered 0 0
/dev /dev tmpfs rw 0 0
/proc /proc proc rw 0 0
/sys /sys sysfs rw 0 0
/proc/bus/usb /proc/bus/usb usbfs rw 0 0
devpts /dev/pts devpts rw 0 0
/dev/VolGroup00/var /var/log ext3 rw,data=ordered 0 0
/dev/sda1 /boot ext3 rw,data=ordered 0 0
tmpfs /dev/shm tmpfs rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
So now what does this have to do with XFS and EXT3? With the defaults of EXT3, it may not always give you the best performance due to the extra overhead of the way it does its journaling, if you were to change the options to
data=writeback mode, you'll notice that this is simliar journaling method found in
XFS,JFS and ResierFS as described here:
http://www.ibm.com/developerworks/library/l-fs8.html
I believe that there are some additional overheads that maybe caused by using NFS on EXT3, which may explain the issue that users are seeing and that moving from EXT3 to XFS has "seemed" to solve the issues for those who have implemented the change. Though, before you go out and make this change, you need to understand your environment and what this change would exactly mean, please refer to differences in EXT2/XFS and EXT3, especially around a power interruption or system crash.
In conclusion, I'm not recommending one way or the other, it really depends on your environment. Though this has shown to resolve the issue as users have implemented to get around this problem, but it's definitely worth while to really understand the problem in your environment and figure out why and decide on the best solution. I would also recommend users perform their own tests to further isolate the problem in their environment and try local storage, using iSCSI to see if they can pin-point the problem.
If you have found this script or other scripts to be useful and would like to contribute back, please click
here to donate! Thank You
07/20/10 - build 4.3
Enhancements:
Bug Fixes:
03/27/10 - build 4.2
Enhancements:
03/27/10 - build 4.1
Enhancements:
02/24/09 - build 4.1
Bug Fixes:
05/26/09 - build 2.5.1
Enhancements:
Bug Fixes:
04/25/09 - build 2.5
Enhancements:
Bug Fixes:
02/21/09 - build 2.0
Initial release
You did not carefully read my last comment. I know it's a little hard to follow the threads in the comments section, but I do expect that users read through my replies carefully and answer all questions posed.
The comment referred to your manual execution of the backup:
Your execution:
vmkfstools --server 134.134.20.2 --username root -i "\[datastore1\] vMA-ovf-4.0.0-161993/vMA-ovf-4.0.0-161993.vmdk" -d zeroedthick -a LSIlogic "\[datastore2\] vma/vma.vmdk"
and I'm saying that your *-a" (adapter) format is incorrect. It should be lsilogic and NOT LSIlogic
Try this command on vMA:
vmkfstools --server 134.134.20.2 --username root -i "\[datastore1\] vMA-ovf-4.0.0-161993/vMA-ovf-4.0.0-161993.vmdk" -d zeroedthick -a lsiogic "\[datastore2\] vma/vma.vmdk"
The other question I have about the VM you're trying to clone is it looks like it's the default vMA 4.0 VMDK that's downloaded from VMare's site? If that is the case, the VMDK may not be supported format directly on ESX, generally you'll need to import which does a conversion OR using something like VMware converter.
I would recommend you use an actual VMDK that is fully functional or if you want to do testing, create a dummy VM on the ESXi host itself.
Please report back on the comments made from above
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Tnaks again William,
the Vm we are trying to clone manually was imported from the .OVF of vmware and it was working fine. We turn off the Vm and test to clone it.. We don´t have more Vm´s that we can turn off..
The previous LSIlogic was a copy/paste failure, i had written lsilogic really..
I run it again with the same result:
# vmkfstools --server 134.134.20.2 --username root -i "[datastore1] vMA-ovf-4.0.0-161993/vMA-ovf-4.0.0-161993.vmdk" -d zeroedthick -a lsilogic "[datastore2] vma/vma.vmdk"
Enter password:
Unable to clone virtual disk : General fault caused by file.
One of the parameters supplied is invalid
On all our Esxi server, we have open SSH and telnet to run GhettoVCB with Plink from Windows, any problem with this?
Thanks
You're executing this on vMA 4.0 right? and not on ESXi?
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Of course William,
i am executing this on Vma 4.0, not on Esxi.. But all servers managed by this Vma are Esxi 3.5 U2 and U3. I am using the same Vma to run vmwarevSphereHealthCheck.pl without any problem
Thanks
That's quite odd ... basically what we need to do is to be able to issue the vmkfstools clone from vMA 4.0 ....if you can't manually do that, then there's no reason to troubleshoot further into the backup script. I'm just wondering if it's the VM that you're trying to clone (which I agree should not have issues as I've done clones of vMA many times) OR there's something funny that's going on in your environment. Once you can do this, then you should be able to use the script w/o issues.
If you have a lower priority system, you can take a snapshot fo the VM and see if you can clone, if you can start the clone process w/o issues. You can ctrl+c the clone and remove the snapshot and commit it back.
This is what we need to do as next steps, else I won't be able to help more.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I created a new vm with a small 2 Gb disk and the result is the same, i tried with absolute and relative paths..
# vmkfstools --server 134.134.20.2 --username root -i "[datastore1] /vmfs/volumes/4975853e-4f77538d-9ce8-002219860e82/test/test.vmdk" -d zeroedthick -a lsilogic "[datastore2] /vmfs/volumes/493fc889-4d865435-e8af-002219860e84/prueba/prueba.vmdk"
Enter password:
Unable to clone virtual disk : General fault caused by file.
The system cannot find the file specified
Sorry you're doing something wrong or you're missing something.
When you're issuing the clone operation, it accepts the following format:
"\[datastore\] vmfolder/vm.vmdk"
Please take screenshots of the available datastores on your ESXi host and also on the VM that is to be cloned, click into it's configuration and select the Hard Disk and take a screenshot, to the right of that, I should see a box containing the diskFile path to the VMDK
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
just to let u know, i've seen the same thing as jlorenzo with a bunch of esx 3.5 u3 hosts.
different vms, newest vma, it was all correct... i could not get rid of the problem.
i didn't invetigate a lot of time. we just decided to try again after an upgrde to vphere 4...
Have you guys tried it with VIMA 1.0? I know there's been significant amount of API changes and it would not surprise me if there might be something funky going on. The last time I tested against ESX 3.5 was on Update4 which worked using vMA 4.0 (unfortunately the testing was very limited) and I don't have additional resources to host any 3.5 systems.
I agree, it's definitely better to move to vSphere, there are huge benefits but I can see why some may want to stay on 3.5 ...specifically on U2/U3
I think for those that don't want to move to ESX(i) 4.0, a better backup system is probably using ghettoVCB.sh
Thanks for the comments
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
It´s possible use ghettoVCB from Vma?, we are using it from Windows with plink..
We are not using vsphere version for hardware requirements that are not met.
Thanks
No, ghettoVCB runs on either the ESX Service Console or ESXi unsupported console. Please go through documentation carefully for usage. You can however issue the backup from a Windows system using plink, again this is referenced in the 'additional info' in the documentation.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
We are using ghettovcb from Windows with plink, but we want to improve performance with ghettovcbg2 from Vma 4 without success at this time.
Could you see the screenshots?
I've not really tested performance between the two scripts, with g2, you're utilizing the API's to make the backup which might be slightly slower than issuing the commands from the Service Console of ESX or unsupported busybox console with ESXi. Performance is always an iffy subject and much of it will depend on the setup, you may or may not get better performance. The real point of g2 is to provide similar backup functionality of ghettoVCB without the require the use of SSH console, more importantly on ESXi where this console is unsupported.
As mentioned earlier from another user, you may or may not see issues working with u2/u3 ... I would recommend either using VIMA 1.0 and see if you still have these issues or moving to ESXi 4.0 (though this script will not work w/o proper licenses)
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
i currently fixed the issue by adding this function and
relogin on each backup iteration
sub loginESX {
&log("info", "loginESX: Login by vi-fastpass to: " . $host);
VIFPLib::login_by_fastpass($host);
my $viuser = vifplib_perl::CreateVIUserInfo();
my $vifplib = vifplib_perl::CreateVIFPLib();
eval { $vifplib->QueryTarget($host, $viuser); };
if(!$@) {
$host_username = $viuser->GetUsername();
$host_password = $viuser->GetPassword();
}
}
sub backUpVMs {
my @vm_backup_list = @_;
my $vm_backup_dir;
my $snapshot_name;
my $original_vm_state;
my $returnval;
my $count = 0;
foreach my $vm_name (@vm_backup_list) {
if($success_backups{$vm_name} ne -1 && $success_backups{$vm_name} ne 1) {
loginESX(); <==== ADDED
my $vm_view = Vim::find_entity_view(view_type => 'VirtualMachine',filter => {"config.name" => $vm_name});
if(defined($vm_view)) {
this way each backup start with a login into the ESX server, and failed connections are restored.
this is now running since 23/12/2009 without any problem. before i couldn't run it 2/3 times without getting errors.
luc
Interesting .... I'm wondering if this only affects ESX(i) 3.5 hosts? This is the first time I've heard of this where you may have a potential connection issue as this handled automatically with the most updated version of the script. Good to hear you've found a work around, let me know if this does fix the issue permanently.
Thanks for the info
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
With VIMA 1.0 we obtain identical results, the .vmx file is copied in the target datastore, the snapshot is created and after one minute is deleted without copy it to the target..
Any idea?
Thanks
From all the comments/replies, it sounds like you cannot use either VIMA 1.0 or vMA 4.0 to clone a VMs VMDK that's running on a supposedly ESX(i) 3.5u2/u3 hosts? I can see why the .vmx gets copied because it uses vifs which is basically utilizing HTTP to do the transfer, if you're using the free version of ESXi and you're not on ESXi 3.5u2 or u3, then the APIs are locked down.
I remember asking you much earlier in the initial replies what version and you stated you were running ESXi 3.5 u2 or u3, can you please confirm that is the case and you've not updated or patched? This would make sense if you're outside of either of those builds causing you not to use the script as it requires a properly licensed version of ESXi.
Please confirm
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I am using ESXi 3.5 Update 2 Foundation version in one server and Free Esxi 3.5 Update 3 in another 3 servers without any additional patch or updates.
Thanks
William, I have read the information about the license requirement to use this script, but i still have doubts..
what is the minimum required license to run ghettoVCBg2 in a Esxi 3.5 Update 4 or 5? Foundation license no longer exists.. and in a Vsphere 4 server?.. i suppose that Standard license is valid like the essentials.. if it isn´t valid, what version of essentials it´s necessary, normal or plus?
Ghettovcbg2 uses vstorage APIs for backups?
Thanks
Basically any license for your ESXi host that is NOT the free version will work. I know VMware has changed and updated additional license types and it can get confusing.
Also, no, the script does not use the vStorage API which is basically a marketing term from VMware. vStorage API is composed of vSphere API (which ghettoVCBg2 uses) + VDDK (which I don't make use of).
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
In Esxi (free license and Foundation license) update 2 and 3, is not possible for me to use different disk format than thin with this script.. with zeroedthick fails, and my vm´s has the default disk format (zeroedthick) so if i use thin format in case of having to restore i will to convert the format disk again to zeroedthick..
With large size files often fails, any workaround or new version available to correct this?
Thanks
Remember the ESXi 3.5u2/u3 functionality is more of a hack due to a VMware bug and also note that their were some fixes in vmkfstools script pre vSphere. This script is primarily supporting licensed version of ESX(i) and I'm not sure why you're running into these issues but it's working for the majority of the users. I'm thinking this is more specific to u2/u3 builds than the script. I would recommend going to ESXi 4.0 as it does provide further stability and feature enhancements.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Thanks for the script, works well towards ESX datastores.
Ideal would be, if the vm files would be written towards a cifs share, in my case on mount point /mnt/backup.
Problem is, that nothing happens. The log says:
02-02-2010 17:54:03 -- info: ============================== ghettoVCBg2 LOG START ==============================
02-02-2010 17:54:03 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
02-02-2010 17:54:03 -- info: CONFIG - VM_BACKUP_DATASTORE = /mnt/backup
02-02-2010 17:54:03 -- info: CONFIG - VM_BACKUP_DIRECTORY = backups
02-02-2010 17:54:03 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
02-02-2010 17:54:03 -- info: CONFIG - ADAPTER_FORMAT = buslogic
02-02-2010 17:54:03 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
02-02-2010 17:54:03 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
02-02-2010 17:54:03 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
02-02-2010 17:54:03 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2010-02-02
02-02-2010 17:54:03 -- info: CONFIG - VM_VMDK_FILES = all
02-02-2010 17:54:09 -- info: Initiate backup for !
I know the mount is there, mount through mount -t cifs -o, username=.....etc.
Any ideas about that? Or are only ESX Datastores supported?
BK
Only valid ESX(i) datastores are supported in the form of FC/iSCSI LUN or NFS Datastore ... even though cifs can be mounted, it's not understood by the APIs.
If you want to backup to cifs share that is mounted on your ESX host, you may want to look at which is the same script but using the Service Console for classic ESX or unsupported Busybox console for ESXi. Since this run's on one of the consoles, you just need to specify a valid path.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I solved it by moving all te files to the cifs share in a seperate action.
Thanks for the script!
BK
np.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hi William,
thank you very much for your absolutely great script. I've implemented it in external locations with one ore two ESX servers and it's running very well.
But on one location I have a special effect:
The script starts and create a folder in the mounted nfs datastor. After finishing the vmx file is created, but unfortunately no vmdk file.
The script is running on vMA 4 on a ESX 3.5U5.
In the logfile is no error or warnig.
I've tried to copy the vmdk file with vmkfstools to the nfs datastor. This is working fine.
Here the output of a dryrun:
02-09-2010 11:24:36 -- info: ============================== ghettoVCBg2 LOG START ==============================
02-09-2010 11:24:36 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
02-09-2010 11:24:36 -- info: CONFIG - VM_BACKUP_DATASTORE = nfs_hee_backup
02-09-2010 11:24:36 -- info: CONFIG - VM_BACKUP_DIRECTORY = backup
02-09-2010 11:24:36 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
02-09-2010 11:24:36 -- info: CONFIG - ADAPTER_FORMAT = lsilogic
02-09-2010 11:24:36 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
02-09-2010 11:24:36 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
02-09-2010 11:24:36 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
02-09-2010 11:24:36 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2010-02-09
02-09-2010 11:24:36 -- info: CONFIG - VM_VMDK_FILES = all
02-09-2010 11:24:36 -- debug: Main: Login by vi-fastpass to: desheeesx1.xxx.com
02-09-2010 11:24:36 -- debug: copyTask: Task START
02-09-2010 11:24:36 -- debug: copyTask: waiting for next job and sleep ...
02-09-2010 11:24:39 -- debug: Main: Disconnect from: desheeesx1.xxx.com
02-09-2010 11:24:39 -- debug: Main: Login by vi-fastpass to: desheeesx2.xxx.com
02-09-2010 11:24:40 -- info: -
DRYRUN DEBUG INFO desheetst1v -
02-09-2010 11:24:40 -- info: DEBUG - Host Build: VMware ESX Server 3.5.0 build-207095
02-09-2010 11:24:40 -- info: DEBUG - Host: desheeesx2.xxx.com
02-09-2010 11:24:40 -- info: DEBUG - Virtual Machine: desheetst1v
02-09-2010 11:24:40 -- info: DEBUG - VM ConfigPath: desheetst1v/desheetst1v.vmx
02-09-2010 11:24:40 -- info: DEBUG - VMX File: desheetst1v.vmx
02-09-2010 11:24:40 -- info: DEBUG - BackupConfigPath: backup/desheetst1v/desheetst1v-2010-02-09/desheetst1v.vmx
02-09-2010 11:24:40 -- info: DEBUG - BackupPath: backup/desheetst1v/desheetst1v-2010-02-09
02-09-2010 11:24:40 -- info: DEBUG - VM Datastore: diskfiles_heeesx2_1
02-09-2010 11:24:40 -- info: DEBUG - VMDK(s):
02-09-2010 11:24:40 -- info: DEBUG - desheetst1v/desheetst1v.vmdk
02-09-2010 11:24:40 -- info: -
DRYRUN DEBUG INFO desheetst1v -
02-09-2010 11:24:40 -- debug: Main: Disconnect from: desheeesx2.xxx.com
02-09-2010 11:24:40 -- debug: Main: Calling final clean up
02-09-2010 11:24:40 -- debug: cleanUP: Thread clean up starting ...
02-09-2010 11:24:40 -- debug: cleanUp: CopyTask was never started, send copyTaskStart
02-09-2010 11:24:40 -- debug: cleanUp: Send exit to copyThread
02-09-2010 11:24:40 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit
02-09-2010 11:24:40 -- debug: copyTask: die ...
02-09-2010 11:24:40 -- debug: cleanUp: Join passed
02-09-2010 11:24:40 -- info: ============================== ghettoVCBg2 LOG END ==============================
Perhaps I'm too blind to find the error?!
Thanks for the script. Works great. Any chance of an option to shutdown the vm, take a snapshot once the vm is powered off and then power the vm back on while the backup is occuring? This would minimize the downtime while allowing a "perfect" backup.
No, that is not a feature that is available. Though you probably can easily modify the script to support this option, majority of the backup use cases is for online backups. Unless you're backing up databases, if you have VMware Tools installed and enable quiescing, you'll get a good backup.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Thanks for the great script!
It works great but I have problems with deleting old versions of backups.
I running the script vSphereManagent Assistant and an ESXI4 (actual Patchlevel).
I can backup one vm without problems but if I want to backup more than one vm in one turn the second one cannot be found.
After hours of checking I found out it only appears when old backups have to be deleted between backing up the first and the second vm.
If I delete all older backups of the first vm manually before running the script everything works fine.
I checked my nfs settings with the new nfs-faq changed a few parameters but it didn't help.
Another thing that made me think there may be a bug in the script is the following:
I made backup of a vm with rotation count set to 1. The backup crashed for some reason that do not matter here.
After the system was restarted. I started the backup again. The backup broke with a "file exists error". Could that mean there is only one backup per day possible (with rotation count set to 1)?
Another thing that made me worry is: I could not identify from looking at the backuped files that the backup had crashed. Everything looked "normal". Am I missing something?
Maybe the script could write a backup successful message into the backup directory to make sure everything is ok?
One more thank you for your excellent work!!
Hi,
This script does not support multi-vm backup (this is because the backup process is on the host side which can degrade performance, hence I'm only doing one backup at a time), you might be able to run multiple instances and if they contain different VM lists, you might be able to get it to work but this is not something I support.
Yes you're correct that if you set the rotation to 1, then at most you'll have 1 backup per day. If you want to have multiple backups, then you need to increment the rotation count, that's what the variable is there to control. I've just tested this with 2 VMs and having the count set to 1 and I did not receive any issues, I'm guessing this might have something to do with you trying to run more than one backup and potentially having duplicated VM list between the two.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hello!
You answered "This script does not support multi-vm backup..." that was a missunderstood probably because of my "german-english" I meant if I have two vms listet in my --vmlist file the deleting of the older backups does not work and as a cause of that I get an error message that the next vm in the list cannot be found.
I'm still a little confused, you should only have 1 instance of ghettoVCBg2 running, supplied with only 1 vmlist. You should not manually be deleting any files, the script will handle the rotation based on what you've configured. Can you walk me through exactly what you're seeing with the backups, how many VMs, their names and what you're seeing.
Thanks
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I have only one instance of the script running.
I start the script "./ghettoVCBg2.pl --vmlist myvms". The contents of the file myvms are two entries representing the names of two vms I wan to backup listed in separate lines. The rotation count is set to 1.
The backup starts. The first vm is backuped successfully. When the script wants to delete the older version of the backup (because rotation count is set to 1) an error occurs and the older backup is not deleted. Then the script continues and tries to backup the second vm. After a few seconds it tells the vm could not be found and it exits.
That means deleting of old backups does not work and as cause of that I can only put one vm in vmlist-file because the second one never gets backuped.
I checked the settings of nfs server. Change a few parameters but deleting still doesn't work.
Do you have sufficient capacity to test a single backup on local volume, not on NFS? I'm wondering if you're experiencing an issue that has been seen more so with ghettoVCB and NFS where deletion process is taking much longer and there is period in which the host is unable to communicate to the NFS server causing some issues during deletion. I would like to see if we can isolate the NFS server and see if you have the same problem
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I think you are right.
The problem only occurs on NFS.
I get an input/output erorr while deleting large files. I already changed the parameters on my nfs-server (debian lenny) according to your nfs-faq.
But that didn't help?
What else could I do?
Hi William,
I have been successfully using your script for some time now. Works a treat. Thanks.
I have just started to use it in a more complicated environment and it looks like there are some issues when a VM has multiple VMDKs in multiple datastores. If a VM has two identically named VMDKs (obviously, in different datastore) then it would appear the script is not able to handle that and the second copy of the file overwrites the first. The VM in question has 5 VMDKs spread over 3 datastores like so;
"[VMFS3-SAN-ROC2] NEMPOC2/NEMPOC2.vmdk
"[VMFS3-SAN-ROC3] NEMPOC/NEMPOC_1.vmdk"
"[ATA-VMFS] NEMPOC/NEMPOC_1.vmdk"
"[VMFS3-SAN-ROC3] NEMPOC/NEMPOC.vmdk"
"[ATA-VMFS] NEMPOC/NEMPOC.vmdk"
Notice that there are 2 VMDKs with name NEMPOC.vmdk and 2 with name NEMPOC_1.vmdk. Also notice that the first VMDK from the ATA datastore gets backup up to the VMFS3-SAN-ROC3 directory instead of the script creating a new directory for each datastore.
Any help would be appreciated.
Log file below;
02-24-2010 13:48:13 -- info: ============================== ghettoVCBg2 LOG START ==============================
02-24-2010 13:48:13 -- debug: Main: Login by vi-fastpass to: brirts02.example.COM
02-24-2010 13:48:13 -- debug: copyTask: Task START
02-24-2010 13:48:13 -- debug: copyTask: waiting for next job and sleep ...
02-24-2010 13:48:18 -- debug: Main: Disconnect from: brirts02.example.COM
02-24-2010 13:48:18 -- debug: Main: Login by vi-fastpass to: brirts01.example.COM
02-24-2010 13:48:22 -- debug: Main: Disconnect from: brirts01.example.COM
02-24-2010 13:48:22 -- debug: Main: Login by vi-fastpass to: rocesx07.example.COM
02-24-2010 13:48:27 -- debug: reConfigureBackupParams: VM - NEMPOC2
02-24-2010 13:48:27 -- info: CONFIG - USING CONFIGURATION FILE = NEMPOC2
02-24-2010 13:48:27 -- info: CONFIG - BACKUP_LOG_OUTPUT = /home/vi-admin/log/ghetto.log
02-24-2010 13:48:27 -- info: CONFIG - VM_BACKUP_DATASTORE = T1ISCSI1
02-24-2010 13:48:27 -- info: CONFIG - VM_BACKUP_DIRECTORY = backups
02-24-2010 13:48:27 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
02-24-2010 13:48:27 -- info: CONFIG - ADAPTER_FORMAT = lsilogic
02-24-2010 13:48:27 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
02-24-2010 13:48:27 -- info: CONFIG - VM_SNAPSHOT_MEMORY = YES
02-24-2010 13:48:27 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
02-24-2010 13:48:27 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2010-02-24
02-24-2010 13:48:27 -- info: CONFIG - VM_VMDK_FILES = all
02-24-2010 13:48:27 -- info: Initiate backup for NEMPOC2 found on rocesx07.example.COM
02-24-2010 13:48:38 -- debug: NEMPOC2 original powerState: poweredOff
02-24-2010 13:48:38 -- info: NEMPOC2 has 5 VMDK(s)
02-24-2010 13:48:38 -- debug: backupVMDK: Backing up "[VMFS3-SAN-ROC2] NEMPOC2/NEMPOC2.vmdk" to "[T1ISCSI1] backups/NEMPOC2/NEMPOC2-2010-02-24/NEMPOC2.vmdk"
02-24-2010 13:48:38 -- debug: backupVMDK: Signal copyThread to start
02-24-2010 13:48:38 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
02-24-2010 13:48:38 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
02-24-2010 13:48:38 -- debug: CopyThread: Start backing up VMDK(s) ...
02-24-2010 14:03:44 -- debug: backupVMDK: Backup progress: Elapsed time 15 min
02-24-2010 14:08:03 -- debug: copyTask: send copySuccess message ...
02-24-2010 14:08:03 -- debug: copyTask: waiting for next job and sleep ...
02-24-2010 14:08:46 -- debug: backupVMDK: Successfully completed backup for NEMPOC2/NEMPOC2.vmdk Elapsed time: 20 min
02-24-2010 14:08:49 -- debug: backupVMDK: Backing up "[VMFS3-SAN-ROC3] NEMPOC/NEMPOC_1.vmdk" to "[T1ISCSI1] backups/NEMPOC2/NEMPOC2-2010-02-24/VMFS3-SAN-ROC2/NEMPOC_1.vmdk"
02-24-2010 14:08:49 -- debug: backupVMDK: Signal copyThread to start
02-24-2010 14:08:49 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
02-24-2010 14:08:49 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
02-24-2010 14:08:49 -- debug: CopyThread: Start backing up VMDK(s) ...
02-24-2010 14:12:16 -- debug: copyTask: send copySuccess message ...
02-24-2010 14:12:16 -- debug: copyTask: waiting for next job and sleep ...
02-24-2010 14:12:51 -- debug: backupVMDK: Successfully completed backup for NEMPOC/NEMPOC_1.vmdk Elapsed time: 4 min
02-24-2010 14:12:53 -- debug: backupVMDK: Backing up "[ATA-VMFS] NEMPOC/NEMPOC_1.vmdk" to "[T1ISCSI1] backups/NEMPOC2/NEMPOC2-2010-02-24/VMFS3-SAN-ROC2/NEMPOC_1.vmdk"
02-24-2010 14:12:53 -- debug: backupVMDK: Signal copyThread to start
02-24-2010 14:12:53 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
02-24-2010 14:12:53 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
02-24-2010 14:12:53 -- debug: CopyThread: Start backing up VMDK(s) ...
02-24-2010 14:13:02 -- debug: copyTask: send copySuccess message ...
02-24-2010 14:13:02 -- debug: copyTask: waiting for next job and sleep ...
02-24-2010 14:13:54 -- debug: backupVMDK: Successfully completed backup for NEMPOC/NEMPOC_1.vmdk Elapsed time: 1 min
02-24-2010 14:13:56 -- debug: backupVMDK: Backing up "[VMFS3-SAN-ROC3] NEMPOC/NEMPOC.vmdk" to "[T1ISCSI1] backups/NEMPOC2/NEMPOC2-2010-02-24/VMFS3-SAN-ROC2/NEMPOC.vmdk"
02-24-2010 14:13:56 -- debug: backupVMDK: Signal copyThread to start
02-24-2010 14:13:56 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
02-24-2010 14:13:56 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
02-24-2010 14:13:56 -- debug: CopyThread: Start backing up VMDK(s) ...
02-24-2010 14:29:04 -- debug: backupVMDK: Backup progress: Elapsed time 15 min
02-24-2010 14:44:10 -- debug: backupVMDK: Backup progress: Elapsed time 30 min
02-24-2010 14:49:44 -- debug: copyTask: send copySuccess message ...
02-24-2010 14:49:44 -- debug: copyTask: waiting for next job and sleep ...
02-24-2010 14:50:14 -- debug: backupVMDK: Successfully completed backup for NEMPOC/NEMPOC.vmdk Elapsed time: 36 min
02-24-2010 14:50:17 -- debug: backupVMDK: Backing up "[ATA-VMFS] NEMPOC/NEMPOC.vmdk" to "[T1ISCSI1] backups/NEMPOC2/NEMPOC2-2010-02-24/VMFS3-SAN-ROC2/NEMPOC.vmdk"
02-24-2010 14:50:17 -- debug: backupVMDK: Signal copyThread to start
02-24-2010 14:50:17 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
02-24-2010 14:50:17 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
02-24-2010 14:50:17 -- debug: CopyThread: Start backing up VMDK(s) ...
02-24-2010 14:50:26 -- debug: copyTask: send copySuccess message ...
02-24-2010 14:50:26 -- debug: copyTask: waiting for next job and sleep ...
02-24-2010 14:51:18 -- debug: backupVMDK: Successfully completed backup for NEMPOC/NEMPOC.vmdk Elapsed time: 1 min
02-24-2010 14:51:21 -- debug: checkVMBackupRotation: Starting ...
02-24-2010 14:51:30 -- info: Backup completed for NEMPOC2!
02-24-2010 14:51:30 -- debug: Main: Disconnect from: rocesx07.example.COM
02-24-2010 14:51:30 -- debug: Main: Calling final clean up
02-24-2010 14:51:30 -- debug: cleanUP: Thread clean up starting ...
02-24-2010 14:51:30 -- debug: cleanUp: Send exit to copyThread
02-24-2010 14:51:30 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit
02-24-2010 14:51:30 -- debug: copyTask: die ...
02-24-2010 14:51:30 -- debug: cleanUp: Join passed
02-24-2010 14:51:30 -- info: ============================== ghettoVCBg2 LOG END ==============================
I think I found the issue, there was a typo in handling the case where a VM has it's VMDK(s) spread across multiple datastores.
I've just updated the fix, check out the latest version on Sourceforge: http://vghetto.svn.sourceforge.net/viewvc/vghetto/scripts/ghettoVCBg2.pl?view=log
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Yea, unfortunately no one has found what is causing this odd issue with NFS datastores. I don't know what is the commonality between the various NFS servers that has been having this issue. Though with that test, you just verified as many others have, that it's the NFS server and not the script.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hello !!!
It seems I have found a solution for the "deleting of old backups" problem. I changed the filesystem from ext3 to xfs.
Now deleting works fine!
I will go on testing on a few other locations and I will tell you about the results.
But I am very optimistic!
Cool. Let me know your findings, if it does indeed resolve the issue. I'm also interested to know what OS version you're using to export your NFS volume.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hello!
Success!
I had the problem of "not deleting old backups" on every location I used the script.
On all these locations I was using Debian Lenny with ext3 filesystem as NFS-Server. Now I changed the filesystem on every location to XFS and the problem is gone!
One more time a big thank you for that wonderful script!
Very cool to hear and very interesting solution. I know there are quite a few users running into similar issues with ghettoVCB, guess I'll suggest your fix and see if it resolves their issues as well.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
William,
Is there any way to have the script not do any rotation ? I would like to manually manage cleaning up backups etc.
You can comment out the function which does the rotation on line 645
&checkVMBackupRotation($vm_view,"[$VM_BACKUP_DATASTORE] $VM_BACKUP_DIRECTORY/$vm_name","[$VM_ BACKUP_DATASTORE] $VM_BACKUP_DIRECTORY/$vm_name/$vm_name\-$VM_BACKUP_DIR_NAMING_CONVENTION",$vm_name,$vmx_file);
However, the default naming scheme is , so if you decide to run more than 1 backup per day, you'll run into some issues. A work around is to change line 82
from
my $VM_BACKUP_DIR_NAMING_CONVENTION = timeStamp('YMD');
to
my $VM_BACKUP_DIR_NAMING_CONVENTION = timeStamp('MDYHMS');
which is much granular and you'll have unique backup directories created.
The other thing you could do is just set a very high number for the rotation count.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Excellent. Thanks.
Setting a high number for rotation count causes other subtle problems in my environment i.e. Currently I 'rsync' the entire backup directory to another server, and when the rotation count renames the directory numbers, rsync thinks everything has changed and copies everything to the other location. Far from optimal.
I was also wondering what setting rotation=0 actually does.
Cheers.
Hello William!
One more info for you.
You suggested that eventually using ext3 with the writethrough option would also solve the problem with the not deleting of old backups.
I have tried this but that didn't help! The only solution that worked for me was changing to xfs.
Interesting, thanks for the info. I'm still awaiting from other users that had reported this problem to see if they're seeing the same results or if they had to go XFS to really solve it.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
So what is the deal with ESXi 3u4+ that ghettoVCBg2 no longer works? It is something that VMware changed? I've read that at VMware's urging vendors have pulled the free versions of their backup tools.
This really puts those of us who run on a shoe string budget in an bit of a bind.
In order to get backups, I'm actually downgrading my ESXi servers to 3u3.
What a pain.
Thanks
Take a look at this blog post for your answer: http://vmetc.com/2009/03/31/esxi-u4-ends-free-version-read-and-write-access-from-the-rcli/
I wish VMware had their API fully functional for free ESXi ... but then again, I can see their reasoning for not doing so. At the end of the day, you pay for what you get and for a fully functionally hyperivsor for free, that's pretty damn good. You always have the option of using
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".