This tool is the follow up release of the ghettoVCB
backup utility which allows users to perform backups of virtual machines residing on ESX(i) 3.5+/4.x+
servers using methodology similar to VMware's VCB
tool. By incorporating highly constructive feedback from the VMware community and utilizing the existing VI API, ghettoVCB’s framework was completely rewritten to be harder, better, faster, stronger
.
The primary motivation for ghettoVCBg2 was to provide ESXi users with access to the utility without relying on unlocking and utilizing the unsupported console. To satisfy this requirement, the rebuilt framework takes advantage of the
VI Perl Toolkit/vSphere SDK for Perl which is present in the
VIMA/vMA virtual appliance provided by VMware. As a result, ghettoVCBg2 provides a more proper backup solution that administrators can utilize in their virtual infrastructure.
As it stands, it is a requirement that ghettoVCB be invoked directly on each of the ESX servers hosting virtual machines in need of a backup. By taking advantage of VIMA/vMA, the entire backup process becomes centralized in turn eliminating the abovementioned requirement. Additionally, by leveraging the vi-fastpass library, unattended authentication between VIMA/vMA and target ESX servers is made possible. This alleviates the need to continually pass in credentials for each ESX server.
By centralizing the backup process, individual virtual machine backup lists pertaining to their respective ESX(i) servers are no longer required. ghettoVCBg2 has the ability to identify virtual machines and their respective hosts, potentially eliminating any past confusion associated with maintaining multiple lists.
In its current configuration, the script will allow up to 3 unique backups of the Virtual Machine before it will overwrite the previous backups; this however, can be modified to fit procedures if need be. Please be diligent in running the script in a test or staging environment before using it on production live Virtual Machines; this script functions well within our environment but there is a chance that it may not fit well into other environments.
VMs running on ESX(i) 3.5u2+ or ESX(i) 4.x+
VMware VIMA 1.0 or VMware vMA 4.x
Version | Support | Alternative |
ESX 3.5u2 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 3.5u3 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 3.5u4 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESXi 3.5u2 | Works for free | ghettoVCB\ |
ESXi 3.5u3 | Works for free | ghettoVCB\ |
ESXi 3.5u4 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 4.0 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESXi 4.0 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESX 4.1 | Requires at least Foundation/Essentials license | ghettoVCB\ |
ESXi 4.1 | Requires at least Foundation/Essentials license | ghettoVCB\ |
1) Download ghettoVCBg2.tar.gz
to your VIMA 1.0/vMA 4.x system
2) Extract the contents of the tarball
[vi-admin@scofield ~]$ tar -zxvf ghettoVCBg2.tar.gz
ghettoVCBg2/
ghettoVCBg2/ghettoVCBg2-vm_backup_configuration_template
ghettoVCBg2/ghettoVCBg2.pl
3) The script should be in a directory called
ghettoVCBg2
Prior to starting, ensure that the VMware VIMA/vMA host is managing the appropriate ESX(i) 3.5u2/4.x+ host(s)
1. Add ESXi hosts to VIMA/vMA managment using vifp
A password prompt for the vi-admin account and root password to the pertinent ESXi host(s) will be presented.
[vi-admin@vima-primp-industries ~]$ sudo vifp addserver olga.resnet.ucsb.edu
root@olga.resnet.ucsb.edu's password:
[vi-admin@vima-primp-industries ~]$
2. Verify that the appropriate hosts are being managed by VIMA/vMA
[vi-admin@vima-primp-industries ~]$ sudo vifp listservers
olga.resnet.ucsb.edu
himalaya.primp-industries.com
The following variables (similar to that of ghettoVCB) need to be defined within the script prior to execution.
Defining the datastore in which the backups are stored:
Defining the name of folder that will contain backups (if folder does not exist, it will automatically be created):
Defining the backup rotation per VM:
Defining the backup disk format (zeroedthick, eagerzeroedthick, thin, and 2gbsparse are available):
Defining the adapter type for backed up VMDK (buslogic, and lsilogic are available):
Defining whether the VM is powered down or not prior to backup (1 = enable, 0 = disable):
Note: VM(s) that are powered off will not require snapshoting
Defining whether virtual machine memory is snapped and if quiescing is enabled (1 = enable, 0 = disable):
Note: By default both are disabled
Defining the detail of logging (verbose or normal):
*Defining VMDK(s) to backup from a particular VM either a list of vmdks or "all"
VM_VMDK_FILES="myvmdk.vmdk"
Experimental Support Configs:
Note: Port 25 by default
my $SEND_MAIL = "no";
my $EMAIL_HOST = "emailserver";
my $EMAIL_DOMAIN = "localhost.localdomain";
my $EMAIL_TO = 'William Lam <william@primp-industries.com.com>';
my $EMAIL_FROM = 'ghettoVCBg2 <ghettoVCBg2@primp-industries.com.com>';
Ensure that you do not edit past this section:
########################## DO NOT MODIFY PAST THIS LINE ##########################
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl
Required command option 'vmlist' not specified.
Synopsis: ./ghettoVCBg2.pl OPTIONS
Command-specific options:
--config_dir
Name of directory containing VM(s) backup configurations
--dryrun (default '0')
Set to 1 to enable dryrun mode (default 0)
--output (default '/tmp/ghettoVCBg2.log')
Full path to output log (default /tmp/ghettoVCBg2.log)
--vmlist (required)
A file containing a list of virtual machine(s) to be backed up on host
Common VI options:
--config (variable VI_CONFIG)
Location of the VI Perl configuration file
--credstore (variable VI_CREDSTORE)
Name of the credential store file defaults to <HOME>/.vmware/credstore/vicredentials.xml on Linux and <APPDATA>/VMware/credstore/vicredentials.xml on Windows
--encoding (variable VI_ENCODING, default 'utf8')
Encoding: utf8, cp936 (Simplified Chinese), iso-8859-1 (German), shiftjis (Japanese)
--help
Display usage information for the script
--passthroughauth (variable VI_PASSTHROUGHAUTH)
Attempt to use pass-through authentication
--passthroughauthpackage (variable VI_PASSTHROUGHAUTHPACKAGE, default 'Negotiate')
Pass-through authentication negotiation package
--password (variable VI_PASSWORD)
Password
--portnumber (variable VI_PORTNUMBER)
Port used to connect to server
--protocol (variable VI_PROTOCOL, default 'https')
Protocol used to connect to server
--savesessionfile (variable VI_SAVESESSIONFILE)
File to save session ID/cookie to utilize
--server (variable VI_SERVER, default 'localhost')
VI server to connect to. Required if url is not present
--servicepath (variable VI_SERVICEPATH, default '/sdk/webService')
Service path used to connect to server
--sessionfile (variable VI_SESSIONFILE)
File containing session ID/cookie to utilize
--url (variable VI_URL)
VI SDK URL to connect to. Required if server is not present
--username (variable VI_USERNAME)
Username
--verbose (variable VI_VERBOSE)
Display additional debugging information
--version
Display version information for the script
Note: There are some additional command line arguments that can be provided upon runtime: --debug, --output and --vmlist. --vmlist is the only parameter required. The rest will have defaults if not explicitly defined.
--dryrun
--output
--config_dir
The input to this script is a file that contains the display name of the virtual machine(s) separated by a newline. When creating this file on a non-Linux/UNIX system, you may introduce a ^M character which may cause the script to misbehave. To ensure that this does not occur, please create the file on the ESX/ESXi host.
Provided is a sample of what the file should contain:
[vi-admin@scofield ~]$ cat virtual_machine_backup_list
VIMA
vMA-2
DISK_FORMAT | 2gbsparse |
ADAPTER_FORMAT | buslogic |
LOG_LEVEL | info |
VM_SNAPSHOT_MEMORY | 0 |
VM_SNAPSHOT_QUIESCE | 0 |
VMS_TO_BACKUP | (VIMA,vMA-2) |
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --vmlist virtual_machine_backup_list
[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
11-13-2009 23:34:53 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-13-2009 23:34:53 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-13-2009 23:34:53 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-13-2009 23:34:53 -- info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
11-13-2009 23:34:53 -- info: CONFIG - DISK_BACKUP_FORMAT = 2gbsparse
11-13-2009 23:34:53 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-13-2009 23:34:53 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-13-2009 23:34:53 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-13-2009 23:34:53 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-13-2009 23:34:53 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-13
11-13-2009 23:34:53 -- info: CONFIG - VM_VMDK_FILES = all
11-13-2009 23:34:58 -- info: Initiate backup for VIMA found on himalaya.primp-industries.com
11-13-2009 23:35:18 -- info: VIMA has 1 VMDK(s)
11-13-2009 23:37:37 -- info: Backup completed for VIMA!
11-13-2009 23:37:37 -- info: Initiate backup for vMA-2 found on himalaya.primp-industries.com
11-13-2009 23:37:47 -- info: vMA-2 has 1 VMDK(s)
11-13-2009 23:40:07 -- info: Backup completed for vMA-2!
11-13-2009 23:40:10 -- info: ============================== ghettoVCBg2 LOG END ==============================
VIMA
VM_BACKUP_DATASTORE | dlgCore-NFS-bigboi.VM-Backups |
VM_BACKUP_DIRECTORY | WILLIAM_BACKUPS |
DISK_FORMAT | zeroedthick |
ADAPTER_FORMAT | buslogic |
LOG_LEVEL | debug |
VM_SNAPSHOT_MEMORY | 1 |
VM_SNAPSHOT_QUIESCE | 1 |
VM_VMDK_FILES | VIMA_1.vmdk |
vMA-2
VM_BACKUP_DATASTORE | dlgCore-NFS-bigboi.VM-Backups |
VM_BACKUP_DIRECTORY | vGHETTO_BACKUPS |
DISK_FORMAT | thin |
ADAPTER_FORMAT | lsilogic |
LOG_LEVEL | info |
VM_SNAPSHOT_MEMORY | 0 |
VM_SNAPSHOT_QUIESCE | 0 |
VM_VMDK_FILES | all |
1. Create folder to hold individual VM backup policies (can be named anything):
[vi-admin@scofield ~]$ mkdir backup_config
2. Create individual VM backup policies for each VM and ensure each file is named exactly as the display name of the VM being backed up (use provided template to create duplicates):
[vi-admin@scofield backup_config]$ cp ghettoVCBg2-vm_backup_configuration_template VIMA
[vi-admin@scofield backup_config]$ cp ghettoVCBg2-vm_backup_configuration_template vMA-2
4. Using a preferred editor (nano or vi), edit each VM backup policy file
Listing of VM backup policy within backup configuration directory
[vi-admin@scofield backup_config]$ ls
ghettoVCBg2-vm_backup_configuration_template VIMA vMA-2
5. Pass in the value, backup_config, into the --config_dir parameter
(e.g.)
--config_dir backupConfigs
Example of two distinct VM backup policies will be applied to VIMA and vMA-2
Backup policy for "VIMA" (backup only 1 specific VMDKs)
[vi-admin@scofield backup_config]$ cat VIMA
VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups"
VM_BACKUP_DIRECTORY = "WILLIAM_BACKUPS"
VM_BACKUP_ROTATION_COUNT = "3"
DISK_BACKUP_FORMAT = "zeroedthick"
ADAPTER_FORMAT = "buslogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
VM_SNAPSHOT_MEMORY = "1"
VM_SNAPSHOT_QUIESCE = "1"
LOG_LEVEL = "debug"
VM_VMDK_FILES = "VIMA_1.vmdk"
Backup policy for VM "vMA-2" (backup all VMDKs found)
VM_BACKUP_DATASTORE = "dlgCore-NFS-bigboi.VM-Backups"
VM_BACKUP_DIRECTORY = "vGHETTO_BACKUPS"
VM_BACKUP_ROTATION_COUNT = "3"
DISK_BACKUP_FORMAT = "thin"
ADAPTER_FORMAT = "lsilogic"
POWER_VM_DOWN_BEFORE_BACKUP = "0"
VM_SNAPSHOT_MEMORY = "0"
VM_SNAPSHOT_QUIESCE = "0"
LOG_LEVEL = "info"
VM_VMDK_FILES = "all"
The following is an output of the script when utilizing the new --config_dir parameter with debug mode enabled.
Note: --vmlist is still a required parameter with the exception that backup policy files must exist for each VM in the list else the VM will be ignored in the backup process.
Execution of backup
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --config_dir backup_config --vmlist virtual_machine_backup_list
[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
11-14-2009 00:18:41 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-14-2009 00:18:45 -- info: CONFIG - USING CONFIGURATION FILE = VIMA
11-14-2009 00:18:45 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:18:45 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:18:45 -- info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
11-14-2009 00:18:45 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
11-14-2009 00:18:45 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-14-2009 00:18:45 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:18:45 -- info: CONFIG - VM_SNAPSHOT_MEMORY = YES
11-14-2009 00:18:45 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
11-14-2009 00:18:45 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:18:45 -- info: CONFIG - VM_VMDK_FILES = VIMA_1.vmdk
11-14-2009 00:18:45 -- info: Initiate backup for VIMA found on himalaya.primp-industries.com
11-14-2009 00:18:55 -- debug: VIMA original powerState: poweredOn
11-14-2009 00:18:55 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2009-11-14" for VIMA
11-14-2009 00:19:45 -- info: VIMA has 2 VMDK(s)
11-14-2009 00:19:45 -- debug: findVMDKFile: Found VMDK File: VIMA_1.vmdk
11-14-2009 00:19:45 -- debug: backupVMDK: Backing up "[http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk" to "[http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14/VIMA_1.vmdk"
11-14-2009 00:19:45 -- debug: backupVMDK: Signal copyThread to start
11-14-2009 00:19:45 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
11-14-2009 00:19:45 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
11-14-2009 00:19:45 -- debug: CopyThread: Start backing up VMDK(s) ...
11-14-2009 00:19:52 -- debug: copyTask: send copySuccess message ...
11-14-2009 00:19:52 -- debug: copyTask: waiting for next job and sleep ...
11-14-2009 00:20:46 -- debug: backupVMDK: Successfully completed backup for [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk Elapsed time: 1 min
11-14-2009 00:20:49 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2009-11-14" for VIMA
11-14-2009 00:20:51 -- debug: checkVMBackupRotation: Starting ...
11-14-2009 00:20:57 -- debug: Purging [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-13--3 due to rotation max
11-14-2009 00:21:05 -- info: Backup completed for VIMA!
11-14-2009 00:21:05 -- debug: reConfigureBackupParams: VM - vMA-2
11-14-2009 00:21:05 -- info: CONFIG - USING CONFIGURATION FILE = vMA-2
11-14-2009 00:21:05 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:21:05 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:21:05 -- info: CONFIG - VM_BACKUP_DIRECTORY = vGHETTO_BACKUPS
11-14-2009 00:21:05 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
11-14-2009 00:21:05 -- info: CONFIG - ADAPTER_FORMAT = lsilogic
11-14-2009 00:21:05 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:21:05 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-14-2009 00:21:05 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-14-2009 00:21:05 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:21:05 -- info: CONFIG - VM_VMDK_FILES = all
11-14-2009 00:21:05 -- info: Initiate backup for vMA-2 found on himalaya.primp-industries.com
11-14-2009 00:21:15 -- info: vMA-2 has 1 VMDK(s)
11-14-2009 00:23:22 -- info: Backup completed for vMA-2!
11-14-2009 00:23:25 -- info: ============================== ghettoVCBg2 LOG END ==============================
[vi-admin@scofield ~]$ ./ghettoVCBg2.pl --config_dir backup_config --vmlist virtual_machine_backup_list --dryrun 1
[vi-admin@scofield ~]$ cat /tmp/ghettoVCBg2.log
11-14-2009 00:26:29 -- info: ============================== ghettoVCBg2 LOG START ==============================
11-14-2009 00:26:34 -- info: CONFIG - USING CONFIGURATION FILE = VIMA
11-14-2009 00:26:34 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIRECTORY = WILLIAM_BACKUPS
11-14-2009 00:26:34 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
11-14-2009 00:26:34 -- info: CONFIG - ADAPTER_FORMAT = buslogic
11-14-2009 00:26:34 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_MEMORY = YES
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:26:34 -- info: CONFIG - VM_VMDK_FILES = VIMA_1.vmdk
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO VIMA ----------
11-14-2009 00:26:34 -- info: DEBUG - Host Build: VMware ESX 4.0.0 build-164009
11-14-2009 00:26:34 -- info: DEBUG - Host: himalaya.primp-industries.com
11-14-2009 00:26:34 -- info: DEBUG - Virtual Machine: VIMA
11-14-2009 00:26:34 -- info: DEBUG - VM ConfigPath: [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA.vmx
11-14-2009 00:26:34 -- info: DEBUG - VMX File: VIMA.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupConfigPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14/VIMA.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] WILLIAM_BACKUPS/VIMA/VIMA-2009-11-14
11-14-2009 00:26:34 -- info: DEBUG - VM Datastore: himalaya-local-SAS.Savvio
11-14-2009 00:26:34 -- info: DEBUG - VMDK(s):
11-14-2009 00:26:34 -- info: DEBUG - [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA.vmdk
11-14-2009 00:26:34 -- info: DEBUG - [http://himalaya-local-SAS.Savvio|http://himalaya-local-SAS.Savvio] VIMA/VIMA_1.vmdk
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO VIMA ----------
11-14-2009 00:26:34 -- debug: reConfigureBackupParams: VM - vMA-2
11-14-2009 00:26:34 -- info: CONFIG - USING CONFIGURATION FILE = vMA-2
11-14-2009 00:26:34 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DATASTORE = dlgCore-NFS-bigboi.VM-Backups
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIRECTORY = vGHETTO_BACKUPS
11-14-2009 00:26:34 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
11-14-2009 00:26:34 -- info: CONFIG - ADAPTER_FORMAT = lsilogic
11-14-2009 00:26:34 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
11-14-2009 00:26:34 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2009-11-14
11-14-2009 00:26:34 -- info: CONFIG - VM_VMDK_FILES = all
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO vMA-2 ----------
11-14-2009 00:26:34 -- info: DEBUG - Host Build: VMware ESX 4.0.0 build-164009
11-14-2009 00:26:34 -- info: DEBUG - Host: himalaya.primp-industries.com
11-14-2009 00:26:34 -- info: DEBUG - Virtual Machine: vMA-2
11-14-2009 00:26:34 -- info: DEBUG - VM ConfigPath: [http://himalaya-local-SAS.Constellation|http://himalaya-local-SAS.Constellation] vMA-2/vMA-2.vmx
11-14-2009 00:26:34 -- info: DEBUG - VMX File: vMA-2.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupConfigPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] vGHETTO_BACKUPS/vMA-2/vMA-2-2009-11-14/vMA-2.vmx
11-14-2009 00:26:34 -- info: DEBUG - BackupPath: [http://dlgCore-NFS-bigboi.VM-Backups|http://dlgCore-NFS-bigboi.VM-Backups] vGHETTO_BACKUPS/vMA-2/vMA-2-2009-11-14
11-14-2009 00:26:34 -- info: DEBUG - VM Datastore: himalaya-local-SAS.Constellation
11-14-2009 00:26:34 -- info: DEBUG - VMDK(s):
11-14-2009 00:26:34 -- info: DEBUG - [http://himalaya-local-SAS.Constellation|http://himalaya-local-SAS.Constellation] vMA-2/vMA-2.vmdk
11-14-2009 00:26:34 -- info: ---------- DRYRUN DEBUG INFO vMA-2 ----------
11-14-2009 00:26:37 -- info: ============================== ghettoVCBg2 LOG END ==============================
0Q: I'm having some issues or would like to provide feedback
0A: Please join the ghettoVCBg2 Group to post your comments/questions
1Q: I'm using ESXi and free licensed version ( I did not pay for anyting ), will this script still work?
1A: No, you need to have a licensed version of ESXi to use this script.
2Q: I have a VM listed in my backup file but no backup of that VM took place. The log stated that the VM could not be found. How could this be? I know that it’s running on host X…
2A: Ensure the ESX/ESXi host is being managed by VMware VIMA/vMA (sudo vifp listservers to verify)
3Q: Can I schedule backups to take place hourly, daily, monthly, yearly?
3A: Yes, do a search online for crontab.
4Q: I want to schedule my backup on Windows, how do I do this?
4A: Do a search for
plink. Make sure you have paired SSH keys setup between your Windows system and VIMA/vMA.
5Q: Do I have to use VMware VIMA/vMA to initiate the backup?
5A: Yes, vi-fastpass is being used and only VIMA/vMA provides this capability. However, you can kickoff the process from another host (*nix/win)
4Q: I only have a single ESXi host. I want to take backups and store them somewhere else. The problem is: I don't have NFS, iSCSI nor FC SAN. What can I do?
4A: You can use local storage to store your backups assuming that you have enough space on the destination datastore. Afterwards, you can use scp (WinSCP/FastSCP) or vifs (vCLI) to transfer the backups from the ESXi host to your local desktop.
5Q: I’m pissed; the backup is taking too long. My datastore is of type X?
5A: YMMV, take a look at your storage configuration and make sure it is optimized.
6Q: I noticed that the backup rotation is occurring after a backup. I don't have enough local storage space, can the process be changed?
6A: This is primarily done to ensure that you have at least one good backup in case the new backup fails. If you would like to modify the script, you're more than welcome to do so.
7Q: What is the best storage configuration for datastore type X?
7A: Search the VMTN forums; there are various configurations for the different type of storage/etc.
8Q: I want to setup NFS Server to run my backups. Which is the best and should it be virtual or physical?7
8A: Please refer to answer 7A. From experience, we’ve seen physical instances of NFS servers to be faster than their virtual counterparts. As always, YMMV.
9Q: When I try to use the script I get an error stating: "restricted version". What the heck is this?
9A: Please take a look at the ESX/ESXi Version Support Table in this document. The reason is explained here.
10Q: I'm getting error X when using the script or I'm not getting any errors, the backup didn’t even take place. Oh what could be wrong?
10A: First, you'll want to check the log (default /tmp/ghettoVCBg2) and see if there are any obvious errors. If not, please provide the entire output of the log, a short description of your environment and the type of VM(s) you're trying to backup—I will try to provide some assistance.
11Q: I have VMs that have snapshots. I want to back these things up but the script doesn’t let me do it. How do I fix that?
11A: VM snapshots are not meant to be kept for long durations. When backing up a VM that contains a snapshot, you should ensure all snapshots have been committed prior to running a backup. No exceptions will be made…ever.
12Q: I would like to restore from backup, what is the best method?
12A: The restore process will be unique for each environment and should be determined by your backup/recovery plans. At a high level you have the option of mounting the backup datastore and registering the VM in question or copy the VM from the backup datastore to the ESX/ESXi host. The latter is recommended so that you're not running a VM living on the backup datastore or inadvertently modifying your backup VM(s).
13Q: When using the experimental feature: Individual VM backup policy, if I don't have a policy will it use the default configurations defined in the script?
13A: No, this use case will not be supported, ever. If you decide to use the per VM policy, then the global configuration in the script is ignored. If a VM policy config is not found, the VM will not be backed up even if it's in the –-vmlist.
14Q: In vMA 4.0, vCenter target will be supported, can I just add vCenter target and assume this will work?
14A: At the moment, this is not a supported feature. You will need to add each ESX/ESXi host for the vi-fastpass library to be able to locate the host(s).
15Q: When I try to run the script I get: "-bash: ./ghettoVCBg2.pl: Permission denied", what is wrong?
15A: You need to change the permission on the script to be executable, chmod +x ghettoVCBg2.pl
16Q: Where can I download the latest version of the script?
16A: The latest version is available on this page at the bottom. To get the version of the script run
./ghettoVCBg2.pl --version
17Q: I'm running into an issue when I backup my VMs using NFS on Linux, I'm able to either get 1 successful backup and all subsequent backups fail OR I'm having issues where snapshots are not being removed upon completition and future backups fail.
17A: This problem/issue has been reported by quite a few users in the community and only affects a very small percentage of users overall under a specific configuration. The problem as I understand it today which is still an on-going investigation by myself and the community is as follows:
Setup: NFS on Linux where the underlying volume is EXT3 filesystem
and backups are failing primarily around larger sized VMs ( > 150GB).
The issue is during the final phase of a VM being backed-up in the rotation function where a request is sent out to the NFS server to either A) rotate/rename a directory B) delete an old backup. There is either a disconnect or delay that severely affects the backup process that may lead to a whole slew of "unexpected" behavior from having only 1 backup complete successful and all subsquent backups failing/etc.
First off, there is nothing wrong with the script from what we've found it and has to do with NFS server and specifically the volume that the backup datastore is hosted on. Thanks to community user
ralfoertner who had a simliar problem using
ghettoVCBg2 using Debian with EXT3 base filesystem for his NFS volume. He decided to do some additional troubleshooting to further isolate the problem and actually found by changing the underlying filesystem from EXT3 to XFS that the issue went away and resolve all issues he had ran into regarding an NFS on Linux backup solution.
Now, we're still 100% sure why the above solution has resolved the problem or what the actual problem is but I'll explain the current theory:
EXT3 is a journaled filesystem, you can do a search online for more information. There's a few advantages with EXT3 from it's predcessor EXT2, but one difference is in the approach it takes with journaling as explained here in greater detail:
http://www.ibm.com/developerworks/library/l-fs7.html
Basically EXT2 does only "metadata journling" whereas EXT3 does both data and metadata journaling, this would ensure that all changes to the filesystem where it's data or metadata is consistent in the event of an unexpected system crash. With EXT3, there are 3 journaling options: (
data=writeback mode,data=ordered mode,data=journal mode ) and by default it uses
data=ordered mode to ensure the consistency as described from above.
One easy method to check what mount options are configured, on a Linux system you can check /proc/mounts:
[vi-admin@scofield ~]$ cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw,data=ordered 0 0
/dev /dev tmpfs rw 0 0
/proc /proc proc rw 0 0
/sys /sys sysfs rw 0 0
/proc/bus/usb /proc/bus/usb usbfs rw 0 0
devpts /dev/pts devpts rw 0 0
/dev/VolGroup00/var /var/log ext3 rw,data=ordered 0 0
/dev/sda1 /boot ext3 rw,data=ordered 0 0
tmpfs /dev/shm tmpfs rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
So now what does this have to do with XFS and EXT3? With the defaults of EXT3, it may not always give you the best performance due to the extra overhead of the way it does its journaling, if you were to change the options to
data=writeback mode, you'll notice that this is simliar journaling method found in
XFS,JFS and ResierFS as described here:
http://www.ibm.com/developerworks/library/l-fs8.html
I believe that there are some additional overheads that maybe caused by using NFS on EXT3, which may explain the issue that users are seeing and that moving from EXT3 to XFS has "seemed" to solve the issues for those who have implemented the change. Though, before you go out and make this change, you need to understand your environment and what this change would exactly mean, please refer to differences in EXT2/XFS and EXT3, especially around a power interruption or system crash.
In conclusion, I'm not recommending one way or the other, it really depends on your environment. Though this has shown to resolve the issue as users have implemented to get around this problem, but it's definitely worth while to really understand the problem in your environment and figure out why and decide on the best solution. I would also recommend users perform their own tests to further isolate the problem in their environment and try local storage, using iSCSI to see if they can pin-point the problem.
If you have found this script or other scripts to be useful and would like to contribute back, please click
here to donate! Thank You
07/20/10 - build 4.3
Enhancements:
Bug Fixes:
03/27/10 - build 4.2
Enhancements:
03/27/10 - build 4.1
Enhancements:
02/24/09 - build 4.1
Bug Fixes:
05/26/09 - build 2.5.1
Enhancements:
Bug Fixes:
04/25/09 - build 2.5
Enhancements:
Bug Fixes:
02/21/09 - build 2.0
Initial release
HI William,
I'm using your script (and it is beautiful), but i'm running into a problem that I can't figure out...
I've created a backup list with all of the vmware machines that need to be backed up (about 15). The backup starts normally and files are being copied.
After some random time, the script indicates that several virtual machines can't be found (always random). When I create a new backup list with these virtual machines (same names) and run that, the backup script can find them and starts with the backup.
This second run also usually stops with the message that one or more virtual machines can't be found. Then i have to adjust the list again until all of the virtual machines have been backed up.
I'm using 3 vmware machines, all of them are licensed properly (vmware foundation). They can all access the backup medium (nfs share).
It is not a host related problem, when i create a list of 3 virtual machines that are all on the same server, the same thing happens.
This is quite annoying because now I have to keep an eye on the backup log the entire weekend to ensure it has done it's job..
Got any advice for me?
Thanks in advance! I truly love your work!
update
I've found something interesting.. I do a 3 VM (test) job (first located on host1, second located on host2 and third located on host1).
(host1: 10.10.1.51, host2: 10.10.1.52, host3: 10.10.1.54)
1. The script starts looking at host1 and finds the first vm, backup starts and finishes. The script searches for the second VM, doesn't find it on host1
2. The script goes on to host2, finds the second vm, backup starts and finishes. The script searches for the third VM, doesn't find it on host2, goes onto host 3. Doesn't find it on host 3 and says: error: getFinalList: ERROR - Unable to locate VM. Then the script stops
So the script isn't searching for the VM on the entire vmware farm, only on the same host and the next. Not the previous hosts.
Is this behaviour normal? In a non-vmotion environment, it's just a case of putting all of the backups in the right order. But when you do Vmotion, you never know where your VM's are at a specific time.
Below you can see the two runs that i did. First was with all three of the VM's in the backup list, second is with the one that didn't go at the first run.
FIRST RUN
06-23-2010 12:50:13 -- info: CONFIG - ADAPTER_FORMAT = buslogic
06-23-2010 12:50:13 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
06-23-2010 12:50:13 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
06-23-2010 12:50:13 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
06-23-2010 12:50:13 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2010-06-23
06-23-2010 12:50:13 -- info: CONFIG - VM_VMDK_FILES = all
06-23-2010 12:50:13 -- debug: Main: Login by vi-fastpass to: 10.10.1.51
06-23-2010 12:50:13 -- debug: copyTask: Task START
06-23-2010 12:50:13 -- debug: copyTask: waiting for next job and sleep ...
06-23-2010 12:50:16 -- info: Initiate backup for Topcon found on 10.10.1.51
06-23-2010 12:50:25 -- debug: Topcon original powerState: poweredOn
06-23-2010 12:50:25 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2010-06-23" for Topcon
06-23-2010 12:50:35 -- info: Topcon has 1 VMDK(s)
06-23-2010 12:50:35 -- debug: backupVMDK: Backing up "[OpenFiler 1] Topcon/Topcon.vmdk" to "[synw1] BACKUPS/Topcon/Topcon-2010-06-23/Topcon.vmdk"
06-23-2010 12:50:35 -- debug: backupVMDK: Signal copyThread to start
06-23-2010 12:50:35 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
06-23-2010 12:50:35 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
06-23-2010 12:50:35 -- debug: CopyThread: Start backing up VMDK(s) ...
06-23-2010 13:05:52 -- debug: backupVMDK: Backup progress: Elapsed time 15 min
06-23-2010 13:06:21 -- debug: copyTask: send copySuccess message ...
06-23-2010 13:06:21 -- debug: copyTask: waiting for next job and sleep ...
06-23-2010 13:06:53 -- debug: backupVMDK: Successfully completed backup for Topcon/Topcon.vmdk Elapsed time: 16 min
06-23-2010 13:06:56 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2010-06-23" for Topcon
06-23-2010 13:06:57 -- debug: checkVMBackupRotation: Starting ...
06-23-2010 13:08:01 -- info: Backup completed for Topcon!
06-23-2010 13:08:01 -- debug: Main: Disconnect from: 10.10.1.51
06-23-2010 13:08:01 -- debug: Main: Login by vi-fastpass to: 10.10.1.52
06-23-2010 13:08:05 -- info: Initiate backup for Monitoring Proxy found on 10.10.1.52
06-23-2010 13:08:13 -- debug: Monitoring Proxy original powerState: poweredOn
06-23-2010 13:08:13 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2010-06-23" for Monitoring Proxy
06-23-2010 13:08:23 -- info: Monitoring Proxy has 1 VMDK(s)
06-23-2010 13:08:23 -- debug: backupVMDK: Backing up "[OpenFiler 2] Monitoring Proxy/Monitoring Proxy.vmdk" to "[synw1] BACKUPS/Monitoring Proxy/Monitoring Proxy-2010-06-23/Monitoring Proxy.vmdk"
06-23-2010 13:08:23 -- debug: backupVMDK: Signal copyThread to start
06-23-2010 13:08:23 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
06-23-2010 13:08:23 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
06-23-2010 13:08:23 -- debug: CopyThread: Start backing up VMDK(s) ...
06-23-2010 13:14:59 -- debug: copyTask: send copySuccess message ...
06-23-2010 13:14:59 -- debug: copyTask: waiting for next job and sleep ...
06-23-2010 13:15:31 -- debug: backupVMDK: Successfully completed backup for Monitoring Proxy/Monitoring Proxy.vmdk Elapsed time: 7 min
06-23-2010 13:15:36 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2010-06-23" for Monitoring Proxy
06-23-2010 13:15:37 -- debug: checkVMBackupRotation: Starting ...
06-23-2010 13:15:59 -- info: Backup completed for Monitoring Proxy!
06-23-2010 13:15:59 -- debug: Main: Disconnect from: 10.10.1.52
06-23-2010 13:15:59 -- debug: Main: Login by vi-fastpass to: 10.10.1.54
06-23-2010 13:16:01 -- debug: Main: Disconnect from: 10.10.1.54
06-23-2010 13:16:01 -- error: getFinalList: ERROR - Unable to locate VM: Wiki
06-23-2010 13:16:01 -- debug: Main: Calling final clean up
06-23-2010 13:16:01 -- debug: cleanUP: Thread clean up starting ...
06-23-2010 13:16:01 -- debug: cleanUp: Send exit to copyThread
06-23-2010 13:16:01 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit
06-23-2010 13:16:01 -- debug: copyTask: die ...
06-23-2010 13:16:01 -- debug: cleanUp: Join passed
06-23-2010 13:16:01 -- info: ghettoVCBg2 LOG END
SECOND RUN
06-23-2010 15:24:20 -- debug: Main: Login by vi-fastpass to: 10.10.1.51
06-23-2010 15:24:24 -- info: Initiate backup for Wiki found on 10.10.1.51
06-23-2010 15:24:47 -- debug: Wiki original powerState: poweredOn
06-23-2010 15:24:47 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2010-06-23" for Wiki
06-23-2010 15:24:57 -- info: Wiki has 1 VMDK(s)
06-23-2010 15:24:57 -- debug: backupVMDK: Backing up "[OpenFiler 2] Confluence/Confluence.vmdk" to "[synw1] BACKUPS/Wiki/Wiki-2010-06-23/Confluence.vmdk"
06-23-2010 15:24:57 -- debug: backupVMDK: Signal copyThread to start
06-23-2010 15:24:57 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
06-23-2010 15:24:57 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
06-23-2010 15:24:57 -- debug: CopyThread: Start backing up VMDK(s) ...
06-23-2010 15:40:08 -- debug: copyTask: send copySuccess message ...
06-23-2010 15:40:08 -- debug: copyTask: waiting for next job and sleep ...
06-23-2010 15:40:16 -- debug: backupVMDK: Successfully completed backup for Confluence/Confluence.vmdk Elapsed time: 15 min
06-23-2010 15:40:20 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2010-06-23" for Wiki
06-23-2010 15:40:22 -- debug: checkVMBackupRotation: Starting ...
06-23-2010 15:41:12 -- info: Backup completed for Wiki!
06-23-2010 15:41:12 -- debug: Main: Disconnect from: 10.10.1.51
06-23-2010 15:41:12 -- debug: Main: Login by vi-fastpass to: 10.10.1.52
06-23-2010 15:41:14 -- debug: Main: Disconnect from: 10.10.1.52
06-23-2010 15:41:14 -- debug: Main: Login by vi-fastpass to: 10.10.1.54
06-23-2010 15:41:15 -- debug: Main: Disconnect from: 10.10.1.54
06-23-2010 15:41:15 -- debug: Main: Calling final clean up
06-23-2010 15:41:15 -- debug: cleanUP: Thread clean up starting ...
06-23-2010 15:41:15 -- debug: cleanUp: Send exit to copyThread
06-23-2010 15:41:15 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit
06-23-2010 15:41:15 -- debug: copyTask: die ...
06-23-2010 15:41:15 -- debug: cleanUp: Join passed
Hi Joriz, this seems to be about the ext3 issue that William documented above, do you have a nas with ext3 or ext4 file system?
Regards
Christian
Hi Christian,
I do have a NAS with ext3... I also have the async option enabled.. What is still don't understand is, what is the downside of using XFS as a storage filesystem?
Regards,
Joris
Jorizzzzzz we have the same problem (i described it at the head of topic) and find solution:
Run backup only for one vm (in vmlist) = "one vm per backup session".
For many vm`s we write simple shell script, that sequential run backup job for each vm in text file.
Switched over from the ghettoVCB to this g2 and nice, like the email feature.
Wondering...
1. Will the new Paravirtual SCSI controller type be added to .pl config file someday?
2. any ETA on FAQ #14 for vCenter support? without having to add each ESXi host?
3. forgot...can we have option to use an exclude list instead of include list?
4. like ghettoVCB, can this option offer compression and NON-PERSISTANT NFS?
Overall, looking good.
Joris, we use XFS with Openfiler and no issues.
Also...make sure ASYNC is enabled on the share for better performance.
Not a script expert so...
Trying to automate...anyone know how to create a sh script on the VMA to:
1. first gather VM names from all the ESXi hosts configured, output to a file with just the true VM name, nothing else, (drop the .vmx extension and volume path)
2. then parse through that file and remove VM's from a skip file (like good ole' ghettoVCB), that would then create a finished file for use in the ghettoVCBg2 command line option --vmlist
Thinking....
vmware-cmd -H server1 -U username -P password >> vm_to_backup_listing
vmware-cmd -H server2 -U username -P password >> vm_to_backup_listing
etc...
...which would append output to the vm_to_backup_listing file.
find VM's from skip.txt, script could then:
A. remove VM names from original "vm_to_backup_listing"
B. or create new polishedlist file.
then we'd run:
$ ./ghettoVCBg2.pl --vmlist polishedlist
Done!
If anyone can figure out, I'll place you on our Christmas list.
You're better off writing your own script querying exactly what you need vs trying to hack around existing scripts
Luckily, this is a pretty simple task and I already have snippets of code that does these basic operations.
Take a look at: generateVMList.pl which can be run on vMA and connects to an ESX or ESXi host.
To execute you'll need to provide 2 inputs: --vm_exclusionfile (which is list of VMs to exclude) and --vm_outputfile (which is the name of the output file)
FYI - Donations are always welcome - http://www.virtuallyghetto.com/p/how-you-can-help.html
Thanks
BTW - Due to limited amount of free cycles and multiple projects I have going on both on and off work, I may or may not get to all the other questions and provide answers. Do know, I am very active on all 80+ scripts that I maintain, may take some time for me to get around
=========================================================================
William Lam
VMware vExpert 2009,2010
VMware scripts and resources at:
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Awesome! All Hail William!
Just so you know I was "trying" on my own script (for loop, strings, awk, etc....) but no expert.
You got all my requests, won't add anymore, promise.
Thanks again!
For those who would want a quick way to scan for all virtual machine names from all your ESXi hosts registered with VMA and place into 1 file then backup them up, see below, thanks to lamw (get his pl script above first). Make sure you place VM's to skip in exclusion file.
call this script "runbackup.sh"
#!/bin/bash
scans all registered ESXi hosts in VMA for virtual machine names to backup
#VARIABLES FOR ESXi hosts
ESXHOSTS="server1 server2 server3 server4 server5"
USER=user
PASSWORD=password
#REMOVE OLD OUTPUTFILE to REFRESH LIST
rm vm_outputfile
Getting VM info per esx host
for esxhost in $; do
./generateVMList.pl --vm_exclusionfile vm_exclusionfile --vm_outputfile vm_outputfile --server $ --username $ --password $
done
#just remove "--dryrun 1" to run the backup live!
./ghettoVCBg2.pl --vmlist vm_outputfile --dryrun 1
Hello,
i've exactly the same problem as Jorizzzzzz.
I also tested to mount my ext3 NFS share with "data=writeback" mode, but that is more worse then before.
Backup storage is a Openfiler NAS system with ext3+NFS. The VMs are at 15 - 40 GBytes.
If i remember right the problem exist since the update from ESXi 4.0 to 4.0u1.
Ditto that. Compression on the radar for g2 in 2010? NON-PERSISTANT NFS too? please... : )
Hi William, is there a chance to get a ghettoVCBg2 script for vMA 4.1 and ESXi 4.1?
Thank you.
It should work for vSphere 4.1 (ESX,ESXi and vMA 4.1) but will require a quick update to the script to allow for the new build number as it checks for 4.0.0 explicitly. I've not seen any changes in the API during the beta to cause the script to break but I've just not had the time to validate these scripts. Been having a lot of other things to take care of on my plate. If you want, you can try editing the script and just add and "OR" statement to allow for 4.1.0 and that should work, definitely test before rolling it out. I plan on pushing updates to both ghettoVCB and ghettoVCBg2 but not sure when that might be because I'm pretty booked for the next 3months with various projects.
=========================================================================
William Lam
VMware vExpert 2009,2010
VMware scripts and resources at:
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I tested it in vMA 4.1 and it doesn't work, there was some API functions (about creating user or user information) that raise an error. I'm sorry that I haven't more information now, but my vMA was in production at the time I test it and I rolled back to a 4.0 snapshot.
In vMA 4.0 and esxi 4.1 servers works fine
Did you perform an upgrade from vMA 4.0 -> 4.1 using vima-update?
afaik, it should migrate your exiting credstore to the new format and the hosts within the vi-fastpass should still work via the API calls.
If you can test with a dev environment, I would like to see what errors you're seeing. It may take me some time to upgrade our environment to vSphere 4.1, not rushing it since there are quite a few issues out in the while.
Thanks for your patience.
=========================================================================
William Lam
VMware vExpert 2009,2010
VMware scripts and resources at:
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Yes, I performed the upgrade using vima-update
After the upgrade I tested the script and it raised the error about the version so I fixed it.
After that, it raised an error with a api function, if I'm not wrong the error was in line 286 with the function vifplib_perl::CreateVIUserInfo()
To stay sure, I unregistered and registered again my esxi servers in vma and I tested vi-fastpass but nothing change (the error still there)
Yea, I actually had a moment to take a look and looks like VMware updated the vi-fastpass client libs and did not mention anything in their release notes. They did not even provide proper backwards compatibility mappings to their methods which is why you're seeing an error with that function call. I'll need to spend some time updating the script but it's not looking too pretty at the moment =/
Worse case, I'll have to put up a 4.1 specific version which is not ideal from a code management perspective. I'll see if I can get something out this weekend, but for now I would recommend staying on vMA 4.0. With respect to ghettoVCBg2 and vMA 4.1, there are 0 updates ... so there's no benefits in updating if you're just using it for this script.
This vSphere 4.1 release in general was not a smooth one by far .... there's been so many bugs, issues incompatibilities/etc. been a pain in the ass
Thanks for reporting the issue early and stay tuned.
=========================================================================
William Lam
VMware vExpert 2009,2010
VMware scripts and resources at:
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Firstly, thanks for what looks to be a very useful tool, and particularly for a speedy update to make it work with vMA4.1, which I installed yesterday and then cursed because the script didn't work...
I have one problem so far and a couple of questions:
As with another user above, I'm finding that if I configure the script to use either zeroedthickor eagerzeroedthick, it doesn't back up the vmdk file, but if I configure the same VM to back up using 'thin', it works as expected. I have no problem with leaving it like that, as long as there are no disadvantages to doing so, or any problems that will crop up later? (haven't got as far as testing restores yet) A colleague has also been having exactly the same problem with ghettoVCB version 1 with a standalone ESXi box.
My hardware is a pair of ESX 3.5 hosts, with vCenter 2.5 - all licensed, and running on vMA 4.1. VMs are all stored on an iSCSI array, and using an NFS datastore for backups. The test VM has no snapshots and was powered off.
Questions next (I've tried searching but haven't found specific answers):
Is there actually a difference with setting lsilogic vs buslogic in the configuration? I always thought that a virtual disk was a virtual disk, and that just denoted what virtual SCSI card would be emulated in the guest? What does changing the controller type actually do?
As well as my main ESX boxes, I also have an ESXi box for testing. The vMA machine is running on the licensed ESX cluster, and configured to authenticate and talk to the ESXi machine. Am I right in assuming that as the ESXi box does not specifically have a license, I can't backup VMs on it, even though the ESX hosts running the vMA machine are licensed? (I've tried, and the backups fail with 'Can't get vim_service') The FAQ only talks about licensed vs unlicensed, not a mix of the two. It's not an issue if that's so, but if you think it should work I'll need to dig further into the problem..
Again, many thanks for developing this tool..
Good points on the 2 below:
>zeroedthickor eagerzeroedthick
Maybe someday we can have an auto-detect, pull from the VM setings, vmx file maybe? which one would yield faster backups?
>Is there actually a difference with setting lsilogic vs buslogic in the configuration? I always thought that a virtual disk was a virtual disk, and that just denoted what virtual SCSI card would be emulated in the guest? What does changing the controller type actually do?
again, could script detect from VMX file or other? we have a mix of lsi and bus, maybe others in the future.
Too many variables. LOL, other then that, ghettoVCBg2 is awesome!
lamw,
Problem:
Been using script for over a week now, with a 1.36TB NFS, we use a rotation of 1 copy - each VM since we can't compress.
Today I had our file server VM complete it's backup within 2 minutes, all that is left is the vmx file, there was prior copy on NFS before backup started. I had info and not debug configured in script, that has now been changed, ; ( , but... wondering if script is checking to make sure backup is 100% completed successfully before wiping out the older copy, still using the ESXi 4.0 script, not the latest one. Guessing the snapshot quiece failed mid-stream, there was no prior snapshot but your script checks that and works fine. Script does not log anything on ESXi host or vCenter when it does the snapshot.
Besides configuring "debug" and "keeping" 2 copies, any other ideas?
FYI:
Script needs a slight tweak.
Here is debug output. No failure message in email but it failed. The real issue is we did not have enough space to backup the VM, so, script needs to tell user "Failed to start backup due to not enough space to backup VM", deleted some VM's and it finally backed up. The white rabbit must have ate the message cause I did not get it. LOL.
07-23-2010 08:47:20 -- info: ============================== ghettoVCBg2 LOG START ==============================
07-23-2010 08:47:20 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
07-23-2010 08:47:20 -- info: CONFIG - VM_BACKUP_DATASTORE = VDRBACKUP
07-23-2010 08:47:20 -- info: CONFIG - VM_BACKUP_DIRECTORY =
07-23-2010 08:47:20 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
07-23-2010 08:47:20 -- info: CONFIG - ADAPTER_FORMAT = lsilogic
07-23-2010 08:47:20 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
07-23-2010 08:47:20 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
07-23-2010 08:47:20 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = YES
07-23-2010 08:47:20 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2010-07-23
07-23-2010 08:47:20 -- info: CONFIG - VM_VMDK_FILES = all
07-23-2010 08:47:20 -- debug: Main: Login by vi-fastpass to: server1
07-23-2010 08:47:20 -- debug: copyTask: Task START
07-23-2010 08:47:20 -- debug: copyTask: waiting for next job and sleep ...
07-23-2010 08:47:23 -- debug: Main: Disconnect from: server1
07-23-2010 08:47:23 -- debug: Main: Login by vi-fastpass to: server2
07-23-2010 08:47:25 -- debug: Main: Disconnect from: server2
07-23-2010 08:47:25 -- debug: Main: Login by vi-fastpass to: server3
07-23-2010 08:47:26 -- info: Initiate backup for PC601 FILE SERVER found on server3
07-23-2010 08:47:34 -- debug: PC601 FILE SERVER original powerState: poweredOn
07-23-2010 08:47:34 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2010-07-23" for PC601 FILE SERVER
07-23-2010 08:47:59 -- info: PC601 FILE SERVER has 1 VMDK(s)
07-23-2010 08:47:59 -- debug: backupVMDK: Backing up "[DataStore3] File Server/File Server.vmdk" to "[VDRBACKUP] /PC601 FILE SERVER/PC601 FILE SERVER-2010-07-23/File Server.vmdk"
07-23-2010 08:47:59 -- debug: backupVMDK: Signal copyThread to start
07-23-2010 08:47:59 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
07-23-2010 08:47:59 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
07-23-2010 08:47:59 -- debug: CopyThread: Start backing up VMDK(s) ...
07-23-2010 08:48:04 -- debug: copyTask: send copySuccess message ...
07-23-2010 08:48:04 -- debug: copyTask: waiting for next job and sleep ...
07-23-2010 08:48:59 -- debug: backupVMDK: Successfully completed backup for File Server/File Server.vmdk Elapsed time: 1 min
07-23-2010 08:49:08 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2010-07-23" for PC601 FILE SERVER
07-23-2010 08:49:09 -- debug: checkVMBackupRotation: Starting ...
07-23-2010 08:49:12 -- info: Backup completed for PC601 FILE SERVER!
07-23-2010 08:49:12 -- debug: Main: Disconnect from: server3
07-23-2010 08:49:12 -- debug: Main: Login by vi-fastpass to: server4
07-23-2010 08:49:14 -- debug: Main: Disconnect from: server4
07-23-2010 08:49:14 -- debug: Main: Login by vi-fastpass to: server5
07-23-2010 08:49:15 -- debug: Main: Disconnect from: server5
07-23-2010 08:49:15 -- debug: Main: Calling final clean up
07-23-2010 08:49:15 -- debug: cleanUP: Thread clean up starting ...
07-23-2010 08:49:15 -- debug: cleanUp: Send exit to copyThread
07-23-2010 08:49:15 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit
07-23-2010 08:49:15 -- debug: copyTask: die ...
07-23-2010 08:49:15 -- debug: cleanUp: Join passed
07-23-2010 08:49:15 -- info: ============================== ghettoVCBg2 LOG END ==============================
Disk capacity and management is something I'm going to assume users are taking into consideration. There's nothing stopping you from creating a snapshot that fills up a datastore volume while the backup in place. I'll consider adding some checks but this can be difficult. I suggest you ensure that you actually have sufficient space for backups.
=========================================================================
William Lam
VMware vExpert 2009,2010
VMware scripts and resources at:
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
...actually, your script MUST BE checking the space "needed" cause it goes to the next VM within minutes. I think it is an easy fix, the script just needs to say "failed" not "successful" backup when that occurs.
also...we really need the VMware vMA appliance to do some sort of CBT, inline compression, (maybe dedupe but that may be asking too much) in the future.
Again, I'll take your feedback into consideration and I'll see what I can do as time permits. Though you as an administrator should ensure that you have sufficient space for backups and setup proper alerts in case you hit thresholds before the volume fills up.
also...we really need the VMware vMA appliance to do some sort of CBT
Note, this is not a function of vMA but a function of vSphere. If you're looking for CBT, you need to look at true enterprise backup solution Check out vDR, Veeam, etc. this is not an easy functionality to implement and would be a nice to have but for a free solution, it is out of the question.
Remember it's important to note that I manage more than 100+ scripts and I have requests and Q/A everyday, so I try to split my "free" time on the various projects to provide scripts and utilities to the community free of charge. Hopefully you understand and if you have found any of my scripts useful in your day to day life, feel free to support us - http://www.virtuallyghetto.com/p/how-you-can-help.html
Thanks
=========================================================================
William Lam
VMware vExpert 2009,2010
VMware scripts and resources at:
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
When executing ghettoVCBg2.pl by hand on vMA 4.1 it works like a charm, but when setting up a cronjob via crontab -e i get some strange errors, even when using /etc/crontab edited by sudo vi.
the error reads as follows:
Can't load '/usr/lib/perl5/site_perl/5.8.8/libvmatargetlib_perl.so' for module vmatargetlib_perl: libtypes.so: cannot open shared object file: No such file or directory at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/DynaLoader.pm line 230.
at /usr/lib/perl5/site_perl/5.8.8/VMware/VmaTargetLib.pm line 10
Compilation failed in require at /usr/lib/perl5/site_perl/5.8.8/VMware/VIFPLib.pm line 10.
BEGIN failed--compilation aborted at /usr/lib/perl5/site_perl/5.8.8/VMware/VIFPLib.pm line 10.
Compilation failed in require at /DATEN/ghettoVCBg2/ghettoVCBg2.pl line 16.
BEGIN failed--compilation aborted at /DATEN/ghettoVCBg2/ghettoVCBg2.pl line 16.
i already made a sh-script where the path is set that is also used in the vi-admin-bash but it still does not work...
anyone any idea ?
What's most likely happening is the path to the Perl modules that are being used is not defined in the cron entry, remember by default cron will have a very limited default path defined.
Do a search online for setting the PATH for cron job and take a look at the existing PATH and you should be good once that is set
=========================================================================
William Lam
VMware vExpert 2009,2010
VMware scripts and resources at:
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I added the SHELL and LD_LIBRARY_PATH entries found in vi-admin's /usr/bin/env to the cron job and it seems to have solved the problem.
How exactly did you do ?
adding this per crontab -e or did you add this to /etc/crontab ?
i tried to call this .sh file per crontab -e, but it still does not work
SHELL=/bin/bash
PATH=/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/java/jre-vmware/bin:/sbin:/usr/sbin:/home/vi-admin/bin
LD_LIBRARY_PATH=:/opt/vmware/vma/lib64:/opt/vmware/vma/lib
/ghettoVCBg2/ghettoVCBg2.pl --dryrun --verbose --vmlist /ghettoVCBg2/virtual_machine_back_list&> /vcbg2.log
"#!/bin/bash -l" as shebang makes it work "-l"
makes bash act as if it had been invoked as a login shell and it reads the .profile
We seem to have a problem with the latest version. Everything seems to be working properly, the backups are being written to the designated location. But once every single backup is finished the backup files get deleted. The folders are still there, but it contents have disappeared. I changed the destination from iSCSI to NFS and back, and also the backup format (from thin to zeroedthick) to no avail. I also thought it to be possible that there might be something wrong with the number of backups that are saved but changing that number does not help either. Is there anyone that can shed a light on this, or maybe has the same issue?
See also the log below where we do a backup of the vMA itself.
JW
08-12-2010 18:13:09 -- info: ============================== ghettoVCBg2 LOG START ==============================
08-12-2010 18:13:09 -- debug: copyTask: Task START
08-12-2010 18:13:09 -- debug: copyTask: waiting for next job and sleep ...
08-12-2010 18:13:09 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCBg2.log
08-12-2010 18:13:09 -- info: CONFIG - VM_BACKUP_DATASTORE = sun01-nfs-share01
08-12-2010 18:13:09 -- info: CONFIG - VM_BACKUP_DIRECTORY = ghettoVCB
08-12-2010 18:13:09 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
08-12-2010 18:13:09 -- info: CONFIG - ADAPTER_FORMAT = buslogic
08-12-2010 18:13:09 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = NO
08-12-2010 18:13:09 -- info: CONFIG - VM_SNAPSHOT_MEMORY = NO
08-12-2010 18:13:09 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = NO
08-12-2010 18:13:09 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2010-08-12
08-12-2010 18:13:09 -- info: CONFIG - VM_VMDK_FILES = all
08-12-2010 18:13:09 -- debug: Main: Login by vi-fastpass to: ndims-beh02.****.****
08-12-2010 18:13:10 -- warn: Unable to login to: ndims-beh02.****.**** - maybe offline or unreachable - skipping
08-12-2010 18:13:10 -- debug: Main: Login by vi-fastpass to: vmhost14.****.****
08-12-2010 18:13:14 -- debug: Main: Disconnect from: vmhost14.****.****
08-12-2010 18:13:14 -- debug: Main: Login by vi-fastpass to: vmhost13.****.****
08-12-2010 18:13:20 -- debug: Main: Disconnect from: vmhost13.****.****
08-12-2010 18:13:20 -- debug: Main: Login by vi-fastpass to: vmhost12.****.****
08-12-2010 18:13:26 -- debug: Main: Disconnect from: vmhost12.****.****
08-12-2010 18:13:26 -- debug: Main: Login by vi-fastpass to: vmhost11.****.****
08-12-2010 18:13:29 -- info: Initiate backup for vSphere Management Assistant (vMA) found on vmhost11.****.****
08-12-2010 18:13:33 -- debug: vSphere Management Assistant (vMA) original powerState: poweredOn
08-12-2010 18:13:33 -- debug: Creating Snapshot "ghettoVCBg2-snapshot-2010-08-12" for vSphere Management Assistant (vMA)
08-12-2010 18:13:43 -- info: vSphere Management Assistant (vMA) has 1 VMDK(s)
08-12-2010 18:13:43 -- debug: backupVMDK: Backing up "[sun01-nfs-share01] vSphere Management Assistant (vM/vSphere Management Assistant (vM.vmdk" to "[sun01-nfs-share01] ghettoVCB/vSphere Management Assistant (vMA)/vSphere Management Assistant (vMA)-2010-08-12/vSphere Management Assistant (vM.vmdk"
08-12-2010 18:13:43 -- debug: backupVMDK: Signal copyThread to start
08-12-2010 18:13:43 -- debug: backupVMDK: Backup progress: Elapsed time 0 min
08-12-2010 18:13:43 -- debug: copyTask: Wake up and follow the white rabbit, with status: doCopy
08-12-2010 18:13:43 -- debug: CopyThread: Start backing up VMDK(s) ...
08-12-2010 18:14:57 -- debug: copyTask: send copySuccess message ...
08-12-2010 18:14:57 -- debug: copyTask: waiting for next job and sleep ...
08-12-2010 18:15:44 -- debug: backupVMDK: Successfully completed backup for vSphere Management Assistant (vM/vSphere Management Assistant (vM.vmdk Elapsed time: 2 min
08-12-2010 18:15:46 -- debug: Removing Snapshot "ghettoVCBg2-snapshot-2010-08-12" for vSphere Management Assistant (vMA)
08-12-2010 18:15:47 -- debug: checkVMBackupRotation: Starting ...
08-12-2010 18:15:54 -- info: Backup completed for vSphere Management Assistant (vMA)!
08-12-2010 18:15:54 -- debug: Main: Disconnect from: vmhost11.****.****
08-12-2010 18:15:54 -- debug: Main: Login by vi-fastpass to: vmhost10.****.****
08-12-2010 18:15:57 -- debug: Main: Disconnect from: vmhost10.****.****
08-12-2010 18:15:57 -- debug: Main: Calling final clean up
08-12-2010 18:15:57 -- debug: cleanUP: Thread clean up starting ...
08-12-2010 18:15:57 -- debug: cleanUp: Send exit to copyThread
08-12-2010 18:15:57 -- debug: copyTask: Wake up and follow the white rabbit, with status: exit
08-12-2010 18:15:57 -- debug: copyTask: die ...
08-12-2010 18:15:57 -- debug: cleanUp: Join passed
08-12-2010 18:15:57 -- info: ============================== ghettoVCBg2 LOG END ==============================
As it appears the folder in which the backup first appears is renamed after the backup. A dash and a number are being added to the end of the name. When one refreshes the vSphere client that I used to browse the datastore it shows an empty folder, where it should be saying that the folder does not exist anymore. When one goes up one level, refreshes and drills down one level, the backup files are there. A whole lot less complicated than I thought it was, but still rather confusing.
JW
I am running ESXi 4.1 booting to a USB. I have no native OS running on the bare metal. Do I need ESXi running on a host OS in order to use ghettoVCBg2?
I've not tested it with ESXi running off a USB ... though the backup does not rely on where ESXi image is running off of, it's more of where the backups will be stored. As long as a valid datastore is seen by the host and you're using a licensed version of ESXi, this script should function. Please take a look at the documentation for more details around the backup
=========================================================================
William Lam
VMware vExpert 2009,2010
VMware scripts and resources at:
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Can test ghettovcb with an unlicensed version? or will it not even function in anyway?
Please take a look at the documentation.
=========================================================================
William Lam
VMware vExpert 2009,2010
VMware scripts and resources at:
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hello!
First of all tanks for the great script!
I'am using the script for quite a while and I was really happy with it.
Then the system disk of my nfs-server crashed. I had no backup, so I had to reinstall the whole server sytem.
After I was finished and made all the connections to the esxi server the backup failed to backup the second VM.
I changed to the newest version of the script but nothing changed. The log always says that the the second VM could not be found.
I changed the destination datastore - no change.
I changed the first vm to be the second in the backup - no change.
Also the performance of the backup is much more slow than before.
Thank you for your help!
Thank you for making this script. We appreciate having an affordable option for backing up VMware guests. Like several other users we are also experiencing the "Unable to locate VM:" errors. Our datastore for backups is also an NFS share with ext3 filesystem. Any plans for a fix to the ext3/NFS issue in the near future? Will switching to xfs definitely fix the problem?
Hi, sean.carolan, in my case I'm still having that issue also with XFS. The only way I found to definitely solve the issue is to do a bash script where I run the Getto script once for each VM.
Wow...that's sort of...non-ideal. I will consider taking the one-script-instance per VM route if there are no other options.
Great script, as usual, but can I run the script against a single host and not all hosts registered in the vMA? I didn't find an option to do this (--server doesn't).
lamw,
Any tips for us on killing the PID ghettoVCBg2 creates? I know you have solution for the "other" ghettoVCB (storing PID in log)
I want to stop ghettoVCBg2 at 6am on Mondays thru Sat. Don't want backups running during business hours.
Thinking, we could edit your script and add (somewhat, not sure perl code method):
echo $$ > pid.pid
I could then read this file "pid.pid" (containing PID) in my taskkill.sh script inside our crontab, then stop the backup.
Any thoughts?
MKguy,
Per the documentation, the script does not run against any specific ESX(i) host, it relies on vi-fastpass targets being added into vMA and based on the VM backup list, it'll traverse through all hosts locating the VMs for you. If it finds all the VMs in the first say 5 out of 10 hosts, it'll disconnect and quit after the 5th and not try to go on. If it doesn't find the VM, it will state a warning at the very end.
Hopefully this makes more sense
=========================================================================
William Lam
VMware vExpert 2009,2010
VMware scripts and resources at:
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
vmbru,
This is something I could look into adding, though it's pretty trivial to get the PID but you will not be using any type of "echo" statement as that is not how you do it in Perl. You'll use the same syntax, the "$$" will give you the PID of the script which can then be printed in the logs, similar to how is doing it today.
Though the real issue looks like you're trying to "kill" the script during a certain period of time? Why not just use cron to schedule your backups so that it does not conflict with your business hours? I think that is the right way of doing it versus trying to kill the script, the latter should only be used if an issue occurs or you need to stop the script. In either case, you should be able to look up the PID by running "ps -ef" and locating the ghettoVCBg2 script being executed.
=========================================================================
William Lam
VMware vExpert 2009,2010
VMware scripts and resources at:
Getting Started with the vMA (tips/tricks)
Getting Started with the vSphere SDK for Perl
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
lamw,
Trust me, I'd rather not try to kill the ghetto script/process, I just have many VM's and TB's of data to backup and it could run thru business hours. I also use VDR and it shuts down further backups when the backup window is exceeded, just trying to add that feature to your script. My crontab is setup to only start your script at 6:35pm M-F and 1:35pm on Sat.
I was able to edit your script right before the EMAIL CONF section and added:
open (FILE, ">>/home/vi-admin/pid.pid");
print FILE "$$\n";
close (FILE);
...this creates file called pid.pid, with the initial PID (there are other PIDS, calls to esxi hosts, etc...) but figure safer to kill the original 1st generated one), that my taskkill.sh script (below) reads:
taskkill.sh
arr=(`cat pid.pid`)
kill -15 ${arr[*]}
exit 0
I have a kickoff script that generates the list of VM's to backup that clears this PID file too.
Anyway, this seems to work for me, it kills the remainder of process and allows the current backup to finish (which is ok)...as far as I can tell from testing for past 2 days.
So...guess I'm good for now.
Well, my problem is that I want to use the local datastore of all my hosts to store the VM backups on, which are named differently for each host.
So what I'm doing currently looks like this:
- prior to the backups, run a custom script which connects to each host and outputs which VMs run on this hostX (of which I want to backup) into textfileX
- have a ghettovcbg2.pl scripts for each host, only difference is the $VM_BACKUP_DATASTORE variable
- run ghettovcbg2-hostX.pl --vmlist textfileX for each host respectively
I know this system isn't perfect, even if I could specify hosts; in fact it may seem even dumb, but do you know a better way to achieve this?
Hi all,
I have a problem running g2. I currently use ghettoVCB on my esxi servers with no problem, but I would like to upgrade to g2 and found a bizzarre problem... Here what I did:
- downloaded and installed vma 4.1 via ovf
- setup vma
- run vifp to add a server of mine, and it correctly shows in vifp listservers
- created a file with only one line containing "XP" without quotes (an xp pro test machine)
- downloaded g2 and untarred in its default directory
- modified variables to my needs
- run ghettoVCBg2.pl --vmlist mylist
The problem is that is does nothing. It only passes to the next line and the shell prompt returns. On my ESXi 4.1 (licensed) nothing runs.
Also the original script, with default variables, does nothing. No errors. Loglevel is set to Debug, but nothing happens.
If I run the script with no options it gives me the man page. If I put a wrong machine name in the list-file it does nothing, if I put the wrong name for the list file it complains about it.
I'm probably missing something very big, but I read all the FAQs and posts and I'm stuck here. Simply nothing happens.
Specifying a server with --server does not help. If I put a dummy server name I obtain no error.
--version is
vSphere SDK for Perl version: 4.1
Script 'ghettoVCBg2.pl' version: 4.3
A thread exited while 2 threads were running
Any ideas?
Any idea on how to use this with VMA4.1? I am getting an error: Use of inherited AUTOLOAD for non-method vifplib_perl::CreateVIUserInfo() is depreciated at ./ghettoVCBg2.pl line 305
Thanks!
nevermind - found the problem with the .pl script. Since I did an in-place upgrade of vMA both the vima-release and vma-release files existed in the /etc directory. I modified the script to look for vma-release first and the above error went away.
William, Thanks for all that you do!
Hi tschneider,
I'm having the same issue and would be grateful if you could let me know what changes you made to the script to get this to work.
Thanks
Paul