This script performs backups of virtual machines residing on ESX(i) 3.5/4.x/5.x/6.x/7.x servers using methodology similar to VMware's VCB tool. The script takes snapshots of live running virtual machines, backs up the master VMDK(s) and then upon completion, deletes the snapshot until the next backup. The only caveat is that it utilizes resources available to the Service Console of the ESX server or Busybox Console (Tech Support Mode) of the ESXi server running the backups as opposed to following the traditional method of offloading virtual machine backups through a VCB proxy.
This script has been tested on ESX 3.5/4.x/5.x and ESXi 3.5/4.x/5.x/6.x/7.x and supports the following backup mediums: LOCAL STORAGE, SAN and NFS. The script is non-interactive and can be setup to run via cron. Currently, this script accepts a text file that lists the display names of virtual machine(s) that are to be backed up. Additionally, one can specify a folder containing configuration files on a per VM basis for granular control over backup policies.
Additionally, for ESX(i) environments that don't have persistent NFS datastores designated for backups, the script offers the ability to automatically connect the ESX(i) server to a NFS exported folder and then upon backup completion, disconnect it from the ESX(i) server. The connection is established by creating an NFS datastore link which enables monolithic (or thick) VMDK backups as opposed to using the usual *nix mount command which necessitates breaking VMDK files into the 2gbsparse format for backup. Enabling this mode is self-explanatory and will evidently be so when editing the script (Note: VM_BACKUP_VOLUME variable is ignored if ENABLE_NON_PERSISTENT_NFS=1 ).
In its current configuration, the script will allow up to 3 unique backups of the Virtual Machine before it will overwrite the previous backups; this however, can be modified to fit procedures if need be. Please be diligent in running the script in a test or staging environment before using it on production live Virtual Machines; this script functions well within our environment but there is a chance that it may not fit well into other environments.
If you have any questions, you may post in the dedicated ghettoVCB VMTN community group.
If you have found this script to be useful and would like to contribute back, please click here to donate.
Please read ALL documentation + FAQ's before posting a question about an issue or problem. Thank You
1) Download ghettoVCB from github by clicking on the ZIP button at the top and upload to either your ESX or ESXi system (use scp or WinSCP to transfer the file)
2) Extract the contents of the zip file (filename will vary):
# unzip ghettoVCB-master.zip
Archive: ghettoVCB-master.zip
creating: ghettoVCB-master/
inflating: ghettoVCB-master/README
inflating: ghettoVCB-master/ghettoVCB-restore.sh
inflating: ghettoVCB-master/ghettoVCB-restore_vm_restore_configuration_template
inflating: ghettoVCB-master/ghettoVCB-vm_backup_configuration_template
inflating: ghettoVCB-master/ghettoVCB.conf
inflating: ghettoVCB-master/ghettoVCB.sh
3) The script is now ready to be used and is located in a directory named ghettoVCB-master
# ls -l
-rw-r--r-- 1 root root 281 Jan 6 03:58 README
-rw-r--r-- 1 root root 16024 Jan 6 03:58 ghettoVCB-restore.sh
-rw-r--r-- 1 root root 309 Jan 6 03:58 ghettoVCB-restore_vm_restore_configuration_template
-rw-r--r-- 1 root root 356 Jan 6 03:58 ghettoVCB-vm_backup_configuration_template
-rw-r--r-- 1 root root 631 Jan 6 03:58 ghettoVCB.conf
-rw-r--r-- 1 root root 49375 Jan 6 03:58 ghettoVCB.sh
4) Before using the scripts, you will need to enable the execute permission on both ghettoVCB.sh and ghettoVCB-restore.sh by running the following:
chmod +x ghettoVCB.shchmod +x ghettoVCB-restore.sh
The following variables need to be defined within the script or in VM backup policy prior to execution.
Defining the backup datastore and folder in which the backups are stored (if folder does not exist, it will automatically be created):
VM_BACKUP_VOLUME=/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS
Defining the backup disk format (zeroedthick, eagerzeroedthick, thin, and 2gbsparse are available):
DISK_BACKUP_FORMAT=thin
Note: If you are using the 2gbsparse on an ESXi 5.1 host, backups may fail. Please download the latest version of the ghettoVCB script which automatically resolves this or take a look at this article for the details.
Defining the backup rotation per VM:
VM_BACKUP_ROTATION_COUNT=3
Defining whether the VM is powered down or not prior to backup (1 = enable, 0 = disable):
Note: VM(s) that are powered off will not require snapshoting
POWER_VM_DOWN_BEFORE_BACKUP=0
Defining whether the VM can be hard powered off when "POWER_VM_DOWN_BEFORE_BACKUP" is enabled and VM does not have VMware Tools installed
ENABLE_HARD_POWER_OFF=0
If "ENABLE_HARD_POWER_OFF" is enabled, then this defines the number of (60sec) iterations the script will before executing a hard power off when:
ITER_TO_WAIT_SHUTDOWN=3
The number (60sec) iterations the script will wait when powering off the VM and will give up and ignore the particular VM for backup:
POWER_DOWN_TIMEOUT=5
The number (60sec) iterations the script will wait when taking a snapshot of a VM and will give up and ignore the particular VM for backup:
Note: Default value should suffice
SNAPSHOT_TIMEOUT=15
Defining whether or not to enable compression (1 = enable, 0 = disable):
ENABLE_COMPRESSION=0
NOTE: With ESXi 3.x/4.x/5.x, there is a limitation of the maximum size of a VM for compression within the unsupported Busybox Console which should not affect backups running classic ESX 3.x,4.x or 5.x. On ESXi 3.x the largest supported VM is 4GB for compression and on ESXi 4.x the largest supported VM is 8GB. If you try to compress a larger VM, you may run into issues when trying to extract upon a restore. PLEASE TEST THE RESTORE PROCESS BEFORE MOVING TO PRODUCTION SYSTEMS!
Defining the adapter type for backed up VMDK (DEPERCATED - NO LONGER NEEDED😞
ADAPTER_FORMAT=buslogic
Defining whether virtual machine memory is snapped and if quiescing is enabled (1 = enable, 0 = disable):
Note: By default both are disabled
VM_SNAPSHOT_MEMORY=0
VM_SNAPSHOT_QUIESCE=0
NOTE: VM_SNAPSHOT_MEMORY is only used to ensure when the snapshot is taken, it's memory contents are also captured. This is only relevant to the actual snapshot and it's not used in any shape/way/form in regards to the backup. All backups taken whether your VM is running or offline will result in an offline VM backup when you restore. This was originally added for debugging purposes and in generally should be left disabled
Defining VMDK(s) to backup from a particular VM either a list of vmdks or "all"
VMDK_FILES_TO_BACKUP="myvmdk.vmdk"
Defining whether or not VM(s) with existing snapshots can be backed up. This flag means it will CONSOLIDATE ALL EXISTING SNAPSHOTS for a VM prior to starting the backup (1 = yes, 0 = no):
ALLOW_VMS_WITH_SNAPSHOTS_TO_BE_BACKEDUP=0
Defining the order of which VM(s) should be shutdown first, especially if there is a dependency between multiple VM(s). This should be a comma seperate list of VM(s)
VM_SHUTDOWN_ORDER=vm1,vm2,vm3
Defining the order of VM(s) that should be started up first after backups have completed, especially if there is a dependency between multiple VM(s). This should be a comma seperate list of VM(s)
VM_STARTUP_ORDER=vm3,vm2,vm1
Defining NON-PERSISTENT NFS Backup Volume (1 = yes, 0 = no):
ENABLE_NON_PERSISTENT_NFS=0
NOTE: This is meant for environments that do not want a persisted connection to their NFS backup volume and allows the NFS volume to only be mounted during backups. The script expects the following 5 variables to be defined if this is to be used: UNMOUNT_NFS, NFS_SERVER, NFS_MOUNT, NFS_LOCAL_NAME and NFS_VM_BACKUP_DIR
Defining whether or not to unmount the NFS backup volume (1 = yes, 0 = no):
UNMOUNT_NFS=0
Defining the NFS server address (IP/hostname):
NFS_SERVER=172.51.0.192
Defining the NFS export path:
NFS_MOUNT=/upload
Defining the NFS datastore name:
NFS_LOCAL_NAME=backup
Defining the NFS backup directory for VMs:
NFS_VM_BACKUP_DIR=mybackups
NOTE: Only supported if you are running vSphere 4.1 and this feature is experimental. If you are having issues with sending mail, please take a look at Email Backup Log section
Defining whether or not to email backup logs (1 = yes, 0 = no):
EMAIL_LOG=1
Defining whether or not to email message will be deleted off the host whether it is successful in sending, this is used for debugging purposes. (1 = yes, 0 = no):
EMAIL_DEBUG=1
Defining email server:
EMAIL_SERVER=auroa.primp-industries.com
Defining email server port:
EMAIL_SERVER_PORT=25
Defining email delay interval (useful if you have slow SMTP server and would like to include a delay in netcat using -i param, default is 1second):
EMAIL_DELAY_INTERVAL=1
Defining recipient of the email:
EMAIL_TO=auroa@primp-industries.com
Defining from user which may require specific domain entry depending on email server configurations:
EMAIL_FROM=root@ghettoVCB
Defining to support RSYNC symbolic link creation (1 = yes, 0 = no):
RSYNC_LINK=0
Note: This enables an automatic creation of a generic symbolic link (both a relative & absolution path) in which users can refer to run replication backups using rsync from a remote host. This does not actually support rsync backups with ghettoVCB. Please take a look at the Rsync Section of the documentation for more details.
# cat ghettoVCB.conf
VM_BACKUP_VOLUME=/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS
DISK_BACKUP_FORMAT=thin
VM_BACKUP_ROTATION_COUNT=3
POWER_VM_DOWN_BEFORE_BACKUP=0
ENABLE_HARD_POWER_OFF=0
ITER_TO_WAIT_SHUTDOWN=3
POWER_DOWN_TIMEOUT=5
ENABLE_COMPRESSION=0
VM_SNAPSHOT_MEMORY=0
VM_SNAPSHOT_QUIESCE=0
ALLOW_VMS_WITH_SNAPSHOTS_TO_BE_BACKEDUP=0
ENABLE_NON_PERSISTENT_NFS=0
UNMOUNT_NFS=0
NFS_SERVER=172.30.0.195
NFS_MOUNT=/nfsshare
NFS_LOCAL_NAME=nfs_storage_backup
NFS_VM_BACKUP_DIR=mybackups
SNAPSHOT_TIMEOUT=15
EMAIL_LOG=0
EMAIL_SERVER=auroa.primp-industries.com
EMAIL_SERVER_PORT=25
EMAIL_DELAY_INTERVAL=1
EMAIL_TO=auroa@primp-industries.com
EMAIL_FROM=root@ghettoVCB
WORKDIR_DEBUG=0
VM_SHUTDOWN_ORDER=
VM_STARTUP_ORDER=
To override any existing configurations within the ghettoVCB.sh script and to use a global configuration file, user just needs to specify the new flag -g and path to global configuration file (For an example, please refer to the sample execution section of the documenation)
Running multiple instances of ghettoVCB is now supported with the latest release by specifying the working directory (-w) flag.
By default, the working directory of the ghettoVCB instance is /tmp/ghettoVCB.work and you can run another instance by providing an alternate working directory. You should try to minimize the number of ghettoVCB instances running on your ESXi host as it does consume some amount of resources when running in the ESXi Shell. This is considered an experimental feature, so please test in a development environment to ensure everything is working prior to moving to production system.
Ensure that you do not edit past this section:
########################## DO NOT MODIFY PAST THIS LINE ##########################
# ./ghettoVCB.sh
###############################################################################
#
# ghettoVCB for ESX/ESXi 3.5, 4.x+ and 5.x
# Author: William Lam
# http://www.virtuallyghetto.com/
# Documentation: http://communities.vmware.com/docs/DOC-8760
# Created: 11/17/2008
# Last modified: 2012_12_17 Version 0
#
###############################################################################
Usage: ghettoVCB.sh [options]
OPTIONS:
-a Backup all VMs on host
-f List of VMs to backup
-m Name of VM to backup (overrides -f)
-c VM configuration directory for VM backups
-g Path to global ghettoVCB configuration file
-l File to output logging
-w ghettoVCB work directory (default: )
-d Debug level [info|debug|dryrun] (default: info)
(e.g.)
Backup VMs stored in a list
./ghettoVCB.sh -f vms_to_backup
Backup a single VM
./ghettoVCB.sh -m vm_to_backup
Backup all VMs residing on this host
./ghettoVCB.sh -a
Backup all VMs residing on this host except for the VMs in the exclusion list
./ghettoVCB.sh -a -e vm_exclusion_list
Backup VMs based on specific configuration located in directory
./ghettoVCB.sh -f vms_to_backup -c vm_backup_configs
Backup VMs using global ghettoVCB configuration file
./ghettoVCB.sh -f vms_to_backup -g /global/ghettoVCB.conf
Output will log to /tmp/ghettoVCB.log (consider logging to local or remote datastore to persist logs)
./ghettoVCB.sh -f vms_to_backup -l /vmfs/volume/local-storage/ghettoVCB.log
Dry run (no backup will take place)
./ghettoVCB.sh -f vms_to_backup -d dryrun
The input to this script is a file that contains the display name of the virtual machine(s) separated by a newline. When creating this file on a non-Linux/UNIX system, you may introduce ^M character which can cause the script to miss-behave. To ensure this does not occur, plesae create the file on the ESX/ESXi host.
Here is a sample of what the file would look like:
[root@himalaya ~]# cat vms_to_backup
vCOPS
vMA
vCloudConnector
Debug Mode
Note: This execution mode provides a qucik summary of details on whether a given set of VM(s)/VMDK(s) will be backed up. It provides additional information such as VMs that may have snapshots, VMDK(s) that are configured as independent disks, or other issues that may cause a VM or VMDK to not backed up.
[root@himalaya ghettoVCB]# ./ghettoVCB.sh -f vms_to_backup -d dryrun
Logging output to "/tmp/ghettoVCB-2011-03-13_15-19-57.log" ...
2011-03-13 15:19:57 -- info: ============================== ghettoVCB LOG START ==============================
2011-03-13 15:19:57 -- info: CONFIG - VERSION = 2011_03_13_1
2011-03-13 15:19:57 -- info: CONFIG - GHETTOVCB_PID = 30157
2011-03-13 15:19:57 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS
2011-03-13 15:19:57 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 3
2011-03-13 15:19:57 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2011-03-13_15-19-57
2011-03-13 15:19:57 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
2011-03-13 15:19:57 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0
2011-03-13 15:19:57 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0
2011-03-13 15:19:57 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3
2011-03-13 15:19:57 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5
2011-03-13 15:19:57 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15
2011-03-13 15:19:57 -- info: CONFIG - LOG_LEVEL = dryrun
2011-03-13 15:19:57 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCB-2011-03-13_15-19-57.log
2011-03-13 15:19:57 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0
2011-03-13 15:19:57 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0
2011-03-13 15:19:57 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all
2011-03-13 15:19:57 -- info: CONFIG - EMAIL_LOG = 0
2011-03-13 15:19:57 -- info:
2011-03-13 15:19:57 -- dryrun: ###############################################
2011-03-13 15:19:57 -- dryrun: Virtual Machine: scofield
2011-03-13 15:19:57 -- dryrun: VM_ID: 704
2011-03-13 15:19:57 -- dryrun: VMX_PATH: /vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/scofield/scofield.vmx
2011-03-13 15:19:57 -- dryrun: VMX_DIR: /vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/scofield
2011-03-13 15:19:57 -- dryrun: VMX_CONF: scofield/scofield.vmx
2011-03-13 15:19:57 -- dryrun: VMFS_VOLUME: himalaya-local-SATA.RE4-GP:Storage
2011-03-13 15:19:57 -- dryrun: VMDK(s):
2011-03-13 15:19:58 -- dryrun: scofield_3.vmdk 3 GB
2011-03-13 15:19:58 -- dryrun: scofield_2.vmdk 2 GB
2011-03-13 15:19:58 -- dryrun: scofield_1.vmdk 1 GB
2011-03-13 15:19:58 -- dryrun: scofield.vmdk 5 GB
2011-03-13 15:19:58 -- dryrun: INDEPENDENT VMDK(s):
2011-03-13 15:19:58 -- dryrun: TOTAL_VM_SIZE_TO_BACKUP: 11 GB
2011-03-13 15:19:58 -- dryrun: ###############################################
2011-03-13 15:19:58 -- dryrun: ###############################################
2011-03-13 15:19:58 -- dryrun: Virtual Machine: vMA
2011-03-13 15:19:58 -- dryrun: VM_ID: 1440
2011-03-13 15:19:58 -- dryrun: VMX_PATH: /vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/vMA/vMA.vmx
2011-03-13 15:19:58 -- dryrun: VMX_DIR: /vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/vMA
2011-03-13 15:19:58 -- dryrun: VMX_CONF: vMA/vMA.vmx
2011-03-13 15:19:58 -- dryrun: VMFS_VOLUME: himalaya-local-SATA.RE4-GP:Storage
2011-03-13 15:19:58 -- dryrun: VMDK(s):
2011-03-13 15:19:58 -- dryrun: vMA-000002.vmdk 5 GB
2011-03-13 15:19:58 -- dryrun: INDEPENDENT VMDK(s):
2011-03-13 15:19:58 -- dryrun: TOTAL_VM_SIZE_TO_BACKUP: 5 GB
2011-03-13 15:19:58 -- dryrun: Snapshots found for this VM, please commit all snapshots before continuing!
2011-03-13 15:19:58 -- dryrun: THIS VIRTUAL MACHINE WILL NOT BE BACKED UP DUE TO EXISTING SNAPSHOTS!
2011-03-13 15:19:58 -- dryrun: ###############################################
2011-03-13 15:19:58 -- dryrun: ###############################################
2011-03-13 15:19:58 -- dryrun: Virtual Machine: vCloudConnector
2011-03-13 15:19:58 -- dryrun: VM_ID: 2064
2011-03-13 15:19:58 -- dryrun: VMX_PATH: /vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/vCloudConnector/vCloudConnector.vmx
2011-03-13 15:19:58 -- dryrun: VMX_DIR: /vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/vCloudConnector
2011-03-13 15:19:58 -- dryrun: VMX_CONF: vCloudConnector/vCloudConnector.vmx
2011-03-13 15:19:58 -- dryrun: VMFS_VOLUME: himalaya-local-SATA.RE4-GP:Storage
2011-03-13 15:19:58 -- dryrun: VMDK(s):
2011-03-13 15:19:59 -- dryrun: vCloudConnector.vmdk 3 GB
2011-03-13 15:19:59 -- dryrun: INDEPENDENT VMDK(s):
2011-03-13 15:19:59 -- dryrun: vCloudConnector_1.vmdk 40 GB
2011-03-13 15:19:59 -- dryrun: TOTAL_VM_SIZE_TO_BACKUP: 3 GB
2011-03-13 15:19:59 -- dryrun: Snapshots can not be taken for indepdenent disks!
2011-03-13 15:19:59 -- dryrun: THIS VIRTUAL MACHINE WILL NOT HAVE ALL ITS VMDKS BACKED UP!
2011-03-13 15:19:59 -- dryrun: ###############################################
2011-03-13 15:19:59 -- info: ###### Final status: OK, only a dryrun. ######
2011-03-13 15:19:59 -- info: ============================== ghettoVCB LOG END ================================
In the example above, we have 3 VMs to be backed up:
Note: This execution modes provides more in-depth information about environment/backup process including additional storage debugging information which provides information about both the source/destination datastore pre and post backups. This can be very useful in troubleshooting backups
[root@himalaya ghettoVCB]# ./ghettoVCB.sh -f vms_to_backup -d debug
Logging output to "/tmp/ghettoVCB-2011-03-13_15-27-59.log" ...
2011-03-13 15:27:59 -- info: ============================== ghettoVCB LOG START ==============================
2011-03-13 15:27:59 -- debug: Succesfully acquired lock directory - /tmp/ghettoVCB.lock
2011-03-13 15:27:59 -- debug: HOST VERSION: VMware ESX 4.1.0 build-260247
2011-03-13 15:27:59 -- debug: HOST LEVEL: VMware ESX 4.1.0 GA
2011-03-13 15:27:59 -- debug: HOSTNAME: himalaya.primp-industries.com
2011-03-13 15:27:59 -- info: CONFIG - VERSION = 2011_03_13_1
2011-03-13 15:27:59 -- info: CONFIG - GHETTOVCB_PID = 31074
2011-03-13 15:27:59 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS
2011-03-13 15:27:59 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 3
2011-03-13 15:27:59 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2011-03-13_15-27-59
2011-03-13 15:27:59 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
2011-03-13 15:27:59 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0
2011-03-13 15:27:59 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0
2011-03-13 15:27:59 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3
2011-03-13 15:27:59 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5
2011-03-13 15:27:59 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15
2011-03-13 15:27:59 -- info: CONFIG - LOG_LEVEL = debug
2011-03-13 15:27:59 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCB-2011-03-13_15-27-59.log
2011-03-13 15:27:59 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0
2011-03-13 15:27:59 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0
2011-03-13 15:27:59 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all
2011-03-13 15:27:59 -- info: CONFIG - EMAIL_LOG = 0
2011-03-13 15:27:59 -- info:
2011-03-13 15:28:01 -- debug: Storage Information before backup:
2011-03-13 15:28:01 -- debug: SRC_DATASTORE: himalaya-local-SATA.RE4-GP:Storage
2011-03-13 15:28:01 -- debug: SRC_DATASTORE_CAPACITY: 1830.5 GB
2011-03-13 15:28:01 -- debug: SRC_DATASTORE_FREE: 539.4 GB
2011-03-13 15:28:01 -- debug: SRC_DATASTORE_BLOCKSIZE: 4
2011-03-13 15:28:01 -- debug: SRC_DATASTORE_MAX_FILE_SIZE: 1024 GB
2011-03-13 15:28:01 -- debug:
2011-03-13 15:28:01 -- debug: DST_DATASTORE: dlgCore-NFS-bigboi.VM-Backups
2011-03-13 15:28:01 -- debug: DST_DATASTORE_CAPACITY: 1348.4 GB
2011-03-13 15:28:01 -- debug: DST_DATASTORE_FREE: 296.8 GB
2011-03-13 15:28:01 -- debug: DST_DATASTORE_BLOCKSIZE: NA
2011-03-13 15:28:01 -- debug: DST_DATASTORE_MAX_FILE_SIZE: NA
2011-03-13 15:28:01 -- debug:
2011-03-13 15:28:02 -- info: Initiate backup for scofield
2011-03-13 15:28:02 -- debug: /usr/sbin/vmkfstools -i "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/scofield/scofield_3.vmdk" -a "buslogic" -d "thin" "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/scofield/scofield-2011-03-13_15-27-59/scofield_3.vmdk"
Destination disk format: VMFS thin-provisioned
Cloning disk '/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/scofield/scofield_3.vmdk'...
Clone: 37% done.
2011-03-13 15:28:04 -- debug: /usr/sbin/vmkfstools -i "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/scofield/scofield_2.vmdk" -a "buslogic" -d "thin" "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/scofield/scofield-2011-03-13_15-27-59/scofield_2.vmdk"
Destination disk format: VMFS thin-provisioned
Cloning disk '/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/scofield/scofield_2.vmdk'...
Clone: 85% done.
2011-03-13 15:28:05 -- debug: /usr/sbin/vmkfstools -i "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/scofield/scofield_1.vmdk" -a "buslogic" -d "thin" "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/scofield/scofield-2011-03-13_15-27-59/scofield_1.vmdk"
2011-03-13 15:28:06 -- debug: /usr/sbin/vmkfstools -i "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/scofield/scofield.vmdk" -a "buslogic" -d "thin" "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/scofield/scofield-2011-03-13_15-27-59/scofield.vmdk"
Destination disk format: VMFS thin-provisioned
Cloning disk '/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/scofield/scofield.vmdk'...
Clone: 78% done.
2011-03-13 15:29:52 -- info: Backup Duration: 1.83 Minutes
2011-03-13 15:29:52 -- info: Successfully completed backup for scofield!
2011-03-13 15:29:54 -- debug: Storage Information after backup:
2011-03-13 15:29:54 -- debug: SRC_DATASTORE: himalaya-local-SATA.RE4-GP:Storage
2011-03-13 15:29:54 -- debug: SRC_DATASTORE_CAPACITY: 1830.5 GB
2011-03-13 15:29:54 -- debug: SRC_DATASTORE_FREE: 539.4 GB
2011-03-13 15:29:54 -- debug: SRC_DATASTORE_BLOCKSIZE: 4
2011-03-13 15:29:54 -- debug: SRC_DATASTORE_MAX_FILE_SIZE: 1024 GB
2011-03-13 15:29:54 -- debug:
2011-03-13 15:29:54 -- debug: DST_DATASTORE: dlgCore-NFS-bigboi.VM-Backups
2011-03-13 15:29:54 -- debug: DST_DATASTORE_CAPACITY: 1348.4 GB
2011-03-13 15:29:54 -- debug: DST_DATASTORE_FREE: 296.8 GB
2011-03-13 15:29:54 -- debug: DST_DATASTORE_BLOCKSIZE: NA
2011-03-13 15:29:54 -- debug: DST_DATASTORE_MAX_FILE_SIZE: NA
2011-03-13 15:29:54 -- debug:
2011-03-13 15:29:55 -- debug: Storage Information before backup:
2011-03-13 15:29:55 -- debug: SRC_DATASTORE: himalaya-local-SATA.RE4-GP:Storage
2011-03-13 15:29:55 -- debug: SRC_DATASTORE_CAPACITY: 1830.5 GB
2011-03-13 15:29:55 -- debug: SRC_DATASTORE_FREE: 539.4 GB
2011-03-13 15:29:55 -- debug: SRC_DATASTORE_BLOCKSIZE: 4
2011-03-13 15:29:55 -- debug: SRC_DATASTORE_MAX_FILE_SIZE: 1024 GB
2011-03-13 15:29:55 -- debug:
2011-03-13 15:29:55 -- debug: DST_DATASTORE: dlgCore-NFS-bigboi.VM-Backups
2011-03-13 15:29:55 -- debug: DST_DATASTORE_CAPACITY: 1348.4 GB
2011-03-13 15:29:55 -- debug: DST_DATASTORE_FREE: 296.8 GB
2011-03-13 15:29:55 -- debug: DST_DATASTORE_BLOCKSIZE: NA
2011-03-13 15:29:55 -- debug: DST_DATASTORE_MAX_FILE_SIZE: NA
2011-03-13 15:29:55 -- debug:
2011-03-13 15:29:55 -- info: Snapshot found for vMA, backup will not take place
2011-03-13 15:29:57 -- debug: Storage Information before backup:
2011-03-13 15:29:57 -- debug: SRC_DATASTORE: himalaya-local-SATA.RE4-GP:Storage
2011-03-13 15:29:57 -- debug: SRC_DATASTORE_CAPACITY: 1830.5 GB
2011-03-13 15:29:57 -- debug: SRC_DATASTORE_FREE: 539.4 GB
2011-03-13 15:29:57 -- debug: SRC_DATASTORE_BLOCKSIZE: 4
2011-03-13 15:29:57 -- debug: SRC_DATASTORE_MAX_FILE_SIZE: 1024 GB
2011-03-13 15:29:57 -- debug:
2011-03-13 15:29:57 -- debug: DST_DATASTORE: dlgCore-NFS-bigboi.VM-Backups
2011-03-13 15:29:57 -- debug: DST_DATASTORE_CAPACITY: 1348.4 GB
2011-03-13 15:29:57 -- debug: DST_DATASTORE_FREE: 296.8 GB
2011-03-13 15:29:57 -- debug: DST_DATASTORE_BLOCKSIZE: NA
2011-03-13 15:29:57 -- debug: DST_DATASTORE_MAX_FILE_SIZE: NA
2011-03-13 15:29:57 -- debug:
2011-03-13 15:29:58 -- info: Initiate backup for vCloudConnector
2011-03-13 15:29:58 -- debug: /usr/sbin/vmkfstools -i "/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/vCloudConnector/vCloudConnector.vmdk" -a "buslogic" -d "thin" "/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/vCloudConnector/vCloudConnector-2011-03-13_15-27-59/vCloudConnector.vmdk"
Destination disk format: VMFS thin-provisioned
Cloning disk '/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/vCloudConnector/vCloudConnector.vmdk'...
Clone: 97% done.
2011-03-13 15:30:45 -- info: Backup Duration: 47 Seconds
2011-03-13 15:30:45 -- info: WARN: vCloudConnector has some Independent VMDKs that can not be backed up!
2011-03-13 15:30:45 -- info: ###### Final status: ERROR: Only some of the VMs backed up, and some disk(s) failed! ######
2011-03-13 15:30:45 -- debug: Succesfully removed lock directory - /tmp/ghettoVCB.lock
2011-03-13 15:30:45 -- info: ============================== ghettoVCB LOG END ================================
[root@himalaya ~]# ./ghettoVCB.sh -f vms_to_backup
# ./ghettoVCB.sh -m MyVM
/ghettoVCB # ./ghettoVCB.sh -a
/ghettoVCB # ./ghettoVCB.sh -a -e vm_exclusion_list
1. Create folder to hold individual VM backup policies (can be named anything):
[root@himalaya ~]# mkdir backup_config
2. Create individual VM backup policies for each VM that ensure each file is named exactly as the display name of the VM being backed up (use provided template to create duplicates):
[root@himalaya backup_config]# cp ghettoVCB-vm_backup_configuration_template scofield
[root@himalaya backup_config]# cp ghettoVCB-vm_backup_configuration_template vCloudConnector
Listing of VM backup policy within backup configuration directory
[root@himalaya backup_config]# ls
scofield vCloudConnector
ghettoVCB-vm_backup_configuration_template
Backup policy for "scofield" (backup only 2 specific VMDKs)
[root@himalaya backup_config]# cat scofield
scofield_2.vmdk,scofield_1.vmdk
VM_BACKUP_VOLUME=/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS
DISK_BACKUP_FORMAT=thin
VM_BACKUP_ROTATION_COUNT=3
POWER_VM_DOWN_BEFORE_BACKUP=0
ENABLE_HARD_POWER_OFF=0
ITER_TO_WAIT_SHUTDOWN=4
POWER_DOWN_TIMEOUT=5
SNAPSHOT_TIMEOUT=15
ENABLE_COMPRESSION=0
VM_SNAPSHOT_MEMORY=0
VM_SNAPSHOT_QUIESCE=0
VMDK_FILES_TO_BACKUP=""
Backup policy for VM "vCloudConnector" (backup all VMDKs found)
[root@himalaya backup_config]# cat
vCloudConnectorVM_BACKUP_VOLUME=/vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS
vCloudConnector.vmdk
DISK_BACKUP_FORMAT=thin
VM_BACKUP_ROTATION_COUNT=3
POWER_VM_DOWN_BEFORE_BACKUP=0
ENABLE_HARD_POWER_OFF=0
ITER_TO_WAIT_SHUTDOWN=4
POWER_DOWN_TIMEOUT=5
SNAPSHOT_TIMEOUT=15
ENABLE_COMPRESSION=0
VM_SNAPSHOT_MEMORY=0
VM_SNAPSHOT_QUIESCE=0
VMDK_FILES_TO_BACKUP=""
Note: When specifying -c option (individual VM backup policy mode) if a VM is listed in the backup list but DOES NOT have a corresponding backup policy, the VM will be backed up using the default configuration found within the ghettoVCB.sh script.
Execution of backup
[root@himalaya ~]# ./ghettoVCB.sh -f vms_to_backup -c backup_config -l /tmp/ghettoVCB.log
2011-03-13 15:40:50 -- info: ============================== ghettoVCB LOG START ==============================
2011-03-13 15:40:51 -- info: CONFIG - USING CONFIGURATION FILE = backup_config//scofield
2011-03-13 15:40:51 -- info: CONFIG - VERSION = 2011_03_13_1
2011-03-13 15:40:51 -- info: CONFIG - GHETTOVCB_PID = 2967
2011-03-13 15:40:51 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS
2011-03-13 15:40:51 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 3
2011-03-13 15:40:51 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2011-03-13_15-40-50
2011-03-13 15:40:51 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
2011-03-13 15:40:51 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0
2011-03-13 15:40:51 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0
2011-03-13 15:40:51 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 4
2011-03-13 15:40:51 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5
2011-03-13 15:40:51 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15
2011-03-13 15:40:51 -- info: CONFIG - LOG_LEVEL = info
2011-03-13 15:40:51 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCB.log
2011-03-13 15:40:51 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0
2011-03-13 15:40:51 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0
2011-03-13 15:40:51 -- info: CONFIG - VMDK_FILES_TO_BACKUP = scofield_2.vmdk,scofield_1.vmdk
2011-03-13 15:40:51 -- info: CONFIG - EMAIL_LOG = 0
2011-03-13 15:40:51 -- info:
2011-03-13 15:40:53 -- info: Initiate backup for scofield
Destination disk format: VMFS thin-provisioned
Cloning disk '/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/scofield/scofield_2.vmdk'...
Clone: 100% done.
Destination disk format: VMFS thin-provisioned
Cloning disk '/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/scofield/scofield_1.vmdk'...
Clone: 100% done.
2011-03-13 15:40:55 -- info: Backup Duration: 2 Seconds
2011-03-13 15:40:55 -- info: Successfully completed backup for scofield!
2011-03-13 15:40:57 -- info: CONFIG - VERSION = 2011_03_13_1
2011-03-13 15:40:57 -- info: CONFIG - GHETTOVCB_PID = 2967
2011-03-13 15:40:57 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS
2011-03-13 15:40:57 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 3
2011-03-13 15:40:57 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2011-03-13_15-40-50
2011-03-13 15:40:57 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
2011-03-13 15:40:57 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0
2011-03-13 15:40:57 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0
2011-03-13 15:40:57 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3
2011-03-13 15:40:57 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5
2011-03-13 15:40:57 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15
2011-03-13 15:40:57 -- info: CONFIG - LOG_LEVEL = info
2011-03-13 15:40:57 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCB.log
2011-03-13 15:40:57 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0
2011-03-13 15:40:57 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0
2011-03-13 15:40:57 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all
2011-03-13 15:40:57 -- info: CONFIG - EMAIL_LOG = 0
2011-03-13 15:40:57 -- info:
2011-03-13 15:40:59 -- info: Snapshot found for vMA, backup will not take place
2011-03-13 15:40:59 -- info: CONFIG - USING CONFIGURATION FILE = backup_config//vCloudConnector
2011-03-13 15:40:59 -- info: CONFIG - VERSION = 2011_03_13_1
2011-03-13 15:40:59 -- info: CONFIG - GHETTOVCB_PID = 2967
2011-03-13 15:40:59 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS
2011-03-13 15:40:59 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 3
2011-03-13 15:40:59 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2011-03-13_15-40-50
2011-03-13 15:40:59 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
2011-03-13 15:40:59 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0
2011-03-13 15:40:59 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0
2011-03-13 15:40:59 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 4
2011-03-13 15:40:59 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5
2011-03-13 15:40:59 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15
2011-03-13 15:40:59 -- info: CONFIG - LOG_LEVEL = info
2011-03-13 15:40:59 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCB.log
2011-03-13 15:40:59 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0
2011-03-13 15:40:59 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0
2011-03-13 15:40:59 -- info: CONFIG - VMDK_FILES_TO_BACKUP = vCloudConnector.vmdk
2011-03-13 15:40:59 -- info: CONFIG - EMAIL_LOG = 0
2011-03-13 15:40:59 -- info:
2011-03-13 15:41:01 -- info: Initiate backup for vCloudConnector
Destination disk format: VMFS thin-provisioned
Cloning disk '/vmfs/volumes/himalaya-local-SATA.RE4-GP:Storage/vCloudConnector/vCloudConnector.vmdk'...
Clone: 100% done.
2011-03-13 15:41:51 -- info: Backup Duration: 50 Seconds
2011-03-13 15:41:51 -- info: WARN: vCloudConnector has some Independent VMDKs that can not be backed up!
2011-03-13 15:41:51 -- info: ###### Final status: ERROR: Only some of the VMs backed up, and some disk(s) failed! ######
2011-03-13 15:41:51 -- info: ============================== ghettoVCB LOG END ================================
Please take a look at FAQ #25 for more details before continuing
To make use of this feature, modify the variable ENABLE_COMPRESSION from 0 to 1. Please note, do not mix uncompressed backups with compressed backups. Ensure that directories selected for backups do not contain any backups with previous versions of ghettoVCB before enabling and implementing the compressed backups feature.
nc (netcat) utility must be present for email support to function, this utility is a now a default with the release of vSphere 4.1 or greater, previous releases of VI 3.5 and/or vSphere 4.0 does not contain this utility. The reason this is listed as experimental is it may not be compatible with all email servers as the script utlizes nc (netcat) utility to communicate to an email server. This feature is provided as-is with no guarantees. If you enable this feature, a separate log will be generated along side any normal logging which will be used to email recipient. If for whatever reason, the email fails to send, an entry will appear per the normal logging mechanism.
Users should also make note due to limited functionality of netcat, it uses SMTP pipelining which is not the most ideal method of communicating with an SMTP server. Email from ghettoVCB may not work if your email server does not support this feature.
You can define an email recipient in the following two ways:
EMAIL_TO=william@virtuallyghetto.com
OR
EMAIL_TO=william@virtuallyghetto.com,tuan@virtuallyghetto.com
If you are running ESXi 5.1, you will need to create a custom firewall rule to allow your email traffic to go out which I will assume is default port 25. Here are the steps for creating a custom email rule.
Step 1 - Create a file called /etc/vmware/firewall/email.xml with contains the following:
<ConfigRoot>
<service>
<id>email</id>
<rule id="0000">
<direction>outbound</direction>
<protocol>tcp</protocol>
<porttype>dst</porttype>
<port>25</port>
</rule>
<enabled>true</enabled>
<required>false</required>
</service>
</ConfigRoot>
Step 2 - Reload the ESXi firewall by running the following ESXCLI command:
~ #
esxcli network firewall refresh
Step 3 - Confirm that your email rule has been loaded by running the following ESXCLI command:
~ # esxcli network firewall ruleset list | grep email
email true
Step 4 - Connect to your email server by usingn nc (netcat) by running the following command and specifying the IP Address/Port of your email server:
~ # nc 172.30.0.107 25
220 mail.primp-industries.com ESMTP Postfix
You should recieve a response from your email server and you can enter Ctrl+C to exit. This custom ESXi firewall rule will not persist after a reboot, so you should create a custom VIB to ensure it persists after a system reboot. Please take a look at this article for the details.
To make use of this feature, modify the variable RSYNC_LINK from 0 to 1. Please note, this is an experimental feature request from users that rely on rsync to replicate changes from one datastore volume to another datastore volume. The premise of this feature is to have a standardized folder that rsync can monitor for changes to replicate to another backup datastore. When this feature is enabled, a symbolic link will be generated with the format of "<VMNAME>-symlink" and will reference the latest successful VM backup. You can then rely on this symbolic link to watch for changes and replicate to your backup datastore.
Here is an example of what this would look like:
[root@himalaya ghettoVCB]# ls -la /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/vcma/
total 0
drwxr-xr-x 1 nobody nobody 110 Sep 27 08:08 .
drwxr-xr-x 1 nobody nobody 17 Sep 16 14:01 ..
lrwxrwxrwx 1 nobody nobody 89 Sep 27 08:08 vcma-symlink -> /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS/vcma/vcma-2010-09-27_08-07-37
drwxr-xr-x 1 nobody nobody 58 Sep 27 08:04 vcma-2010-09-27_08-04-26
drwxr-xr-x 1 nobody nobody 58 Sep 27 08:06 vcma-2010-09-27_08-05-55
drwxr-xr-x 1 nobody nobody 58 Sep 27 08:08 vcma-2010-09-27_08-07-37
FYI - This feature has not been tested, please provide feedback if this does not work as expected.
To recover a VM that has been processed by ghettoVCB, please take a look at this document: Ghetto Tech Preview - ghettoVCB-restore.sh - Restoring VM's backed up from ghettoVCB to ESX(i) 3.5, ...
There may be a situation where you need to stop the ghettoVCB process and entering Ctrl+C will only kill off the main ghettoVCB process, however there may still be other spawn processes that you may need to identify and stop. Below are two scenarios you may encounter and the process to completely stop all processes related to ghettoVCB.
Step 1 - Press Ctrl+C which will kill off the main ghettoVCB instance
Step 2 - Search for any existing ghettoVCB process by running the following:
# ps -c | grep ghettoVCB | grep -v grep
3360136 3360136 tail tail -f /tmp/ghettoVCB.work/ghettovcb.Cs1M1x
Step 3 - Here we can see there is a tail command that was used in the script. We need to stop this process by using the kill command which accepts the PID (Process ID) which is identified by the first value on the far left hand side of the command. In this example, it is 3360136.
# kill -9 3360136
Note: Make sure you identify the correct PID, else you could accidently impact a running VM or worse your ESXi host.
Step 4 - Depending on where you stopped the ghettoVCB process, you may need to consolidate or remove any existing snapshots that may exist on the VM that was being backed up. You can easily do so by using the vSphere Client.
Step 1 - Search for the ghettoVCB process (you can also validate the PID from the logs)
~ # ps -c | grep ghettoVCB | grep -v grep
3360393 3360393 busybox ash ./ghettoVCB.sh -f list -d debug
3360790 3360790 tail tail -f /tmp/ghettoVCB.work/ghettovcb.deGeB7
Step 2 - Stop both the main ghettoVCB instance & tail command by using the kill command and specifying their respective PID IDs:
kill -9 3360393
kill -9 3360790
Step 3 - If a VM was in the process of being backed up, there is an additional process for the actual vmkfstools copy. You will need to identify the process for that and kill that as well. We will again use ps -c command and search for any vmkfstools that are running:
# ps -c | grep vmkfstools | grep -v grep
3360796 3360796 vmkfstools /sbin/vmkfstools -i /vmfs/volumes/himalaya-temporary/VC-Windows/VC-Windows.vmdk -a lsilogic -d thin /vmfs/volumes/test-dont-use-this-volume/backups/VC-Windows/VC-Windows-2013-01-26_16-45-35/VC-Windows.vmdk
Step 4 - In case there is someone manually running a vmkfstools, make sure you take a look at the command itself and that it maps back to the current VM that was being backed up before kill the process. Once you have identified the proper PID, go ahead and use the kill command:
# kill -9 3360796
Step 5 - Depending on where you stopped the ghettoVCB process, you may need to consolidate or remove any existing snapshots that may exist on the VM that was being backed up. You can easily do so by using the vSphere Client.
Please take a moment to read over what is a cronjob and how to set one up, before continuing
The task of configuring cronjobs on classic ESX servers (with Service Console) is no different than traditional cronjobs on *nix operating systems (this procedure is outlined in the link above). With ESXi on the other hand, additional factors need to be taken into account when setting up cronjobs in the limited shell console called Busybox because changes made do not persist through a system reboot. The following document will outline steps to ensure that cronjob configurations are saved and present upon a reboot.
Important Note: Always redirect the ghettoVCB output to /dev/null and/or to a log when automating via cron, this becomes very important as one user has identified a limited amount of buffer capacity in which once filled, may cause ghettoVCB to stop in the middle of a backup. This primarily only affects users on ESXi, but it is good practice to always redirect the output. Also ensure you are specifying the FULL PATH when referencing the ghettoVCB script, input or log files.
e.g.
0 0 * * 1-5 /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/ghettoVCB.sh -f /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/backuplist > /dev/null
or
0 0 * * 1-5 /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/ghettoVCB.sh -f /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/backuplist > /tmp/ghettoVCB.log
Task: Configure ghettoVCB.sh to execute a backup five days a week (M-F) at 12AM (midnight) everyday and send output to a unique log file
Configure on ESX:
1. As root, you'll install your cronjob by issuing:
[root@himalaya ~]# crontab -e
2. Append the following entry:
0 0 * * 1-5 /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/ghettoVCB.sh -f /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/backuplist > /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/ghettoVCB-backup-$(date +\%s).log
3. Save and exit
[root@himalaya dlgCore-NFS-bigboi.VM-Backups]# crontab -e
no crontab for root - using an empty one
crontab: installing new crontab
4. List out and verify the cronjob that was just created:
[root@himalaya dlgCore-NFS-bigboi.VM-Backups]# crontab -l
0 0 * * 1-5 /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/ghettoVCB.sh -f /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/backuplist > /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/ghettoVCB-backup-$(date +\%s).log
You're ready to go!
Configure on ESXi:
1. Setup the cronjob by appending the following line to /var/spool/cron/crontabs/root:
0 0 * * 1-5 /vmfs/volumes/simplejack-local-storage/ghettoVCB.sh -f /vmfs/volumes/simplejack-local-storage/backuplist > /vmfs/volumes/simplejack-local-storage/ghettoVCB-backup-$(date +\%s).log
If you are unable to edit/modify /var/spool/cron/crontabs/root, please make a copy and then edit the copy with the changes
cp /var/spool/cron/crontabs/root /var/spool/cron/crontabs/root.backup
Once your changes have been made, then "mv" the backup to the original file. This may occur on ESXi 4.x or 5.x hosts
mv /var/spool/cron/crontabs/root.backup /var/spool/cron/crontabs/root
You can now verify the crontab entry has been updated by using "cat" utility.
2. Kill the current crond (cron daemon) and then restart the crond for the changes to take affect:
On ESXi < 3.5u3
kill $(ps | grep crond | cut -f 1 -d ' ')
On ESXi 3.5u3+
~ # kill $(pidof crond)
~ # crond
On ESXi 4.x/5.0
~ # kill $(cat /var/run/crond.pid)
~ # busybox crond
On ESXi 5.1 to 6.x
~ # kill $(cat /var/run/crond.pid)
~ # crond
On ESXi 7.x
~ # kill $(cat /var/run/crond.pid)
~ # /usr/lib/vmware/busybox/bin/busybox crond
3. Now that the cronjob is ready to go, you need to ensure that this cronjob will persist through a reboot. You'll need to add the following two lines to /etc/rc.local (ensure that the cron entry matches what was defined above). In ESXi 5.1, you will need to edit /etc/rc.local.d/local.sh instead of /etc/rc.local as that is no longer valid.
On ESXi 3.5
/bin/kill $(pidof crond)
/bin/echo "0 0 * * 1-5 /vmfs/volumes/simplejack-local-storage/ghettoVCB.sh -f /vmfs/volumes/simplejack-local-storage/backuplist > /vmfs/volumes/simplejack-local-storage/ghettoVCB-backup-\$(date +\\%s).log" >> /var/spool/cron/crontabs/root
crond
On ESXi 4.x/5.0
/bin/kill $(cat /var/run/crond.pid)
/bin/echo "0 0 * * 1-5 /vmfs/volumes/simplejack-local-storage/ghettoVCB.sh -f /vmfs/volumes/simplejack-local-storage/backuplist > /vmfs/volumes/simplejack-local-storage/ghettoVCB-backup-\$(date +\\%s).log" >> /var/spool/cron/crontabs/root
/bin/busybox crond
On ESXi 5.1 to 6.x
/bin/kill $(cat /var/run/crond.pid)
/bin/echo "0 0 * * 1-5 /vmfs/volumes/simplejack-local-storage/ghettoVCB.sh -f /vmfs/volumes/simplejack-local-storage/backuplist > /vmfs/volumes/simplejack-local-storage/ghettoVCB-backup-\$(date +\\%s).log" >> /var/spool/cron/crontabs/root
crond
On ESXi 7.x
/bin/kill $(cat /var/run/crond.pid) > /dev/null 2>&1
/bin/echo "0 0 * * 1-5 /vmfs/volumes/simplejack-local-storage/ghettoVCB.sh -f /vmfs/volumes/simplejack-local-storage/backuplist > /vmfs/volumes/simplejack-local-storage/ghettoVCB-backup-\$(date +\\%s).log" >> /var/spool/cron/crontabs/root
/usr/lib/vmware/busybox/bin/busybox crond
Afterwards the file should look like the following:
~ # cat /etc/rc.local
#! /bin/ash
export PATH=/sbin:/bin
log() {
echo "$1"
logger init "$1"
}
#execute all service retgistered in /etc/rc.local.d
if [http:// -d /etc/rc.local.d |http:// -d /etc/rc.local.d ]; then
for filename in `find /etc/rc.local.d/ | sort`
do
if [ -f $filename ] && [ -x $filename ]; then
log "running $filename"
$filename
fi
done
fi
/bin/kill $(cat /var/run/crond.pid)
/bin/echo "0 0 * * 1-5 /vmfs/volumes/simplejack-local-storage/ghettoVCB.sh -f /vmfs/volumes/simplejack-local-storage/backuplist > /vmfs/volumes/simplejack-local-storage/ghettoVCB-backup-\$(date +\\%s).log" >> /var/spool/cron/crontabs/root
/bin/busybox crond
This will ensure that the cronjob is re-created upon a reboot of the system through a startup script
2. To ensure that this is saved in the ESXi configuration, we need to manually initiate an ESXi backup by running:
~ # /sbin/auto-backup.sh
config implicitly loaded
local.tgz
etc/vmware/vmkiscsid/vmkiscsid.db
etc/dropbear/dropbear_dss_host_key
etc/dropbear/dropbear_rsa_host_key
etc/opt/vmware/vpxa/vpxa.cfg
etc/opt/vmware/vpxa/dasConfig.xml
etc/sysconfig/network
etc/vmware/hostd/authorization.xml
etc/vmware/hostd/hostsvc.xml
etc/vmware/hostd/pools.xml
etc/vmware/hostd/vmAutoStart.xml
etc/vmware/hostd/vmInventory.xml
etc/vmware/hostd/proxy.xml
etc/vmware/ssl/rui.crt
etc/vmware/ssl/rui.key
etc/vmware/vmkiscsid/initiatorname.iscsi
etc/vmware/vmkiscsid/iscsid.conf
etc/vmware/vmware.lic
etc/vmware/config
etc/vmware/dvsdata.db
etc/vmware/esx.conf
etc/vmware/license.cfg
etc/vmware/locker.conf
etc/vmware/snmp.xml
etc/group
etc/hosts
etc/inetd.conf
etc/rc.local
etc/chkconfig.db
etc/ntp.conf
etc/passwd
etc/random-seed
etc/resolv.conf
etc/shadow
etc/sfcb/repository/root/interop/cim_indicationfilter.idx
etc/sfcb/repository/root/interop/cim_indicationhandlercimxml.idx
etc/sfcb/repository/root/interop/cim_listenerdestinationcimxml.idx
etc/sfcb/repository/root/interop/cim_indicationsubscription.idx
Binary files /etc/vmware/dvsdata.db and /tmp/auto-backup.31345.dir/etc/vmware/dvsdata.db differ
config implicitly loaded
Saving current state in /bootbank
Clock updated.
Time: 20:40:36 Date: 08/14/2009 UTC
Now you're really done!
If you're still having trouble getting the cronjob to work, ensure that you've specified the correct parameters and there aren’t any typos in any part of the syntax.
Ensure crond (cron daemon) is running:
ESX 3.x/4.0:
[root@himalaya dlgCore-NFS-bigboi.VM-Backups]# ps -ef | grep crond | grep -v grep
root 2625 1 0 Aug13 ? 00:00:00 crond
ESXi 3.x/4.x/5.x:
~ # ps | grep crond | grep -v grep
5196 5196 busybox crond
Ensure that the date/time on your ESX(i) host is setup correctly:
ESX(i):
[root@himalaya dlgCore-NFS-bigboi.VM-Backups]# date
Fri Aug 14 23:44:47 PDT 2009
Note: Careful attention must be noted if more than one backup is performed per day. Backup windows should be staggered to avoid contention or saturation of resources during these periods.
0Q: I'm getting error X when using the script or I'm not getting any errors, the backup didn’t even take place. What can I do?
0A: First off, before posting a comment/question, please thoroughly read through the ENTIRE documentation including the FAQs to see if your question has already been ansered.
1Q: I've read through the entire documentation + FAQs and still have not found my answer to the problem I'm seeing. What can I do?
1A: Please join the ghettoVCB Group to post your question/comment.
2Q: I've sent you private message or email but I haven't received a response? What gives?
2A: I do not accept issues/bugs reported via PM or email, I will reply back, directing you to post on the appropriate VMTN forum (that's what it's for). If the data/results you're providing is truely senstive to your environment I will hear you out, but 99.99% it is not, so please do not messsage/email me directly. I do monitor all forums that contain my script including the normal VMTN forums and will try to get back to your question as soon as I can and as time permits. Please do be patient as you're not the only person using the script (600,000+ views), thank you.
3Q: Can I schedule backups to take place hourly, daily, monthly, yearly?
3A: Yes, do a search online for crontab.
4Q: I would like to setup cronjob for ESX(i) 3.5 or 4.0?
4A: Take a look at the Cronjob FAQ section in this document.
5Q: I want to schedule my backup on Windows, how do I do this?
5A: Do a search for plink. Make sure you have paired SSH keys setup between your Windows system and ESX/ESXi host.
6Q: I only have a single ESXi host. I want to take backups and store them somewhere else. The problem is: I don't have NFS, iSCSI nor FC SAN. What can I do?
6A: You can use local storage to store your backups assuming that you have enough space on the destination datastore. Afterwards, you can use scp (WinSCP/FastSCP) to transfer the backups from the ESXi host to your local desktop.
7Q: I’m pissed; the backup is taking too long. My datastore is of type X?
7A: YMMV, take a look at your storage configuration and make sure it is optimized.
8Q: I noticed that the backup rotation is occurring after a backup. I don't have enough local storage space, can the process be changed?
8A: This is primarily done to ensure that you have at least one good backup in case the new backup fails. If you would like to modify the script, you're more than welcome to do so.
9Q: What is the best storage configuration for datastore type X?
9A: Search the VMTN forums; there are various configurations for the different type of storage/etc.
10Q: I want to setup an NFS server to run my backups. Which is the best and should it be virtual or physical?
10A: Please refer to answer 7A. From experience, we’ve seen physical instances of NFS servers to be faster than their virtual counterparts. As always, YMMV.
11Q: I have VMs that have snapshots. I want to back these things up but the script doesn’t let me do it. How do I fix that?
11A: VM snapshots are not meant to be kept for long durations. When backing up a VM that contains a snapshot, you should ensure all snapshots have been committed prior to running a backup. No exceptions will be made…ever.
12Q: I would like to restore from backup, what is the best method?
12A: The restore process will be unique for each environment and should be determined by your backup/recovery plans. At a high level you have the option of mounting the backup datastore and registering the VM in question or copy the VM from the backup datastore to the ESX/ESXi host. The latter is recommended so that you're not running a VM living on the backup datastore or inadvertently modifying your backup VM(s). You can also take a look at ghettoVCB-restore which is experimentally supported.
13Q: When I try to run the script I get: "-bash: ./ghettoVCB.sh: Permission denied", what is wrong?
13A: You need to change the permission on the script to be executable, chmod +x ghettoVCB.sh
14Q: Where can I download the latest version of the script?
14A: The latest version is available on on github - https://github.com/lamw/ghettoVCB/downloads
15Q: I would like to suggest/recommend feature X, can I get it? When can I get it? Why isn't it here, what gives?
15A: The general purpose of this script is to provide a backup solution around VMware VMs. Any additional features outside of that process will be taken into consideration depending on the amount of time, number of requests and actual usefulness as a whole to the community rather than to an individual.
16Q: I have found this script to be very useful and would like to contribute back, what can I do?
16A: To continue to develop and share new scripts and resources with the community, we need your support. You can donate here Thank You!
17Q: What are the different type of backup uses cases that are supported with ghettoVCB?
17A: 1) Live backup of VM with the use of a snapshot and 2) Offline backup of a VM without a snapshot. These are the only two use cases supported by the script.
18Q: When I execute the script on ESX(i) I get some funky errors such as ": not found.sh" or "command not found". What is this?
18A: Most likely you have some ^M characters within the script which may have come from either editing the script using Windows editor, uploading the script using the datastore browser OR using wget. The best option is to either using WinSCP on Windows to upload the script and edit using vi editor on ESX(i) host OR Linux/UNIX scp to copy the script into the host. If you still continue to have the issue, do a search online on various methods of removing this Windows return carriage from the script
19Q: My backup works fine OR it works for a single backup but I get an error message "Input/output error" or "-ash: YYYY-MM-DD: not found" during the snapshot removal process. What is this?
19A: The issue has been recently identified by few users as a problem with user's NFS server in which it reports an error when deleting large files that take longer than 10seconds. VMware has recently released a KB article http://kb.vmware.com/kb/1035332 explaining the details and starting with vSphere 4.1 Update 2 or vSphere 5.0, a new advanced ESX(i) parameter has been introduced to increase the timeout. This has resolved the problem for several users and maybe something to consider if you are running into this issue, specifically with NFS based backups.
20Q: Will this script function with vCenter and DRS enabled?
20Q: No, if the ESX(i) hosts are in a DRS enabled cluster, VMs that are to be backed up could potentially be backed up twice or never get backed up. The script is executed on a per host basis and one would need to come up a way of tracking backups on all hosts and perhaps write out to external file to ensure that all VMs are backed up. The main use case for this script are for standalone ESX(i) host
21Q: I'm trying to use WinSCP to manually copy VM files but it's very slow or never completes on huge files, why is that?
21A: WinSCP was not designed for copying VM files out of your ESX(i) host, take a look at Veeam's FastSCP which is designed for moving VM files and is a free utility.
22Q: Can I use setup NFS Server using Windows Services for UNIX (WSFU) and will it work?
22A: I've only heard a handful of users that have successfully implemented WSFU and got it working, YMMV. VMware also has a KB article decribing the setup process here: http://kb.vmware.com/kb/1004490 for those that are interested. Here is a thread on a user's experience between Windows Vs. Linux NFS that maybe helpful.
23Q: How do VMware Snapshots work?
23A: http://kb.vmware.com/kb/1015180
24Q: What files make up a Virtual Machine?
24A: http://virtualisedreality.wordpress.com/2009/09/16/quick-reminder-of-what-files-make-up-a-virtual-ma...
25Q: I'm having some issues restoring a compressed VM backup?
25A: There is a limitation in the size of the VM for compression under ESXi 3.x & 4.x, this limitation is in the unsupported Busybox console and should not affect classic ESX 3.x/4.x. On ESXi 3.x, the maximum largest supported VM is 4GB for compression and on ESXi 4.x the largest supported VM is 8GB. If you try to compress a larger VM, you may run into issues when trying to extract upon a restore. PLEASE TEST THE RESTORE PROCESS BEFORE MOVING TO PRODUCTION SYSTEMS!
26Q: I'm backing up my VM as "thin" format but I'm still not noticing any size reduction in the backup? What gives?
2bA: Please refer to this blog post which explains what's going on: http://www.yellow-bricks.com/2009/07/31/storage-vmotion-and-moving-to-a-thin-provisioned-disk/
27Q: I've enabled VM_SNAPSHOT_MEMORY and when I restore my VM it's still offline, I thought this would keep it's memory state?
27A: VM_SNAPSHOT_MEMORY is only used to ensure when the snapshot is taken, it's memory contents are also captured. This is only relavent to the actual snapshot itself and it's not used in any shape/way/form in regards to the backup. All backups taken whether your VM is running or offline will result in an offline VM backup when you restore. This was originally added for debugging purposes and in generally should be left disabled
28Q: Can I rename the directories and the VMs after a VM has been backed up?
28A: The answer yes, you can ... but you may run into all sorts of issues which may break the backup process. The script expects a certain layout and specific naming scheme for it to maintain the proper rotation count. If you need to move or rename a VM, please take it out of the directory and place it in another location
29Q: Can ghettoVCB support CBT (Change Block Tracking)?
29A: No, that is a functionality of the vSphere API + VDDK API (vSphere Disk Development Kit). You will need to look at paid solutions such as VMware vDR, Veeam Backup & Recovery, PHD Virtual Backups, etc. to leverage that functionailty.
30Q: Does ghettoVCB support rsync backups?
30A: Currently ghettoVCB does not support rsync backups, you either obtain or compile your own static rsync binary and run on ESXi, but this is an unsupported configuration. You may take a look at this blog post for some details.
31Q: How can I contribute back?
31A: You can provide feedback/comments on the ghettoVCB Group. If you have found this script to be useful and would like to contribute back, please click here to donate.
32Q: How can select individual VMDKs to backup from a VM?
32A: Ideally you would use the "-c" option which requires you to create individual VM configuration file, this is where you would select specific VMDKs to backup. Note, that if you do not need to define all properties, anything not defined will adhere from the default global properties whether you're editing the ghettoVCB.sh script or using ghettoVCB global configuration file. It is not recommended that you edit the ghettoVCB.sh script and modify the VMDK_FILES_TO_BACKUP variable, but if you would like to keep everything in one script, you may add the extensive list of VMDKs to backup but do know this can get error prone as script may be edited frequently and lose some flexibility to support multiple environments.
33Q: Why is email not working when I'm using ESXi 5.x but it worked in ESXi 4.x?
33A: ESXi 5.x has implemented a new firewall which requires the email port that is being used to be opened. Please refer to the following articles on creating a custom firewall rule for email:
http://www.virtuallyghetto.com/2012/09/creating-custom-vibs-for-esxi-50-51.html
How to Create Custom Firewall Rules in ESXi 50
How to Persist Configuration Changes in ESXi 4.x/5.x Part 1
How to Persist Configuration Changes in ESXi 4.x/5.x Part 2
34Q: How do I stop the ghettoVCB process?
34A: Take a look at the Stopping ghettoVCB Process section of the documentation for more details.
Many have asked what is the best configuration and recommendation for setting up a cheap NFS Server to run backups for VMs. This has been a question we've tried to stay away from just because the possiblities and solutions are endless. One can go with physical vs. virtual, use VSA (Virtual Storage Appliances) such as OpenFiler or Lefthand Networks, Windows vs. Linux/UNIX. We've not personally tested and verify all these solutions and it all comes down to "it depends" type of answer. Though from our experience, we've had much better success with a physical server than a virtual.
It is also well known that some users are experiencing backup issues when running specifically against NFS, primarily around the rotation and purging of previous backups. The theory from what we can tell by talking to various users is that when the rotation is occuring, the request to delete the file(s) may take awhile and does not return within a certain time frame and causes the script to error out with unexpected messages. Though the backups were successful, it will cause unexpected results with directory structures on the NFS target. We've not been able to isolate why this is occuring and maybe due to NFS configuration/exports or hardware or connection not being able to support this process.
We'll continue to help where we can in diagonising this issus but we wanted to share our current NFS configuration, perhaps it may help some users who are new or trying to setup their system. ( Disclaimer: These configurations are not recommendations nor endorsement for any of the components being used)
UPDATE: Please also read FAQ #19 for details + resolution
Server Type: Physical
Model: HP DL320 G2
OS: Arch linux 2.6.28
Disks: 2 x 1.5TB
RAID: Software RAID1
Source Host Backups: ESX 3.5u4 and ESX 4.0u1 (We don't run any ESXi hosts)
uname -a output
Linux XXXXX.XXXXX.ucsb.edu 2.6.28-ARCH #1 SMP PREEMPT Sun Jan 18 20:17:17 UTC 2009 i686 Intel(R) Pentium(R) 4 CPU 3.06GHz GenuineIntel GNU/Linux
NICs:
00:05.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5702X Gigabit Ethernet (rev 02)
00:06.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5702X Gigabit Ethernet (rev 02)
NFS Export Options:
/exports/vm-backups XXX.XXX.XXX.XXX/24(rw,async,all_squash,anonuid=99,anongid=99)
*One important thing to check is to verify that your NFS exportion options are setup correctly, "async" should be configured to ensure that all IO requests are processed and reply back to the client before waiting for the data to be written to the storage.
*Recently VMware released a KB article describing the various "Advanced NFS Options" and their meanings and recommendations: http://kb.vmware.com/kb/1007909 We've not personally had to touch any of these, but for other vendors such as EMC and NetApp, there are some best practices around configuring some of these values depending on the number of NFS volumes or number of ESX(i) host connecting to a volume. You may want to take a look to see if any of these options may help with NFS issue that some are seeing
*Users should also try to look at their ESX(i) host logs during the time interval when they're noticing these issues and see if they can find any correlation along with monitoring the performance on their NFS Server.
*Lastly, there are probably other things that can be done to improve NFS performance or further optimization, a simple search online will also yield many resources.
Windows utility to email ghettoVCB Backup Logs - http://www.waldrondigital.com/2010/05/11/ghettovcb-e-mail-rotate-logs-batch-file-for-vmware/
Windows front-end utility to ghettoVCB - http://www.magikmon.com/mkbackup/ghettovcb.en.html
Note: Neither of these tools are supported, for questions or comments regarding these utilities please refer to the author's pages.
Enhancements:
Fixes:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Enhancements:
Fixes:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Enhancements:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Enhancements:
Fixes:
Enhancements:
Fixes:
Big thanks to Alain Spineux and his contributions to the ghettoVCB script and helping with debugging and testing.
Enhancements:
Fixes:
Enhancements:
Enhancements:
Enhancements:
Fixes:
Fixes:
Enhancements:
Fixes:
Fixes:
Big thanks goes out to the community for the suggested features and to those that submitted snippet of their modifications.
Updated FAQ #20-24 for common issues/questions. Also included a new section about our "personal" NFS configuration and setup.
Fix the crontab section to reflect the correct syntax + updated FAQ #17,#18 and #19 for common issues.
The following enhancements and fixes have been implemented in this release of ghettoVCB. Special thanks goes out to all the ghettoVCB BETA testers for providing time and their environments to test features/fixes of the new script!
Enhancements:
Fixes:
The VM has only 512 Mb of RAM
I manually created a snapshot in less than 3 minutes so no timeout problems
Interesting, something must have caused it to fail on the snapshot creation. You mentioned you backed up 10 other VMs perfectly fine, when this issue occurred was this part of an overall backup with 10 systems within a list? Have you tried to isolate it down to just this VM and do a single backup with just this VM?
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Yes it is a part of an overall backup with 10 VMs.
I tried to isolate it but the problem still remain
Please post a an execute run with only this single VM with both debug and dryrun mode. Could you also elaborate on what this VM is running? OS, etc?
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
The VM is Windows server 2003 enterprise with 512 mb ram
I isolated the backup
In debug the log seems ok
2009-12-14 07:33:28 -- info: ============================== ghettoVCB LOG START ==============================
2009-12-14 07:33:28 -- debug: HOST BUILD: VMware ESXi 4.0.0 build-181792
2009-12-14 07:33:28 -- debug: HOSTNAME: DELLESX.gruppoanthea.local
2009-12-14 07:33:29 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS
2009-12-14 07:33:29 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 1
2009-12-14 07:33:29 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
2009-12-14 07:33:29 -- info: CONFIG - ADAPTER_FORMAT = buslogic
2009-12-14 07:33:29 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0
2009-12-14 07:33:29 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0
2009-12-14 07:33:30 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3
2009-12-14 07:33:30 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5
2009-12-14 07:33:31 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15
2009-12-14 07:33:31 -- info: CONFIG - LOG_LEVEL = debug
2009-12-14 07:33:31 -- info: CONFIG - BACKUP_LOG_OUTPUT = web01Debug.log
2009-12-14 07:33:31 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0
2009-12-14 07:33:31 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0
2009-12-14 07:33:32 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all
2009-12-14 07:33:35 -- info: Initiate backup for web01
2009-12-14 07:33:35 -- info: Creating Snapshot "ghettoVCB-snapshot-2009-12-14" for web01
2009-12-14 07:33:38 -- debug: Waiting for snapshot "ghettoVCB-snapshot-2009-12-14" to be created
2009-12-14 07:33:38 -- debug: Snapshot timeout set to: 900 seconds
2009-12-14 07:33:39 -- debug: Waiting for snapshot creation to be completed - Iteration: 0 - sleeping for 60secs (Duration: 0 seconds)
Destination disk format: VMFS zeroedthick
Cloning disk '/vmfs/volumes/datastore1/web01/web01.vmdk'...
Clone: 0% done.
Clone: 1% done.
Clone: 2% done.
Clone: 3% done.
Clone: 4% done.
Clone: 5% done.
Clone: 6% done.
Clone: 7% done.
Clone: 8% done.
Clone: 9% done.
Clone: 10% done.
Clone: 11% done.
Clone: 12% done.
Clone: 13% done.
Clone: 14% done.
Clone: 15% done.
Clone: 16% done.
Clone: 17% done.
Clone: 18% done.
Clone: 19% done.
Clone: 20% done.
Clone: 21% done.
Clone: 22% done.
Clone: 23% done.
Clone: 24% done.
Clone: 25% done.
Clone: 26% done.
Clone: 27% done.
Clone: 28% done.
Clone: 29% done.
Clone: 30% done.
Clone: 31% done.
Clone: 32% done.
Clone: 33% done.
Clone: 34% done.
Clone: 35% done.
Clone: 36% done.
Clone: 37% done.
Clone: 38% done.
Clone: 39% done.
Clone: 40% done.
Clone: 41% done.
Clone: 42% done.
Clone: 43% done.
Clone: 44% done.
Clone: 45% done.
Clone: 46% done.
Clone: 47% done.
Clone: 48% done.
Clone: 49% done.
Clone: 50% done.
Clone: 51% done.
Clone: 52% done.
Clone: 53% done.
Clone: 54% done.
Clone: 55% done.
Clone: 56% done.
Clone: 57% done.
Clone: 58% done.
Clone: 59% done.
Clone: 60% done.
Clone: 61% done.
Clone: 62% done.
Clone: 63% done.
Clone: 64% done.
Clone: 65% done.
Clone: 66% done.
Clone: 67% done.
Clone: 68% done.
Clone: 69% done.
Clone: 70% done.
Clone: 71% done.
Clone: 72% done.
Clone: 73% done.
Clone: 74% done.
Clone: 75% done.
Clone: 76% done.
Clone: 77% done.
Clone: 78% done.
Clone: 79% done.
Clone: 80% done.
Clone: 81% done.
Clone: 82% done.
Clone: 83% done.
Clone: 84% done.
Clone: 85% done.
Clone: 86% done.
Clone: 87% done.
Clone: 88% done.
Clone: 89% done.
Clone: 90% done.
Clone: 91% done.
Clone: 92% done.
Clone: 93% done.
Clone: 94% done.
Clone: 95% done.
Clone: 96% done.
Clone: 97% done.
Clone: 98% done.
Clone: 99% done.
Clone: 100% done.
2009-12-14 09:54:26 -- info: Removing snapshot from web01 ...
But in the console I can see this error
./ghettoVCB.sh: line 735: syntax error: /vmfs/volumes/VMBACKUP/ESX15/web01/web01-2009-11-14+1
This is the dryrun log
2009-12-11 16:43:40 -- info: ============================== ghettoVCB LOG START ==============================
2009-12-11 16:43:40 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/WILLIAM_BACKUPS
2009-12-11 16:43:41 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 1
2009-12-11 16:43:41 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick
2009-12-11 16:43:41 -- info: CONFIG - ADAPTER_FORMAT = buslogic
2009-12-11 16:43:41 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0
2009-12-11 16:43:42 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0
2009-12-11 16:43:42 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3
2009-12-11 16:43:43 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5
2009-12-11 16:43:43 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15
2009-12-11 16:43:44 -- info: CONFIG - LOG_LEVEL = dryrun
2009-12-11 16:43:44 -- info: CONFIG - BACKUP_LOG_OUTPUT = web01DryRun.log
2009-12-11 16:43:44 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0
2009-12-11 16:43:44 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0
2009-12-11 16:43:44 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all
2009-12-11 16:43:46 -- dryrun: ###############################################
2009-12-11 16:43:47 -- dryrun: Virtual Machine: web01
2009-12-11 16:43:47 -- dryrun: VM_ID: 64
2009-12-11 16:43:47 -- dryrun: VMX_PATH: /vmfs/volumes/datastore1/web01/web01.vmx
2009-12-11 16:43:48 -- dryrun: VMX_DIR: /vmfs/volumes/datastore1/web01
2009-12-11 16:43:48 -- dryrun: VMX_CONF: web01/web01.vmx
2009-12-11 16:43:49 -- dryrun: VMFS_VOLUME: datastore1
2009-12-11 16:43:49 -- dryrun: VMDK(s):
2009-12-11 16:43:49 -- dryrun: web01.vmdk
2009-12-11 16:43:49 -- dryrun: ###############################################
2009-12-11 16:43:51 -- info: ============================== ghettoVCB LOG END ================================
Yea I don't see anything wrong with the logs either, but when you get anything relating to "+X" where X is some number, it's probably around the rotation function where it's renaming and deleting older backups. I don't see anything odd about your VM backup, so I'll have to try to reproduce this if I can in my lab.
I see this as your ESXi build: VMware ESXi 4.0.0 build-181792
What ESXi 4 version is this? update 1? is it patched? I'll need to try to duplicate the exact version to see if there's been any odd changes in the busybox console (which I hope isn't the case).
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I've tried searching through the comments for an answer.. sorry If this is a repeat question and thank you for this great script.
I have 4 ESX 3.5 servers using this script with great success so far. My question is how can I generate the vm_backup_list dynamically. VMGuests often move around from host to host and each time I have to update the backup list on all 4 hosts same when I create a new VMGuest. is there a way to generate the backup list dynamically?
thnks....
Majority of the users out there do not have vCenter and the script is executed on a per host basis w/o vMotion. Having said that, I know there are a few out there that run into this problem and to tell you the truth, this is not something I've not tested.
You can dynamically generate the list, but my fear is that if could be the case that the VM is backed up, it migrates and then some other host now sees this VM and tries to back it up. Basically you would some logic to keep track of what VMs have been backed up, perhaps an exclusion list for the given day/etc.
To dump out all the VMs residing on a given host is pretty trivial and if you're on the classic ESX, vmware-cmd -l can do it but if you just want the VM name which is eventually feed into the script to process, you can just use vimsh by doing something like:
vmware-vim-cmd vmsvc/getallvms | grep -v Vmid | awk '{print $2}' | sed '/^$/d'
and just redirect that out to a file and then have that be read into ghettoVCB to process
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
That's a Vmware ESXi patched but I had this problem also with older release.
I can try with the update 1 in the next days but I don't think that's a problem releated with the release of vmware.
I think that the problem is when renaming and deleting. Infact it create the backup files that are perfect but it leave the old backup folder.
hi,
i still have the problems with the snapshots:
sometimes the snapshots will be removed and sometimes not...
here the logs:
2009-12-15 10:25:01 -- info: ============================== ghettoVCB LOG START ============================== 2009-12-15 10:25:01 -- debug: HOST BUILD: VMware ESXi 4.0.0 build-171294 2009-12-15 10:25:01 -- debug: HOSTNAME: ESXi01-AIC.Domain.de 2009-12-15 10:25:01 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/QNAP01-AIC 2009-12-15 10:25:01 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 3 2009-12-15 10:25:01 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick 2009-12-15 10:25:01 -- info: CONFIG - ADAPTER_FORMAT = buslogic 2009-12-15 10:25:01 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0 2009-12-15 10:25:01 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0 2009-12-15 10:25:01 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3 2009-12-15 10:25:01 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5 2009-12-15 10:25:01 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15 2009-12-15 10:25:01 -- info: CONFIG - LOG_LEVEL = debug 2009-12-15 10:25:01 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCB2.log 2009-12-15 10:25:01 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0 2009-12-15 10:25:01 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0 2009-12-15 10:25:01 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all 2009-12-15 10:25:03 -- info: Initiate backup for AIC-KAV01 2009-12-15 10:25:03 -- info: Creating Snapshot "ghettoVCB-snapshot-2009-12-15" for AIC-KAV01 2009-12-15 10:25:03 -- debug: Waiting for snapshot "ghettoVCB-snapshot-2009-12-15" to be created 2009-12-15 10:25:03 -- debug: Snapshot timeout set to: 900 seconds Destination disk format: VMFS zeroedthick Cloning disk '/vmfs/volumes/datastore1/AIC-KAV01/AIC-KAV01.vmdk'... Clone: 100% done. 2009-12-15 10:38:44 -- info: Removing snapshot from AIC-KAV01 ... 2009-12-15 10:38:50 -- info: Backup Duration: 13.78 Minutes 2009-12-15 10:38:50 -- info: Successfully completed backup for AIC-KAV01! 2009-12-15 10:38:50 -- info: Snapshot found for VS01-Versand, backup will not take place 2009-12-15 10:38:50 -- info: ============================== ghettoVCB LOG END ================================
2009-12-15 10:51:32 -- info: ============================== ghettoVCB LOG START ============================== 2009-12-15 10:51:32 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/QNAP01-AIC 2009-12-15 10:51:32 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 3 2009-12-15 10:51:32 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick 2009-12-15 10:51:32 -- info: CONFIG - ADAPTER_FORMAT = buslogic 2009-12-15 10:51:32 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0 2009-12-15 10:51:32 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0 2009-12-15 10:51:32 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3 2009-12-15 10:51:32 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5 2009-12-15 10:51:32 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15 2009-12-15 10:51:32 -- info: CONFIG - LOG_LEVEL = dryrun 2009-12-15 10:51:32 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCB3.log 2009-12-15 10:51:32 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0 2009-12-15 10:51:32 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0 2009-12-15 10:51:32 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all 2009-12-15 10:51:33 -- dryrun: ############################################### 2009-12-15 10:51:33 -- dryrun: Virtual Machine: AIC-KAV01 2009-12-15 10:51:33 -- dryrun: VM_ID: 16 2009-12-15 10:51:33 -- dryrun: VMX_PATH: /vmfs/volumes/datastore1/AIC-KAV01/AIC-KAV01.vmx 2009-12-15 10:51:33 -- dryrun: VMX_DIR: /vmfs/volumes/datastore1/AIC-KAV01 2009-12-15 10:51:33 -- dryrun: VMX_CONF: AIC-KAV01/AIC-KAV01.vmx 2009-12-15 10:51:33 -- dryrun: VMFS_VOLUME: datastore1 2009-12-15 10:51:33 -- dryrun: VMDK(s): 2009-12-15 10:51:33 -- dryrun: AIC-KAV01.vmdk 2009-12-15 10:51:33 -- dryrun: ############################################### 2009-12-15 10:51:33 -- dryrun: ############################################### 2009-12-15 10:51:33 -- dryrun: Virtual Machine: VS01-Versand 2009-12-15 10:51:33 -- dryrun: VM_ID: 48 2009-12-15 10:51:33 -- dryrun: VMX_PATH: /vmfs/volumes/datastore1/VS01-Versand/VS01-Versand.vmx 2009-12-15 10:51:33 -- dryrun: VMX_DIR: /vmfs/volumes/datastore1/VS01-Versand 2009-12-15 10:51:33 -- dryrun: VMX_CONF: VS01-Versand/VS01-Versand.vmx 2009-12-15 10:51:33 -- dryrun: VMFS_VOLUME: datastore1 2009-12-15 10:51:33 -- dryrun: VMDK(s): 2009-12-15 10:51:33 -- dryrun: VS01-Versand-000002.vmdk 2009-12-15 10:51:33 -- dryrun: ############################################### 2009-12-15 10:51:33 -- info: ============================== ghettoVCB LOG END ================================
when i delete all snapshots with vspehre and start backup again, i get these both errors in the console:
rm: cannot remove '/vmfs/volumes/QNAP01-AIC/AIC-KAV01/AIC-KAV01-2009-12-14--3/AIC-KAV01-flat.vmdk': No such file or directory
rm: cannot remove '/vmfs/volumes/QNAP01-AIC/VS01-Versand/VS01-Versand-2009-12-15--3/VS01-Versand-flat.vmdk': Input/output error
and here is the log from backup after deleting snapshots with vsphere:
2009-12-15 12:07:22 -- info: ============================== ghettoVCB LOG START ============================== 2009-12-15 12:07:22 -- debug: HOST BUILD: VMware ESXi 4.0.0 build-171294 2009-12-15 12:07:22 -- debug: HOSTNAME: ESXi01-AIC.Domain.de 2009-12-15 12:07:22 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/QNAP01-AIC 2009-12-15 12:07:22 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 3 2009-12-15 12:07:22 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick 2009-12-15 12:07:22 -- info: CONFIG - ADAPTER_FORMAT = buslogic 2009-12-15 12:07:22 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0 2009-12-15 12:07:22 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0 2009-12-15 12:07:22 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3 2009-12-15 12:07:22 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5 2009-12-15 12:07:22 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15 2009-12-15 12:07:22 -- info: CONFIG - LOG_LEVEL = debug 2009-12-15 12:07:22 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCB.log 2009-12-15 12:07:22 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0 2009-12-15 12:07:22 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0 2009-12-15 12:07:22 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all 2009-12-15 12:07:23 -- info: Initiate backup for AIC-KAV01 2009-12-15 12:07:23 -- info: Creating Snapshot "ghettoVCB-snapshot-2009-12-15" for AIC-KAV01 2009-12-15 12:07:24 -- debug: Waiting for snapshot "ghettoVCB-snapshot-2009-12-15" to be created 2009-12-15 12:07:24 -- debug: Snapshot timeout set to: 900 seconds Destination disk format: VMFS zeroedthick Cloning disk '/vmfs/volumes/datastore1/AIC-KAV01/AIC-KAV01.vmdk'... Clone: 100% done. 2009-12-15 12:21:05 -- info: Removing snapshot from AIC-KAV01 ... 2009-12-15 12:21:38 -- info: Backup Duration: 14.25 Minutes 2009-12-15 12:21:38 -- info: Successfully completed backup for AIC-KAV01! 2009-12-15 12:21:39 -- info: Initiate backup for VS01-Versand 2009-12-15 12:21:39 -- info: Creating Snapshot "ghettoVCB-snapshot-2009-12-15" for VS01-Versand 2009-12-15 12:21:40 -- debug: Waiting for snapshot "ghettoVCB-snapshot-2009-12-15" to be created 2009-12-15 12:21:40 -- debug: Snapshot timeout set to: 900 seconds Destination disk format: VMFS zeroedthick Cloning disk '/vmfs/volumes/datastore1/VS01-Versand/VS01-Versand.vmdk'... Clone: 100% done. 2009-12-15 12:39:23 -- info: Removing snapshot from VS01-Versand ... 2009-12-15 12:40:00 -- info: Backup Duration: 18.35 Minutes 2009-12-15 12:40:00 -- info: Successfully completed backup for VS01-Versand! 2009-12-15 12:40:00 -- info: ============================== ghettoVCB LOG END ================================
JLeyva,
I personally use 1 list only shared for my ESX servers. I saved it in shared storage (NAS) and in that way I only update 1 list.
Is the destination datastore that you're backing up to an NFS datastore? If so, do you have sufficient space locally that we can try the same exact backup? I've started to notice that users that run into issues with either removing snapshots and deleting files sometimes run into issues due to their NFS setup and it sometimes produces input/output errors in either timeouts or errors, generally saturation of the link or non-optimal configuration.
To ensure we're removing that out of the picture, lets try a local backup where it's writing to a local VMFS volume and if you're still having the same issues, we can diagnose further. As you've noted, this is probably not an issue with not being on update1 and this has worked for many on previous releases.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I've already mentioned this to @bonibaz:
Is the destination datastore that you're backing up to an NFS datastore? If so, do you have sufficient space locally that we can try the same exact backup? I've started to notice that users that run into issues with either removing snapshots and deleting files sometimes run into issues due to their NFS setup and it sometimes produces input/output errors in either timeouts or errors, generally saturation of the link or non-optimal configuration.
To ensure we're removing that out of the picture, lets try a local backup where it's writing to a local VMFS volume and if you're still having the same issues, we can diagnose further. As you've noted, this is probably not an issue with not being on update1 and this has worked for many on previous releases.
Let me know if you're in the same boat?
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
hi william,
yes, the destination datastore is an NFS. It´s a "QNAP"-NAS with more then 9TB. Actually i only backup these both VMs, so ther is enough space unused.
At the local datastore i have only 94GB, so i can create two or maximal three backups from both VMs.
Tonight i backup to the NFS. I´ll change my configuration tomorrow for backup to local datastore.
I'm not so much worried about the available space versus the setup of the NFS server. As mentioned to help isolate the issue, I would like you to run a comparison backup between the NFS datastore and local datastore and see if local is consistently successful without having any issues. If you don't have space locally to support multiple backups, then lets try it with one VM and set the rotate to max of 1 and check back and see if we see any similarities/differences. You might also want to look at the logs under /var/log/* and see if there are any issues reported during the time of the backup, that may also tell us what might be going on
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I reported back in september that my thin provisioned VMs were taking up the full allocated size on the NFS where they were being backed up to. I suspected this might be a problem with the NFS server I was using (Services for Unix on Windows Server 2003).
I can now confirm that that appears to have been the case. With no other changes other that switching to a new NFS server (openfiler 2.3) the VMs are now being backed up to their appropriate size. (i.e. 1 that was taking up the full 40 GB on the Win2003 server is now only taking up 8 GB on the openfiler NAS).
Very glad to hear the issue has been resolved!
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I will try a local backup
@Rob98:
This I already reported in this thread. You can work around this by using the '2gbsparse' format instead. This will only occupy the space which is used by the vmdk on Windows NFS.
@All:
How about your experiences with NFS datastore at all.
Mine are not as good.
For me this seems not to be very reliable implemented in ESXi.
I tried different NFS servers (all on Windows plattforms) in completely different environments also with very fast Hardware, but from time to time I still get input/output errors in all environments (mostly on restore). Server which seems to work best so far is AllegroNFS
Hey William, nice script.
I downloaded LAST_MODIFIED_DATE=11/14/2009 and I think either I am doing something wrong or the -f command is missing from the cron examples.
Dont you need to use the -f also as part of the cron command ?
Also for the log file you are using %s. I will suggest using $(date +
%F)-ghetto-backup.log instead as it will be easier to tail based on a date than having the ghetto part first, easier to sort, etc.
Also, /bin/busybox crond does not seem to work in ESXi 4.0.0 build-208167, if I use just crond it works.
Regards,
Paul Aviles
today i changed the backuppath to the local datastore with "rotation: 1"
i think, next week i can post more...
Hi, when I runs the script appear:
/vmfs/volumes/ee8e6fb6-4774615c/script # ./ghettoVCB.sh -f vmtobackup
./ghettoVCB.sh: line 165: syntax error: "elif" unexpected (expecting "then")
Coud you help me?
Regards,
There should not be any errors when executing, usually users will run into this issue if they downloaded the script and somehow got 'Windows' specific characters into the script that may cause issues within the execution. You'll want to only edit the script within ESX(i) host. How did you download,upload and edit the script?
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I will say to copy the script again and make sure you dont open it in windows first. That is known to mess the scripts.
Regards,
Paul Aviles
Thanks William for the help.
The script download to my Windows PC, without editing it upload with vSphere Client and modify it using "vi" in ESXi. Is it OK?
Regards,
Gustavo Zolfo
NO, that is not okay. If you use the vSphere Client to upload ... it does its funkyness with Windows and add ^M characters to the script which causes issue.
Please download the script to your local system using any browser, do not edit the file and upload using SCP and for those that are on Windows system, use winscp (free utility). Then edit using VI
I may need to update the documents to state this explicitly as I find users are either editing the script in Windows text editor OR uploading using vSphere Client which can mess up the contents of the script.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
This recent thread: reminded me of your question awhile back about calling the shrink option from with a VM using VMware Tools, afaik, this is not possible and there's few threads on VMTN and on google how it's not supported.
However, from this thread, it sounds like you can potentially call your own script prior to the backup and snaphot ... you might be able to use this open source version of shrink tool vmshrink which actually makes an RPC call to undocumented API to start the shrink process. You should be able to create two .bat scripts following the thread mentioned above (.bat ext. of course) and make a call to vmshrink tool before snapshot is taken.
Would be interesting to see if this could be a potential work around and help shorten the duration of backup process.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Thank you again!!
It´s working....
If you are on Windows you can edit the script from within WinSCP. This one takes care of converting EOL characters.
Another option is to use Scite. With this editor you can change EOL character automatically as needed.
Hello William, hello all!
I have installed the ghettoVCB backup yesterday and need to say that I really love it. It does exactly what I had hoped, I only need to test the restore but I'm very optimistic.
One minor improvement came to my mind during the setup: The backup parameters require the adapter type (I guess to correctly read the data from the disk-file) which makes it impossible to use the default parameters if Win2000 and Win2003+ vms are in use.
I dont know much about unix shell scripts and also not about the internal structures of the *.vmdk files but by looking briefly over your scripts and my .vmdk file I thought of something like the following:
this is a section out of your ghettoVCB.sh:
...
#support for vRDM and deny pRDM
grep "vmfsPassthroughRawDeviceMap" "$" > /dev/null 2>&1 if \[ $? -eq 1 ]; then ==> ADAPTER_FORMAT=$(grep -i "ddb.adapterType" "$" | awk -F '=' '{print $2}' | sed 's/"//g' | sed -e 's/^\[\[:blank:]]//;s/\[\[:blank:]]$//')
if \[ "$" == "zeroedthick" ]; then
if \[ "$" == "4" ]; then
$ -i "$" -a "$" -d zeroedthick "$" 2>&1 | tee -a -i "$"
else
...
The line with ===> is inserted by me.
I copied the command from another section of your scripts because I think it extracts the right part of the line from a file (It is just a guess; to be honest, I dont understand even a part of what it really does ;-(
Could this save me from specifying the adapter format each time and still get me the correct data? Do you think it works with all type of disks or only the .vmdk/-flat.vmdk combination that I looked at?
Thanks for any feedback. Markus
These were all additional parameters that were presented so that users may choose how their backups were taken place, some had asked if they could change the adapter/etc. I don't have access to my test environment, but from the logic, it sounds reasonable to get the adapter information from the .vmdk descriptor file.
Thanks for the feedback
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Ah, I see, the adapter type is required to write the destination disk file not to read the source file ? Thus it can be converted and backed-up at the same time.
I'll give it a try and see... Thank you! Have a successful start into 2010!
hi william,
i changed the backuppath to a local datastore before christmas.
but my problem is still the same. here the log:
2009-12-23 04:30:01 -- info: ============================== ghettoVCB LOG START ============================== 2009-12-23 04:30:01 -- debug: HOST BUILD: VMware ESXi 4.0.0 build-171294 2009-12-23 04:30:01 -- debug: HOSTNAME: ESXi01-AIC.XXXXXXXXXXX.de 2009-12-23 04:30:01 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/datastore1/testbackup 2009-12-23 04:30:01 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 2 2009-12-23 04:30:01 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick 2009-12-23 04:30:01 -- info: CONFIG - ADAPTER_FORMAT = buslogic 2009-12-23 04:30:01 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0 2009-12-23 04:30:01 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0 2009-12-23 04:30:01 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3 2009-12-23 04:30:01 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5 2009-12-23 04:30:01 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15 2009-12-23 04:30:01 -- info: CONFIG - LOG_LEVEL = debug 2009-12-23 04:30:01 -- info: CONFIG - BACKUP_LOG_OUTPUT = stdout 2009-12-23 04:30:01 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0 2009-12-23 04:30:01 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0 2009-12-23 04:30:01 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all 2009-12-23 04:30:03 -- info: Initiate backup for VS01-Versand 2009-12-23 04:30:03 -- info: Creating Snapshot "ghettoVCB-snapshot-2009-12-23" for VS01-Versand 2009-12-23 04:30:04 -- debug: Waiting for snapshot "ghettoVCB-snapshot-2009-12-23" to be created 2009-12-23 04:30:04 -- debug: Snapshot timeout set to: 900 seconds Destination disk format: VMFS zeroedthick Cloning disk '/vmfs/volumes/datastore1/VS01-Versand/VS01-Versand.vmdk'... Clone: 100% done. 2009-12-23 04:37:55 -- info: Removing snapshot from VS01-Versand ... 2010-01-11 06:49:23 -- info: Backup Duration: 27499.33 Minutes 2010-01-11 06:49:23 -- info: Successfully completed backup for VS01-Versand! 2010-01-11 06:49:23 -- info: ============================== ghettoVCB LOG END ================================ done. 2010-01-11 06:49:23 -- info: Backup Duration: 27499.33 Minutes 2010-01-11 06:49:23 -- info: Successfully completed backup for VS01-Versand! 2010-01-11 06:49:23 -- info: ============================== ghettoVCB LOG END ================================
i removed the snapshot today by hand.
2009-12-24 04:30:01 -- info: ============================== ghettoVCB LOG START ============================== 2009-12-24 04:30:01 -- debug: HOST BUILD: VMware ESXi 4.0.0 build-171294 2009-12-24 04:30:01 -- debug: HOSTNAME: ESXi01-AIC.XXXXXXXXXXXXX.de 2009-12-24 04:30:01 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/datastore1/testbackup 2009-12-24 04:30:01 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 2 2009-12-24 04:30:01 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick 2009-12-24 04:30:01 -- info: CONFIG - ADAPTER_FORMAT = buslogic 2009-12-24 04:30:01 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0 2009-12-24 04:30:01 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0 2009-12-24 04:30:01 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3 2009-12-24 04:30:01 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5 2009-12-24 04:30:01 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15 2009-12-24 04:30:01 -- info: CONFIG - LOG_LEVEL = debug 2009-12-24 04:30:01 -- info: CONFIG - BACKUP_LOG_OUTPUT = stdout 2009-12-24 04:30:01 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0 2009-12-24 04:30:01 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0 2009-12-24 04:30:01 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all 2009-12-24 04:30:02 -- info: Snapshot found for VS01-Versand, backup will not take place 2009-12-24 04:30:02 -- info: ============================== ghettoVCB LOG END ================================
do you have any idea why the snapshot will not be removed?
Hi,
I also got some problems using the compression features of the script. I tested with 2 VM's, one large one with 3 disks, one of 50GB, one of 5GB and one of 250GB. Usage on this disks is very low, maybe 20GB in total. After backup I could not get the backup unzipped, on both the ESXi machine and another "normal" linux machine. The same goes for a VM with a single 40GB disk with 20GB usage. The error is: "Invalid tar magick". Tried backuping to both local storage and remote NFS storage.
Because I needed to transfer the big VM to a remote VMWare Server machine I looked for another way to compress the backup. I ended up compressing it disk-by-disk with the gzip command, and afterwards I could unzip them just fine. Maybe it is an idea to rewrite the script to just zip the single disk files in stead of the whole VM directory?
Very odd situation, this is the only case I've seen with this type of persisted issue. The only thing I can think of off the top of my head is that you're potentially overloading your ESXi host? How are the physical resources (cpu/memory) on the ESXi host? Do you see any spikes in performance during the backup window that may cause the snapshots to fail to commit? Could you also trace the backup timestamps with ESXi logs (/var/log/messages,/var/log/vmware/hostd.log) and see if there are any errors?
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
What was the exact syntax you used to extract the compressed contents? I've not had the time to work on the restoration script supported compressed backups, there's few issues I just need to work out the kinks on.
Regarding individual compressed VMDK, I don't think it makes sense as it'll be more complicated for the restoration to extract each VMDK versus just extracting 1 file which has the contents of the entire VM. In general, you'll want to recover the entire VM including it's configuration file and all it's disk and not individual files.
I've not done extensive testing on larger VMDK files, we personally don't use compression when we issue our backups but there have been a few in the community that wanted this option. I believe the current method of compression is the most efficient but if you have data proving other wise, I would love to hear it and could potentially consider it in a future release of ghettoVCB
Thanks for the feedback
--William
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
hi william,
here is my log from tonight:
2010-01-12 04:30:01 -- info: ============================== ghettoVCB LOG START ============================== 2010-01-12 04:30:01 -- debug: HOST BUILD: VMware ESXi 4.0.0 build-171294 2010-01-12 04:30:01 -- debug: HOSTNAME: ESXi01-AIC.XXXXXXXXXXXXX.de 2010-01-12 04:30:01 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/datastore1/testbackup 2010-01-12 04:30:01 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 2 2010-01-12 04:30:01 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick 2010-01-12 04:30:01 -- info: CONFIG - ADAPTER_FORMAT = buslogic 2010-01-12 04:30:01 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0 2010-01-12 04:30:01 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0 2010-01-12 04:30:01 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3 2010-01-12 04:30:01 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5 2010-01-12 04:30:01 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15 2010-01-12 04:30:01 -- info: CONFIG - LOG_LEVEL = debug 2010-01-12 04:30:01 -- info: CONFIG - BACKUP_LOG_OUTPUT = stdout 2010-01-12 04:30:01 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0 2010-01-12 04:30:01 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0 2010-01-12 04:30:01 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all 2010-01-12 04:30:03 -- info: Initiate backup for VS01-Versand 2010-01-12 04:30:03 -- info: Creating Snapshot "ghettoVCB-snapshot-2010-01-12" for VS01-Versand 2010-01-12 04:30:03 -- debug: Waiting for snapshot "ghettoVCB-snapshot-2010-01-12" to be created 2010-01-12 04:30:04 -- debug: Snapshot timeout set to: 900 seconds Destination disk format: VMFS zeroedthick Cloning disk '/vmfs/volumes/datastore1/VS01-Versand/VS01-Versand.vmdk'... Clon2010-01-12 05:03:58 -- info: Removing snapshot from VS01-Versand ... Clone: 100% done. 95 bytes) 2010-01-12 04:37:58 -- info: Removing snapshot from VS01-Versand ... /tmp #
backup faild again at "removing snapshot"
here are my host-resources:
Dell PowerEdge 1950
CPU: Intel Xeon E5430 @2.66GHz
Ram: 12GB
I have two VM´s on my ESX.
the errors i can find in the logs ar:
[2010-01-12 04:30:08.095 66081B90 info 'Vmsvc'] Failed to consolidate disks in Foundry: Error: (15) The file is already in use [2010-01-12 04:30:03.826 145C8B90 error 'vm:/vmfs/volumes/4ae6e3da-baeabf38-2950-001ec9b33af1/VS01-Versand/VS01-Versand.vmx'] Invalid transition request [2010-01-12 04:30:03.827 145C8B90 info 'TaskManager'] Task Completed : haTask-48-vim.VirtualMachine.createSnapshot-7829 Status error [2010-01-12 04:30:03.869 1464AB90 error 'vm:/vmfs/volumes/4ae6e3da-baeabf38-2950-001ec9b33af1/VS01-Versand/VS01-Versand.vmx'] Invalid transition request [2010-01-12 04:30:03.869 1464AB90 info 'TaskManager'] Task Completed : haTask-48-vim.VirtualMachine.createSnapshot-7831 Status error
backup starts at 4:30am and snapshot should be removed 4:37am.
if you want i can send you the logmessages from backup between start and end with email.
The only thing I can think of off the top of my head is that you're potentially overloading your ESXi host? How are the physical resources (cpu/memory) on the ESXi host? Do you see any spikes in performance during the backup window that may cause the snapshots to fail to commit?
Please verify that your host is not under constraint and take a look at your performance charts during the period of backup
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
I've downloaded and tested the latest version of ghettoVCB.sh (11/14/2009)... and it works very well as long as the NFS configuration is set within the program; not the configuration file. It seems that the NFS data (i.e.:
ENABLE_NON_PERSISTENT_NFS=1
UNMOUNT_NFS=1
NFS_SERVER=192.168.100.125
NFS_MOUNT=/a0/nsnbc/vm_backup
NFS_LOCAL_NAME=nfsbackup
NFS_VM_BACKUP_DIR=CYLI6S30
VMDK_FILES_TO_BACKUP="all" )
does not get set to overlay the defaults. Is there something I am missing here? I execute the script as follows:
./ghettoVCB.sh -f vms_to_backup -c vm_backup_configs -d dryrun
The and the default NFS configs do not get passed thru to the script, but other configuration does. Is there an updated version to the script where the NFS does update? I have started to update this current script, but if
Thanks
That is correct, non-persistent NFS configurations is not a configurable item in the VM Backup Configs, as they primarily pertain to how the VM should be backed up.
VMDK_FILES_TO_BACKUP should be something you can define on a per VM basis. Remember that each VM must have it's own config and if any of these values are not defined in the config, a default will be used.
FYI - The latest version will always be up on the download page and if you want to check out the changes from release to release, that information is found in the change log. There's reason why I spend the time documenting everything, though not all users hit the documentation first
Thanks
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Thank you William. Though I may have several NFS data backup sites, I'll just control the NFS mounts from the programs and the VM client configuration from the configuration files. It works fine now.
One option is to just mount your NFS datastores (persistent) and specify in the VM Configuration file to the datastore in which they should be backed up in.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
hi william,
her is a screenshot from performance during backup.
backup starts at 10:02am.
her the log:
2010-01-18 09:02:01 -- info: ============================== ghettoVCB LOG START ============================== 2010-01-18 09:02:01 -- debug: HOST BUILD: VMware ESXi 4.0.0 build-171294 2010-01-18 09:02:01 -- debug: HOSTNAME: ESXi01-AIC.XXXXXXXXXXXX.de 2010-01-18 09:02:01 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/datastore1/testbackup 2010-01-18 09:02:01 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 2 2010-01-18 09:02:01 -- info: CONFIG - DISK_BACKUP_FORMAT = zeroedthick 2010-01-18 09:02:01 -- info: CONFIG - ADAPTER_FORMAT = buslogic 2010-01-18 09:02:01 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0 2010-01-18 09:02:01 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0 2010-01-18 09:02:01 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3 2010-01-18 09:02:01 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5 2010-01-18 09:02:01 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15 2010-01-18 09:02:01 -- info: CONFIG - LOG_LEVEL = debug 2010-01-18 09:02:01 -- info: CONFIG - BACKUP_LOG_OUTPUT = stdout 2010-01-18 09:02:01 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0 2010-01-18 09:02:01 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0 2010-01-18 09:02:01 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all 2010-01-18 09:02:03 -- info: Initiate backup for VS01-Versand 2010-01-18 09:02:03 -- info: Creating Snapshot "ghettoVCB-snapshot-2010-01-18" for VS01-Versand 2010-01-18 09:02:04 -- debug: Waiting for snapshot "ghettoVCB-snapshot-2010-01-18" to be created 2010-01-18 09:02:04 -- debug: Snapshot timeout set to: 900 seconds Destination disk format: VMFS zeroedthick Cloning disk '/vmfs/volumes/datastore1/VS01-Versand/VS01-Versand.vmdk'... Clone: 0% done. Clone: 1% done. Clone: 2% done. Clon2010-01-18 09:36:02 -- info: Removing snapshot from VS01-Versand ... Clone: 8% done. Clone: 9% done. Clone: 100% done. 2010-01-18 09:10:16 -- info: Removing snapshot from VS01-Versand ...
Very nice script, thanks!
One thing I've noticed since updating to the most recent version; the Cronjob FAQ no longer works correctly, because the script expects explicit arguments with flags. I have a cronjob working that looks like this:
0 16 * * 6 /vmfs/volumes/main/ghettoVCB.sh -f /vmfs/volumes/main/backup_list -l /var/log/$(date +%y%m%d)-backup.log
The script runs successfully with this cronjob defined, but the log file looks like every line is written twice. Maybe it is writing both stdout and stderr to the log file?
Anyways, minor inconvenience, but overall great. Thanks again!
EDIT: How do you post formatted code to these forums?
Yea I'm not sure ... it's odd that every time the script is called automatically the time that it takes to create a snapshot is pretty long but from what I remember if you do it via manually, it's pretty quick. The other question would be is the application/VM doing anything during this period that may cause it to have these issues? I would also take a look at your graphs on all 4 resources during the backup window.
Another thought, have you tried this during a different period of time? Does this always consistently happen? This might be a tough one to figure based on the information I'm seeing. May need to setup some type of troubleshooting session though I'm quite busy this week. If you potentially can setup a remote session (WebEx/etc) then you can drive and we can do further deep dive but I'm out of ideas other than something is going on with your VMs that's environmental versus an issue with the script
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Thanks for the comments.
The cron FAQ is actually out of date (my fault)...forgot to update this section when I rolled out the latest rev of the script. You're correct, you need to specify the correct flags as you would if you were to manually run it. It's been on my to-do list but been so busy with work both day and night .... I'll eventually get to it.
Regarding writing the lines out twice, this should not be the case. I've not seen this issue before but I can look into it when I get some free time.
To use code tags its actually using
Take a look at this VMTN text markup guide: http://communities.vmware.com/markuphelpfull.jspa
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
Hi
Not sure if i am doing something wrong?
Getting the following output.
/vmfs/volumes/4b4f58b6-4836d164-7575-f4ce46ae9e83/Backups # ./ghettoVCB.sh -f vmlist
###############################################################################
#
ghettoVCB for ESX/ESXi 3.5 & 4.x+
Author: William Lam
Created: 11/17/2008
Last modified: 11/14/2009
#
###############################################################################
Usage: ./ghettoVCB.sh -f -c -l
OPTIONS:
-f List of VMs to backup
-c Configuration directory for VM backups
-l File to output logging
-d Debug level info (default: info)
(e.g.)
Backup VMs stored in a list
./ghettoVCB.sh -f vms_to_backup
Backup VMs based on specific configuration located in directory
./ghettoVCB.sh -f vms_to_backup -c vm_backup_configs
Output will log to /tmp/ghettoVCB.log
./ghettoVCB.sh -f vms_to_backup -l /tmp/ghettoVCB.log
Dry run (no backup will take place)
./ghettoVCB.sh -f vms_to_backup -d dryrun
/vmfs/volumes/4b4f58b6-4836d164-7575-f4ce46ae9e83/Backups #
Forgot to mention trying to run this manually to test.
Then i will add it to the cron.
Running ESXi 4.0 21xxx
Licensed with essentials license.
Backing up to a 2nd sata hard drive called Datastore2.
vm is running on datastore1.
as for as i know, there is no job running on vm during backup. i tested backup at different times (4:00am, 10:00am, 3:00am). allways the same result.
i will test this week with the other vm. an i will test shutdown vm before backup... it´s no problem to shut down the second vm.
i also noticed, when i creat snapshot manuall in vmsphere and delet all snapshots after creation, it takes 2 or 3 minutes for dleteing the snapshot.
deleting the snapshots from second vm takes only 10 seconds...
i have never setup a remote session on esx
When I said remote troubleshooting session, I meant you would share out your desktop via WebEx/etc. and I would be able to view what you're seeing and walk you through some troubleshooting steps and looking at logs/etc. Again, this is something I would not be able to do until next week at the earliest. Again, I'm not sure why you're seeing this issue. I would have to believe it's something environmental.
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".
You should definitely see some output, can you past the contents of your vmlist and also run it using -d which is debug mode. Please wrap all your output using tags for readability
=========================================================================
William Lam
VMware vExpert 2009
VMware ESX/ESXi scripts and resources at: http://engineering.ucsb.edu/~duonglt/vmware/
VMware Code Central - Scripts/Sample code for Developers and Administrators
If you find this information useful, please award points for "correct" or "helpful".