.ultrageek. Computers, Music, and Thoughts

20Sep/111

Backing up VM Server using the CLI

I do work for a company that has a VM Server with multiple hosts running on it. DNS, Mail, Web, these tasks generally work best on dedicated hosts so that when they go bump in the night they don't take everything with it or complain when you have to reboot or restart services to fix them. Isolation worked before VM's and it has only gained ground afterwards. The question comes, how to back them up on the cheap? Shutting them down, zipping and/or copying them is not an option as it takes a bit of time (30-120 minutes) to get copies of the vmdk's.

So what I'm about to share with you is not a new idea as it was obtained from the usual amount of Google searching. What I will show though is an end to end process for admins that have VM Server 2.x running on a Microsoft Windows platform to back up, while the machine is running, to a file that can then be backed up either through FTP or other means if you have physical access to the box. It will basically snapshot, copy, and then zip up the vmdk files. Experts can skip to the end where the final example is assembled, the rest please trudge along so that the "why" and "how" is answered.

Step One - Preparation

  • Storage - Determine where you are going to house the backups. Make sure it has at least twice or three times as much as space as the VM currently takes on your storage.
  • VMware Command Line Tools - You will need to download and install the following package in order to use the "VMrun" command called by the script you will create used to list your VM's, create and delete snapshots, and use to perform other command line maintenance chores. Here is the link to the VMware site to download: http://www.vmware.com/support/developer/vix-api/. You can also Google "VMrun" for additional info.
  • Compression - You will need a command line compression program in order to take the files that you have copied and archive them to a single compressed image for backup and storage. My recommendation for this problem is to download and use the latest version of 7zip which is open source. The good news is that you will not have to download a specific command line version, just install the standard package and make sure that the folder is in the system path so that you can call it from a batch file without having to explicitly call where the EXE is located. Here is the 7zip website: http://www.7-zip.org/
  • Robocopy - You can use anything to copy this but I have found that through the years robocopy is tried and true for this type of copying. I would recommend using the Windows 2003 Resource Kit for the latest version: http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=17657. I'm sure there are other sources but would recommend getting it from Microsoft if possible. Don't worry about the age, command copying is pretty old technology, not a lot has changed though robocopy does have some flexibility not found in your standard xcopy toolset. This is a good reference: http://ss64.com/nt/robocopy.html
  • Text Editor - Make sure you know how to use a text editor such as notepad or Editpad Lite (plug). Would highly recommend that your text editor of choice resides in your "Send To" menu when right clicking a file. If that concept is greek to you, we might be in trouble.

Step Two - Batch File Set up

  • Directory - Create a specific directory to store all of your backups. For this example we're going to assume that you have a second drive called "D:" and that we have created a folder underneath called Backup. This is where we will be copying all the VM files as well as creating the archive folders.
  • Batch File - Create a batch file called "backup.bat". You can do this from the command line by navigating to the directory and going "notepad backup.bat" and notepad will ask if you would like to create the file. You can create a text file and rename. When you rename or create the file and view it in explorer it should look like an EXE and not a text file icon otherwise you have your extensions hidden and all you have done is created a backup.bat.txt file. If this hangs you up, drop to a command line (CMD.EXE) and do either a move backup.bat.txt backup.bat or substitute move for ren, either works.
  • Backup Log - We will be piping console output to a file called "Backup.log" in order to debug and check the execution of our backup job. You do not have to create this file, when we run the batch file it will automatically get created.

Step Three - Batch File Creation

  • Beer - Get one or two, you might need it if this makes your brain hurt. For others, continue on, once you do one host it is just a matter of rubber stamping out the rest of the hosts.
  • VM List - Drop to a command line (start --> run --> CMD) and type the following "vmrun -T server -h https://localhost:8333/sdk -u username -p password list" without the quotes of course. This will give you a list of VM's running that hopefully makes sense to you. You will want to copy and paste that list in to notepad for future reference.
  • Batch File Lines
    • Start Log: Echo %date% - %time% - Starting Email Back Process >> backup.log
      • This gives you an idea of when the process started which coupled with the stop log line you can get the delta of time it took to complete the task. Your server might run slow while this is happening but at least your VM's did not have to be suspended or stopped to complete the backup task.
    • Create Snapshot: vmrun -T server -h https://localhost:8333/sdk -u username -p password snapshot "[Standard] Virtual Machine.vmx" Backup
      • -T is for the type of VMware server.
      • Snapshot is the action
      • Backup is the name of the snapshot and can be named anything
    • Copy: robocopy "C:\Virtual Machine" "D:\Backup\email" /E /R:0 /W:0 /XF *00001.vmdk* /XF *.vmsn /XD *.lck* /XF *.vmem
      • /e for subfolders
      • /r:0 sets retries to zero
      • /w:0 sets the time between retries to zero
      • /XF excludes files, /XD excludes directories
    • Delete Snapshot: vmrun -T server -h https://localhost:8333/sdk -u username -p password deletesnapshot "[Standard] Virtual Machine.vmx" Backup
    • Compress File: 7z a -r -oD:\backup -t7z %date:~4,2%%date:~7,2%%date:~10,4%\email_%date:~4,2%%date:~7,2%%date:~10,4%.7z -v2g D:\Backup\mail\*.* -mmt -mx9
      • a and -r sets the program to archive and recursive
      • -o is the directory
      • -tyz sets the archive type to 7zip, this can be just zip
      • %date:~4,2%%date:~7,2%%date:~10,4%\email_%date:~4,2%%date:~7,2%%date:~10,4%.7z - This creates a sub directory with today's date as well as the archive with today's date
      • -v2g - Sets the volume archive size to 2gigs
      • D:\Backup\mail\*.* - Source
      • -mmt - Turns on multi-threading. This can put your machine on its knee's so determine if speed or CPU power is more important, might be a none starter if done after hours. This will default to 2 threads I believe, some of the other compression methods support more cores like bzip2
      • -mx9 - Compression level, 9 being the highest
    • Stop Log: Echo %date% - %time% - Finished Email Backup Process >> backup.log

Step Four - Putting it all together (or the EZ example)

Echo %date% - %time% - Starting Email Back Process >> backup.log
vmrun -T server -h https://localhost:8333/sdk -u username -p password snapshot "[Standard] Virtual Machine.vmx" Backup
robocopy "C:\Virtual Machine" "D:\Backup\email" /E /R:0 /W:0 /XF *00001.vmdk* /XF *.vmsn /XD *.lck* /XF *.vmem
7z a -r -oD:\backup -t7z %date:~4,2%%date:~7,2%%date:~10,4%\email_%date:~4,2%%date:~7,2%%date:~10,4%.7z -v2g D:\Backup\mail\*.* -mmt -mx9
Stop Log: Echo %date% - %time% - Finished Email Backup Process >> backup.log

You will want to make sure all of your commands are in your environment path, this might require some tweaking to get this to run with task schedule, and you might want to tweak some of the 7zip settings as needed. I had the source page that I used years ago to create this script but it is gone, even from google cache. If someone wants to email a source that closely matches this I will be more than happy to give credit where do. I've seen a lot of posts of scripts that are pretty fancy but a lot involve suspending the machine which is just not an option when the job could easily take 30-120 minutes to complete.

 


Comments (1) Trackbacks (0)
  1. In the robocopy portion of the script how did you come up with ‘C:\Virtual Machine’ as a copy source? How would it translate the statement if the source is still on the datastore that the host(s) is connected to?

    Location of ‘backup’ snapshot: [Standard] Virtual Machine

    ‘robocopy “C:\Virtual Machine” “D:\Backup\email” /E /R:0 /W:0 /XF *00001.vmdk* /XF *.vmsn /XD *.lck* /XF *.vmem’

    Any assistance you can offer would be greatly appreciated.

    ~Ben


Leave a comment

No trackbacks yet.