First, here’s a recap of the problem. It’s written in terms of VMware Fusion but should be equally valid for other VM engines such as Parallels Desktop. If you’ve already read the previous article then you might want to skip to the meat of this article.

Large backups

When you’ve been using Fusion’s virtual machine (in my case, to run Windows), you will find that Time Machine’s backup takes longer than expected — sometimes much longer.

This is because Fusion stores its virtual machines’ disks as a bunch of large files. When you run your virtual machine using Fusion (or Parallels Desktop), the contents of the VM’s virtual disk changes, and even if you haven’t made many explicit changes to files, all sorts of small changes happen to the virtual disk’s contents.

Incremental can still be too much

Time Machine performs incremental backups. That is, it copies only files that have changed since the last backup. Unfortunately those large files that Fusion uses to store the virtual disk’s contents? They’ve probably all changed, at least a little.

So, instead of your incremental being a few megabytes and taking a few tens of seconds, it’s ten gigabytes, takes ages, and rapidly fills your backup disk. (Which causes its own problems: described here.)

One solution: back up the VM in another way

One solution is to keep all your precious files in the Mac file-store, tell Time Machine to avoid backing up your virtual machine(s), and then back them up some other way. This results in the smallest backups and is useful if you don’t need sophisticated incremental backups of your virtual machines.

If this sounds like you, then I’ve shown how this can be achieved in an earlier article.

An alternative solution: use a sparse bundle

If you need proper incremental backups of your VMs, then one approach is to use a sparse bundle. This is a disk image which is stored  as a collection of small files (typically 8MB), so when Time Machine backs up the disk image, it only needs to copy the files that have changed.

This approach still isn’t perfect — when (for example) Windows does anything, it tends to scatter its writes over the disk, so incremental backups are still large, but they’re not as large as when Time Machine backs up the VM directly. Typically, your incrementals may be a few hundred megabytes with no VMs, a few gigabytes with a VM stored on a sparse bundle (for Windows. Linux may be better behaved), and a few tens of gigabytes with the same VM stored in the standard way.

Here’s how

First lets create a new disk image based on a sparse bundle and move our VM onto it. We’ll need to know how large to make the VM, so go to Documents and look in Virtual Machines. In there is one folder per virtual machine and you can find the VM’s size by clicking on the corresponding folder and hitting cmd+I:

Create the disk image

Now create the sparse bundle. The point-and-click way of doing this is to use Disk Utility, so start that up (I use spotlight to do so):

and click on New Image:

Then fill in the details. I made my sparse bundle file-system for the Windows XP VM twice as large as the space it currently occupies, to allow for future expansion — so 50GB in my case (don’t worry: it only uses as much real disk as is needed to store the actual content). Here’s all the fields you need to change:

• The Save As is the filename for the sparse bundle whereas the Volume Name is the name the disk will have when mounted — I made them the same;
• I saved my sparse bundle in Documents->Virtual Machines;
• Image Format needs to be sparse bundle disk image, as that’s the whole point of the exercise.

Now click on Create, and

a few seconds later you have a sparse bundle file-system image.

Now double-click that file (mine’s called VM disk.sparsebundle).

Mac OS will mount it…

and the new file system will appear on the desktop.

Move the VM into the new disk image

Next, drag the folder(s) corresponding to the VM(s) you’d like backed up by Time Machine onto the new disk. Hold down cmd as you drag and drop, so that you move the folders (rather than copying them):

Ensure the new disk image is mounted at login

The .sparsebundle file needs to be mounted every time you log in, so that the file-store is available. To do that, go into System Preferences->Accounts, click on your username and then on Login items. Now drag your sparsebundle file into the list of login items:

and that’s it.

Using your newly-homed VM(s)

To use your new VM(s), just double-click on the corresponding folder — perhaps put an alias to it on your desktop or in the dock. When you next use it you’ll get a message:

Just click on I moved it and you’re sorted.

Warning

As with any method of backup which relies on backing up the virtual machine’s disk image using the host operating system, your backup may not be of any use if it’s made whilst the VM is running, so I suggest you make sure that Fusion/Parallels is not running when Time Machine is backing up your data. If you don’t always manage to do so then some of your backups may not be valid.

Why?

Because if you backup the files that Fusion (or Parallels) is using for a disk image, then you’re backing up what’s on disk, but not what’s in memory, in disk buffers, etc. So when you come to restore from backup, you may find that the disk image is corrupted. In the worst case scenario you won’t be able to start up the VM’s operating system successfully at all, or restore any files contained within the VM’s file-store.

By backing up when the VM isn’t running, you ensure that all data is on disk in a safe, consistent state.

No related posts.

Are you using the excellent VMware Fusion and backing up your Mac with Time Machine? If so, you may notice a problem.

Large backups

When you’ve been using Fusion’s virtual machine (in my case, to run Windows), you will find that Time Machine’s next backup takes longer than expected — sometimes much longer.

This is because Fusion stores its virtual machines’ disks as a bunch of large files. When you run your virtual machine using Fusion, the contents of the VM’s virtual disk changes, and even if you haven’t made many explicit changes to files, all sorts of small changes happen to the virtual disk’s contents.

Incremental can still be too much

Time Machine performs incremental backups. That is, it copies only files that have changed since the last backup. Unfortunately those large files that Fusion uses to store the virtual disk’s contents? They’ve probably all changed, at least a little.

So, instead of your incremental being a few megabytes and taking a few tens of seconds, it’s ten gigabytes, takes ages, and rapidly fills your backup disk. (Which causes its own problems: described here.)

An alternative fix

If you need incremental backups of your VMs, or your VMs are running an operating system that doesn’t have support for accessing theMac OS file-store (Linux, for example), then you’ll likely find this approach is the best for you.

If your VM(s) are running Windows (so you can access the Mac OS file-store from within the VM) and you don’t need a sophisticated incremental backup of your VMs then read on, as the following approach produces by far the most efficient backups.

This fix

The fix described here is to ensure the files you edit in Fusion’s virtual world are in the Mac’s natural file-store (so they continue to be backed up by Time Machine), stop using Time Machine to back up your virtual machines, and then back up your virtual machines in a different way. I’ll cover each of these steps in turn below.

Put your files somewhere safe

Instead of putting your Windows (or whatever) files in the Windows file-store, place them in the Mac’s file-store (which is available to Windows as the Z: drive by default). This means that your precious files have a proper, incremental backup using Time Machine, and you can restore earlier versions should the need arise.

Stop Time Machine backing up your VMs

Open Time Machine Preferences…, click on Options… and then the small + to add a Folder to the Do not back up list:

Then navigate to Documents below your home directory and there you will find a folder called Virtual Machines — add that and Time Machine will no longer backup your Fusion VMs.

And backup your VMs in a different way

If, like me, your virtual machine is a secondary system, then you probably have no need of an incremental, sophisticated backup. What’s important is the files you edit are protected, and they are now protected by Time Machine as part of the Mac’s natural file-store.

So, simply drag your Documents/Virtual Machines folder onto the backup drive every week or so, and you’re sorted.

Warning

As with any method of backup which relies on backing up the virtual machine’s disk image using the host operating system, your backup may not be of any use if it’s made whilst the VM is running, so I suggest you make sure that Fusion/Parallels is not running when you copy the data.

Why?

Because if you copy the files that Fusion (or Parallels) is using for a disk image, then you’re backing up what’s on disk, but not what’s in memory, in disk buffers, etc. So when you come to restore from backup, you may find that the disk image is corrupted. In the worst case scenario you won’t be able to start up the VM’s operating system successfully at all, or restore any files contained within the VM’s file-store.

By backing up when the VM isn’t running, you ensure that all data is on disk in a safe, consistent state.

No related posts.

However, you may not want a backup every hour — the backup disk is a limited resource. If you’re regularly changing large files then Time Machine is regularly writing large amounts of data to the backup disk, and if that’s the case then you won’t have backups going back very far into the past.

So there will be lots of revisions of files that have changed recently, but nothing much from, say, a couple of months ago.

So how do you change the interval?

The standard Time Machine options don’t allow you to change the backup interval (the time between backups), but the preferences are there to be changed if you know how.

From the command line

If you don’t want to install any extra software then you can change Time Machine’s backup interval from a shell prompt. Start up a Terminal window and then type:

sudo defaults write /System/Library/LaunchDaemons/com.apple.backupd-auto StartInterval -int 18000

The 18000 above is the required backup interval in seconds — five hours in this example. And note that the above command is all on one line.

Point-and-click

There are several full Mac OS applications out there that allow you to manipulate Time Machine’s hidden preferences. The two most popular are Time Machine Scheduler and Time Machine Editor. I haven’t used either (being someone who leans toward using the command line), but they look like they’ll do the job. However…

My recommendation was, but is no more:

I did suggest using the Secrets prefPane to change Time Machine’s backup interval (see below), but unfortunately that turned out to be bad advice. You can set the interval using the Secrets prefPane but it doesn’t stick. Quit the System Preferences application and start it up again and you’ll find the interval has gone back to the default of 3600 seconds.

So it seems Secrets prefPane doesn’t work for this after all! I’ll leave the rest of this article in place and perhaps someone will tell me when the problem’s fixed?

In the mean time, please use one of the options suggested above.

Here’s the rest of the original article, which at the moment doesn’t work:

I suggest you use the Secrets Prefpane from blacktree. Just download it from the link near the top of the web page, unzip it and then install it by double-clicking on the resulting file (Secrets.prefPane).

Now, when you start up System Preferences you’ll find a new preferences pane:

The Secrets preferences pane allows you to set the hidden Time Machine backup interval:

but — and this is why I recommend the Secrets prefPane – it also allows you to change all sorts of other hidden preferences in Mac OS:

which is great fun — and who knows, may even prove useful

No related posts.