The trouble with snapshots: how can you copy them?

One day we’ll look back and remember fondly when it was possible to clone any Mac volume to an identical copy. Sadly, since the introduction of APFS in High Sierra, that has now stopped working for those volumes formatted using the new file system. Try making a clone of an APFS volume now and you may not even notice what’s missing from the copy: snapshots.

Until Big Sur, this didn’t make a great deal of difference. Few apps other than Time Machine made much use of snapshots, and in Time Machine they were only made on APFS volumes being backed up, and were automatically removed after 24 hours anyway. Big Sur changes this with the introduction of Time Machine backing up to APFS volumes, which uses snapshots to store those backups. What had been a minor and essentially silent problem suddenly became more significant: how can you make a copy of your new Time Machine backup on an APFS volume, then? The answer, as far as I can see, is that you can’t. Not yet, at least.

Look in the master cloning utility, Carbon Copy Cloner, and you’ll be politely informed that copying to or from an APFS Time Machine backup volume isn’t supported.

cccnocopy

Look in the documentation for command tools like diskutil and asr, and there’s no mention of imaging, copying or restoring snapshots within an APFS volume.

To see what the problem is, we need to go back and consider what it means to make a copy of the two different types of backup used by Time Machine.

Prior to Big Sur, all Time Machine backups were made to volumes (or images) in HFS+ format. Those backups consist of a single file system containing millions of files, folders, and both file and directory hard links. Although a slow and time-consuming task, making an exact replica of the original is straightforward. Thus copying an HFS+ backup to another HFS+ disk is just a matter of patience, and hoping that there are no errors in amongst all those millions of file system records. It’s also essential to remember that HFS+ backups can only be copied to another HFS+ file system, whether on a physical or virtual disk, as without support for directory hard links, the whole scheme fails.

To understand what’s required to copy a snapshot within a Time Machine backup to APFS, or anywhere else for that matter, I’ll consider a simple scenario.

tmsnapshot

In this example, a snapshot was made a little time ago. That contains the file system metadata (red), which includes the two folders (pale yellow) and references the data for two files: OldDoc (blue) and ChangedDoc. At that time, the latter included two blocks of data, ChangedDoc1 (red) and ChangedDoc2 (blue).

The user then creates a new document, NewDoc (green), and edits ChangedDoc so that it’s stored in one new block (green) and one old block (blue).

At that stage, rolling back to the snapshot would remove NewDoc, which was created since the snapshot was made, and it would restore the old (red) first block of ChangedDoc. The data specific to that snapshot therefore consists of the file system, the snapshot itself, and the original first block of ChangedDoc.

Consider trying to make an identical copy of that volume, with its single snapshot. There are two sets of file system metadata to be copied, one for the current state, and that for the snapshot. Then the data for each has to be copied for those snapshots to be usable. Some files are common to both, some only to one, and in the case of ChangedDoc there’s one block which differs in each, and one which is common.

Restoring from either the current file system or the snapshot is straightforward. Duplicating them both onto another disk is considerably more complex if they’re both to remain fully functional.

Now imagine if the disk in question contains half a million or more files, and a hundred or more Time Machine snapshots, each with its own file system metadata and intermeshed file data. Constructing an accurate duplicate of that lot could take as long as creating all those Time Machine backups in the first place.

Until Big Sur started to create backups to APFS volumes using snapshots, there was precious little demand for copying snapshots in this way. There’s clearly increasing demand now that so many of us are benefitting from backing up to APFS. I suspect that it’s only a matter of time and engineering effort before it will be possible to make copies of these new Time Machine backup volumes, but that’s unlikely to happen next week.

Postscript

Some have spotted that it is possible to make an identical copy of most external disks, either using a hardware disk duplicator, or even (if you’re adventurous enough) using dd. Both techniques require you to copy between disks which are very similar if not identical, making them less attractive if you want to move your backups/snapshots to a larger disk. It may be possible then to grow the container, but APFS volumes in the backup role can behave strangely, so that may not work. Another problem you face is that APFS relies on UUIDs and other identifiers which must be unique, so one of the two disks must then be wiped completely, lest they be mounted together and put APFS into a severe sulk. I wish anyone who wants to try any of this the very best of luck: it looks like they’ll need it.