Time Machine to APFS: Backup structure and access

Time Machine backups made to HFS+ volumes pull few tricks. Because their backups are a combination of ‘real’ files and hard links, what you see in the Finder is essentially consistent with what you get, although there’s no obvious way to distinguish what is just a hard link, or inside a hard-linked directory.

Time Machine backups to APFS volumes (TMA) are considerably more complex because of their reliance on snapshots. This article tries to explain how what you see of them varies according to what you use, and why that is so. It’s based on my experience of examining several different backup sets created by TMA, and your experience may well differ. As usual, there’s essentially no documentation available, and tmutil, the primary command tool for working with Time Machine, doesn’t appear to have been updated to support TMA in any systematic way.

Snapshots on volumes being backed up

As with previous versions of Time Machine going back to High Sierra, when backing up volumes in APFS format (but not HFS+), a volume snapshot is made and stored on each volume at the start of the backup process, every hour. Regardless of how much might be excluded from backups of that volume, this is a full volume snapshot, and can be accessed in the normal way, following mounting.

tmavolsnap

Once mounted, the user has full access to the whole contents of that snapshot, including folders and files which are excluded from the TMA backup. This is an important feature of all versions of Time Machine since High Sierra: the snapshots made on each source volume include all folders which are in the backup exclusion list, and can only exclude whole volumes.

Snapshots in the backup store

These are normally unmounted, and only contain the folders of the volume which aren’t in the backup exclusions. Whether that’s achieved by pruning the file system tree or only copying the data in the folders to be backed up isn’t documented, and has little practical consequence.

Access to these snapshots and their paths differs according to how the user views them, and this is where the confusion starts.

In the Time Machine app, these are shown as backups containing just the backed-up folders, with other excluded backups being shown greyed out. This makes it more likely that the snapshot file system is complete and hasn’t been pruned.

tmatimemac

In the Finder, these are shown when viewing the contents of the backup volume, either in its entry in Locations or through its mountpoint in /Volumes/[backupstore volname]. They are thus presented in the Finder path
/Volumes/[backupstore volname]/[datestamp]//…

tmafinderbackvol

Note that the path information given by the Finder is quite different, and has the form
[backupstore volname]@snap-[snapshotXID]/[datestamp].backup//…
which gives the formal path through the snapshot itself. The snapshotXID is a positive integer which is unique to that snapshot on the backup store.

This is a custom Finder view, with a button to Enter Time Machine at its upper right, and what might appear to be mounted backup snapshots are in fact synthesised, probably from the backup_manifest.plist file at the top level of the backup store. Those backups are therefore not so accessible to apps or the command line.

tmaopenfile

In their open and save file dialogs, apps can only see one .previous backup on the backup store, which appears in the path
[backupstore volname]/[datestamp].previous//…
or in Terminal
/Volumes/[backupstore volname]/[datestamp].previous//…

For apps and Terminal to be able to access backup snapshots, they have to be mounted. A convenient way to do this in the GUI is using Carbon Copy Cloner. They are then listed, as mounted, in
/Volumes/.timemachine/[UUID]

tmabackupsnap

In Terminal, the command
diskutil apfs listSnapshots volume
where volume might be disk7s2, for example, lists the backup snapshots available on the backup store at volume. Before any of them can be accessed, that snapshot must be mounted. They might then typically be accessed through
/Volumes/[backupstore volname]/[datestamp].previous//…

Structure of backup snapshots

The backup store has a distinctive volume structure, consisting of hidden Spotlight indexes, any mounted backup snapshots listed by their datestamp, a property list containing details of all the backups in that store, and an optional property list giving information about any inheritance of those backups.

Backup snapshots themselves contain the eventdb constructed during their original backup process, a checkpoint file, a property list containing all the exclusions which applied to TMA backups when that backup was made, and the backup itself listed by volume name.

Those are summarised in the diagram.

TMAbackupstruc

The Backup Manifest backup_manifest.plist is particularly interesting. It’s a persistently hidden file even when copied elsewhere, although its name doesn’t start with a period/stop. It contains an array of data about each of the backups in the store, marked by their datestamp, start date and an ‘xid’ which differs from the snapshotXID. Two dictionaries of data are provided for each backup: a statistical summary of that backup (key “stats”), including the number of items changed and those propagated from the previous backup; information about the backup in the Volume Store, including its UUID, the snapshotXID, and the name and role of each source volume. This appears to be the best way for apps to access TMA backup information, prior to mounting any backup snapshot.

Consequences

TMA backups provide good access to the user through the Time Machine app and the Finder. However, their ability to manipulate backups is severely limited. As these rely on snapshots, individual files can’t be removed from them, a feature restricted to TMH. Whole backups, complete with their snapshots, can be removed, though. That isn’t supported by the Time Machine app, but can be performed directly in the Finder.

Before other apps or commands can access a backup, its snapshot has to be mounted, which is a non-trivial task. The paths used in the Finder’s synthetic view of backups are quite different from those seen when an app, command or script tries to access the same files, and need considerable care if they are to work.

Finally, tmutil provides very little support for TMA backups at present, and needs an overhaul and revision of its man page.