Last Week on My Mac: Time Machine and APFS

We’re at the beginning of a long road for Apple’s new file system APFS, and the way ahead is far from level. Once Apple does roll out support for Fusion Drives and traditional rotating disks, it then has to start addressing the problem of Time Machine backups.

For the moment, you can’t use Time Machine to back up to an APFS volume. This is because a central part of Time Machine’s magic depends on hard links. When your Mac ‘backs up’ a file which hasn’t changed since the last backup, it doesn’t move any data, but creates a hard link to the last version of that file. This creates the illusion that each Time Machine backup is a complete copy of the backed up volume, when in fact only a tiny proportion of those ‘files’ are distinct from the last backup.

Only APFS doesn’t support hard links [clarification: it doesn’t support directory hard links, on which Time Machine backups depend – see comments below]. Even worse, APFS uses its own cunning tricks to work different kinds of magic with clones and sparse storage strategies – which are not supported by good old HFS+, the Mac Extended file system we’ve been using for nearly twenty years. For the moment, Time Machine works from APFS to an HFS+ backup volume, but it’s not a good fit. As the new features in APFS are used more by macOS and apps, each backup will be like knocking square pegs into round holes. Thousands of times every hour.

Consider a single large file, which is initially written to two areas of storage on an APFS volume. When that is backed up to HFS+, it is written as that single file to the backup.

CoS1

We then clone that file (as might be used in a versioning system), and change a small part of the copy. APFS is smart: rather than writing out a whole new version of that file with those small changes as HFS+ would, it only writes out those changes. The original version is made up from the first two storage areas, the edited version from one of those and a new area containing the changes. These are now two different files which share some common data.

CoS2

HFS+ cannot (of itself) mimic this behaviour: if it were to link the edited file to the original, the changes would be lost as the link would go back to the original; if it were to link the original to the edited version, then the original would be lost instead. When Time Machine comes to make a backup of the changed clone, it has to make a complete copy of the changed file. Thus backups to HFS+ are likely to be larger than their APFS originals, a problem explored by Dave Nanian in his article on Shirt Pocket Watch.

Things get worse when you then try to restore from a backup made on HFS+. There is no inherent facility in HFS+ to associate the original and modified versions of this file, nor to recognise the sections which differ. Reconstructing the file structure on the APFS volume is thus a difficult problem, which Time Machine may not currently try to solve. Restoring the backup to an HFS+ volume, on the other hand, results in something quite different, because of the differences between the two file systems.

These issues appear worst in APFS snapshots, the basis for the new Mobile Time Machine likely to be popular with anyone using a laptop away from their normal Time Machine backup volume. As an APFS snapshot essentially consists of file system metadata, it is hard to see how it can be backed up to HFS+ at all, without consuming much more space and losing structure. Perhaps such snapshots are simply lost, in which case they cannot be restored from an HFS+ backup.

The Time Machine backup volume is also one widely used macOS storage system which is crying out for a modern file system. You don’t have to back up from a modern, large Fusion Drive or SSD for long before your backups contain many millions of files, and occupy several terabytes of space. These backups have to be maintained every hour, thinning out the old, and on conclusion of each backup session its Spotlight metadata have to be re-indexed. It’s amazing that HFS+ has coped so well for so long.

Unfortunately, the great majority of Time Machine backup drives also still use rotating disks, and APFS has been designed primarily for SSDs. This isn’t necessarily a problem, but through the summer it became clear that early releases of APFS were not faring particularly well on hard disks, and Apple openly admitted that it needed more work to improve its performance on them. Time Machine backups will be a good test of that.

This will also be a good test of Apple’s commitment to Macs and macOS, as Time Machine doesn’t exist in iOS. If Apple is as serious about Macs as it keeps claiming, Time Machine will be revised soon to support APFS backup volumes, no later than macOS 10.14. Whether it will prove feasible to convert existing HFS+ backups to APFS is another matter which Apple must already have in its master plan.

(Thanks to Michael Tsai for drawing attention to the original post.)