You’d be forgiven for thinking that Time Machine backs up everything a user could possibly want. After all, isn’t that what Apple tells us? Only that’s not entirely correct: Time Machine backs up most of what we want, but is surprisingly forgetful when it comes to some very important files.
This isn’t any easy matter to investigate, because of all the variables which do vary between different users, but recent articles by Maxwell @mxswd have raised several important questions. They fall into three main areas: Photos Libraries, iCloud Drive, and everything else.
Apple’s advice on backing up Photos Libraries is to either use Time Machine, or make a manual copy of the library to external storage. At present, only the latter will actually replicate your library in full. When Time Machine backs up a Photos Library, it completely omits the database folder within it, where all its vital metadata are stored.
Maxwell has described this in detail here.
Apple’s Photos User Guide refers to potential problems with permissions if your Photos Library is on an external drive which is also backed up to that same drive. I’ve previously commented that this remark is bizarre, as no one should ever contemplate backing up one volume on a drive to another volume on the same drive: that’s not a backup but a duplicate, which is just as vulnerable as the original should anything go wrong with that drive.
Apple has apparently been informed of this problem with Time Machine, but hasn’t got round either to addressing it, or to warning users that their Time Machine backup will lose their metadata when restored.
This is a strange behaviour on the part of Photos and Time Machine, given that manually copying Photos Libraries works around this issue perfectly well. It happens because the database folder inside a Photos Library has an extended attribute of type
com.apple.metadata:com_apple_backup_excludeItem which prevents it from being backed up by Time Machine. In other words, this is deliberate, perhaps because of a bug in Photos. Apple needs to come clean over this.
This is a more complex problem. Maxwell notes that Time Machine doesn’t back up folders which are found in iCloud Drive, but in my experience it’s not as simple as that. The rule seems to be that Time Machine does back up the contents of your current iCloud Drive, but only those items which are currently stored locally. Any items which have been evicted to iCloud, and are only represented by local stub files, aren’t backed up.
Here Apple has a sound rationale, although I can’t recall it explaining this to users anywhere. Quite a few users now have a great many files evicted to their iCloud Drives, far more than they could store locally. This is perhaps most common among those who work with their Desktop & Documents folders in iCloud.
For Time Machine to be able to back up a file evicted to iCloud Drive, that file would need to be downloaded to local storage first. Imagine having a 512 GB internal SSD, and over a terabyte of documents in your iCloud Drive, almost all of which were evicted from local storage. Even if Time Machine were to download these in rotation so as not to exceed local space available, how long might it take to perform that?
iCloud Drive files which are already in local storage are fair game, though, and as far as I can tell, that’s all you’ll get in your Time Machine backup. So if you want a document backed up locally, and its iCloud icon shows it to have been evicted, select it and ensure that it downloads before a backup is made. You may find my free app Cirrus or its lightweight sibling Bailiff a help with that.
Most of the other files which are omitted from Time Machine backups are ephemeral caches and similar which are readily rebuilt after a restore. Those are listed in full in the Standard Exclusions list, at /System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions.plist (thanks to Hannes @hjuutilainen for reminding me of this). Maxwell notes that the Voice Memos app omits its recordings, which seems a bit strange.
If you want to check whether any given item should be backed up by Time Machine, it’s easy to browse your own backups. You can also inspect the extended attributes of the original folder or file using my free xattred: if anything has an extended attribute of type
com.apple.metadata:com_apple_backup_excludeItem, then Time Machine will exclude it. And if you were to remove that extended attribute, then it should be backed up.
I’m not infrequently asked by users whether a specific item is backed up by Time Machine so that it can be restored when required. The very best way to test this is what you should be doing periodically with any backup: tuck the current copy away somewhere safe, and try to restore it from a backup.
Only when you can have confidence that your documents can be restored should you trust your backup, whether it’s Time Machine or anything else.
Thanks to Maxwell for reminding me to look at this more explicitly, and for his research, and to Hannes for reminding me of the Standard Exclusions list. Amended at 1725 26 September 2019 to add that useful path.