What doesn’t Time Machine back up?

You’d be forgiven for thinking that Time Machine backs up everything a user could possibly want. After all, isn’t that what Apple tells us? Only that’s not entirely correct: Time Machine backs up most of what we want, but is surprisingly forgetful when it comes to some very important files.

This isn’t any easy matter to investigate, because of all the variables which do vary between different users, but recent articles by Maxwell @mxswd have raised several important questions. They fall into three main areas: Photos Libraries, iCloud Drive, and everything else.

Photos Libraries

Apple’s advice on backing up Photos Libraries is to either use Time Machine, or make a manual copy of the library to external storage. At present, only the latter will actually replicate your library in full. When Time Machine backs up a Photos Library, it completely omits the database folder within it, where all its vital metadata are stored.

Maxwell has described this in detail here.

Apple’s Photos User Guide refers to potential problems with permissions if your Photos Library is on an external drive which is also backed up to that same drive. I’ve previously commented that this remark is bizarre, as no one should ever contemplate backing up one volume on a drive to another volume on the same drive: that’s not a backup but a duplicate, which is just as vulnerable as the original should anything go wrong with that drive.

Apple has apparently been informed of this problem with Time Machine, but hasn’t got round either to addressing it, or to warning users that their Time Machine backup will lose their metadata when restored.

This is a strange behaviour on the part of Photos and Time Machine, given that manually copying Photos Libraries works around this issue perfectly well. It happens because the database folder inside a Photos Library has an extended attribute of type com.apple.metadata:com_apple_backup_excludeItem which prevents it from being backed up by Time Machine. In other words, this is deliberate, perhaps because of a bug in Photos. Apple needs to come clean over this.

iCloud Drive

This is a more complex problem. Maxwell notes that Time Machine doesn’t back up folders which are found in iCloud Drive, but in my experience it’s not as simple as that. The rule seems to be that Time Machine does back up the contents of your current iCloud Drive, but only those items which are currently stored locally. Any items which have been evicted to iCloud, and are only represented by local stub files, aren’t backed up.

Here Apple has a sound rationale, although I can’t recall it explaining this to users anywhere. Quite a few users now have a great many files evicted to their iCloud Drives, far more than they could store locally. This is perhaps most common among those who work with their Desktop & Documents folders in iCloud.

For Time Machine to be able to back up a file evicted to iCloud Drive, that file would need to be downloaded to local storage first. Imagine having a 512 GB internal SSD, and over a terabyte of documents in your iCloud Drive, almost all of which were evicted from local storage. Even if Time Machine were to download these in rotation so as not to exceed local space available, how long might it take to perform that?

iCloud Drive files which are already in local storage are fair game, though, and as far as I can tell, that’s all you’ll get in your Time Machine backup. So if you want a document backed up locally, and its iCloud icon shows it to have been evicted, select it and ensure that it downloads before a backup is made. You may find my free app Cirrus or its lightweight sibling Bailiff a help with that.

Everything else

Most of the other files which are omitted from Time Machine backups are ephemeral caches and similar which are readily rebuilt after a restore. Those are listed in full in the Standard Exclusions list, at /System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions.plist (thanks to Hannes @hjuutilainen for reminding me of this). Maxwell notes that the Voice Memos app omits its recordings, which seems a bit strange.

If you want to check whether any given item should be backed up by Time Machine, it’s easy to browse your own backups. You can also inspect the extended attributes of the original folder or file using my free xattred: if anything has an extended attribute of type com.apple.metadata:com_apple_backup_excludeItem, then Time Machine will exclude it. And if you were to remove that extended attribute, then it should be backed up.

One lesson

I’m not infrequently asked by users whether a specific item is backed up by Time Machine so that it can be restored when required. The very best way to test this is what you should be doing periodically with any backup: tuck the current copy away somewhere safe, and try to restore it from a backup.

Only when you can have confidence that your documents can be restored should you trust your backup, whether it’s Time Machine or anything else.

Thanks to Maxwell for reminding me to look at this more explicitly, and for his research, and to Hannes for reminding me of the Standard Exclusions list. Amended at 1725 26 September 2019 to add that useful path.

8Comments

Add yours

1

Valdo on September 26, 2019 at 7:27 am

Very interesting acumen, but as non native English speaking person I need a bit help.

* The Photo backup issue is related to the libraries within the Photos app or to the Pictures folder? I suppose that’s the first one but not sure.
* iCloud backup is clear except how to find out which folders are stored locally and which only on iCloud Drive?

Thank you!

LikeLiked by 1 person
- 2
  
  hoakley on September 26, 2019 at 7:53 am
  
  Thanks.
  Photos Libraries have the extension .photoslibrary, and are actually folders posing as files. Other items in the Pictures folder are fine.
  To see which iCloud Drive items are evicted into iCloud, just browse them in the Finder and the icon by them shows whether they’re stored remotely or have been downloaded.
  Howard.
  
  LikeLike
3

Valdo on September 26, 2019 at 8:02 am

Thank you very much Howard !
On my Mac all files seems to be stored locally, I saw this small cloud icon only during browsing the iCloud Drive with my iPad.

LikeLiked by 1 person
- 4
  
  hoakley on September 26, 2019 at 8:08 am
  
  They will only be evicted if you have Optimise Storage turned on in your iCloud settings.
  Howard
  
  LikeLike
5

Paul on September 26, 2019 at 9:07 am

Thanks once again for the intelligence. TBH although I have TimeMachine active on my Macs to an external drive in each case ( WD Passport which are fairly robust) I also run regular clones using Carbon Copy Cloner to a central RAID which uses high quality drives with a 5+ year guarantee. I don’t trust TM in other words although it’s a handy localised facility whereas CCC has never let me down and has good support. For super critical files like my archive folio (my design work for the last 20 years) I have 3 copies, one off Mac in a physical box. These backups are also in two physical locations as kind of personal disaster recovery plan – if the office where I work floods, burns down, I have another option. Call me paranoid.

LikeLiked by 1 person
- 6
  
  hoakley on September 26, 2019 at 5:29 pm
  
  Thanks. That’s not paranoia, but a thought-through backup policy.
  I wish more users were as thorough.
  Howard.
  
  LikeLike
7

Michael Tsai - Blog - What Time Machine Doesn’t Back Up on September 26, 2019 at 5:23 pm

[…] Howard Oakley says iCloud Drive does get backed up, and I do, in fact, see the files in my Time Machine […]

LikeLike
8

gastropod on September 26, 2019 at 11:49 pm

The Photos issue is probably because backing up a database that is in use will often result in a corrupted backup. Databases usually have an internal way to export the data to a recoverable format, and that locks records while the export is in progress. Aperture does this explicitly with the Vault feature. You initiate an update to the Vault by hand and until it’s finished nothing else can be done in Aperture. The Vault can then be safely backed up by Time Machine or other backup program (assuming it isn’t being updated at the time, but that’s much less likely than Aperture being in use.) I wouldn’t trust even a Finder copy of a Photos/Aperture library to be fully intact if it was copied while Photos/Aperture was running and active.

Apple really needs to do something about this other than closing their eyes and sticking their fingers in their ears. It’s not as if they don’t understand the issues. Although, given the fragility of much of their metadata, they do seem to think that our time spent working on metadata wasn’t really all that important and we can just do it all over again, right?

LikeLiked by 1 person

Share this:

Related