What doesn’t Time Machine back up?

If you use Time Machine you’ll be aware that its backups don’t contain everything that they could. One obvious example is that, at least when backing up to APFS in Big Sur and later, you can’t back up your System volume any more. This article looks at what is and isn’t backed up by Time Machine in recent versions of macOS, in particular in backups made to APFS. If you want to read about Mojave and earlier, this article should help.

Time Machine has several mechanisms by which items are designated as not to be backed up.

Exclusion lists

The first of those exclusion mechanisms used to be its Standard Exclusions list, at /System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions.plist, but that disappeared without trace in Big Sur. Currently the only way to inspect that list is by locating the hidden file .exclusions.plist at the top level of any backup.

This lists several categories of exclusions, of which the first are apiExclusionPaths, which are added by individual third-party apps. The equivalent of what used to appear in StdExclusions.plist is now given under the standardExclusionPaths key, which contains:

.DocumentRevisions-V100 – the version database on each volume, added in Big Sur,
.Spotlight-V100 – Spotlight metadata,
.Trashes – contents of Trash folders,
.fseventsd – the File System Events database,
/Library/Logs – traditional text log files,
/Users/Guest – guest user files,
/private/var (partial) – various transient files,

among other ephemeral items.

Of those, only one results in any significant loss of data, the version database. Although this was dutifully copied by Time Machine into backups for several years, the current structure of that database appears to make it impossible to restore successfully, even when restoring a complete volume. By the middle of the Catalina cycle, it had become a frequent cause of Time Machine choking, so was added to the standardExclusionPaths for Big Sur. As far as I’m aware, though, this was never fixed in Catalina.

Following those are stickyExclusionPaths, which exclude various photos library contents in Big Sur. This has expanded considerably in Monterey to include language modelling files and more. The key systemFilesExcluded is set to true to ensure that the whole System volume is always excluded, following which are any userExclusionPaths which you have set using tmutil or in the Time Machine pane, as explained here.

Per-item custom exclusions

In addition to the rules given in the Standard Exclusion list, any file or folder can have an extended attribute of type com.apple.metadata:com_apple_backup_excludeItem attached to it as a ‘sticky’ exclusion, which may add it to the stickyExclusionPaths list and prevents it from being backed up by Time Machine. If you then remove that extended attribute, it should be backed up normally again.

iCloud Drive

Items stored in iCloud are a more complex problem. The rule here seems to be that Time Machine does back up the contents of your current iCloud Drive, but only those items which are currently stored locally. Any items which have been evicted to iCloud, and are only represented by local stub files, aren’t backed up.

There’s a sound rationale for this, although I can’t recall it being explained by Apple. Quite a few users now have a great many files evicted to their iCloud Drives, far more than they could store locally. This is perhaps most common among those who work with their Desktop & Documents folders in iCloud. For Time Machine to be able to back up a file evicted to iCloud Drive, that file would need to be downloaded to local storage first. Imagine having a 512 GB internal SSD, and over a terabyte of documents in your iCloud Drive, almost all of which were evicted from local storage. Even if Time Machine were to download these in rotation so as not to exceed local space available, how long might it take to perform that for a backup?

iCloud Drive files which are already in local storage are fair game, though, and as far as I can tell, that’s all you’ll get in your Time Machine backup. So if you want a document backed up locally, and its iCloud icon shows it to have been evicted, select it and ensure that it downloads before a backup is made. You may find my free app Cirrus or its lightweight sibling Bailiff a help with that.

Photos libraries

Photos libraries are something of an oddity here, as it has been known for several years that Time Machine doesn’t back up some of the folders inside Photos libraries, although for some time this was considered to be a bug.

Apple’s advice on backing up Photos libraries is to either use Time Machine, or make a manual copy of the library to external storage, although only the latter will replicate your library in full. When Time Machine backs up a Photos library, it completely omits the database folder within it because it has an extended attribute of type com.apple.metadata:com_apple_backup_excludeItem. This assumes that the contents of that folder can be successfully regenerated by Photos after a library has been restored from backup.

Carbon Copy Cloner

Mike Bombich gives a thorough and detailed account of what CCC doesn’t copy on this page. As far as I can see, his list is essentially the same as Time Machine’s standardExclusionPaths.

5Comments

Add yours

1

Michael Tsai - Blog - What Time Machine Doesn’t Back Up on September 8, 2021 at 3:29 pm

[…] Update (2021-09-08): See also: Howard Oakley. […]

LikeLike
2

2J on September 10, 2021 at 12:40 am

Howard, thank you for your continued contributions to the community of valuable information and tools. Two quick questions:

1) Any idea what the “V100” suffix means re the names of several invisible files?

2) As far as I’m aware Apple has tinkered with Time Machine in most (if not each and every) new release of macOS since 10.6. However, and rather maddeningly, Apple usually provides little or even no information about these changes. Last year was no exception, except that the scale and extent of changes was massive. Now that macOS 12 is nearing release, has any information emerged about any improvements to Time Machine in Monterey? Aside from tweaks to exclusions lists, are there any substantive changes? Not surprisingly, Apple’s product marketing web pages are silent on the matter, and unfortunately I also haven’t seen any relevant information in the beta release notes. Though neither of those things means that nothing has changed…

LikeLiked by 1 person
- 3
  
  hoakley on September 10, 2021 at 6:40 am
  
  Thank you.
  I believe that V100 means ‘version 1.00’, and is used in the event that one day there might also be a V200.
  Time Machine is almost invariably omitted from details of changes in new versions of macOS, as it falls between three stools. It isn’t one of the parts of macOS which concerns many developers, so is usually omitted from WWDC coverage, although there have been occasional exceptions. It isn’t normally considered to be important to sysadmins either, so doesn’t get detailed in ‘enterprise’ information. It’s primarily for regular users, but there priorities seem controlled by marketing, who don’t appear to think that backups are ‘popular’ enough to mention. The introduction of TM to APFS in Big Sur was the only exception, though, which did get a small mention.
  It’s also one of the features of beta-releases which is least tested, as many of those testing betas don’t back them up using TM.
  I can’t tell you what Monterey brings in terms of changes to TM, but will be explaining them once it has been released, and will support them in T2M2 2.0.
  Howard.
  
  LikeLike
4

melinportland on January 21, 2022 at 12:21 am

Hi there, great article. I am trying to learn more about how Time Machine behaves so I can understand what *is* being backed-up. Do you find that Contacts that are synced with iCloud are backed up by Time Machine? I regularly back-up my computer (Mac Book Pro 2021, 4TB hard drive, OS Monterey v 12.1) with Time Machine. My contacts are synced with iCloud. I was not able to restore Contacts via Time Machine, although I could navigate to a previously saved version of contacts on Time Machine and see the cards and even get as far as the Import screen. You mention, “The rule here seems to be that Time Machine does back up the contents of your current iCloud Drive, but only those items which are currently stored locally.” The data *does* seem to be in ~/Library/Application Support/AddressBook on Time Machine but not restorable via the usual process of opening Contacts, entering Time Machine, navigating to desired version, click Restore, Import. Would love to hear about your experience with restoring iCloud-synced Contacts from Time Machine.

LikeLiked by 1 person
- 5
  
  hoakley on January 21, 2022 at 8:32 pm
  
  Thank you.
  The basic rule for Time Machine backups with iCloud is that it does back up contents of iCloud Drive which are stored locally on that Mac, but when they are evicted to iCloud storage, it no longer does so.
  However, shared databases stored in iCloud, such as your Address Book, aren’t backed up, as they don’t exist as shared files in your iCloud Drive, but as a shared database. As with iCloud Keychain, you need to temporarily disable sharing of that data, allow it to download locally, and then you should be able to back it up.
  Howard.
  
  LikeLike

Share this:

Like this:

Related