Lots of Mac users now have very large Photos Libraries, and many are moving them to external storage. If you have done that, use Time Machine to back up that external drive, and have now upgraded to High Sierra, you’ll surely be shocked to learn that Apple says that won’t work: try restoring from a Time Machine backup of a Photos Library kept on an external disk in High Sierra, and the restored Library won’t work.

I often moan about Apple’s dearth of documentation; in this case, it is Apple which has drawn attention to the problem, in its Help page on backing up Photos Libraries:
WARNING: If a Photos library is located on an external drive, don’t back up the drive using Time Machine. The permissions for your Photos library may conflict with those for the Time Machine backup.

Oddly, this warning isn’t mirrored in this Photos Help page explaining how to restore a Photos Library from a Time Machine backup.

So what is this problem with ‘conflicting permissions’?

Time Machine backups are performed by a system process running as root, and regularly handle all sorts of restrictive permissions, including folders and files which are root-only and protected by SIP. Backups also work fine for all users, even though different users cannot see inside one another’s Home folders. Time Machine should, and does, set the permissions of your backups to those of the folders and files being backed up.

Neither does Time Machine appear to have a problem with ‘extended permissions’, or Access Control Lists (ACLs). Although these are not used particularly widely in user files, they do appear in places, and seem to work fine through backing up and restoring.

Furthermore, Apple still recommends using Time Machine to back up Photos Libraries which are kept on internal storage. It is hard to see how Photos might have to use different permissions for any part of its Libraries only when they are stored externally.

With the problem apparently confined to High Sierra, and not troubling Sierra, it is most likely an issue with Apple’s new file system, APFS, or Time Machine in High Sierra, or its new version of Photos.

APFS looks an appealing scapegoat, but doesn’t fit into any logical pattern. After upgrading to High Sierra, internal storage could have been converted to APFS (if an SSD), or remain in HFS+ (Fusion Drive, or hard disk). Neither APFS nor HFS+ should behave any differently on external storage: if Time Machine can restore reliably to internal storage, so it should be able to to external storage. Neither does Apple mention any difference in reliability between the two file systems.

Similar arguments apply to Time Machine and to Photos: if they handle backups from internal storage correctly, what could possibly go wrong with an external Photos Library?

Could the term ‘permissions’ here be a gloss for non-technical users and actually mean something different, like extended attributes (xattrs)? Although that seems most unlikely, Photos does use xattrs in certain components of a Photos Library to designate items which should not be backed up by Time Machine.

In both Sierra and High Sierra, the com.apple.metadata:com_apple_backup_excludeItem xattr is attached to the Library’s databases such as the whole /database folder inside a Photos Library, and to those in /private/com.apple.photoanalysisd/GraphService/PhotosGraph. This doesn’t appear to have changed.

Browsing a High Sierra Photos Library doesn’t reveal any obvious problems which might trip up Time Machine either. A couple of folders (/private and /resources) have hidden files (.metadata_never_index) to prevent their metadata from being indexed for Spotlight, but that is used elsewhere apparently without causing problems with Time Machine.

Apple’s warning leaves us in limbo. There are alternative ways of backing up a Photos Library on an external drive, but it is a serious admission that the most popular, almost universal, backup software for macOS cannot reliably restore these quite widely-used Libraries.

More worrying still, given that Apple leaves the cause obscure, is the fact that other content stored on external drives might be similarly affected. If the cause of the problem is ‘conflicting permissions’ between those used inside a Photos Library and those of a Time Machine backup, then third party products could suffer the same issue, preventing users from restoring their documents from Time Machine backups to an external drive.

Apple clearly knows what is going wrong here, but tells us too little to be of any real help. If it’s a bug in Photos, Time Machine, or a subsystem which they are using, then it should have been fixed by now, given its severe implications. If it’s a feature of High Sierra, then it must be properly documented so that others (if not Apple) can work around it.

In the meantime, let me repeat Apple’s warning lest you forget:
WARNING: If a Photos library is located on an external drive, don’t back up the drive using Time Machine. The permissions for your Photos library may conflict with those for the Time Machine backup.

Perhaps we should all broadcast this problem far and wide, in the hope that Apple might be shamed into detailing it properly, and perhaps even fixing it.

Postscript: in the couple of days between writing this and it being published here, Apple has changed that Help file. It now says something quite different. Instead of the above, it states:

“WARNING: If a Photos library is located on an external drive, don’t use Time Machine to store a backup on that external drive. The permissions for your Photos library may conflict with those for the Time Machine backup.”

This is, of course, totally different to the original warning which I have quoted verbatim above. It still makes no sense at all, though, and who would be dumb enough to back up anything to the same disk? If you are going to warn users that doing that might (somehow) result in permissions problems, shouldn’t you actually warn them that backing anything up to the same disk is a waste of time in the first place?

Apple needs to come clean and explain what the problem is, and start making sense.