Last Week on My Mac: Will Disk Utility ever work properly?

When Apple developed APFS, it decided neither to make it open source, nor to document it sufficiently to help third parties develop their own utilities for its maintenance. In doing so, it made a commitment to its users that Apple itself would provide all the tools necessary to work with its new file system. After all, any file system without sufficient tools to check, repair and maintain it isn’t fit for release, is it?

Disk Utility and its command line check and repair tool fsck_apfs are thus among the most important utilities in the whole of macOS. While there are several excellent alternatives which can perform the same tasks on HFS+ volumes, Apple’s policies on APFS have successfully locked all competition out of this market. Yet, over five years after the new file system’s release on 27 March 2017, Disk Utility is still riddled with bugs, and can’t check or repair disks containing Time Machine backups unless the Mac is booted in Recovery Mode, or without resorting to skulduggery with hidden snapshots.

This is easily demonstrated if you already make Time Machine backups to an APFS volume, and are running macOS 12.4. Once Time Machine has completed one of its hourly backup sessions, open Disk Utility, select that backup volume and click First Aid.

diskutil1

You’ll almost certainly see this error which perversely leads you to believe that repair failed because of problems found on the volume, and recommends that you “back up the data on this volume”. As it still appears impossible to do that with Time Machine APFS backup volumes, users are tempted to erase their backups and start afresh, in the hope that the problem doesn’t recur.

In fact, what has happened is that Disk Utility hasn’t attempted to unmount the backups stored on that volume properly. Instead of calling fsck_apfs using its -l option to attempt a live volume check and repair, the action has simply failed.

There is a workaround, but you have to be careful or you’ll end up racing against Time Machine, which will be busy trying to stop you from running First Aid. First, select the backup volume and unmount it, and repeat that with any other volumes in the same container. Then, in the Finder, press Command-Shift-. (period or full stop) to reveal all hidden items. Navigate your way to the hidden folder containing backups at /Volumes/.timemachine/[UUID] and unmount all the snapshots listed in that folder.

diskutil11

You can’t select them all and eject them together, but have to eject them individually. You should then be able to run First Aid without any errors. If you prefer, you can instead use the command
diskutil verifyvolume /dev/diskNsM
where diskNsM is the device identifier as shown in Disk Utility, such as disk7s1, or use fsck_apfs if you want to employ its additional options, for example to exclude checks of snapshots.

Once any repair is complete, select the backup volume (still unmounted) or its container and click on the tool to mount it again. Over the next few minutes, Time Machine will successively remount its backup snapshots, so they will become available to Spotlight search, and access through the Finder and Time Machine app.

The simpler alternative is to resign yourself to starting your Mac up in Recovery, and using Disk Utility there to run First Aid on the backup volume.

What’s strange about this, of course, is that First Aid can’t run because the container in which the backup volume is located can’t be unmounted while all those snapshots are mounted. Yet APFS snapshots are strictly read-only, and in the case of the SSV containing the boot system, are considered stable enough to verify using cryptographic hashes.

For most users, this problem is compounded by the fact that macOS cunningly conceals the many snapshots it has mounted in a hidden folder. This isn’t the first time that Apple’s ploys to hide details from the user have had unintended consequences. As neither the user nor Disk Utility can normally see all these mounted snapshots, the user has no idea why Disk Utility can’t unmount the container in which their backup volume is located. Indeed, Disk Utility plays along with the deception in offering to mount that container, as if it has been unmounted successfully. Deliberately deceiving users can come back to bite them when things don’t work out.

I’ll be most surprised if Disk Utility is ever fixed in Monterey. All we can hope for now is that Apple finally meets its commitment to users in macOS 13, more than five years after it released the first version of APFS, and two years since Time Machine has been making backups on APFS volumes. Has it really been that difficult to get Disk Utility to work properly, or was it just that Apple didn’t care to?

Postscript

Several people have asked me whether the same fundamental problem accounts for failure to eject an external disk containing Time Machine backups on an APFS volume. The answer is almost certainly yes. If you try to eject a disk and get a refusal:

  1. Check that Time Machine isn’t making a backup, or due to make one soon.
  2. Eject each volume on the disk in the Finder or Disk Utility.
  3. Open /Volumes/.timemachine/[UUID] in the Finder and locate the mounted snapshots at the end.
  4. Eject each of those mounted snapshots in the Finder.
  5. Try ejecting the disk again.

I wish you success every time.

I’m very grateful to kapitainsky for pointing me in the right direction with this workaround.