macOS superimposes additional space management features over APFS to make some file system contents purgeable when it runs short of free space. These aren’t explained in any coherent way in user documentation, nor can I find any explanation provided to developers. Furthermore, information given to users is inconsistent across the Finder and Disk Utility, and appears riddled with bugs that only confound the user. This article demonstrates these in an edge case for clarity.
This space management is based on the concept of purgeable files, as mentioned in Disk Utility’s Help pages and discussed in detail here. Although Apple provides a conceptual guide to programming the file system, it hasn’t been updated for over four years and doesn’t cover macOS space management at all.
At its heart is the equation (whose origin is obscure)
Capacity = Available + (Used – Purgeable)
Should a file operation, such as copying a file to a volume, require more space than is currently free, then macOS is claimed to delete items deemed purgeable to augment free space and enable the operation to complete. Hence, when declaring space available in a volume, macOS will usually give that currently free plus that designated purgeable. There are innumerable reports of that available space not being realised in practice, and of huge errors in reporting purgeable and available space. This demonstration provides a reproducible sequence to demonstrate them, and the confusion this causes in users.
Demonstration
This is best performed on an external disk formatted in APFS, containing no other data. Although you can do this using a single partition including the whole of the disk, it’s simplest to repartition the disk with a relatively small APFS container, such as 100 GB, using Disk Utility.
Add a second volume to your test container, also in plain APFS format. In this case, I have named those two volumes TestVolume1 and TestVolume2. Then write large files to TestVolume1 to almost fill the container. Here, I used my free app Stibium to write two files each of 47 GB for a total of 94 GB. Inspect these using Disk Utility.
The container has a Capacity of 100 GB, with 94 GB Used and 6 GB Free, which is correct.
TestVolume1 has a Capacity of 100 GB, 6 GB Available, 94 GB Used, and none Purgeable, which is also correct.
TestVolume2 has a Capacity of 100 GB, 6 GB Available, and Other Volumes have 94 GB Used. Everything appears in order.
The next step is to turn the current Used space on TestVolume1 into Purgeable space, which is a little more complicated. This requires making a snapshot of at least TestVolume1, then deleting the files making up the 94 GB of Used space. One way to do that is to:
- Unmount all current Time Machine backup storage, to ensure test files aren’t backed up to it.
- Open Time Machine settings and remove TestVolume1 and TestVolume2 from the list of excluded volumes, enabling Time Machine backups for those two volumes.
- Open Terminal and enter the command
tmutil localsnapshot
, which will then make snapshots of all volumes being backed up, including both TestVolume1 and TestVolume2. Note that snapshots made by third-party tools may not behave the same as those of Time Machine, and may not be included in Purgeable space. - Using Disk Utility, confirm that a snapshot has been made of TestVolume1.
- Drag the large files in TestVolume1 to the Trash and empty it, leaving both volumes empty.
- Restore Time Machine settings by adding the two volumes back to the exclusion list before remounting Time Machine backup storage.
When you now inspect the two volumes in Disk Utility, their status has changed.
TestVolume1 now has a Capacity of 100 GB, 100 GB Available, 94 GB Used, and 94 GB Purgeable. Although this might appear puzzling, as all the space Used is considered Purgeable, it remains arithmetically correct.
TestVolume2 now has a Capacity of 100 GB, 6 GB Available, Other Volumes have 94 GB Used, and only 160 KB Purgeable, which is also correct.
Look in the Finder’s Get Info dialogs for those two volumes, and the figures given are completely different, and confusing.
Most significantly, Available and Purgeable space for TestVolume1 is given incorrectly as the same as those for TestVolume2. This is arithmetically correct, but doesn’t reveal the 94 GB of Purgeable space, and the only clue given is in the large Used space reported. As the user can’t see any files in the volume, there is insufficient information to inform the user that those 94 GB are in fact purgeable from TestVolume1, and can be deleted by the user or by macOS if it purges the volume.
Now try copying a large test file, in this example 10 GB, to TestVolume2.
The Finder then reports that there isn’t sufficient disk space, and refers the user to the Storage section in About This Mac.
Information given in Storage deepens the confusion, as the two volumes are there shown as having 100 GB available for use, and no clue is given as to the existence or location of Purgeable space.
If the user does guess what the cause of this problem is, and tries copying the 10 GB file to TestVolume1 rather than TestVolume2, macOS will then purge the snapshot, and both volumes will have sufficient free space to accommodate that file.
Following purging and successful copying of the 10 GB file, TestVolume1 has a Capacity of 100 GB, 90 GB Available, 10 GB Used, and only 35 MB Purgeable, although it’s unclear what that might represent.
TestVolume2 now has a Capacity of 100 GB, 90 GB Available, Other Volumes have 10 GB Used, but curiously its remaining snapshot of 414 KB isn’t declared as being Purgeable, although it appears to be included in its own Used space of 1.3 MB.
If this isn’t confusing enough for you, then stop short of purging the snapshot, rename TestVolume1 with a leading . to hide it from the Finder, and give the disk to someone else to try to sort out.
Conclusions
- macOS space management features can be extremely confusing.
- They require careful and explicit documentation for users, support staff, and developers. Currently, there is essentially no documentation of any help, and Disk Utility’s Help page is incorrect and opaque. Purgeable space taken by snapshots can be recovered by the user deleting those snapshots.
- The only GUI tool in macOS that reports space consistently is Disk Utility. The Finder and Storage give figures that are incorrect, and only confuse further.
- Although purgeable space can prove useful, its current implementation per volume conflicts with APFS volumes sharing free space within a container. Per volume control of purging leads to blocking of available space, as shown in the demonstration.
- macOS space management needs to be rethought, redesigned, implemented without inconsistencies or bugs, and thoroughly documented for users, support staff and developers. Anything short of that will leave us all in the same or worse mess.
- Space management isn’t leading-edge or sexy and doesn’t bring any new emoji, but used to be what Apple was so good at.
I am extremely grateful to kapitainsky for testing and extensive comments that led me to this demonstration, and understanding how this really doesn’t work at all.