hoakley April 27, 2023 Macs, Technology

Where does macOS get its volume free space figures from?

Have you ever wondered where apps like Disk Utility get their figures for free space on volumes, particularly those for purgeable and available space? If you thought they got them from APFS, then I’m sorry to disappoint, but their source is the space management feature in macOS, its CacheDelete subsystem.

To look at this in more depth, I generated a file system event sufficient to trigger CacheDelete to recalculate its measures of free space, on a VM with log privacy disabled, no iCloud connection, and with Time Machine backups turned off.

CacheDelete’s service daemon is deleted, which also has a helper process deleted_helper. They rely on a set of definitions stored in /System/Library/CacheDelete, currently 32 short property lists used to define the services most likely to have purgeable data, and set how CacheDelete should purge those files. As those property lists are stored on the System volume, they can’t be changed or augmented by the user or third-parties.

CacheDelete forceRefresh

In this example, the file system event was deleting a large file by emptying the Finder’s Trash. The sequence starts with the Finder posting an EmptyingTrash event to the deleted daemon. This becomes a CacheDelete event key of CACHE_DELETE_EMPTY_TRASH and a forceRefresh request for deleted. Once it has validated mountpoints for volumes, deleted starts work, setting the urgency to 3 and the query path to SLOW, to perform a thorough refresh.

cachedeletetrash1

deleted then calls its helper to evaluate the fsPurgeable total, the number and size of files for 10 standard services that are marked as being purgeable in the file system, using an inode flag. This is reported back in a total broken down by service:

/System/Volumes/Data freespace: 30861533184
CACHE_DELETE_AMOUNT = 10989568
CACHE_DELETE_ITEMIZED_PURGEABLE =
- CACHE_DELETE_TOTAL_FSPURGEABLE = 10989568
- com.apple.fspurgeable_data = 10756096
- com.apple.fspurgeable_document = 233472
[others zero].

cachedeletetrash2

In the next phase, the deleted daemon checks a longer list of around 39 services, including those in the first list, with the Service Info Amount for each. These figures represent the space required for items considered to be nonpurgeable and those that are shared purgeable, and CacheDelete updates each in its list of nonpurgeable space, to arrive at a total for the volume.

CACHE_DELETE_NONPURGEABLE_AMOUNT = 49331272
CACHE_DELETE_SHARED_PURGEABLE = 14124800
CACHE_DELETE_TOTAL_AVAILABLE = 14124800
CACHE_DELETE_ITEMIZED_NONPURGEABLE =
- com.apple.logd.cachedelete = 49331272
com.apple.AssetCache.builtin.CacheDelete = 73728
com.apple.coresymbolicationd.cache-delete = 49152
[others zero].

That’s followed by a further round of checks of purgeable space by each service, and another set of reports on space, before deleted calls its helper to re-evaluate fsPurgeable for the ten standard services.

cachedeletetrash3

CacheDelete next builds a list of recently used apps, which is used by deleted to check for further purgeable space, including that inside their containers. This inevitably involves negotiating TCC and privacy protection, depending on the app. That’s followed by a round of checks on plug-ins, based on PlugInKit.

With all the services and apps checked, deleted arrives at a summary report giving totals and a detailed breakdown.

CACHE_DELETE_FREESPACE = 30861533184
CACHE_DELETE_SHARED_PURGEABLE = 15972096
CACHE_DELETE_TOTAL_AVAILABLE = 15972096
CACHE_DELETE_TOTAL_FSPURGEABLE = 10989568
CACHE_DELETE_NONPURGEABLE_AMOUNT = 49331272
CACHE_DELETE_ITEMIZED_NONPURGEABLE =
- com.apple.logd.cachedelete = 49331272
com.apple.AssetCache.builtin.CacheDelete = 73728
com.apple.cache_delete_app_container_caches = 12288
com.apple.coresymbolicationd.cache-delete = 18
others zero.

cachedeletetrash0

On the fairly minimal Data volume of a VM running Ventura 13.3.1 without iCloud or Time Machine snapshots, the whole sequence took 1.3 seconds and thousands of log entries. The total space occupied by files marked as being purgeable was just under 11 MB, with slightly less than 16 MB designated as shared purgeable, and 49 MB of nonpurgeable space. On most Data volumes, those figures will be increased substantially by Time Machine snapshots, but not those made by third-party apps (as they aren’t purgeable by CacheDelete). Purgeable iCloud space remains more of a mystery.

Checking free space

Apps, including Disk Utility and those third-party apps that use the URLResourceValue volumeAvailableCapacityForImportantUsage and similar API calls, derive purgeable and available space from CacheDelete’s latest figures. This is easy to confirm, as when the app calls for those figures, CacheDelete’s entries in the log reveal that it’s providing them. Use the predicate
subsystem == "com.apple.cache_delete"
to see its numerous entries.

You may be surprised to learn that when you open the Finder’s Get Info dialog on a volume, no call is made to CacheDelete to obtain its latest figures for purgeable or available space. As there’s no indication in the log that those figures are obtained from anywhere comparable, it’s most likely that the Finder maintains its own values, and updates them periodically from CacheDelete. What is strange, though, is that when the Finder posts a file system event that’s inevitably going to result in CacheDelete refreshing its authoritative figures, the Finder doesn’t appear to obtain the updated figures once CacheDelete has compiled them.

This may explain why the Finder’s Get Info figures can become badly out of kilter with those shown in Disk Utility and third-party apps. It emphasises the fact that, if you want accurate and up-to-date figures for free space, you should ignore the Finder and check in Disk Utility or elsewhere. The Finder is lost.

23Comments

Add yours

1

Graham on April 27, 2023 at 7:51 am

Interesting findings, thanks. And some interesting discrepancies. On my system I get the following results from different methods:

Log method:

sudo log show –predicate ‘subsystem == “com.apple.cache_delete” && eventMessage CONTAINS[c] “CACHE_DELETE_FREESPACE :”‘ –info –last 1m | grep “CACHE_DELETE_FREESPACE” | tail -n 1 | awk ‘{ gsub(/\(/,””); print $(NF-1) }’

-> 653.83 (GB)

Purgeable + free space from Log method (hand calculated):

(CACHE_DELETE_FREESPACE + CACHE_DELETE_AMOUNT – CACHE_DELETE_NONPURGEABLE_AMOUNT) / 1073741824

-> 677.12 (GB)

ObJC Foundation method (should also theoretically be the amount of purgeable space):

osascript -l ‘JavaScript’ -e “ObjC.import(‘Foundation’); var freeSpaceBytesRef=Ref(); $.NSURL.fileURLWithPath(‘/’).getResourceValueForKeyError(freeSpaceBytesRef, ‘NSURLVolumeAvailableCapacityForImportantUsageKey’, null); Math.round(ObjC.unwrap(freeSpaceBytesRef[0]) / 1073741824)”

-> 683 (GB)

UNIX method:

df -h

-> 609 (GB)

Finder:

-> 733.61 (GB)

So Finder thinks I have way more space, although similar to the figure obtained if I divide the purgeable bytes by 10^9 (-> 727 GB) instead of 2^30.

LikeLiked by 1 person
- 2
  
  hoakley on April 27, 2023 at 3:11 pm
  
  Thank you.
  As far as I can see, purgeable snapshot space is accounted for separately, and not in the CacheDelete figures given in the log. I assume that all values given by CacheDelete and in the GUI are modern decimal-based bytes, rather than gibibytes etc, but that could easily introduce misunderstandings!
  df shouldn’t give you any allowance for purging, and only unconditionally free space, AFAIK.
  The Finder is probably remembering something from a few days ago, and hasn’t been paying attention since then.
  Howard.
  
  LikeLike
3

Duncan on April 27, 2023 at 1:52 pm

More impressive and thorough detective work, Howard.

I often wonder when Apple lets certain ‘technically-specific’ features become out-of-date or unusable, if their mentality is “Well, in the future storage will be so plentiful it will be ‘too cheap to meter'”, and thus not worth bothering over. In one sense they’d be right, as, for example, no one cares if their analog modem scripts are kept up-to-date.* Fussing over the specifics of disk capacity should become a thing of the past, if we eventually get so much storage that there would be no practical way to fill it all.

But such an ideal future denies that we do have limited capacity now, and run up against hard walls when something like a giant software update or system upgrade runs out of room. We’re in an awkward transition stage between technology available today (which sometimes needs hands-on maintenance) and advanced or abundant technology available tomorrow which can truly recede into the background.

* See: /Library/Modem Scripts for vestigial evidence

LikeLiked by 1 person
- 4
  
  hoakley on April 27, 2023 at 3:01 pm
  
  Thank you, Duncan.
  Maybe, but Apple has still got to manage caches, and that’s the primary purpose of CacheDelete before you even wonder whether to offer the user access to theoretically available space. Maybe it will mean that fewer of us care about purgeable space, or try to eat into it.
  Howard.
  
  LikeLike
5

scheblein on April 27, 2023 at 2:29 pm

Where do the posix-like utilities like du and df get their information? I’m thinking of all of the “cross-platform” utilities and security products that purport to support macOS, but are really just running the posix-like commands.

LikeLiked by 1 person
- 6
  
  hoakley on April 27, 2023 at 3:03 pm
  
  As far as I’m aware, those tools are Posix-compliant and merely report unconditionally free space, rather than purgeable and available space. As I’ve said, there are API calls available too, but I doubt those commands use them. In some ways, that’s the most sane solution, but the Finder does like us to see available space.
  Howard.
  
  LikeLike
  - 7
    
    Bob on April 30, 2023 at 12:20 pm
    
    As an “old school” Unix person, I’m used to using df when checking free space. It seems the machinations of MacOS does affect the accuracy of free space reported by df. I’ve delete entire VMs (via rm) yet df shows no new free space. It’s incredibly frustrating, especially when trying to free space for other needs, and I don’t understand why directly unlinking files doesn’t make their space available. (And I don’t think they’re held open by another process.)
    
    LikeLiked by 1 person
    - 8
      
      hoakley on April 30, 2023 at 12:27 pm
      
      Is that on a volume being backed up using Time Machine or third-party software that makes snapshots as part of its backups?
      Howard.
      
      LikeLike
  - 9
    
    Bob on April 30, 2023 at 12:47 pm
    
    “Is that on a volume being backed up using Time Machine or third-party software that makes snapshots as part of its backups?”
    
    Thanks for the response. For some reason I don’t have a reply option on your reply.
    It’s the internal drive on Time Machine. In the past when needing free space I’ve looked at local snapshots out of desperation. Similarly, when deleting a VM snapshot (VirtualBox) used space goes UP according to df. I don’t necessarily expect to get space back just because the difference file is being removed (in theory every block being copied back to the dynamically allocated .vdi could be a new block) but in the worst case it should be a zero-sum operation. What will happen is that space usage goes up, and mysteriously some time later df will report the savings. As the bard might say, things aren’t what they seem.
    
    LikeLiked by 1 person
    - 10
      
      hoakley on April 30, 2023 at 1:10 pm
      
      Ah. So the deleted VM is in a local snapshot, in which case its space isn’t freed until that snapshot is deleted automatically by Time Machine in 24 hours. I strongly recommend that you either keep your VMs (and other similar files) on a separate volume which isn’t backed up using Time Machine or any other method that makes local snapshots.
      There’s nothing wrong with df, APFS, or macOS: it’s doing exactly what snapshots are supposed to do, otherwise they wouldn’t work.
      Howard.
      
      LikeLike
  - 11
    
    Bob on April 30, 2023 at 1:18 pm
    
    “Ah. So the deleted VM is in a local snapshot…”
    
    Actually they shouldn’t be, unless local snapshots aren’t obeying the exclusions list in the TM config. That directory is excluded (As I recall the VirtualBox install is smart enough to do that for you) since the disk images are changing constantly, and TM would fill up really fast otherwise.
    
    LikeLiked by 1 person
    - 12
      
      hoakley on April 30, 2023 at 1:22 pm
      
      “local snapshots aren’t obeying the exclusions list in the TM config”
      As I have explained in several articles here, you can’t exclude folders from snapshots. Snapshots are whole-volume – the entire volume files system, as they have been since High Sierra.
      At least you know where that space is going now – into the snapshot. And it won’t reappear for 24 hours.
      That’s why I recommend a separate volume – when that’s excluded from TM, then no snapshots are made of it.
      Howard.
      
      LikeLike
  - 13
    
    Bob on April 30, 2023 at 1:32 pm
    
    “Snapshots are whole-volume – the entire volume files system, as they have been since High Sierra.”
    
    For whatever reason, the word “snapshot” in the phrase “local snapshots” didn’t register in my brain for what it is. Now that it has, your explanation makes perfect sense. I may (or may not) have read your articles on this topic in the past, but available free time doesn’t allow me to read all of your fine articles. Thanks very much for the clarification. I learned something.
    
    LikeLiked by 1 person
14

Tsering on April 27, 2023 at 6:13 pm

Great information, what happens when you execute a simple erase from recovery? Is that relatively secure method of destroying data?

LikeLiked by 1 person
- 15
  
  hoakley on April 27, 2023 at 6:54 pm
  
  Thank you.
  I’m not sure which type of Mac you’re referring to, nor how much you want to erase.
  If you want to securely erase the Data volume on internal storage of an Intel Mac with a T2, or an Apple silicon Mac, then use Erase All Content and Settings (EACAS), which throws the keys away for the encrypted volume. That effectively destroys its entire contents.
  If you want to destroy anything less than that, such as a single file, or you’re using a plain Intel Mac without a T2, then its gets more complicated, and the basic answer is that you can’t.
  Howard.
  
  LikeLike
  - 16
    
    Bob on April 30, 2023 at 1:13 pm
    
    Call me paranoid / pedantic, but when I’ve wanted to retire a system I don’t trust just removing keys and prefer to overwrite the storage. For an SSD a single pass overwrite would do. If I’m only concerned with the data, I’ll remove the data, delete local snapshots, and do:
    diskutil secureErase freespace 0 /
    
    Which will overwrite free space with a single pass of zeroes. If a HD, I’d use a value other than “0” as dictated in the man page for diskutil under the secureErase verb. Mac doesn’t make it easy to selectively secure erase a file. In older versions, it had an option in Finder to secure erase a file (overwrite the blocks before deallocation) but these days storage usage seems so obfuscated.
    
    LikeLiked by 1 person
    - 17
      
      hoakley on April 30, 2023 at 1:29 pm
      
      “I don’t trust just removing keys and prefer to overwrite the storage”
      It makes no difference overwriting it. Once the keys have gone, the data in that storage can’t be decrypted. If you think it can, then you’re wasting time and effort if your encryption can be broken.
      Apple explains how FileVault works using AES-XTS in its Platform Security Guide. Has anyone ever broken that? Not as far as I know. Do you?
      Overwriting storage was fine for unencrypted hard disks. All it does to a securely encrypted SSD is wear it out.
      Howard.
      
      LikeLike
18

Bob on April 30, 2023 at 2:09 pm

“Apple explains how FileVault works using AES-XTS in its Platform Security Guide. Has anyone ever broken that? Not as far as I know. Do you?”

No, I do not. But given the history of encryption, as a financial analyst might say, past performance is no indication of future results. Analogously, I haven’t needed the seat belts in my car, but I’m not about to stop using them.

As for wearing out SSDs, there is nuance in the issue. I would like to think that Apple uses SLC flash, although I’ve not looked into it. They certainly don’t use QLC like the garbage portable memory sticks, or it would wear out incredibly fast. Given the constant writing MacOS (or any modern OS) does to the storage and given the premium cost they place on the larger storage specs, and given that their hybrid drive configuration on iMacs uses only 128GB of flash for the most active storage, I am inclined to believe that they use SLC which would have about two orders of magnitude more writes over TLC / QLC flash. In short, I’m not concerned, although your point is taken. Working with VMs and snapshots is hard on storage.

LikeLiked by 1 person
- 19
  
  hoakley on April 30, 2023 at 3:05 pm
  
  Seat belt use is no analogue, I’m afraid. They provide protection against the contingency of events that do happen, but unusually. To the best of anyone’s knowledge, no one has ever cracked the encryption used in FileVault. Extensive analyses have been undertaken by many experts, and are widely accepted by cryptographers. Given the practical difficulties of trying to crack a Mac’s internal SSD, I think you’re banking on an event that’s so unlikely to happen that you can comfortably exclude it.
  BUT
  Have you read man diskutil? Including the entry for secureErase, where it says:
  “NOTE: This kind of secure erase is no longer considered safe. Modern devices have wear-leveling, block-sparing, and possibly-persistent cache hardware, which cannot be completely erased by these commands. The modern solution for quickly and securely erasing your data is encryption. Strongly-encrypted data can be instantly “erased” by destroying (or losing) the key (password), because this renders your data irretrievable in practical terms. Consider using APFS encryption (FileVault).”
  I also have bad news on the Fusion Drive front: Apple’s 128 GB SSDs in Fusion Drives have proved catastrophically short-lived for many users. It’s not uncommon for them to die within 2 years, and many are on their last legs at 3. Apple ceased offering them in Macs almost 3 years ago.
  Although Apple’s internal SSDs are generally accepted as being performant, I know of no evidence that they’re any longer-lived than any premium quality SSD, most of which are now TLC but may use SLC caching.
  Howard.
  
  LikeLike
  - 20
    
    Bob on April 30, 2023 at 4:36 pm
    
    Thanks for the detailed reply, Howard!
    Yes I did read that note in the man page a few years ago. The way the secure erase works is by writing to a file that eventually consumes all of the available space on the device. This can be observed by watching free space while it’s running. I’m not sure how load leveling comes into play under that circumstance. I practice “security in layers.” I certainly encrypt my drives, but I do this as an added layer.
    
    About FileVault, I have no doubt that it’s cryptographically secure in that I’m not questioning the mathematical robustness of it. But my experience tells me that encryption methods get weaker over time (through faster hardware or better attack methods,) or fail quickly in the event of a heretofore undiscovered issue, mathematical or in implementation. In the past three years, we have retired at least as many ssh ciphers because they are no longer considered secure. So I tend not to rely solely on encryption when retiring storage.
    
    Interesting comment about the Fusion drive. I have one of the originals, still no trouble. Use cases are different though, so there’s that. I never liked the Fusion setup all that much because I felt it was an unnecessary complication. Now I wonder if there’s a way to reconfigure to bypass the SSD and just use the HD.
    
    As a final note, when it comes to HDs, I usually “erase” them by opening them up, and using a pair of pliers, bend the platter(s) in several locations, rendering recovery impossible. It will never spin again, and the surface has been stretched rendering the recording ruined. The idea that the platters could be flattened and balanced to necessary tolerances and the recording medium back the way it was is stuff of science fiction. I started this practice at work because we discovered our (NIST-rated) degausser could still leave drives somewhat readable.
    
    LikeLiked by 1 person
    - 21
      
      hoakley on April 30, 2023 at 4:58 pm
      
      “So I tend not to rely solely on encryption when retiring storage”
      I’m sorry – I didn’t realise that you were talking about retiring storage. That’s not what this is intended for, it’s when you’re passing your Mac on to another user, as I did just recently with my M1 Mac mini.
      When you’re retiring storage, you should still use EACAS of course, as that does a great deal more than just throw away the encryption keys. But before sending that Mac for recycling, you should physically destroy its internal storage. That’s not hard to do with SSDs, and usually brings great satisfaction.
      For critical applications, the SSD should be removed (as roughly as possible) and mechanically shredded.
      But there’s certainly no point in wasting time performing any type of erase first!
      Yes, you can split a Fusion Drive relatively easily, but by the time you do that the SSD and hard disk are usually rapidly approaching death and only fit for recycling. I fear the enthusiasm for them was primarily driven by Marketing, who saw how much cheaper they were compared to the same capacity of SSD. I think Apple’s engineers were cringeing for them long before they were discontinued.
      Howard.
      
      LikeLike
22

merdan87f029f0b2 on May 8, 2023 at 4:03 pm

A casual observation I made after accidentally renaming a few gigs of files to have a ._ at the beginning of the filename. On the command line things look right but in finder the files are not counted in get info disk space usage or displayed. It’s something to look into. I tested with with apfs and ExFAT filesystems.

LikeLiked by 1 person
- 23
  
  hoakley on May 8, 2023 at 4:53 pm
  
  I suspect that they’re considered to be extended attributes, the normal use of files with that prefix, particularly on PC file systems like ExFAT. You may find their size counted in that of the file system metadata!
  Howard.
  
  LikeLike