How big is that folder: what happened to APFS Fast Directory Sizing?

It’s been another of those long-standing issues in macOS, and Mac OS before it: select a folder and Get Info. If you happened to choose a large high-level folder such as Applications, you might have some time to wait before your Mac can tell you its size. Surely there must be a better way?

APFS promised one: Fast Directory Sizing. At least that was in the original brief documentation which Apple teased us with.

fastdirsizing

The first paragraph there was full of promise, but by the third, doubt had set in. This seemed ideal for many folders, including ~/Documents as cited, and /Applications too, so the question is whether it has been implemented yet.

I set out to see if Mojave 10.14.3, now running the third release of APFS and maturing well, could return folder sizes more quickly. The answer, for major higher-level folders at least, seems to be no. This is made more complex by the fact that Finder seems to cache folder sizes. The first time that I used Get Info on /Applications, it took several minutes to generate its size. A few minutes later, the result was instant, and couldn’t have been thoroughly recalculated. And don’t be mislead by performance on a complete volume, which is quite a different question.

I turned then to the latest and much fuller APFS documentation, where there is no mention of Fast Directory Sizing at all. But there are some tantalising clues.

First, each directory information record has a data structure j_dir_stats_val_t, which contains a numerical value for total_size, which is described as:
“The total size, in bytes, of all the files stored in this directory and all of this directoryʼs descendants.”
That’s just what we want. But it doesn’t seem that is always up to date. In each inode, there are j_inode_flags which include INODE_MAINTAIN_DIR_STATS. When set for a directory, that inode tracks the size of all its children. And all those child subdirectories must also have this flag set.

(Thanks to Ken for pointing this out, via Michael Tsai.)

So for some folders at least, where this feature is enabled, it should be possible to obtain an immediate estimate of the folder’s total size, provided that they’re on an APFS volume.

The next part of the problem is discovering which folders this Fast Directory Sizing is active on. Here it all gets more difficult, because macOS doesn’t appear to provide high level languages such as Swift or Objective-C with ready access to the j_inode_flags which can tell whether it is enabled, nor does it provide any single call to return the size of a folder which might take advantage of this new feature of APFS.

A little time spent with Mojave’s Finder suggests that nothing has changed there either. If the Finder happens to have it cached, getting folder size is blazingly quick, if probably inaccurate; if it has to calculate it afresh, then you’re in for a long wait as it chunders through the entire folder’s contents doing its arithmetic.

It’s time for FileManager to get a function to return a folder’s total size (and total number of items) making best use of APFS Fast Directory Sizing when it’s available.

(Thanks also to Jeff Johnson @lapcatsoftware for instigating this.)