Is APFS doing its job? Big Sur edition

Later this month Apple File System (APFS) will be four years old. First rolled out in a bold and daring move on 27 March 2017 to unsuspecting iOS users, we’ve been running it on our Macs since upgrading to High Sierra in September 2017. This article considers where APFS is today in Big Sur, which has now reached version 1677.81.1.

HFS+, which APFS has all but replaced, was the successor to HFS, which in turn replaced the Mac’s original MFS in 1985. While MFS was designed for use with 400 KB floppy disks and couldn’t cope with storage larger than 20 MB, HFS+ was intended for hard disks larger than any currently available.

Should APFS volumes need checking?

Among the problems with hard disk storage which HFS+ tried to address were seek times, fragmentation, and limited space. As a result, in common with most file systems designed for hard disks, it defaults to overwriting data when changing it, notably in the file system metadata. This is efficient, but inherently prone to error. Should anything go wrong while changing file system data, such as a crash or even worse a kernel panic, that’s likely to leave the file system metadata with a mixture of the old and new, resulting in an error.

Apple addressed this by introducing journalling, in which the file system makes each change first in a record in its journal, then to the catalogue in the file system metadata. Prior to its introduction, it was common for most crashes to result in at least minor damage to file system metadata, and many of us routinely rebuilt its directories after each crash, or as periodic maintenance, to prevent those accumulating until serious errors resulted. The need for this reduced substantially with journalling, but it was a workaround rather than a full solution, and directory maintenance remained popular.

One of the major changes introduced in APFS is in the way that its file system metadata are changed. Because it’s primarily designed for use with SSDs, for which seek times and fragmentation aren’t a concern, what APFS does is to write changed metadata to a new location on disk, and only when that has been completed successfully is the original metadata released for re-use, a process known as copy-on-write. The disadvantage of doing this is when APFS is used on hard disks, which suffer serious fragmentation in their file system metadata. On SSDs, copy-on-write actually helps the essential process of wear levelling, to ensure that the number of erases is spread evenly across storage blocks.

This can’t of course guarantee that errors never occur in an APFS file system, but should eliminate this process of steady degradation and eliminate the need for such routine maintenance.

Does APFS work?

To assess its success, I have surveyed all the storage accessed by my iMac Pro, looking for errors. Currently, that consists of a total of 11 TB across 7 SSDs, of which around 5 TB contains files. These are used by three different backup systems: Time Machine backs up one large and active folder to APFS each hour, with additional hourly backups made by ChronoSync, and daily backups by Carbon Copy Cloner. There are more than 100 snapshots, made by both Time Machine and Carbon Copy Cloner, and those disks have been running Catalina then Big Sur over the past year. I don’t recall performing any previous file system checks in that year, and haven’t noticed any file system errors.

Running First Aid in Disk Utility, which in turn calls fsck_apfs, all disks could be checked apart from one 2 TB SSD, which is used by my Content Caching Server, so couldn’t be unmouted without shutting that service down. Despite not being checked for the last year, not one error was reported, and all volumes, containers, disks and snapshots were reported as being correct.

Does Disk Utility work?

For Big Sur, Apple advises users to check from the bottom up, starting with APFS volumes, then containers, and finally disks. In the current version of Disk Utility, that isn’t all necessary, as checking a container will normally result in each of its volumes also being checked, together with any snapshots.

diskutilfail

A common problem encountered when trying to check volumes or their containers is that Disk Utility returns an immediate error, reporting that item couldn’t be checked because the volume couldn’t be unmounted. Normally it was sufficient to repeat the request, which was then successful. This problem has plagued Disk Utility since the release of High Sierra, but has now become a minor irritant rather than the major obstacle it once was.

Does APFS require routine maintenance?

File systems do incur errors, and excellent tools for the checking and repair of APFS are among the most important utilities in macOS. Continuing to improve fsck_apfs and Disk Utility is unglamorous but vital for users, and there will always be scope for third-party alternatives, which Apple should encourage.

However, APFS does appear to have achieved its objective of eliminating the routine accumulation of file system errors to which HFS+ was prone. In doing so, it has also removed any need for periodic maintenance. The onus is on those who feel such maintenance utilities are necessary to provide objective evidence of any shortcomings in APFS which they feel such maintenance might address. From where I’m sitting, while there’s always room for improvement in features, APFS does its job excellently.