No one outside Apple knew at the time, but six years ago, on 27 March 2017, it made one of its biggest corporate gambles by rolling out a completely new file system in iOS 10.3. Every upgraded device silently converted its storage to the first release of Apple File System (APFS), but Apple didn’t reveal that until over two months later at WWDC. Six months after iOS 10.3, Apple upped the stakes further when it did the same with its upgrade to macOS 10.13 High Sierra.
Although APFS has certainly had its moments in the following six years, those gambles have paid off, and proved key enablers in the success of Apple silicon Macs. Had there been no APFS, many of the fundamental technologies like Secure Boot and the Signed System Volume (SSV) would have been far tougher if not impossible to implement. Macs and Apple’s devices had been in dire need of a modern file system for years; while there was a time when it looked as if that could have been ZFS, in 2014 Apple decided to write its own file system from scratch, with Dominic Giampaolo as lead engineer.
Prehistory
The very first Macs shipped with MFS, designed for 400 KB floppy disks and unable to cope with storage larger than 20 MB. In 1985 that was replaced with the Hierarchical File System HFS, and that evolved into HFS+ in 1998, intended to address hard disk seek times, fragmentation, and limited storage space. In 2002, journalling was added to improve the reliability of HFS+.
Apple announced APFS at WWDC in 2016, when it was expected to be released in another 18 months if development and testing went smoothly. Those who had hoped for ZFS were, and remain, disappointed, but APFS has overcome most of its initial problems, is now one of the most widely used file systems, and has proved itself more than capable on anything from a watch to large enterprise networks of Macs.
History
macOS Sierra already had a pre-release version for those who wanted to preview it. That ended its life in 10.12.6 as version 0.3, or, expressed in subsequent numbering, 249.7.6. As we discovered when we upgraded to High Sierra, though, that was a far cry from what was to come, and never achieved any useful level of compatibility either.
After a promising period in beta-release, Apple discovered fundamental problems with its popular Fusion Drives. The first release version, 748.1.46, shipped with High Sierra 10.13, therefore suddenly dropped support for those, and went ahead and converted startup volumes only on SSDs and hard disks. Snapshots were still wobbly, and it soon became clear that APFS was never going to perform well on rotating disks.
High Sierra had a stormy early release history, marred by a series of security gaffes. Vulnerabilities were fixed in the Supplemental Update released less than two weeks after 10.13, leaving snapshots to be improved in 10.13.1 on 31 October. Many of us anticipated that the problems with Fusion Drives would be fixed quickly, but those weren’t addressed until the release of Mojave the following September. Another problem that marred the introduction of APFS to all platforms was the refusal during beta-testing to incorporate Unicode normalisation; this had to be resolved in release versions of macOS 10.13 and iOS 10, as explained here and here.
In September 2018, Apple released Mojave 10.14, with support for Fusion Drives at last. This was also the time that the first version of the Apple File System Reference became available. Although a long and detailed document, developers soon realised how incomplete it was, in spite of the great delay in its publication. At last third-party file system developers had some hard information to work with, and users began to assume that third-party disk maintenance and repair tools were imminent.
Catalina brought major changes to APFS, with the use of expanded volume roles to form System Volume Groups, and we’ve all come to live with separate firmlinked System and Data volumes ever since. macOS 10.15.5 fixed a serious bug preventing the transfer of very large amounts of data to RAID volumes. At that time, Apple released the current version of the Apple File System Reference, building expectations that third-party tools were just round the corner.
Further major changes came with Big Sur 11.0.1 when it was released in November 2020. For the first time, the System volume was signed, sealed, and booted as a snapshot. This was also the first release of macOS to support making Time Machine backups to APFS volumes, and to support Apple silicon Macs. Since then, Monterey and now Ventura have continued to take advantage of all the new features in APFS. Apple’s ongoing failure to inform even developers of changes still catches us out: apparently Monterey added a feature to trim UDRW disk images, only discovered earlier this year. Who knows what still remains to be discovered in APFS in Ventura?
Strengths and weaknesses
APFS has brought many valuable new features that are now used so extensively that most aren’t even noticed any more. All the best backup utilities include snapshots, and Time Machine is built around them. They also form the basis for the boot copy of macOS, and coupled with its hash-tree verification have made re-installing macOS largely irrelevant. While features such as sparse and clone files are normally well-hidden from the user and taken for granted, they save significant space on the storage of every Mac, and eke out that required for backups.
The downside to this increased complexity is that free space is dynamic and much of it can be temporary and conditional. Apple still has a way to go before users can have confidence in any of the estimates of free space currently available in macOS. This is worst when it’s most critical, such as when trying to update macOS.
What I’ve never seen quantified is the incidence of minor file system errors, and their impact on long-term disk health. As with many other users of HFS+, I was used to running First Aid in Disk Utility and third-party repair tools to fix minor errors, even after the introduction of journalling. Since migrating to APFS, errors have been exceptional, and I have stopped routine maintenance. That isn’t to say that APFS never fails, but the cases that I hear of are extremely rare and unrecoverable.
From its first announcement, APFS has been criticised for its lack of integrity checks on data storage, and lack of redundancy. While file system metadata in APFS do use checksums, there’s no option to protect data using them, nor does APFS provide any scope for a third-party to add integrity checks or redundancy as an extension.
But by far the biggest limitation to APFS is its proprietary secrecy. Although some of its basics have been reversed for third-party products, much remains closely guarded by Apple. This continues to prevent the development of third-party repair and recovery utilities. After six years with just its own Disk Utility and fsck, Apple remains obstinate in its refusal to allow others full access to APFS internals, so depriving users of alternatives. This remains the biggest weakness in APFS today.
Version numbering
APFS version numbers can be important, as they’re referred to in the file system and may be quoted in terms of which version created a given container or volume, and which last modified it.
APFS major version numbers change with major version of macOS:
- macOS 10.12 has APFS version 0.3 or 249.x.x
- 10.13 has 748.x.x
- 10.14 has 945.x.x
- 10.15 has 1412.x.x
- 11 has 1677.x.x
- 12 has 1933.x.x until 12.2, thereafter 1934.x.x.
- 13 has 2142.x.x.
Minor version numbers increment according to the minor version of macOS, and patch numbers wander without pattern:
- macOS 10.13 went through 748.1.46 (10.13) to 748.51.0 (10.13.4 and later)
- 10.14 went from 945.200.129 (10.14) to 945.275.9 (10.14.6 with SU 2020-005)
- 10.15 went from 1412.11.7 (10.15) to 1412.141.1 (10.15.6 and .7)
- 11 went from 1677.50.1 (11.0.1) to 1677.141.2 (11.6)
- 12 went from 1933.41.2 (12.0.1) to 1934.141.2 (12.5)
- 13 started on 2142.41.2, went to 2142.61.2 in 13.1, and 2142.81.1 in 13.2.
Changing version numbers thus aren’t any indication of the scope or magnitude of changes made to APFS. As Apple seldom provides any information on changes made to APFS, it’s anyone’s guess as to what is going on.
The Eclectic Light Company 