For the third time in succession, this year’s major update to macOS has brought fundamental changes which could so easily have resulted in catastrophe. Two years ago, it was the conversion of all boot SSDs to Apple’s new file system; last year it was a similar conversion of Fusion Drives and a new privacy system. This year, the magician-engineers at Apple have split our boot volumes into two, and once again seem to have pulled it off.

Whether or not you think the new read-only system volume in Catalina achieves anything in terms of security, it’s hardly a novel practice in Unix deployment. It’s a logical extension of system firmware and a boot ROM, extending protection beyond the code required to get the main operating system running at boot time. What our computers used to be able to squeeze into KB or MB is now GB to support modern creature comforts.

What is more than a little adventurous is the idea of rolling out such a major restructuring of the file system to millions of users in a single upgrade, and expecting it to just work. You can almost imagine this plan being proposed at a meeting within Apple, and wonder how many times the word insane must have been used.

The problem those engineers faced is that, in Mojave, there’s no clear division between system files which shouldn’t change except during updates, and those which might change repeatedly during normal use. Simply moving /System and hidden folders like /usr to a read-only volume would render a Mac unbootable. But pulling out all the writeable files from within those and relocating them would break so many apps and tools that would be just as disastrous.

What was needed was some way of knitting together parts of two volumes. Let me illustrate this with two examples.

Your main Applications folder contains several different classes of app. In this context, the main division is between those which are bundled with macOS, like Time Machine and Preview, and those installed separately, including Safari as well as all third-party apps. The first group should be on the protected volume, but the second can’t be. So how do you show a single Applications folder incorporating both?

Another case in point is the hidden top-level folder /usr. Most of its folders should be on the protected volume, but one in particular, /usr/local, is widely used to store command tools and other files installed by the user and third-parties. How can you retain that while protecting all the other contents of that folder?

Trying to knit those different bits together using standard methods of linking, like symbolic or hard links, was also going to cause problems. What was needed was a new type of link, which works in both directions, and can allow the Finder to add a little deception on top to make it look like two folders are in fact one. This new type of link, which is somewhere between the two standard forms, was dubbed a firmlink, and is one of the tricks engineered into Catalina to make its dual boot volumes behave like one.

Having arrived at a solution, the next problem was how to convert tens of millions of Macs to the new arrangement. The most obvious approach, to create a new volume and copy across all the user’s and other writeable files, would have been interminably slow for many systems, and the inverse of the optimum. Instead, the existing volume remains writeable and is renamed with – Data appended to its original name. All those files to be installed on the new system volume are removed, and that protected volume is created. Once macOS system files have been installed on the new volume, all that remains is for the two volumes to be knitted together using firmlinks.

In the end, this elegant conversion-installation takes no longer than any other major macOS upgrade.

There are other fundamental changes to the way that Catalina works which I’ll be examining more closely in the coming weeks and months, such as its handling of notarized software and the implementation of hardening. One thing that you certainly can’t complain about is that Catalina is too similar to Mojave.

If someone had told me five or six years ago that in the autumn/fall of 2019 macOS was to have undergone such change, with a radically new file system, read-only system volume, finely controlled privacy protection, snapshots, and more, my reaction might have been the same as those sat in those meetings in Apple – laughter, and words like insane. Between your moans about the new media apps which replace iTunes, spare a few moments to wonder in awe at this engineering achievement, and to thank Apple’s magicians who made it all happen.