Last Week on My Mac: How not to release a macOS upgrade

Even if you chose not to upgrade to it, the release of High Sierra has dominated the past week. To read most of the reviews, you’d have thought the whole thing went like clockwork, and most who have upgraded are impressed and delighted. From where I’m sitting, it was a succession of bad decisions which have caused serious problems for many of those who have upgraded. In short, High Sierra is currently a lemon.

APFS incomplete

First, there was the promise of the new file system, fuelled by further hype at WWDC in June. I have already pointed out why linking the release of APFS into an upgrade driven by the iPhone marketing calendar was such a bad idea, and events have borne this out.

Yet looking through the promised new features in High Sierra, APFS was the only one which could have justified its status as a major release of macOS. There are plenty of internal changes, and new technologies which should in the coming months take Macs to new places. But, aside from APFS, High Sierra currently looks and works very much like Sierra 10.12.6. If you don’t use Photos much, and don’t use the related new features of iOS 11, it can be quite hard to tell the difference.

Apple then gave its engineers just three months over which to complete beta testing, first with developers, then with those brave Mac users who like to shed their blood at that leading edge. That period coincided with the major summer holiday period for the majority of the population in the northern hemisphere. Apple may have been able to stop its engineers going on vacation over that crucial time, but I suspect most of its beta testers still had commitments which reduced the amount of testing, and its thoroughness.

By the time that people were returning to work in early September, the launch of Apple’s new iPhones was as fixed as Christmas, leaving just three weeks to address remaining problems from testing, and deliver the Golden Master for final checks. By this time, it had become clear that the APFS team’s aspiration of robust and reliable support for Fusion Drives and hard disks had been lost.

This posed Apple a serious problem. It now could not delay High Sierra’s release to allow the problems with APFS to be addressed.

So, as Apple warned us in the final weeks before release, High Sierra would only convert SSDs to use APFS. The huge number of other Macs would just have to get along with plain old – alright, ancient and really creaky – HFS+.

High Sierra installer

This left one remaining task which had to be right: the release installer.

It seems that, during beta testing, Apple had tried two different installer designs. The conventional version was the usual monolithic app, which came complete with everything required, apart perhaps from EFI firmware updates. The other version used a small downloader app, which then obtained the large payload from Apple’s servers, and orchestrated the upgrade. Some beta testers reported issues with this.

Instead of sticking to its tried and tested monolithic installer, when High Sierra was first released, it tried to use the other option, which was less than successful. Faced with many failed attempts to upgrade, Apple pulled the two-part version, and reverted to a monolithic installer app. As none of this was explained in the App Store, many of those trying to upgrade were confused, and this sudden change caused further problems.

For those who had got the first two-part installer to work, they now discovered that what they had been provided with didn’t work with Apple’s instructions for making a bootable installer, for example. Many – myself included – then had to delete their first huge downloads and start again, once Apple made the monolithic installer available.

Both installers behaved in undocumented ways which differed from those detailed in pre-release documentation. For example, when run from Sierra, the installer will only convert internal SSDs to APFS, not external ones. Not only did Apple not explain this, but there still seems no reason for that limitation. It becomes even more puzzling and annoying when you discover that, when booted in High Sierra, the installer is only too happy to install High Sierra on an external SSD formatted in APFS. The installer appears to behave irrationally.

Others stumbled into more serious undocumented behaviours. Encrypting a drive using FileVault 2 has always allowed you to shut your Mac down during the process; encryption then resumes when the Mac is next started up. Try that when High Sierra is encrypting an APFS volume, and that partially-encrypted volume becomes unreadable, and has to be formatted again.

As with HFS+, opting to use case-sensitive APFS seems to invite nightmares. One reader here has already reported that you cannot recover from a Time Machine backup to a case-sensitive APFS volume, unless the source volume was also formatted as case-sensitive.

My own time-waster, which drove me to re-format my Time Machine backup with five years of backups on it, was trying to create a dual-boot system. Each time that you restart back into Sierra, it has lost track of its previous Time Machine backups. At best that forces a time-consuming deep event scan, but more typically Time Machine makes a complete new backup, as if it were the first such backup it had ever made.

Others have tripped up when migrating from Sierra volumes, something which appears not to complete (in finite time) in many cases, and in others fails when the Mac falls asleep – which is another potential cause of failed installations without any migration.

So overall, the High Sierra installer mark 2 appears to be the most troublesome and bug-ridden macOS installer for many years, and the options available for install are undocumented. Still.

Pulling Sierra installer

Having messed up the release installer, Apple then behaved in an unprecedented and profoundly unsupportive way to those who had battled their way through the installation of High Sierra: their previous ‘purchase’ of Sierra was removed from their Purchased tab of the App Store app. Apple had burned their boats for them.

It was bad enough that Apple removes previous macOS / OS X releases from the App Store for new ‘purchasers’, something many of us complained about when Sierra was released. But now, once you have downloaded (and possibly installed?) High Sierra, you cannot, for example, download the Sierra installer and make yourself a bootable installer from it.

Let’s say that I have a Mac mini and a MacBook Pro. Because the mini has an internal 1 TB Fusion Drive, I am currently not interested in High Sierra for it. But I upgrade the MacBook Pro to High Sierra, as it has an internal SSD. Then my Fusion Drive fails, and I replace it myself. I cannot start it up in Recovery mode, as its internal drive is unformatted, and has no Recovery partition. I want to make a bootable installer for Sierra. Simple: download the Sierra installer from the App Store, and follow the instructions. Only Apple has now decided that I cannot. If that Mac mini has to work with third-party hardware or software which is incompatible with High Sierra, it is now worthless.

Apple has not explained why it has behaved in this manner, but it is only too easy to envisage it as a means of forcing users to upgrade, particularly those still running old versions of its Pro apps, which are not compatible with High Sierra, by Apple’s own admission.

The App Store can still provide the Sierra installer, but only if you have already downloaded it on that account, and have not installed a copy of High Sierra.

Is APFS SSD-only?

By this stage, a great many Sierra users are wondering why they would want to enter the hell that Apple has made of High Sierra, and whether APFS will ever support Fusion Drives, or conventional rotating hard disks. A sensible next step would be to reassure those potential upgraders that their hardware will be supported, and give them an idea as to how long that might be. That is a step which Apple has yet to take, as plain and open talk about these problems would be anathema.

I wonder how many of us who got excited by the prospect of a major release of macOS this autumn/fall are still as excited now it has arrived. Yet it could have been so different. It is fundamentally a good upgrade, and will bring great things in time, if it is allowed to.