Imagine one day you sit in front of your Mac and open one of your apps. Maybe it’s Charlie Monroe’s time tracking app Eon, or you need to do some invoicing with his UctoX. Instead of seeing the app open, macOS out of the blue tells you that the app which you’ve been using and trusting “will damage your computer”, and should be moved to the Trash. What do you do next?
Exactly that happened to many thousands of Mac users last week. Among the most shocked was the developer of those and other affected apps, Charlie Monroe. On 4 August 2020, according to Apple, one of its automated security systems decided that one of Charlie’s developer distribution certificates, which allows all those apps to run on Macs around the world, was bad and revoked it without even informing the developer.
For Charlie Monroe, this was the start of a nightmare, worse perhaps than those around him being struck down overnight by Covid-19. [Please read my Postscript below.] His software business had vanished with the revocation of his certificate. He couldn’t even log into his developer account with Apple, which had also been taken away from him.
It was also a nightmare for many thousands who rely on Charlie’s software: note these aren’t entertainment or trivial apps, but tools used by many professionals who need to prepare invoices, track their time so they can bill it to customers, and more. Apple wasn’t even telling all those thousands of users that there was a temporary problem, but that Charlie’s software “will damage your computer”.
You’d have thought that, for such a draconian system, Apple might have a phone number staffed 24/7, but Charlie had to wait until Apple’s convenience in opening for the day. It took almost 24 hours before Apple’s error was rectified, and his certificate and developer account reinstated, as you can read in his account.
From Apple’s explanation, this all occurred because an automated security checking system made an error, and automatically revoked his developer’s certificate and account, without any human checking whether this was the correct action. So it could happen again to someone else, maybe me, and you could suddenly find all my utilities “will damage your computer”. Or it could happen to far more important apps, with which you prepare accounts or taxes, or earn your living.
What Apple did to Charlie Monroe on 4 August is inexcusable. We all make errors, but allowing any automated process to make this type of decision is guaranteed to cause serious harm to everyone involved, particularly to large numbers of Mac users.
There’s also the curious question as to why Apple revoked the certificate, rather than pulled one or more of Charlie’s notarizations. When it introduced notarization, one of Apple’s justifications was that it would provide finer control, rather than the huge and heavy-handed kill switch of revoking a certificate and blocking everything signed with that. Perhaps Apple didn’t really mean that after all, but just wanted another level of control over your Mac?
Apple has since apologised to Charlie Monroe for its error. It hasn’t released any statement to reassure other developers that it’s changing anything which might prevent such as catastrophe from happening again, nor has it explained to the billions who run third-party software on Apple products how it’s going to prevent a recurrence – which could readily prevent any Apple user from using their software on their computer or device.
With the enormous power of certificate revocation like this, Apple assumes two further responsibilities. The first is to exercise due diligence in wielding that power, and the second is transparency of process to both developers and users. For if we can’t have complete confidence that Apple won’t make such a grave error, then we have lost confidence in Apple as a supplier.
Would you expect anything less if you bought a car with a remote kill switch which its manufacturer could use to disable it completely? Wouldn’t you demand to know what processes were involved in hitting that kill switch, and what safeguards were in force to ensure that couldn’t happen in error or even maliciously? Then extend that kill switch so the manufacturer could disable not just one car at a time, but tens or even hundreds of thousands. Would you be happy for that kill switch to be controlled by an opaque and fully automated process?
Neither has Apple explained whether its automated systems could do this to more widely used products, which are used by millions. Or is this just the privilege of the small independent developer, who has less clout with Apple?
What sort of error was responsible? Is this something, like the appearance of a forged certificate purporting to be one of Charlie Monroe’s, which could be manipulated maliciously? Is there a vulnerability in Apple’s certificate revocation system?
Perhaps the most important question for all of us, though, is how Apple came to revoke the certificate of a developer whose apps have passed its malware checks when undergoing notarization. In revoking that certificate, Apple was surely admitting that malicious software signed by that developer had been detected, and that the Notary Service had proved inadequate to prevent its notarization and distribution. Doesn’t that undermine the whole justification for notarization?
Apple will no doubt try to ride this one out in silence, as it usually does in matters of security. For developers and users, that doesn’t answer these fundamental questions. Can we ever have confidence in Apple again?
A couple of you have objected to my comparison between what happened to Charlie Monroe and those around him being struck down by Covid-19, considering it “hyperbole” or “flip”. In my opinion, it is neither, but shockingly accurate, which is exactly how I intend it.
For many independent developers, having their signing certificate revoked without notice or reason is every bit as cataclysmic as the comparison I make. They have pursued a passion, coding long hours, putting everything they have and know into their products. That is also how they pay the bills, feed their family, and keep the roof over their head. When this type of disaster strikes an independent software developer, it can destroy them to the point where they commit suicide.
I hope that you can appreciate the seriousness of the situation that Charlie Monroe was in, and that is what I am trying to convey. For those who feel this is too near the bone, and for those who have lost loved ones and colleagues to Covid-19, I mean no offence, and apologise if you feel it inappropriate. But that is how I feel we should see this, not as a bit of a glitch which was soon fixed. In other cases, it could have ended very differently.