Code signatures: 1 How they work, and stop working

Almost all the time, code signatures just work. Whenever you open an app, there are many requests within macOS to check that app’s signature, but you’re never aware of that going on, it just happens. Then along comes a problem, such as the recent one with HP printer software, and signatures seem the worst things ever invented. In these articles, I’m going to explain what goes on with code signatures in Catalina and Big Sur, as a lot has changed in the last couple of years.

Why sign code at all?

Even Big Sur doesn’t require all executable code to be signed, although Apple Silicon Macs are different as they do. When executable code is signed, it brings two main benefits to the user:

  • The signature traces the authenticity of that code back to the named developer who signed it.
  • With the signature is a set of hashes (cdhashes) which verify the code hasn’t been changed.

Of course, both of those can be abused by malicious actors. Their purpose isn’t to eliminate malware or the subversion of code, but to make it much harder. Malware authors do obtain signing certificates and may sign their products too, but those signing certificates are readily identifiable and revoked quickly, making it impossible to run that malware.

Although code signing is required for notarization, they are quite separate processes, with distinct benefits. To avoid confusion, I’ll consider notarization in a separate article.

Certificate of authenticity

Although there’s nothing to stop anyone using a security certificate from elsewhere, for macOS there’s only one source of the certificates required to sign code for Apple’s operating systems, Apple PKI. This is the team within Apple which issues signing (and other) certificates to Apple itself and its very many third-party developers. Not only do they issue certificates, but they can also revoke them, and have detailed and explicit procedures for doing both.

Code signing certificates are configured for a single purpose, and differ between Apple’s operating systems. Developers normally manage them in Xcode, Apple’s free SDK which is the common development platform for all Apple’s hardware. They can also be managed on Apple’s secure developer website. Developers obtain new certificates from either and, should they need, request Apple to revoke existing ones, normally through the website (although Apple PKI procedures refer to this being performed in Xcode as well).

These certificates are usually issued with a validity of five years, up to a maximum of 15. At the end of that period, they expire automatically and the developer has to use a newer certificate which is still valid. That doesn’t mean that all their existing software stops working: an expired certificate isn’t normally revoked. So long as the certificate was valid at the time the signature was applied to software, it remains valid (in the sense that what it has signed remains acceptable) unless revoked.

As with all security certificates, the developer’s certificate goes back along a chain of trust through intermediate certificates to Apple’s root certificate as the Certificate Authority (CA). At the completion of any certificate check, macOS will know which developer signed that code and when, that the chain of trust remains intact, and goes right back to Apple’s root certificate.

Revocation

The normal life cycle of an app or other code distributed to the public is that the developer will sign it with a certificate which is valid at the time, and users will continue being able to open that code until they remove that software, which could be many years later. The original certificate will remain good, even though it has long expired. Two events can change that.

If it’s discovered that someone else may have had access to that certificate, or has been abusing it by using it to sign malware, then urgent action is required to render that certificate invalid: revocation. Once that revocation has been propagated out to Macs, when signature checks are performed, macOS will then immediately block that software from being opened, as happened with the HP printer software. Revocation as a result of compromise can be initiated by the developer, or by a third-party providing information to Apple PKI.

By far the most common reason given for the revocation of Apple developer certificates isn’t compromise, though, it’s because that certificate has been superseded. Over the last 13 months or so, Apple PKI has revoked more than 5.1 million certificates, of which 12% were revoked because of compromise, and 88% because they had been superseded.

For small developers managing just a handful of signing certificates, this may seem odd. But there are many good uses which are sufficiently temporary that the certificate owner should have that certificate revoked once that requirement has ceased. This might apply to beta-test versions of products for iOS, and for corporate in-house use. In those circumstances, best practice isn’t to leave the certificate lying around unused, but to get it revoked to ensure that software can’t be run again and the certificate is safe from abuse. Also included within the same scope are types of certificate which are only intended to have a short lifetime.

Apple PKI is clearly an extremely busy team, issuing anything from 32,929 to 1,550,665 revocations each month for this sub-CA alone. As a CA, it’s required to publish a periodic list of all revocations made, and it updates that every couple of weeks. But that isn’t how Macs get to hear of these certificate revocations, of which only a small proportion are likely to apply to macOS executable code. Instead, Apple propagates this separately.

We used to think that signature revocations were stored in the ‘Gatekeeper’ databases in /private/var/db/gkopaque.bundle on each Mac, and updated silently. However, Apple doesn’t appear to have used that system for well over a year now. At least on Macs running Mojave and later, notifications of certificate revocations are obtained by the background process com.apple.security.syspolicy.check.revocation which is normally set to run every three days. It also appears able to pick up changes more frequently, but isn’t something the user can control.

When a certificate has been revoked and that Mac is aware of it, any attempt to check that certificate will return the error -2147409652 CSSMERR_TP_CERT_REVOKED and the Mac’s security system will block any attempt to run that code. If this is an app with a GUI which is affected, you’ll normally see a standard alert informing you of this. Catalina is more brutal, and may simply crash the app immediately, and in the crash report you’ll see that it occurred because of a code signing error.

Problems come when the error isn’t in a GUI app but in other executable code such as a command tool buried somewhere. If that code is crashed or blocked, it’s up to the calling software to pass the error up, and it may not do so in a way that gives you any clue as to what is going on. All you might see is a complaint from the app that some component is missing, or isn’t working properly, and not the underlying reason.

Integrity check

Although the signing certificate has been in the limelight recently, just as important are the contents of the CodeResources file in the _CodeSignature folder within each signed app. These contain hashes for the protected parts of the app’s files and folders, known as cdhashes, and enable the macOS security system to verify the integrity of those parts of the app which are deemed to be executable.

Without these, the signing certificate can’t give any assurance that what you now have bears any resemblance to the bundle which the developer signed. Malicious software could easily have replaced all the app’s executable code with its own – which is one of the biggest dangers of running unsigned apps: you simply have no idea what’s in them.

Cdhashes don’t protect everything inside an app, and you’ll find that you can change some of the contents of its Resources folder, such as Help books, without necessarily breaking the signature. These are referred to as unsealed contents, and the cdhashes as seals, much in the way that the Big Sur Sealed System Volume works.

Unlike signing certificates, which need to be checked against lists of revoked certificates and external information, checking cdhashes is usually very quick, and is performed in even briefer forms of signature check. Only with very large and complex apps such as Xcode, or those with huge resources, is this likely to be noticed by the user.

How secure?

Code signatures have potential points of weakness, but have proved an effective means of reducing the number of successful attacks over the years, even though macOS can still run unsigned code. They can never eliminate malware: even if signing were to become mandatory (as it is on Apple Silicon Systems, although a developer certificate isn’t required), there would still be malware which played along with those rules. The burden on Apple is therefore the early detection of signed malware, and swift revocation of any certificates being abused.

Code signing isn’t a complete solution to malware, and can never be. But it raises the bar to attackers, and gives Apple an effective means of dealing with them. We may be smarting at the moment because of very infrequent instances where certificates have been revoked on innocent software. But we must also bear in mind the benefits that we have all enjoyed, and remember those 60,000 revocations that Apple PKI make in a typical month. That’s 2,000 a day, seven days each week.