I owe everyone who has downloaded any of my apps before yesterday an apology: I had messed up the signing of them, which forced you to Open them in the Finder in order to get them past Gatekeeper’s quarantine checks. Let me explain, as it is an object lesson in development and security.
The basic idea behind code signing is simple. Each app (etc.) is signed with the developer’s certificate, issued only by Apple. That signature not only identifies the developer, but it also provides a guarantee that the different components within that app have not been tampered with.
When you download an app from the internet (rather than copy it from removable storage, for example), macOS attaches a quarantine flag to that app. The first time that you open (run) that app, a full check of that app is undertaken; if it is approved, then that app is allowed to run and no further full checks are undertaken. Instead, a quick check of the app is performed each time it is started, which essentially verifies its integrity, and that the certificate hasn’t been revoked by Apple. (That glosses over a lot of detail, but I hope the broad principles are still accurate.)
Prior to Xcode 9 (released last week), there were facilities in Xcode to manage various types of certificate, but they didn’t work properly, and there were significant gaps between the documentation and what actually happened. I found the information on Apple’s website was different again, and the whole matter was greatly confusing. However, Apple recommended that leaving Xcode to manage certificates and signing was by far the best way to address this, and I followed those instructions.
None of this was made any easier by the fact that, although I am one developer, Xcode told me that I was two different entities: a team and a person. Each had their own developer certificates, which I had obtained from Apple and installed in my keychain, as the Xcode feature claimed to do that didn’t work.
A glance at my current keychain now shows a whole bunch of different certificates, with three different IDs, each of which is recognisably mine. Until last week, I had two types of certificate:
- Mac Developer, which I understood to be used for signing apps distributed independently of the App Store;
- Developer ID Installer, which I knew was necessary for signing Installer packages, as used to install my command tools
Xcode 8.x never seemed able to show me my certificates, or relate them to the two different entities which it thought me to be. Xcode 9 now does this properly, and one of the entities now shows four of the certificates which I currently hold.
Prior to Xcode 9, I let Xcode manage the signing of my apps, and used a certificate which appeared to work. I tested it in accordance with Apple’s instructions, using the commands
spctl -a -t exec -vv xattred.app
codesign --verify --deep --strict --verbose=2 xattred.app
However, neither of those forces the full check which results from quarantine, so in each case I uploaded my Zipped app to this blog, downloaded it, and checked that Gatekeeper responded with the correct check and dialog. This is quite time-consuming when updating more than one app. For the Installer packages, I similarly checked that they installed correctly here, and that the installed command tool worked.
I had sporadic reports that downloaded apps failed quarantine checks, but I was never able to reproduce them here. As they were infrequent, I assumed that there was an issue between that user’s Mac and Apple’s checking system, but could never reproduce that here, or even when I downloaded apps from here onto my Macs away from home.
This week, another report took me to look at signing and my certificates again. As a result, I abandoned allowing Xcode to automatically manage signing for me. I also reviewed the new documentation in Xcode 9, which draws a clear distinction between the nine different types of certificate which are in common use; Apple also has its own certificate types (as used by its system installers), and there are special certificates for KEXTs which aren’t listed here.
It now appears that the certificate which I had been using all this time, thanks to Xcode 8’s automatic management, was of the wrong type: for Mac Development, not Developer ID Application. Although I am still unable to get Xcode 9 to use the latter type of certificate using its automatic certificate management, I have created myself a shiny new Developer ID Application certificate and forced that manually.
My apps, as delivered from Downloads here, still pass all the tests that I can perform here. Thankfully, as I can place my own quarantine flags using xattred (free from Downloads), I no longer have to upload each app and download it, in order to test it against Gatekeeper’s full checks. But from your comments (thank you), it looks like this has finally fixed the problem: my apps were all correctly signed, but just using the wrong type of certificate.
The irony behind all this is that, whilst app signing is now so complex and prone to failure, correctly signed malware is now the rule, rather than the exception. It does call into question whether such an elaborate system is worth anything in terms of protecting our security.