One of the surprise announcements at WWDC 2019 last June, which seems to have been largely forgotten, is that Catalina not only requires apps and similar executable code to be notarized, but that single-file command tools must also be notarized. Given that there’s currently no way of stapling a notarization ticket to a command tool, I suspect many have just filed this under Too Difficult. But for anyone who installs third-party command tools, this could pose an insurmountable problem. This article explores how Catalina’s Gatekeeper system handles command tools, and what you can do about it.

Demonstration

If you have a Mac running Catalina and a third-party command tool to hand, here’s a little worked example of what happens. Add a quarantine flag to a copy of that command tool (easily done using my free utility xattred), and install it – either at the command line or using Finder – in /usr/local/bin. Then run it from Terminal. If the command tool isn’t notarized, you should see it being killed at launch, with a Gatekeeper failure dialog.

If you then open the Security & Privacy pane, you should see a button has been added at the foot of the General tab, offering to allow the running of the command tool “anyway”.

If you click on that and run the command again, you should see a different dialog, which allows you to bypass the check, and run that tool using the Open button. If you don’t follow that sequence with the Security & Privacy pane and the dialog, the command tool will remain blocked and won’t be run then, or on any further attempt.

This bypass procedure will eventually clear the quarantine flag after initially adding the command tool to the security exceptions list. Note that the procedure isn’t referred to in the first dialog, nor at the command line. Apple does document this procedure for apps, but doesn’t yet appear to mention that it’s the only way to open command tools in this situation.

If the tool has been properly notarized, even though the ticket isn’t stapled to it, Gatekeeper should validate it, and it should run correctly without any intervention.

So, in Catalina, if you install command tools which aren’t notarized, but have a quarantine flag set, macOS won’t allow you to run them without going through that sequence. One important circumstance in which you can easily encounter this is when you copy a command tool between two Macs using AirDrop: this attaches a quarantine flag, which will trigger first run checks on the destination Mac.

Repeat this demonstration in macOS 10.14.6, and you’ll have no problems running a command tool which hasn’t been notarized, despite having its quarantine flag set.

Packaging effects

Thankfully, command tools supplied in Installer packages don’t have the same problem. If the package isn’t notarized, you can’t start installation by double-clicking the package, as Gatekeeper won’t like that either, but you can work around this using the Finder’s Open command instead, which offers a Gatekeeper bypass. Even though the package has a quarantine flag, the tools which it installs aren’t put into quarantine, so when they’re run for the first time, they aren’t subject to full first run checks.

Command tools delivered from the Internet in most other formats will, though, have quarantine flags attached. This is the case, for example, if they’re enclosed in a Zip archive which is downloaded from a remote site: the archive is put into quarantine, and when you unzip it, the flag is added to all tools extracted from the archive.

What happens on first run

Log entries for the crashing of a non-notarized command tool are short and brutal:
0.351432 com.apple.launchservices code-evaluation syspolicyd LaunchServices handle prompt response=2, op.ident=1B708B14-15E3-48A3-80E9-CFEABA44AB1A, info.ident=1F3CE2F1-0D41-4A59-98E6-C062DCBD2ED3, info={<private>}
0.351438 syspolicyd Code evaluation completed: 2
0.352256 Error syspolicyd Terminating process due to Gatekeeper rejection: 679, <private>
0.352310 kernel AppleSystemPolicy Security policy would not allow process: 679, /usr/local/bin/blowhole

Evaluation is complete within 0.000006 second, and termination performed 0.0008 second later.

When a command tool has been notarized properly, it undergoes full Gatekeeper and XProtect checks, as shown in the following short summary of log entries. This sequence starts with the all-important trust evaluation initiated by AMFI, following which Gatekeeper is engaged:
0.407553 amfid Security SecTrustEvaluateIfNecessary
0.414702 syspolicyd GK process assessment: <private> <-- (<private>, <private>)
0.414747 syspolicyd Gatekeeper assessment rooted at: <private>

TCC assessment starts early too:
0.415068 TCC TCCAccessRequest() IPC
following which there are many entries from TCC as it performs its evaluation.

The Gatekeeper scan progresses:
0.422884 syspolicyd queueing up scan for code: <private>
0.422934 syspolicyd starting work for scan for code: <private>
0.427554 syspolicyd GK performScan: <private>, 1, 0

The tool’s notarization ticket is then sought, something which is apparently performed in iCloud as well:
0.427955 syspolicyd looking up ticket: <private>, 2, 1
0.427983 syspolicyd cloudkit record fetch: <private>, <private>
0.428019 syspolicyd cloudkit request cache info: <private>, max-age=300
0.428070 syspolicyd CFNetwork Task <531C6A30-927F-4120-8114-F85F15448965>.<9> resuming, QOS(0x19) Voucher <private>

An XProtect scan is initiated:
0.428432 XprotectService Xprotect is performing a direct malware and dylib scan: <private>

Remote checks start with
0.428575 syspolicyd libnetwork.dylib nw_connection_create_with_id [C5] create connection to Hostname#f39af287:443
following which there are many network-related log entries.

In this case, with a small command tool, the XProtect result returns early:
0.437688 syspolicyd GK Xprotect results: <private>, <private>

The notarization ticket is checked again:
0.690428 syspolicyd looking up ticket: <private>, 2, 0
0.690677 syspolicyd completing lookup: <private>, 0

Results return, indicating success:
0.690730 syspolicyd GK scan complete: <private>, 4, 0
0.690943 syspolicyd scan finished, waking up any waiters: <private>
0.690985 syspolicyd App skips first launch prompt because responsibility: <private>, <private>
0.690990 syspolicyd GK evaluateScanResult: 1, <private>, 1, 0, 1, 0, 4, 0

The final decision by Gatekeeper is to allow the tool to run, without showing any user prompt:
0.690991 syspolicyd GK eval - was allowed: 1, show prompt: 0
0.690992 syspolicyd Updating flags: <private>, 512

Finally, almost 0.3 second after trying to start the tool, it fires into action:
0.698677 co.eclecticlight.blowhole general blowhole blowhole Blowhole snorted!

Conclusions

If you install and use command tools downloaded from the Internet:

• Whenever possible, download pre-built command tools in notarized Installer packages (or, when appropriate, disk images), which will be easiest and most secure to install.
• As an alternative, download, check and build them from source, if you have sufficient information and knowledge to know that you aren’t building tools which could prove malicious.
• Installing pre-built command tools any other way will involve bypassing Gatekeeper (but not XProtect) checks, therefore will put your Mac at risk. Look for additional checks or evidence that what you’re going to install isn’t malicious.
• Your options then include a non-notarized Installer package with Gatekeeper bypass, or removing quarantine flags to avoid Gatekeeper first run checks altogether. Both carry significant security risks.
• Once a command tool has successfully completed its first run, notarization requirements aren’t (currently) checked again.
• Don’t use AirDrop to copy command tools between Macs, but use File Sharing instead.

If you offer command tools which Catalina users can download from the Internet:

• Prefer, when possible, to supply them in notarized Installer packages (or disk images, when appropriate), and/or in source code with checksums/signatures which the user can verify.
• If you can only supply pre-built tools which aren’t notarized, provide users with checksums/signatures for verification, as they will have to bypass Gatekeeper first run checks in order to use those tools.
• Warn Mac users of any problems they need to address to use your tools in Catalina.

For all Catalina users:

• New first run checks and notarization requirements for command tools significantly increase your Mac’s security protection.
• However, they only apply to command tools and installers with a quarantine flag set. If malicious software avoids the quarantine flag being set, or removes it, then most of these new requirements fall away. The only new check then imposed is an XProtect scan; if malware isn’t detected by XProtect, then Catalina won’t prevent it from being run.
• AirDrop sets quarantine flags: I will explore this later this week.

I am extremely grateful to Bryan Christianson for discovering this workaround described above, and for patience in getting me to understand it. I’m also grateful to @rosyna for reminding me that disk images are also an appropriately notarizable means of delivery.
Updated 0640 UTC 23 October 2019, adding link to Apple article on apps.