hoakley August 28, 2020 Macs, Technology

How notarization works

In Catalina and Big Sur, notarization is no longer a bonus: for some types of software like extensions and most plug-ins, it’s essential. The only two general exceptions are software supplied through Apple’s App Store, and Apple’s own software, including the whole of macOS itself. You can still run apps and command tools which haven’t been notarized, but if they’ve been downloaded from the Internet or moved to your Mac using AirDrop (which also sets a quarantine flag) it’s getting progressively more difficult to do so. In Big Sur, it’s no longer just a matter of opening the unnotarized app in the Finder.

Notarization

When a developer notarizes their software, they have to build it to comply with Apple’s rules, which include signing it fully and correctly, and ‘hardening’ the runtime. They can’t submit the app or command tool as it stands, though: it has to be packaged in a way that’s acceptable to the Notary Service. That includes disk images (in UDIF format), signed flat Installer packages (as explained here), and Zip archives (as used by Xcode to notarize apps). The notarization is then specific to the app or executable contained within that.

The Notary Service checks the submission for malware. If none is found, signatures are in order, and other requirements are met, Apple adds its cryptographic hash and other details to its notarization database, and issues a ‘ticket’, which the developer can download and ‘staple’ (attach) to the software.

Tickets can’t be stapled to single-file Mach-O executables, but they can be stapled to Installer packages containing them. They’re most commonly stapled to an app or bundle in the file named CodeResources inside the bundle. However, because its details are recorded in Apple’s database, the ticket doesn’t have to be present for the app to be recognised as having been successfully notarized.

These procedures are straightforward for simple apps built in Xcode. The more complex the app, with helpers and plug-ins particularly, the more difficult these become. This may involve separate notarization of components, assembly into the whole, and notarization of that. For smaller developers, this is very demanding: please show understanding if they encounter problems.

First run

Whenever you run an app, command tool, or other executable code, macOS looks to see if it has a quarantine flag set. If it finds one, Gatekeeper looks for a notarization ticket. If that’s stapled to the bundle, its validity is checked by sending a cryptographic hash to Apple’s servers. If there’s no ticket, and no record of one in your Mac’s local security database, Gatekeeper sends the cryptographic hash to Apple’s servers to see if they have a ticket on record. This checking has been documented in Catalina by Jeff Johnson, even for unsigned shell scripts.

If Gatekeeper finds a valid ticket for that hash, either locally or on Apple’s servers, and the signature is good, XProtect checks that the code contains no known malware, and you’re invited to proceed with running the app.

notarization2

If the app or code hasn’t been notarized, then the normal process is stopped, and you’re informed of the failure. You can then opt to bypass the notarization check if you wish.

gkdialog02

First run notarization checks can impose significant delays on opening apps, which some users have complained about. They might complete more quickly when the notarization ticket has been stapled to the app, but the main purpose of delivering a stapled ticket with software is to enable Gatekeeper to check the ticket when Internet access isn’t available. Any repeat checks may use cached results of previous checks recorded in the local security database, avoiding further delay.

Mystery notarization

Occasionally, you may come across old software which couldn’t have been notarized when it was released, but which now behaves as if it has been notarized: put it through app first run with a quarantine flag set, and you’ll see the same dialog as for a notarized app. This is because, since its original release, that software has been notarized, and possibly not even by its original developer, as I explained here.

Apple has operated schemes which encouraged developers to retrospectively notarize older versions of their products, and still encourages them to do so whenever possible. Because notarization can also be performed by third parties, as well as the developer, it’s quite possible that the developer isn’t even aware. There’s nothing sinister in this, however spooky it might seem at the time.

Finally, if you do encounter any problems with the notarization of software, please contact its developer, as they’re the only ones that can fix it. But appreciate that they may well have been struggling to solve their notarization problems for quite some time. Sometimes it isn’t straightforward at all.

8Comments

Add yours

1

Albert Godfrind on August 28, 2020 at 7:29 am

I’m using all sorts of tools on my mac that are not macOS “apps”. Meaning they do not come as macOS application packages – the sort with content, resources, etc. They are unix/linux open source tools for which builds are available via managers like Homebrew or MacPorts. Do those need to be notarized too ? They are generally just command-line tools.

I also run quite a lot of java software. Obviously, I expect the java JDKs to be fully notarized. But what about the java libraries I download ? Same for Python: I expect Python to be notarized (including Python 2.7). But what about the python libraries I use (like Pandas etc). Do those (java libs, python packages) require notarizing ? Same stands for R and all other development environments. Also some java-based applications come with shell script to make them easy to launch. Are those shell scripts also to be notarized ?

Do you know anything about those ?

LikeLiked by 1 person
- 2
  
  hoakley on August 28, 2020 at 8:03 am
  
  Thank you.
  In general, the answer is that much of what you’re using not only doesn’t need to be notarized, but it can’t be.
  Notarization applies to executable code, that’s pre-built Mach-O binaries which are loaded by the processor and run. That includes apps, certain executable bundles including extensions and binary plug-ins, and command tools. It excludes anything which isn’t an executable binary – like shell scripts, code for interpreted languages like Python and R.
  Notarization is only required when the quarantine flag is set, and then only at first run. If you’re using a package manager which doesn’t set the quarantine flag on downloads, then it doesn’t apply. If you build a tool locally from source, then it also doesn’t apply. There is one small catch here: if you copy across a built tool or app using AirDrop, then that sets the quarantine flag, and first run rules then apply. So you may wish to use file sharing or iCloud instead, as they don’t do that.
  I think you’ll find that this is well documented for each of the major package managers now, as there’s a lot of experience at working with and around it. If in doubt, read the docs.
  Howard.
  
  LikeLike
3

Albert Godfrind on August 28, 2020 at 10:04 am

The grey area seems to be around java libs. They are executable code: just not binary executables, and just as susceptible to spreading malware or be harmful as binary executables. I fear they fly under the radar: they do not get assigned a quarantine flag, and the JVM ignores the quarantine flag anyway.

Same is true for Python packages: python itself may be an interpreted environment, the packages are downloaded and installed essentially as libraries (including compiled byte code).

It seems to me like those (especially java) are candidates for the next generation of attack vectors after Apple locks binary executables. I’m running several mac applications – packaged as mac apps – but that are actually full java applications, with a shell script to start them up – but without a single macOS binary.

Albert

LikeLiked by 1 person
- 4
  
  hoakley on August 28, 2020 at 11:02 am
  
  Thank you.
  There’s no grey area: notarization isn’t designed to address the security of anything other than binary executable code which has been flagged as being in quarantine. You can install a byte-code interpreter and run malicious code on that. Its security is the responsibility of the interpreter and its runtime environment. There’s no way that macOS can enforce anything else, except by requiring the whole environment to run in a sandbox, perhaps, which then limits what that code can do.
  Where a developer packages interpreted code into an app containing a binary executable, the whole app will still need to be notarized. And there are hardening flags which make allowance for such mechanisms as JIT compilers too.
  As with shell scripts, macOS doesn’t relieve the user of the duty of being careful what they do. But notarization should help prevent malicious apps, command tools, etc., from being run when downloaded from the Internet.
  Howard.
  
  LikeLike
5

Milo on August 28, 2020 at 10:25 am

Could you explain how app bundels or command line tools behave differently when they don’t have a quarantine flag attached to them on the first run?

If I understand it correctly it’s possible to remove the quarantine flag in current macOS versions, or did that possibly change in Catalina or Big Sur?

Most notarized apps are also singned using the developer certificate and in some cases with Apple’s certificates. Since certificates have a valid until date, does that mean that software signed with such a certificate will stop working when those certificates expire?

Thank you for this summary. Your articles have helped me a lot to understand the security aspects of macOS that Apple tries to keep hidden so diligently from the average Mac user.

LikeLiked by 1 person
- 6
  
  hoakley on August 28, 2020 at 11:12 am
  
  Thank you.
  The user can always remove the quarantine flag if they wish. Without a flag, notarization checks don’t reject apps which haven’t been notarized, and they can run. In Catalina and later, every time you open an app or command tool, it still undergoes a signature check (although not as deep as with a quarantine flag), and it’s checked by XProtect for known malicious code.
  The rule on signature date is based not on when the app is run, but when it was signed. If the signature was valid when it was signed, and hasn’t been revoked, then it should be accepted. That’s different, though, for installers, I believe, as we experienced when Apple’s old signatures expired on all its macOS installers last year.
  Of course the new complication to all this is running ARM-native code on Apple Silicon Macs, which will require some sort of signature (even an ad-hoc one) or it won’t be run. But that doesn’t apply to Intel code being run in Rosetta 2, nor to Intel Macs.
  Howard.
  
  LikeLike
  - 7
    
    Milo on August 28, 2020 at 12:25 pm
    
    > The rule on signature date is based not on when the app is run, but when it was signed. If the signature was valid when it was signed, and hasn’t been revoked, then it should be accepted.
    
    That sounds reasonable. Although there seems to be at least one incident where a developers certificate was revoked by accident.
    
    > That’s different, though, for installers
    
    I remeber the problem with macos installers. I had redownload installers for my “vintage” macs just in case. It’s a strange policy to be honest. Hopefully Apple will provide access to those files for many more years.
    
    LikeLiked by 1 person
    - 8
      
      hoakley on August 28, 2020 at 12:32 pm
      
      Yes: Apple didn’t revoke that developer’s signature by accident, but in error. I hope it never ever makes a similar mistake.
      The certificate expiry date on Apple’s installers should never prove a problem, as Apple is the Certificate Authority so makes the rules itself. Quite why it didn’t take earlier action I don’t know, but again it was entirely of Apple’s own making.
      Howard.
      
      LikeLike

·Comments are closed.

Share this:

Related