“I’d just finished writing a long and detailed document in Pages. Having saved the original, I exported it as a PDF for my colleagues to comment on.”

“Like most Mac users, Preview is my default app for opening PDFs. In this case, I wanted greater compatibility and features for annotations, so I opened the Finder’s Get Info dialog, and changed that document’s default app to PDF Expert.”

“When I double-clicked the PDF to open it, macOS refused, with a weird error alert.”

“I checked that on Apple’s support pages, where it tells me I should look for a later version of Pages or an alternative app. But I’m running the current version of Pages, which created the PDF, and of PDF Expert, which should have been able to open it. So what should I do next?”

Although that’s not a genuine reader’s letter to MacFormat or Mac|Life, if you step through that sequence in High Sierra or Mojave, you’ll end up with exactly the same problem. Neither is this an arcane way to use macOS. This is what I have been looking at over the last few days. My conclusion is that this isn’t a bug, but “expected behaviour”.

This occurs when three conditions are met:

• a document has its ‘quarantine flag’ set, by an extended attribute of type com.apple.quarantine;
• the user customises which app should be used to open that document, a setting stored in that document’s OpenWith extended attribute;
• the document is opened using the Finder, Dock, or similar, and not through the app’s Open command.

The macOS sub-system reponsible is XProtect, which, when those three conditions are met, refuses the request to open the document, returning an error code of -67062.

Changing this behaviour isn’t trivial, nor is it possible in the GUI using the tools supplied with macOS, which can neither visualise the quarantine or OpenWith extended attributes, nor remove them. For most Mac users, this means that they’re working blind when trying to deal with this.

The immediate workaround is to open the affected document from within the app, using its Open command, as that doesn’t trigger LaunchServices hence XProtect. A more lasting workaround is to remove the quarantine flag from the file, which isn’t easy unless you use a third-party tool (such as xattred or Sandstrip from here). However, as sandboxed apps insist on writing quarantine flags to every file that they open, even when the user doesn’t save the document using that app, this is only temporary.

This behaviour is becoming more widespread as a result of the increasing use of sandboxed apps in macOS, and their promiscuous behaviour in writing all these quarantine flags.

The purpose in all this seems to be preventing the spread and effect of malware contained in documents, something which doesn’t appear to have ever been common or significant in macOS.

As a security measure, it’s too blunt a tool. XProtect doesn’t discriminate between different types of file, and blocks plain text files exactly the same as PDF, although at least TextEdit doesn’t yet quarantine text files. It doesn’t make any attempt to check whether the document could contain malware, let alone scan for malicious content. It’s entirely unselective and indiscriminate. Indeed, the fact that it most prominently affects documents created by the protected apps supplied with macOS, and those from the App Store, shows that it targets those which should be the safest.

The three conditions required to result in XProtect blocking the opening of a document ensure that, on almost every occasion, it is a false positive, something the user is almost powerless to alter. Any security criteria with such a high false positive rate are at best in serious doubt. It effectively prevents users from taking advantage of a valuable feature of the macOS GUI, a behaviour more harmful to macOS and its users than can be compensated for by any gain in security.

Even if the risk were so high that this behaviour was worthwhile, its implementation is bodged:

• The error code -67062 returned is incorrect. If XProtect can’t return anything more appropriate, a new code should be created for this purpose.
• Because of that error code, the text in the alert displayed to the user makes no sense and only serves to confuse. A correct string should be shown from the appropriate error code.
• Apple’s only advice to users regarding that error alert is in an article about app quarantine. Apple should document this behaviour in an article explaining document quarantine to ordinary Mac users.
• Apple also needs to document this behaviour properly for developers of apps which run in a sandbox, so that they can support their products rather than having to guess what macOS is doing in secret.
• If Apple considers that this behaviour really is necessary for the security of macOS, it should remove the ability of users to change the default app for individual documents as being no longer reliable or secure, and make this clear in macOS release notes for users.

Meanwhile, baffled developers and users have been left to assume that this is yet another indication that macOS continues to fall into disrepair. It hasn’t: Apple has deliberately crippled the Finder without letting anyone know.