hoakley Macs, Technology, xattr

🎗 Quarantine: Documents

In the first of this pair of articles, I looked at how ‘quarantine flags’ work for apps and other executables. I pointed out that these are a small minority of the files on your Mac which bear quarantine flags, although that is the use with which we’re familiar. In this article, I look at quarantine and non-executable files such as documents.

macOS has been attaching quarantine flags, in the form of the com.apple.quarantine extended attribute, to documents for as long as it has been to apps, since macOS 10.5 in 2007, as part of the the same process. If a webpage or other file is downloaded from the Internet and saved on your Mac by an app which adds quarantine flags, then a normal quarantine xattr will be added to it. When you decompress a flagged Zip archive, quarantine flags are automatically attached to all the files extracted from it.

Where these non-app flags differ is in the following:

  • Opening a quarantined document or non-executable file doesn’t trigger a Gatekeeper check, which would in any case be meaningless.
  • Nothing appears to change or remove a flag, unless you use a utility or command to do so. The sole exception to this is with flags attached by sandboxed apps, which can replace one another.
  • macOS also adds its own quarantine flags to documents which haven’t been downloaded from the Internet. Oddly, while quarantine of apps is an opt-in behaviour, you can’t opt out of this behaviour, as it’s built into the macOS sandbox.
  • Performing certain operations with quarantined documents may be forbidden. For example, a flagged shell script can’t normally be executed. Thus using flagged files can result in errors.

A regular document quarantine xattr, attached following download from the Internet, has identical content to those attached to apps:
0083;5991b778;Safari.app;BC4DFC58-0D26-460D-9688-81D119298642
with the components:

  1. the quarantine value in hexadecimal,
  2. the time at which the xattr was attached, in hexadecimal,
  3. the app or agent responsible for creating the xattr (normally the downloading app too),
  4. a UUID referring to the entry for this quarantine flag in the QuarantineEvents database

separated by semi-colons.

Those attached to files which haven’t been downloaded differ, as they aren’t associated with entries in the QuarantineEvents database, so lack the UUID. Quarantine values you’re likely to encounter include:

  • 00000000 10000001 = 81
  • 00000000 10000010 = 82
  • 00000000 10000011 = 83

each of which has the high-order bit set to indicate that the file is still in quarantine. Those with UUIDs may still have extant entries in the QuarantineEvents database which may indicate their quarantine type which can include LSQuarantineTypeSandboxed, but those which lack any UUID are normally given with an LSQuarantineType of LSQuarantineTypeSandboxed.

The addition of quarantine flags to files which have never been downloaded from the Internet appears to be a relatively recent behaviour, but has now been seen to occur in macOS Sierra and later. Apple doesn’t appear to have documented this for users, and references to this behaviour are buried deep in Apple’s now outdated but not replaced Entitlement Key Reference.

Sandboxed apps, which includes many of those bundled with macOS and all delivered by the App Store, will attach quarantine flags to files which Apple describes as executable unless the app has the com.apple.security.files.user-selected.executable entitlement. Apple explains:
“By default, when writing executable files in sandboxed apps, the files are quarantined. Gatekeeper prevents quarantined executable files and other similar files (shell scripts, web archives, and so on) from opening or executing unless the user explicitly launches them from Finder.
If those executables are tools that are intended to run from the command line, such as shell scripts, this presents a problem.”

Note that this refers to a sandboxed app writing executable files, which should only account for a very small number of files having a quarantine flag attached. Jeff Johnson has also suggested that similar behaviour may result from security-scoped bookmarks, a complex feature of sandboxed apps which is discussed in the same outdated document. Those come in two forms, one which provides persistent access to a user-specified file or folder, the other provides persistent access to files needed by a specific document. However, there is no indication as to how a quarantine flag might play any role in security-scoped bookmarks.

The role and purpose of these quarantine flags added by sandboxed apps remains obscure, beyond being used to prevent the execution of shell scripts, web archives, etc. Not only that, but sandboxed apps write them, when permissions allow, to all documents which they open, even though the document may not be formally saved by that app.

One suggested purpose for these flags is in checking of documents to determine whether they might be malicious. Several document types, such as JPEG images and PDFs, can harbour malicious content. However:

  • Although XProtect has been seen in past logs apparently ‘checking’ a JPEG, its Yara data files don’t appear to contain any definitions for malicious document types, and no other tool in macOS appears to check documents for malicious content.
  • These quarantine flags remain set. If checking of any kind does take place, neither the quarantine value nor the QuarantineEvents database ever record the outcome of that check.
  • There is no apparent difference in the logs when opening a document which already has a flag set, compared with the same document without the flag, when using the same sandboxed app. In particular, there is no saving in time when opening a large PDF, which would be expected to take significant time for any checks.

For the moment, then, these many documents with quarantine flags remain a mystery.

As extended attributes, you can view and edit quarantine flags using standard tools including the xattr command and my free utility xattred. These flags are easier to work with than many xattrs as they are UTF-8 text, not encoded binary.

As a developer, you can opt to access them as xattrs, which is easiest using extensions to the URL class in Swift, for instance. They are also accessible much more directly as NSURL Resources, using code such as
var theQuarFlag: AnyObject? = nil
try theNSURL.getResourceValue(&theQuarFlag, forKey: kCFURLQuarantinePropertiesKey as URLResourceKey)
which, if theQuarFlag is non-nil, is a dictionary containing key-value pairs for all known fields in the quarantine flag. These not only include the data in the xattr, but where there’s an extant entry in the QuarantineEvents database, this also returns information from that. Keys are listed in LSQuarantine.h, and include:

  • LSQuarantineAgentNameKey, the name of the downloading agent;
  • LSQuarantineAgentBundleIdentifierKey, the bundle ID of the downloading agent;
  • LSQuarantineTimeStampKey, a CFDateRef to the date and time that the quarantine flag was attached;
  • LSQuarantineTypeKey, one of the values listed in the previous article;
  • LSQuarantineOriginURLKey, a CFURLRef to the original location of the file;
  • LSQuarantineDataURLKey, a CFURLRef to the data source of the file.

Removing a quarantine flag simply requires setting it to nil, as in
try theNSURL.setResourceValue(nil, forKey: kCFURLQuarantinePropertiesKey as URLResourceKey)