You may have gathered that I greatly enjoy exploring macOS. It is an essential activity if you are going to be able to support yourself and others in diagnosing and addressing problems, activities which I also greatly enjoy. Perhaps they bring out some inner Sherlock Holmes persona.

Last week I returned to a topic of particular interest to me because it is almost entirely hidden from the user: extended attributes, or xattrs. We probably know most about the ‘quarantine flag’ attached by browsers and other software that download files from the internet, which forces any enclosed apps to go through full Gatekeeper checks. Even that ‘most’ is precious little.

As I have pointed out before, extended attributes had a glorious, and very public, past in the ‘resources’ used in classic releases of Mac OS. In early versions of Mac OS X it was touch and go whether they would survive at all, but somehow the dark forces who wanted a pure Unix didn’t rule the day, and xattrs have flourished like mushrooms growing in a cavern.

Until this week, I had no idea of their prevalence in Sierra and High Sierra. I had assumed that traditional ‘resource forks’, now extended attributes of type com.apple.ResourceFork, had largely fallen into disuse. But they are still being employed quite widely, and more than half of all the files in my Home folder have at least one extended attribute attached to them.

They are an important part of macOS, beyond their use for Gatekeeper’s checks. The xattr com.apple.rootless is attached to hundreds of system folders and files to ensure that they are protected by System Integrity Protection (SIP), and cannot be tampered with by rogue apps (or users!) or malware. More specifically, that xattr type is used to protect almost all the kernel extensions in your Mac’s /System/Library/Extensions folder.

They are also used extensively by apps. One easy way for an app to tell how text files are encoded – whether they use antiquated Mac or Windows schemes, or Unicode UTF-8, UTF-16 or UTF-32 – is to check their com.apple.TextEncoding xattr, which contains that information.

They can be used to transfer metadata about a file, such as keywords, or a copyright marking. As xattrs are remarkably sticky, and essentially invisible to ordinary users, the information that they contain is likely to remain attached to the file data even when users think that it has been removed. This can also apply to information such as the download link from which a file was obtained.

Overall, of the 1 TB of data stored on the Fusion Drive in my main working iMac, over 1.1 million of the 4.7 million files contain at least one xattr. Yet not one commonly-used app lets me even know of their existence, let alone about the information contained within them.

An even greater concern to me is that there appears to be no accessible resource which lists their types, or divulges what is contained within xattrs, or what they are used for. Many of the more than 150 different types of xattr that I have found have been established by Apple, which does provide a few small glimpses into some. Even the commonplace com.apple.quarantine and com.apple.TextEncoding appear to be completely undocumented.

Having built myself a couple of tools to look at xattrs, their occurrence, contents, and use, I intend to compile some information on the xattrs that I come across. As I have pointed out, they may not cause many problems in macOS, but they are both very useful and extensively used. One of the justifications for such exploration is that you never know what you might find until you start looking.

In the next few days, I am going to create a new Category here named xattr, and will start populating it with pages about the xattrs that I encounter, and as much information as I can gather about them.

I know that many third-party developers use xattrs, and would value your input (as far as you can) to correct and augment this information. Maybe over time we can document a reasonable proportion of the more commonly-encountered types of xattr which seem to be in current use. I’m sure that Sherlock Holmes would have wanted to get engaged.