It has been bugging me for a while that I know so little about extended attributes, xattrs. Even a simple question such as how common they are in macOS files seem impossible to answer. In my earlier article and short summary about them, I only found seven different types. Does anyone have any idea as to how many types are in current use?

I thought that the best way to tackle this was to write a little app to explore folders on my Macs, calculating how many files have xattrs, and what types there are. Originally, I thought the app might steadily crawl through the whole of my 1 TB of macOS, apps, documents, and accumulated junk, but for the moment that may be a little ambitious.

Yesterday, I got my app working, and obtained some of its first results. I thought that you might like to be equally surprised by these initial figures.

xattrs are used very widely by files in macOS

Roughly a quarter of all the files I have checked so far, that’s a total of 4.6 million to date, have xattrs. That’s a little over 1.1 million files with xattrs on my Mac.

Fewest files have xattrs in /System/Library, where only 313 of 322575 files that I was able to check have them, and only a handful have more than one xattr. That’s to be expected, as xattrs are most widely used to store small amounts of transient information about a file, mutable metadata, and all the files in that folder are locked down by SIP.

/Library, with its much higher proportion of writable files, has a higher proportion using xattrs, and many with two or more of them.

As you might expect, the highest proportion of files with xattrs is in ~/Documents, where nearly 60% of files have xattrs, and the average xattr count for those files which use them is 1.5.

There are very many types of xattr in use

I thought that I had done quite well to identify seven different types when I first looked at xattrs back in August. I hadn’t even scratched the surface! In ~/Documents alone, I found 120 different types of xattr. I haven’t measured their frequencies, but many appear to be one-offs used by a single vendor for a single purpose.

I don’t propose going through the huge list of different types here, and my original idea of publishing a listing with examples here seems not to be a particularly good way ahead.

There is also huge variation in the size and content of xattrs. A few are traditional resource forks, containing large slabs of binary data. Apple’s sysdiagnose output uses one xattr type SampleWeight which often contains just the single digit ‘1’. Common types like com.apple.FinderInfo are inevitably widespread. The type com.apple.cs.CodeSignature can contain complete security certificates, although I’m not sure when it is used instead of a separate code signature file.

Apple’s xattr types

One generic approach which Apple uses to name many of its xattr types is to make them qualified sub-types of its generic com.apple.metadata. So com.apple.metadata:kMDItemCopyright is the kMDItemCopyright metadata. Apple has documented some of these in its File Metadata Attributes Reference, but additionally uses many more which do not apparently conform to this scheme, and don’t appear to be documented anywhere.

Apple has further extended this scheme to allow unique types, such as com.apple.metadata:kMDLabel_7u5sdyunqij73ecjzmbzivu4zq, which is the kMDLabel subtype of com.apple.metadata, rendered unique by appending 7u5sdyunqij73ecjzmbzivu4zq. It’s not clear how these are intended to be used, or whether they are for internal use only.

xattrs are also used extensively by third-party developers. You may come across the type com.dropbox.attributes, for example, which contains binary data used for items in a Dropbox, I presume.

You may well encounter a rival metadata naming scheme in xattrs too: that of openmetainfo.org, which uses a similar sub-typing system to Apple. For example, org.openmetainfo:kMDItemKeywords contains keywords which might be attached to an image. Apple and openmetainfo.org have common metadata types, such as kMDItemKeywords and kMDItemStarRating, and some files attach both variants for good measure.

So what? They’re no trouble, are they?

Problems with xattrs do appear very rare, although sometimes they could be the cause of the metadata compiler daemon mdworker to crash repeatedly.

xattrs are, though, storing a lot of information, some of it very relevant in security and forensics situations. As users almost never directly modify xattrs themselves, information contained in xattrs is remarkably persistent, so long as the file in question is not cleaned of its xattrs by passage through a file system which cannot support them.

The metadata contained in xattrs is also of great significance to Spotlight and related searches. Many users complain of poor results from Spotlight searches, which might be attributed to patchy metadata coverage.

Finally, I have so far only looked at xattrs in Sierra, using HFS+. APFS is claimed to offer full support for xattrs, so my next step is to check that out in High Sierra.

The app which I have been using to look at xattrs on this larger scale is XattrXverser. I don’t currently have any plans to offer it in Downloads here, although if you are interested in exploring xattrs it could save you a lot of work in the command shell.