Yesterday I revealed how common and rich in information are extended attributes, xattrs, on a desktop system with years of accumulated rubbish, in Sierra 10.12.6. Today I will compare and contrast that with a much leaner system running High Sierra 10.13.2, and look at the structure and function of some of the xattrs which you are likely to come across on both systems.
Extended attributes are still widespread in High Sierra
Because my High Sierra system is much leaner – it has only 86600 items in my Home folder, compared with over 1.7 million on my Sierra system – there are far fewer files with xattrs than on the Sierra system. But there are still over 47000, of which the great majority are in my Home folder. Of the files in that Home folder, 50% have at least one xattr, and the average number of xattrs per file with xattrs is 1.2.
Since my look at Sierra, I have improved my XattrXverser app, and extended coverage to the /private folder. In both Sierra and High Sierra, there is extensive use of xattrs in items in that folder, although from a limited range of types.
My app was able to examine fewer items in the /System/Library folder in High Sierra than in Sierra, I suspect because of more restrictive permissions. It does not obtain elevated privileges, so is locked out of more folders there in High Sierra.
There are still very many different xattr types
I have now found a total of 152 different xattr types in my Sierra Home folder; in my much leaner High Sierra Home folder, I found a total of 64 in a folder one fifth of the size of that in Sierra.
Neither Apple nor any of the other users of xattrs seems to be reducing the number of xattr types.
So which xattr types look important?
There are few xattr types which are so extensively used that they appear in almost every one of the top-level folders in High Sierra. Those which are even found in /private are:
- com.apple.quarantine – the Gatekeeper quarantine metadata, which I have explored in detail before. This is commonly encountered not only in /Applications, but attached to all manner of files in the Home folder.
- com.apple.metadata:com_apple_backup_excludeItem – this is used by Apple to exclude items from Time Machine backups.
- com.apple.TextEncoding – used sporadically to indicate the text encoding for a plain text file.
- com.apple.rootless – a marker of some folders and files which are protected by SIP.
- com.apple.uuiddb.boot-uuid – used on some log files.
- com.apple.logd.metadata – used on some log files.
I look at these in more detail below.
com.apple.ResourceFork, the traditional HFS+ resource fork, is still widely used even in High Sierra, and appears to work fine (as do xattrs) in APFS. I was surprised to find that 5% of the files in my Home folder still have resource forks stored as xattrs.
Apple’s Mail app has its own suite of xattrs, used to label messages and attachments. These include
- com.apple.metadata:com_apple_mail_dateReceived
- com.apple.metadata:com_apple_mail_dateSent
- com.apple.metadata:com_apple_mail_isRemoteAttachment
Several different xattrs can be used to attach copyright information, which would be opaque to the great majority of users. These include:
- com.apple.metadata:kMDItemCopyright
- org.openmetainfo:kMDItemCopyright
- org.openmetainfo.time:kMDItemCopyright
Another xattr which can give valuable information about the origins of a file is com.apple.metadata:kMDItemWhereFroms.
Finder Comments appear occasionally, using a xattr of type com.apple.metadata:kMDItemFinderComment.
Details of five important xattrs in common use with Sierra and High Sierra system files
com.apple.metadata:com_apple_backup_excludeItem seems to be used to mark files and folders which are to be excluded from Time Machine backups, in addition to the exclusions set in the Time Machine pane. They are typically composed of a property list, in hex form like
62706c69 73743030 5f101163 6f6d2e61 70706c65 2e626163 6b757064 08000000 00000001 01000000 00000000 01000000 00000000 00000000 00000000 1c
which starts with the Unicode characters bplist00_ com.apple.backupd
com.apple.TextEncoding is used quite generally by text files to indicate the encoding of their contents. This is normally given as a Unicode string, such as
utf-8;134217984
com.apple.rootless is used to mark some, but probably not all, items which are locked down by SIP. The contents appear very variable, and it can be empty. When it does contain data, this is Unicode text, such as
SystemPolicyConfiguration
com.apple.uuiddb.boot-uuid is attached to the log’s uuidtext files, and contains a UUID in null-terminated ASCII text.
com.apple.logd.metadata is attached to the log’s tracev3 log files, and contains binary data, such as (in hex)
01000000 19df4284 6c424836 89698cef 414e9d68 00000000 ff95a12e 05000000 2935f936 3f010000
I hope that has given you sufficient information to arouse your interest in xattrs and their uses.
My next addition to XattrXverser is going to be a facility to list all xattrs of a specific type, across files in a chosen folder, so that I can examine their usage in more detail.
