Skip to content

The Eclectic Light Company

Macs, painting, and more
Main navigation
  • Downloads
  • M1 & M2 Macs
  • Mac Problems
  • Mac articles
  • Art
  • Macs
  • Painting
hoakley December 11, 2017 Macs, Technology

An introduction to extended attributes, xattrs

Origin and Purposes

Extended attributes associate metadata with individual files and folders. In Classic MacOS, many files have resource forks to contain structured metadata: a classic app, for example, stores definitions of windows, menus, dialogs, etc., in its resource fork. In macOS, files and folders can also have forks, which are implemented as extended attributes: a resource fork becomes resource metadata in a xattr. Extended attributes are quite widely used in other file systems, in Linux and BSD, for example.

macOS and applications use xattrs for various purposes. The most prominent is implementing the quarantine flag which indicates that an app has been downloaded from the internet and requires full Gatekeeper checks. Other xattrs attach details of the website from which a file was downloaded, and they are used by macOS Server. They are not normally used to store metadata such as EXIF, and those associated with other media files, which are normally part of the file data.

🍌 xattrs are not hidden files with the extension .xattr, but are stored centrally on each volume.

Storage

Extended attributes are not stored in the main data for files, but in the Attributes area of the volume metadata. As such they are out of reach of normal file tools, and can only be accessed using tools specifically intended to work with xattrs. Each file and folder can have an effectively unlimited number of xattrs, each of which can be more than 100 KB.

⚠️ Because they are part of the volume metadata, some versions of macOS may not include the space occupied by them when calculating free and used disk space. It is possible to fill a volume with extended attributes and run it out of free space, and that space may also be ignored by services which manage storage use, for example space allocations for clients in a server system. These should be pathological problems, but could also result from malicious activity.

⚠️ Most file systems to which macOS can write either handle xattrs natively (HFS+, APFS), or macOS uses a scheme to preserve them. NFS is an important exception, and files copied to NFS will have all their xattrs stripped.

Types

xattrs are often named using a type signature, much like preference files. The following types are commonly encountered:

  • com.apple.FinderInfo is found very widely, and contains (a little) Finder information, sometimes including the old file type and creator strings.
  • com.apple.progress.fractionCompleted is sometimes associated with a user’s Home folder.
  • com.apple.ResourceFork is a resource fork.
  • com.apple.rootless marks items which are protected by SIP.
  • com.apple.serverdocs.markup appears to be left on some folders which have been handled by macOS Server.
  • com.apple.quarantine marks files which have been downloaded from the internet, and contains their Gatekeeper status, indicating whether they still require full checking, have passed a full check, and have been run on that Mac. This is detailed in the Gatekeeper topic.
  • com.apple.metadata is for metadata generally, and is usually qualified by a subtype, such as those below.

Metadata subtypes (termed attributes by Apple) commonly encountered include:

  • _kMDItemUserTags contains Finder tag information
  • kMDLabel_ is normally serialised with a string of apparently random letters
  • _kTimeMachineNewestSnapshot gives the latest Time Machine backup timestamp for a major folder such as your Home folder
  • _kTimeMachineOldestSnapshot gives the oldest Time Machine backup timestamp for a major folder such as your Home folder
  • kMDItemDownloadedDate gives the datestamp for when a downloaded item was obtained
  • kMDItemWhereFroms gives the URL from which a downloaded item was obtained
  • kMDItemIsScreenCapture, kMDItemScreenCaptureGlobalRect, and kMDItemScreenCaptureType for screenshots.

Examples of compound types include com.apple.metadata:kMDItemWhereFroms and com.apple.metadata:kMDLabel_jwzqfxqstyro4udfe2psoz7kyi, which has a serialised subtype.

⚠️ However, using a type signature is not a requirement. Other xattr types you may encounter include AppCrashCount and os_version.

Detection

Metadata stored in xattrs is usually hidden from the user, although Finder’s Get Info may show the content of some xattrs such as com.apple.metadata:kMDItemDownloadedDate and kMDItemWhereFroms. Some third-party apps may expose relevant metadata. For example, better text editors show and allow the setting of the information stored in com.apple.TextEncoding, which records the encoding scheme used by many text files.

The most reliable way of discovering whether one or more xattrs are associated with a file or folder is using the command ls -la. Resulting listings append the at sign @ at the end of the permissions flags to indicate that item has associated xattrs:
drwxr-xr-x@ 254 hoakley staff 8636 24 Jul 18:39 miscDocs

Tools

The standard tool for working with xattrs is the command xattr. xattrs are listed with
xattr -l itempath where itempath is the path to the file or folder to be examined.

Copying xattrs using this command is complex, and requires writing the xattr which is printed in hex form, e.g.
xattr -wx com.apple.FinderInfo "`xattr -px com.apple.FinderInfo thisitem`" thatitem
copies the com.apple.FinderInfo xattr from thisitem to thatitem.

xattred from Downloads above is a free GUI tool which displays and edits xattrs, and can be used to set the quarantine xattr for test purposes.

Common Problems

Problems with xattrs are uncommon, but can be very difficult to diagnose.

🍌 Some old commands, including cpio, zip, and pax, may omit xattrs unless they are specifically included using an option: if xattrs are to be preserved when using such commands, you should check with the man page and preferably perform a small-scale test before proceeding.

🍌 Occasionally, bugs in specific apps can attach incorrect or very large xattrs to files quite inappropriately. These can in turn cause some services to fail, most commonly the mdworker daemon responsible for indexing metadata for Spotlight. These usually present with bizarre symptoms, and until they are recognised as being caused by xattrs, they remain baffling.

References

One of the few lists of xattrs is given for com.apple.metadata subtypes in Apple’s File Metadata Attributes Reference.
There are many other articles on this blog, in the Category xattr, with its contents page (also listed to the right, at the top of the widgets there).

Share this:

  • Twitter
  • Facebook
  • Reddit
  • Pinterest
  • Email
  • Print

Like this:

Like Loading...

Related

Posted in Macs, Technology and tagged APFS, extended attributes, forks, HFS+, metadata, NFS, quarantine, resources, xattr. Bookmark the permalink.

8Comments

Add yours
  1. 1
    jaydisc on December 12, 2017 at 6:34 am

    > “The most reliable way of discovering whether one or more xattrs are associated with a file or folder is using the command ls -la”

    Of course, you can also use ls -l@ to show the attribute types inline with the directory listing.

    LikeLiked by 1 person

    • 2
      hoakley on December 12, 2017 at 6:38 am

      Thank you – yes, that generates output like
      drwxr-xr-x@ 9 hoakley staff 306 13 May 2016 audioBooks
      com.apple.serverdocs.markup 117
      -rw-r--r--@ 1 hoakley staff 4779098 13 Oct 09:20 bluetoothreconnect.png
      com.apple.FinderInfo 32
      com.apple.metadata:kMDItemIsScreenCapture 42
      com.apple.metadata:kMDItemScreenCaptureGlobalRect 76
      com.apple.metadata:kMDItemScreenCaptureType 49
      -rw-r--r--@ 1 hoakley staff 932 24 Jul 06:33 coast alias
      com.apple.FinderInfo 32

      Howard.

      LikeLike

  2. 3
    Week 50 – 2017 – This Week In 4n6 on December 17, 2017 at 12:44 am

    […] An introduction to extended attributes, xattrs […]

    LikeLike

  3. 4
    John Gilbert on December 18, 2017 at 9:34 am

    Another warning you might include:

    Many well known online backup applications (e.g. Backblaze and iDrive) fail to backup extended attributes. I feel it is unacceptable that Finder tags (as a minimum) are not preserved by a backup program. Two programs which do preserve (all?) extended attributes are Crashplan and Arq.

    LikeLike

    • 5
      hoakley on December 18, 2017 at 9:38 am

      Thank you.
      I am aware that there have been concerns about Backblaze, but am shocked that it and others still don’t back up and restore xattrs.
      This was an issue which was addressed well over ten years ago in command tools such as pax (my own hfspax was one of the first to tackle it). It is disgraceful that any macOS product today cannot cope with such a long-standing feature.
      Howard.

      LikeLike

  4. 6
    Joss on November 14, 2018 at 12:59 am

    A bit late to the party, but there’s also a xattr tagging application called Rixtag, part of Rixstep’s Apple Core Project (ACP). The xattr they use is “com.rixstep.Rixtag”. As far as I know, attributes like this won’t be picked up by Spotlight, only the Apple default ones will be. So if you want your personal tags with Spotlight support, you need to use “com.apple.metadata:MyTag”. Sadly, the `tag` CLI doesn’t support this kind of tagging yet, so it’s a bit harder to automate. And you can always use xattr… and xattred. 🙂

    LikeLiked by 1 person

    • 7
      hoakley on November 14, 2018 at 6:34 am

      You can make your own xattr up, such as com.myco.joss, and use it any way that you like. But as it won’t be indexed by Spotlight and will get stripped if you share using iCloud, there isn’t any point in doing so.
      Howard.

      LikeLike

  5. 8
    An introduction to extended attributes, xattrs – The Eclectic Light Company – Jerry's Mac Blog on November 16, 2018 at 7:28 pm

    […] Source: An introduction to extended attributes, xattrs – The Eclectic Light Company […]

    LikeLike

·Comments are closed.

Quick Links

  • Downloads
  • Mac Troubleshooting Summary
  • M1 & M2 Macs
  • Mac problem-solving
  • Painting topics
  • Painting
  • Long Reads

Search

Monthly archives

  • January 2023 (67)
  • December 2022 (74)
  • November 2022 (72)
  • October 2022 (76)
  • September 2022 (72)
  • August 2022 (75)
  • July 2022 (76)
  • June 2022 (73)
  • May 2022 (76)
  • April 2022 (71)
  • March 2022 (77)
  • February 2022 (68)
  • January 2022 (77)
  • December 2021 (75)
  • November 2021 (72)
  • October 2021 (75)
  • September 2021 (76)
  • August 2021 (75)
  • July 2021 (75)
  • June 2021 (71)
  • May 2021 (80)
  • April 2021 (79)
  • March 2021 (77)
  • February 2021 (75)
  • January 2021 (75)
  • December 2020 (77)
  • November 2020 (84)
  • October 2020 (81)
  • September 2020 (79)
  • August 2020 (103)
  • July 2020 (81)
  • June 2020 (78)
  • May 2020 (78)
  • April 2020 (81)
  • March 2020 (86)
  • February 2020 (77)
  • January 2020 (86)
  • December 2019 (82)
  • November 2019 (74)
  • October 2019 (89)
  • September 2019 (80)
  • August 2019 (91)
  • July 2019 (95)
  • June 2019 (88)
  • May 2019 (91)
  • April 2019 (79)
  • March 2019 (78)
  • February 2019 (71)
  • January 2019 (69)
  • December 2018 (79)
  • November 2018 (71)
  • October 2018 (78)
  • September 2018 (76)
  • August 2018 (78)
  • July 2018 (76)
  • June 2018 (77)
  • May 2018 (71)
  • April 2018 (67)
  • March 2018 (73)
  • February 2018 (67)
  • January 2018 (83)
  • December 2017 (94)
  • November 2017 (73)
  • October 2017 (86)
  • September 2017 (92)
  • August 2017 (69)
  • July 2017 (81)
  • June 2017 (76)
  • May 2017 (90)
  • April 2017 (76)
  • March 2017 (79)
  • February 2017 (65)
  • January 2017 (76)
  • December 2016 (75)
  • November 2016 (68)
  • October 2016 (76)
  • September 2016 (78)
  • August 2016 (70)
  • July 2016 (74)
  • June 2016 (66)
  • May 2016 (71)
  • April 2016 (67)
  • March 2016 (71)
  • February 2016 (68)
  • January 2016 (90)
  • December 2015 (96)
  • November 2015 (103)
  • October 2015 (119)
  • September 2015 (115)
  • August 2015 (117)
  • July 2015 (117)
  • June 2015 (105)
  • May 2015 (111)
  • April 2015 (119)
  • March 2015 (69)
  • February 2015 (54)
  • January 2015 (39)

Tags

APFS Apple AppleScript Apple silicon backup Big Sur Blake bug Catalina Consolation Console diagnosis Disk Utility Doré El Capitan extended attributes Finder firmware Gatekeeper Gérôme HFS+ High Sierra history of painting iCloud Impressionism iOS landscape LockRattler log logs M1 Mac Mac history macOS macOS 10.12 macOS 10.13 macOS 10.14 macOS 10.15 macOS 11 macOS 12 macOS 13 malware Mojave Monet Monterey Moreau MRT myth narrative OS X Ovid painting Pissarro Poussin privacy realism Renoir riddle Rubens Sargent scripting security Sierra SilentKnight SSD Swift symbolism Time Machine Turner update upgrade Ventura xattr Xcode XProtect

Statistics

  • 13,733,463 hits
Blog at WordPress.com.
Footer navigation
  • About & Contact
  • Macs
  • Painting
  • Language
  • Tech
  • Life
  • General
  • Downloads
  • Mac problem-solving
  • Extended attributes (xattrs)
  • Painting topics
  • Hieronymus Bosch
  • English language
  • LockRattler: 10.12 Sierra
  • LockRattler: 10.13 High Sierra
  • LockRattler: 10.11 El Capitan
  • Updates: El Capitan
  • Updates: Sierra, High Sierra, Mojave, Catalina, Big Sur
  • LockRattler: 10.14 Mojave
  • SilentKnight, silnite, LockRattler, SystHist & Scrub
  • DelightEd & Podofyllin
  • xattred, Metamer, Sandstrip & xattr tools
  • 32-bitCheck & ArchiChect
  • T2M2, Ulbow, Consolation and log utilities
  • Cirrus & Bailiff
  • Taccy, Signet, Precize, Alifix, UTIutility, Sparsity, alisma
  • Revisionist & DeepTools
  • Text Utilities: Nalaprop, Dystextia and others
  • PDF
  • Keychains & Permissions
  • LockRattler: 10.15 Catalina
  • Updates
  • Spundle, Cormorant, Stibium, Dintch, Fintch and cintch
  • Long Reads
  • Mac Troubleshooting Summary
  • LockRattler: 11.0 Big Sur
  • M1 & M2 Macs
  • Mints: a multifunction utility
  • LockRattler: 12.x Monterey
  • VisualLookUpTest
  • Virtualisation on Apple silicon
  • LockRattler: 13.x Ventura
Secondary navigation
  • Search

Post navigation

Thoroughly-new version of LockRattler now available
xattr: template for type page

Begin typing your search above and press return to search. Press Esc to cancel.

  • Follow Following
    • The Eclectic Light Company
    • Join 3,125 other followers
    • Already have a WordPress.com account? Log in now.
    • The Eclectic Light Company
    • Customize
    • Follow Following
    • Sign up
    • Log in
    • Copy shortlink
    • Report this content
    • View post in Reader
    • Manage subscriptions
    • Collapse this bar
%d bloggers like this: