Skip to content

The Eclectic Light Company

Macs, painting, and more
Main navigation
  • Downloads
  • M1 & M2 Macs
  • Mac Problems
  • Mac articles
  • Art
  • Macs
  • Painting
hoakley January 8, 2018 Macs, Technology, xattr

Spotlight: using metadata to improve local searches

They just don’t make beer/cars/Spotlight like they used to, do they? When Spotlight first came out in 2005, it was wonderful, and found all sorts of things hidden away in places you’d never have guessed. Since then, as well as expanding to cover web search, it has only gone downhill, and just isn’t as good as it used to be.

That is what many Mac users think, and it is very hard to come up with objective evidence either for or against that opinion.

One thing that you can do is boost the chances of Spotlight’s searches finding the most important hits is to manipulate the metadata which Spotlight uses. Without actually changing a document’s content, you can do this by tagging files with keywords and other metadata which are accessible to Spotlight. The ideal metadata to use – because they are independent of the data format used for any file – is stored in extended attributes, xattrs.

At its most basic, this may just involve adding Finder Comments, which are stored in a xattr of type com.apple.metadata:kMDItemFinderComment and readily accessed using the Finder’s Get Info dialog.

spotmeta01

However, Apple provides a range of more specific xattrs intended for this purpose, including gems such as com.apple.metadata:kMDItemKeywords, which would seem a logical place to store keywords.

This article looks at the xattrs most suitable for enhancing a file’s Spotlight metadata, across Sierra and High Sierra. To compile this information, I have looked at the performance of a selection of xattrs with Spotlight searches in both versions of macOS, and how well they are preserved when moving files using iCloud Drive.

I have looked at plain text and PDF documents with and without a single use of a very distinctive word syzygy, as my marker. For text files which don’t include that word, I have added it in a xattr, so that it functions as a keyword which doesn’t appear in the document itself. The xattrs I have used include:

  • com.apple.ResourceFork
  • com.apple.metadata:_kMDItemUserTags
  • com.apple.metadata:kMDItemCopyright
  • com.apple.metadata:kMDItemCreator
  • com.apple.metadata:kMDItemDescription
  • com.apple.metadata:kMDItemFinderComment
  • com.apple.metadata:kMDItemHeadline
  • com.apple.metadata:kMDItemKeywords
  • org.openmetainfo.time:kMDItemCopyright
  • org.openmetainfo.time:kMDItemDescription
  • org.openmetainfo.time:kMDItemHeadline
  • org.openmetainfo.time:kMDItemKeywords
  • org.openmetainfo:kMDItemCopyright
  • org.openmetainfo:kMDItemDescription
  • org.openmetainfo:kMDItemHeadline
  • org.openmetainfo:kMDItemKeywords
  • co.eclecticlight.test

For the Resource Fork, I added the magic word as hex content in UTF-8 format; for all the others, I added it as a plain string in a property list:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<string>syzygy</string>
</plist>

Spotlight searches consistently found my magic keyword when it was included in any of the com.apple xattrs apart from com.apple.ResourceFork: Spotlight does not, apparently, index any content held in Resource Forks. It also consistently found the magic word when it was included within the content of plain text and PDF documents. Note that this was just a single occurrence among nearly thirty thousand words – 175 KB of text or 184 KB of PDF – which seems pretty good performance to me.

Spotlight search doesn’t appear to index any third party xattrs, such as org.openmetainfo types, or custom types of any kind.

There were, though, two significant problems.

First, as I have noted elsewhere, moving files between Sierra and High Sierra (in either direction) using iCloud Drive strips most xattrs. The xattrs which both worked for Spotlight search and survived iCloud’s censorship were:

  • com.apple.metadata:kMDItemCopyright
  • com.apple.metadata:kMDItemCreator
  • com.apple.metadata:kMDItemDescription
  • com.apple.metadata:kMDItemFinderComment
  • com.apple.metadata:kMDItemHeadline
  • com.apple.metadata:kMDItemKeywords

Stick to those and you shouldn’t go wrong.

Second, Finder Comments are unreliable, and don’t now behave as regular extended attributes. Add a Finder Comment in the Finder, and it appears as a com.apple.metadata:kMDItemFinderComment xattr, which is not stripped by iCloud. But adding a com.apple.metadata:kMDItemFinderComment xattr doesn’t result in its text appearing as a Finder Comment, even when that xattr has been copied in from a file which had that added in the Finder.

spotmeta02

This gets even more complex when moving files with Finder Comments between Sierra and High Sierra: the xattr moves fine, but the contents are no longer shown as Finder Comments, and have to be recreated using the Get Info dialog.

I suspect that the xattr is no longer being used to contain the comment, which is actually tucked away in a local Finder database instead. So the com.apple.metadata:kMDItemFinderComment xattr can be used to contain metadata which will be indexed by Spotlight, even though it may not appear in the Finder’s Get Info dialog. That is an unnecessary mess.

The next item on the wanted list is, of course, a convenient GUI app to give easier access to these xattrs. If you can wait just a little while, I might be able to help you with that too.

Share this:

  • Twitter
  • Facebook
  • Reddit
  • Pinterest
  • Email
  • Print

Like this:

Like Loading...

Related

Posted in Macs, Technology, xattr and tagged extended attributes, High Sierra, macOS, macOS 10.12, macOS 10.13, metadata, search, Sierra, Spotlight, xattr. Bookmark the permalink.

Quick Links

  • Downloads
  • Mac Troubleshooting Summary
  • M1 & M2 Macs
  • Mac problem-solving
  • Painting topics
  • Painting
  • Long Reads

Search

Monthly archives

  • February 2023 (10)
  • January 2023 (74)
  • December 2022 (74)
  • November 2022 (72)
  • October 2022 (76)
  • September 2022 (72)
  • August 2022 (75)
  • July 2022 (76)
  • June 2022 (73)
  • May 2022 (76)
  • April 2022 (71)
  • March 2022 (77)
  • February 2022 (68)
  • January 2022 (77)
  • December 2021 (75)
  • November 2021 (72)
  • October 2021 (75)
  • September 2021 (76)
  • August 2021 (75)
  • July 2021 (75)
  • June 2021 (71)
  • May 2021 (80)
  • April 2021 (79)
  • March 2021 (77)
  • February 2021 (75)
  • January 2021 (75)
  • December 2020 (77)
  • November 2020 (84)
  • October 2020 (81)
  • September 2020 (79)
  • August 2020 (103)
  • July 2020 (81)
  • June 2020 (78)
  • May 2020 (78)
  • April 2020 (81)
  • March 2020 (86)
  • February 2020 (77)
  • January 2020 (86)
  • December 2019 (82)
  • November 2019 (74)
  • October 2019 (89)
  • September 2019 (80)
  • August 2019 (91)
  • July 2019 (95)
  • June 2019 (88)
  • May 2019 (91)
  • April 2019 (79)
  • March 2019 (78)
  • February 2019 (71)
  • January 2019 (69)
  • December 2018 (79)
  • November 2018 (71)
  • October 2018 (78)
  • September 2018 (76)
  • August 2018 (78)
  • July 2018 (76)
  • June 2018 (77)
  • May 2018 (71)
  • April 2018 (67)
  • March 2018 (73)
  • February 2018 (67)
  • January 2018 (83)
  • December 2017 (94)
  • November 2017 (73)
  • October 2017 (86)
  • September 2017 (92)
  • August 2017 (69)
  • July 2017 (81)
  • June 2017 (76)
  • May 2017 (90)
  • April 2017 (76)
  • March 2017 (79)
  • February 2017 (65)
  • January 2017 (76)
  • December 2016 (75)
  • November 2016 (68)
  • October 2016 (76)
  • September 2016 (78)
  • August 2016 (70)
  • July 2016 (74)
  • June 2016 (66)
  • May 2016 (71)
  • April 2016 (67)
  • March 2016 (71)
  • February 2016 (68)
  • January 2016 (90)
  • December 2015 (96)
  • November 2015 (103)
  • October 2015 (119)
  • September 2015 (115)
  • August 2015 (117)
  • July 2015 (117)
  • June 2015 (105)
  • May 2015 (111)
  • April 2015 (119)
  • March 2015 (69)
  • February 2015 (54)
  • January 2015 (39)

Tags

APFS Apple AppleScript Apple silicon backup Big Sur Blake bug Catalina Consolation Console diagnosis Disk Utility Doré El Capitan extended attributes Finder firmware Gatekeeper Gérôme HFS+ High Sierra history of painting iCloud Impressionism iOS landscape LockRattler log logs M1 Mac Mac history macOS macOS 10.12 macOS 10.13 macOS 10.14 macOS 10.15 macOS 11 macOS 12 macOS 13 malware Mojave Monet Monterey Moreau MRT myth narrative OS X Ovid painting Pissarro Poussin privacy realism Renoir riddle Rubens Sargent scripting security Sierra SilentKnight SSD Swift symbolism Time Machine Turner update upgrade Ventura xattr Xcode XProtect

Statistics

  • 13,789,370 hits
Blog at WordPress.com.
Footer navigation
  • About & Contact
  • Macs
  • Painting
  • Language
  • Tech
  • Life
  • General
  • Downloads
  • Mac problem-solving
  • Extended attributes (xattrs)
  • Painting topics
  • Hieronymus Bosch
  • English language
  • LockRattler: 10.12 Sierra
  • LockRattler: 10.13 High Sierra
  • LockRattler: 10.11 El Capitan
  • Updates: El Capitan
  • Updates: Sierra, High Sierra, Mojave, Catalina, Big Sur
  • LockRattler: 10.14 Mojave
  • SilentKnight, silnite, LockRattler, SystHist & Scrub
  • DelightEd & Podofyllin
  • xattred, Metamer, Sandstrip & xattr tools
  • 32-bitCheck & ArchiChect
  • T2M2, Ulbow, Consolation and log utilities
  • Cirrus & Bailiff
  • Taccy, Signet, Precize, Alifix, UTIutility, Sparsity, alisma
  • Revisionist & DeepTools
  • Text Utilities: Nalaprop, Dystextia and others
  • PDF
  • Keychains & Permissions
  • LockRattler: 10.15 Catalina
  • Updates
  • Spundle, Cormorant, Stibium, Dintch, Fintch and cintch
  • Long Reads
  • Mac Troubleshooting Summary
  • LockRattler: 11.0 Big Sur
  • M1 & M2 Macs
  • Mints: a multifunction utility
  • LockRattler: 12.x Monterey
  • VisualLookUpTest
  • Virtualisation on Apple silicon
  • LockRattler: 13.x Ventura
Secondary navigation
  • Search

Post navigation

Orientalism: the reality
WD My Cloud drives need urgent firmware update

Begin typing your search above and press return to search. Press Esc to cancel.

  • Follow Following
    • The Eclectic Light Company
    • Join 3,133 other followers
    • Already have a WordPress.com account? Log in now.
    • The Eclectic Light Company
    • Customize
    • Follow Following
    • Sign up
    • Log in
    • Copy shortlink
    • Report this content
    • View post in Reader
    • Manage subscriptions
    • Collapse this bar
 

Loading Comments...
 

    %d bloggers like this: