Inconsistent behaviour is the enemy of information retrieval. If some of your text files have their contents indexed, but others don’t, it makes search a gamble in which the user is invariably the loser. This article’s cautionary tale concerns an increasingly popular document format, JSON data saved in text files, which are sometimes processed fully for Spotlight’s volume indexes, but other times their entire contents are wilfully ignored.

My example comes from the JSON files saved by LogUI in its custom co.eclecticlight.loguilog files, a content type that is descended from public.json, thus public.text, and might thus be expected to be indexed by the bundled RichText.mdimporter in macOS. Only they’re not, and it would appear that their indexing uses no mdimporter plugin, despite sometimes extracting their text content.

Search failures

The inconsistent behaviour appeared when I was looking at Spotlight search, ironically, and wanted to discover which log files, saved in text JSON format, contained search terms like SPQUParse, which should be exceedingly unusual even in other similar log extracts. I discovered that Spotlight search (using any method) for that and other related terms consistently found them in one log extract, but not another, although searching those two files in LogUI found those terms in both.

Google assured me that Spotlight is unable to search the contents of JSON files, and recommended I created my own mdimporter using a hack from about 15 years ago that I knew would no longer work. Fortunately I have tools that Google didn’t think of mentioning, to work out why some JSON files are indexed but others aren’t: the Drop Window in SpotTest, and the command tool file. I then found three log files to check, each of which I know have plenty of occurrences of SPQUParse.

Results

Successful search is consistent with LogUI2026_07_06_22-44-00spotQuery1.logui, both on my Mac mini in 26.5.2, and in a VM running on it. It succeeds in the Finder’s Find, when using NSMetadata search in SpotTest, and with mdfind. It’s a large JSON file, and mdimport reports writing text content of 5,416,742 characters to its kMDItemTextContent attribute, effectively the Spotlight content index. According to file, that JSON text is “Unicode text, UTF-8 text, with very long lines (65534), with no line terminators”, an accurate summary.

Search is consistently unsuccessful with LogUI2026_07_06_22-44-00fullSpQuery2.logui whatever I try (short of changing its extension). It’s another large JSON file, reported by mdimport as writing text content of 4,358,313 characters to its kMDItemTextContent attribute. However, file takes a dim view of it, reporting that it’s “ASCII text, with very long lines (65536), with no line terminators”, despite it being recognised everywhere else as conformant UTF-8.

My other consistently unsuccessful file is LogUI2026_07_06_22-44-00 – SPQUP.logui. I created this from LogUI2026_07_06_22-44-00fullSpQuery2.logui in an effort to make its contents searchable. This consists of only the log entries that contain my search term of SPQUParse, making them impossible to miss, and the file a small fraction of the size. mdimport doesn’t report writing any text content at all from this file, so none of its contents are being indexed. file claims that this is “JSON data”, presumably sufficient deterrent to any content being indexed.

The remaining mystery, of course, is how the text content of any of these three files was extracted and indexed, given that in each case mdimport reports that no plugin (mdimporter) was used to do that. As Apple doesn’t appear to document what its bundled mdimporters do, we’re left in the dark as to what is responsible.

Cause

Whatever file thought about these three files, they each have the same kMDItemContentTypeTree, starting from co.eclecticlight.loguilog, rising to public.json and public.text. As with previous example bugs I’ve described in Spotlight indexing, the unpredictable behaviour seen here appears to be the result of macOS placing reliance on the ‘magic’ few bytes at the start of a file as an accurate indicator of file type, and ignoring the file’s UTI (or UTType) altogether.

Maybe I should just cheat, and discover the ‘magic’ bytes with which to prefix the contents of each file, to trick Spotlight into indexing its text contents, however that might happen.

The bigger problem

Whatever the immediate solution for making the contents of LogUI’s documents searchable outside the app, there’s a growing problem with this type of document. In-app search works well, but requires each file to be opened for searching, one at a time, making it useless for identifying which of hundreds or thousands of files contain a search term.

Spotlight metadata search requires a suitable mdimporter to capture file content for inclusion in the volume index. If Apple’s bundled mdimporters are unreliable, as they are here, then developers have to turn to Apple’s Spotlight Importer Programming Guide, last revised on 8 August 2013, almost 13 years ago, just before the release of OS X 10.9 Mavericks.

These problems are largely confined to macOS, though, and are well outside the scope of Siri knowledge, AI, semantic search, or any of the new toys coming in xOS 27. But reliable information retrieval remains their essential foundation.

There is one piece of good news. I also tried exporting these JSON log extracts to unstructured Rich Text, and in every case all search methods were able to find the search terms used. Unfortunately, Rich Text log extracts are far less useful than those serialised into structured JSON, except it would appear for the purpose of search.