Spotlight local search problems are common, and are all too often tackled blindly by forcing a volume’s indexes to be rebuilt. Although that can sometimes resolve the problem, without knowing its cause, it can just waste time and effort. In some cases rebuilding the indexes can worsen the problem, at least temporarily. This article explains how to use SpotTest and other tools to perform systematic testing and arrive at a diagnosis before hazarding a guess at its treatment.

1. Setting up

Before going any further, check that Spotlight settings are in order, and don’t exclude the volume or folder you’re trying to search, or the document type you’re looking for. In Spotlight settings,

• check all Results from System are turned on, particularly Files and Folders,
• click on Search Privacy… and remove any locations you want to include in search.

Then open Activity Monitor and watch its CPU % listing to verify that Spotlight isn’t currently in the process of reindexing or performing other maintenance to its indexes. If it is, delay testing until those have completed. Searching using Spotlight while it’s actively working on its indexes will give odd results, or none at all.

If you’re going to use SpotTest on any location with privacy control, such as your Home Documents folder or an external disk, add the app to the Full Disk Access list in Privacy & Security settings, before opening it.

2. Home folder test

Even if your interest is in a different volume, perform a basic test of a new test folder in your current Home folder, to establish a baseline and confirm that Spotlight is working there.

Open SpotTest and set it to defaults, with a Term of cattle, a Scope of [Home], and both Search Keywords and Search EXIF ticked.

Click the Create Tests tool (the leftmost of the tools) to create the folder of test files at the top level of your Home folder. Make a note of the time to the second that you do this.

About 10 seconds after that, click either the Run NSMetadata test or Run mdfind test tool. You should then see a list of files found, including those in the test folder ~/0_SpotTestFiles, including A, B, C, D, E, F, G, K, L, M.

If you don’t see those listed, open Mints and use the Log Window button in its Spotlight section to obtain a log extract from the time the test files were created, or use LogUI to do the same. You’ll then need to look at that log extract to see if there are clues as to why indexing didn’t take place in that period.

Leave the test folder where it is, and anything from 1 hour to 5 days later, repeat the search using either or both of those tools. Once additional indexing has been undertaken:

• NSMetadata should now find A, B, C, D, E, F, G, I, K, L, M but not H
• mdfind should now find A, B, C, D, E, F, G, H, I, K, L, M.

I is found using Live Text, and H by Visual Look Up, the latter only being found by the mdfind search.

These tests have demonstrated:

• mdworker and mds indexing of files supported by system mdimporters;
• delayed background mediaanalysisd image analysis and mds indexing of Live Text and Visual Look Up content.

To match test files with their importers, click the Check importers tool. Note that file L doesn’t use a plugin, and N uses a plugin but can’t be found because the search term is inside an image in the PDF document, which currently isn’t recoverable content.

3. Metadata indexing

If that test isn’t fully successful, or you’re uncertain whether indexing is complete, inspect the metadata of the test files. Open a Finder window on the contents of ~/0_SpotTestFiles, set it to Column View, and widen the window to provide a suitable Preview pane within it. Select each of three files in turn and confirm their metadata are shown correctly. To inspect all available metadata, click on any Show More text.

SpotTestC.pdf will have different datestamps, but the seven fields of metadata below those should be identical to those shown above.

SpotTestK.jpg will also have different datestamps, but the five fields below should be identical to those above.

SpotTestM.txt should include a final line of Keywords one, cattle, three.

You can also check all indexed metadata for those files in Terminal using the commands
mdls ~/0_SpotTestFiles/SpotTestC.pdf
mdls ~/0_SpotTestFiles/SpotTestK.jpg
mdls ~/0_SpotTestFiles/SpotTestM.txt

Missing metadata on those three files demonstrates that the test folder hasn’t been indexed correctly. You can try restarting your Mac, leaving it a few minutes to update its Spotlight indexes, then repeating the tests using SpotTest.

4. Custom mdimporter

Many apps come with their own custom mdimporter that provides Spotlight indexing support for document types not supported by macOS. In the past, these were normally installed in /Library/Spotlight, but more recent apps typically keep them inside the app bundle in Library/Spotlight. These can be tested easily.

Create and save one of those custom document types, so that it contains the word cattle in a way that should be searchable by Spotlight. Copy that document to the ~/0_SpotTestFiles folder, wait about 10 seconds, then repeat the test search. You may well notice that NSMetadata search doesn’t find your custom test document, but mdfind does, because of the difference in the search criteria they use.

You should also click the Check importers tool to check that the correct mdimporter was recognised and used for the custom document type.

5. Volume test

If Spotlight works correctly with the test folder in your Home folder, you may wish to progress to testing a different volume or location. Having created its test folder in ~/0_SpotTestFiles, copy that to the other location. Before you change the Scope of the search, click on the 🔄 button to list available volumes, then select the volume containing the copied test folder in the Scope menu.

When you perform the two types of search on that volume, the same rules should apply as to which will be found. Note though that finding files I and H can take much longer, or they may not appear at all.

6. Search term test

When you’re confident that a search term of cattle can be found reliably, you may wish to extend your testing to other terms. Take care when choosing custom terms, as you want to be confident that they should be found, but not in such numbers that the results are overwhelming. You will also need to create your own test files containing the custom term.

Diagnosis

SpotTest can thus provide key information on:

• Delay or absence of find following creation of test files. If no indexing activity is seen in the log, that indicates indexing failure. If the test files are indexed promptly, it indicates search failure.
• Delay or absence in finding files H and I, indicating an indexing failure.
• Failure of a custom mdimporter to index a custom document type.
• Failure to index another volume.

Those should fit in with the overall scheme used by Spotlight, as shown below.

Happy hunting!