One enduring complaint about storing documents in iCloud Drive is difficulty in searching them, or rather Spotlight’s apparent inability to return search results including those files. This article looks at why files in iCloud Drive can vanish from search results.

For Spotlight to be able to search for a file, that file has to have its metadata extracted and added to the .Spotlight-V100 indexes on that volume, then for the search to be within scope, as shown in the diagram below.

Testing this is made straightforward by the features in my free utility Mints. To assess what could be going wrong, I therefore created a set of standard test files and moved them to iCloud Drive in Ventura 13.4.1 (c).

Results

Initially, all the test files were still downloaded from iCloud, so those local copies were complete with the contents required for indexing. After a few minutes to allow them to be indexed properly, Mints’ Check Search found all of them successfully.

Using Cirrus, I then evicted all the test files, so that they were only present locally as stubs. After a further few minutes to allow reindexing to occur, and checking with Cirrus that all those test files were still evicted, Mints’ Check Search only found one of them, in which Keywords had been attached to the file in an extended attribute.

A few minutes after the test files had been downloaded again from iCloud Drive, Mints was able to find them all again. Exactly the same results were obtained from another Mac following synchronisation of its iCloud Drive.

Stub files

Careful examination of the evicted test files showed that their stubs were different in that all had two additional extended attributes added to them in the process of eviction:

• com.apple.icloud.itemName containing the regular file name of the original file,
• com.apple.metadata:_kMDItemUserTags containing an empty binary property list.

Furthermore, the stub of the test file whose search target was stored in its com.apple.metadata:kMDItemKeywords extended attribute had that attached to it, explaining why Spotlight had been able to find that after it had been evicted from local storage.

When macOS creates a stub for a file that is being evicted to iCloud Drive, it therefore:

1. creates the stub file with the same name as the original, with a stop/period prefixed, or renames the original;
2. writes the stub file data, about 160 bytes, to that file;
3. if it’s creating a new file for the stub, it copies the original’s extended attributes to it;
4. it adds com.apple.icloud.itemName and com.apple.metadata:_kMDItemUserTags extended attributes to the stub.

Retention of the original extended attributes suggests that the stub file is created by renaming and changing the original, rather than creating a new file, but it hasn’t been possible to confirm this by inspecting inodes.

Unfortunately, Spotlight search normally won’t return any matches in iCloud’s hidden stub files, although the search used by Mints will. So, even if you’re searching for metadata contained within an extended attribute that is retained with a stub file, Spotlight won’t report it.

Spotlight search rules

In these tests, Spotlight search only reported hits in files that were downloaded from iCloud Drive, and were stored locally in full. Metadata for evicted files had either been removed from Spotlight’s index, or was ruled out of scope by Spotlight.

If you set your Mac to Optimise Mac Storage, then files stored in iCloud Drive will be lost from Spotlight search if macOS evicts them. However, files that remain downloaded should still benefit from normal search, provided that iCloud Drive hasn’t been added to the Spotlight Privacy exclusion list in Siri & Spotlight settings. Their metadata should be included in the .Spotlight-V100 indexes of the volume containing that user’s Home folder, where iCloud Drive is located.

Mints’ Spotlight tests can be used with iCloud Drive simply by copying or moving the test files to a folder in iCloud Drive.