Now the dust has settled on the major changes brought about in iCloud Drive by Sonoma, I’ve started to look deeper at how iCloud Drive now works, once you have upgraded to macOS 14. One immediate impact is that existing utilities which present log extracts for iCloud Drive, including my own Cirrus and Mints, no longer work as expected, and need to be revised to cover its extensive use of FileProvider.
FileProvider
Apple introduced its new FileProvider API over two years ago, at WWDC 2021, when Johannes Fortmann of the Cloud File Providers team gave a presentation on what was then being encouraged of those who support third-party cloud services in macOS. For those services, FileProvider extensions deliver the interface between macOS, particularly APFS and the Finder, and the cloud service. As Apple hasn’t documented the internals of iCloud Drive, it hasn’t explained how FileProvider affects iCloud Drive now that they have been integrated in Sonoma.
Prior to Sonoma, one of the features in iCloud Drive that hasn’t behaved as documented is Optimise Mac Storage. This has changed in Sonoma, as it now effectively switches between two different types of file provider: a replicated file provider, which syncs between local and remote copies of all files put in the cloud, and a nonreplicated file provider, which hosts and manages files that can be stored locally or whose data may only exist in the cloud. Because a replicated FileProvider is far simpler and isn’t managed locally, I here concentrate on the nonreplicated form.
It’s essential to remember that, in the new iCloud Drive, changing Optimise Mac Storage results in a major change in behaviour, and as far as FileProvider is concerned, it effectively switches between two different versions of iCloud Drive.
New subsystem
FileProvider makes copious use of the Unified log, and its entries are rich in content, structured, and even use emoji effectively. Trying to understand what is happening in iCloud Drive in Sonoma without seeing entries from com.apple.FileProvider isn’t possible, though.
Currently, both Cirrus and Mints use the following predicate to obtain log extracts:
subsystem == "com.apple.clouddocs" OR subsystem == "com.apple.cloudkit"
OR subsystem == "com.apple.mmcs" OR processImagePath CONTAINS[c] "cloudd"
OR processImagePath CONTAINS[c] "bird" OR processID = 0
which doesn’t gather any for com.apple.FileProvider, so loses much of the information written to the log. I intend changing that by adding entries made by the FileProvider subsystem.
Example messages written by com.apple.FileProvider are:
┏10e3 💡 trigger: itemEvicted(.main(docID(14584)), inParent: .main(fileID(501470)))
┗10e3
These use an initial bracketing character to indicate that they form one response in its series, they use an emoji, and give explicit identities such as as the docID and fileID rather than the all-too-common anonymisation of <private>.
Eviction
The simplest action to analyse is that of eviction, when the local copy of a file held in iCloud Drive is removed, leaving it dataless, an action only available in nonreplicated file providers. This schedules a file provider evict operation, which first has to be handled by File Coordination. When that gives the go-ahead, FileProvider works through a series of steps to fetch the file’s metadata, clean remote versions, update its file system ‘snapshot’ (not an APFS volume snapshot), and change the item’s content status.
These are shown in the diagram below, built from log records obtained from Sonoma 14.1.1 evicting a single file from iCloud Drive.
Involvement of traditional iCloud subsystems such as CloudDocs is now minimal, and the overwhelming majority of significant log entries are made by com.apple.FileProvider. If that looks long-winded, downloading or materialising that file is far longer and more complicated, as is adding a new file to iCloud Drive. The irony in this particular case is that, no sooner is the file evicted, but as its QuickLook thumbnail has to be displayed in the Finder, it’s immediately materialised for that purpose.
Consequences
Sonoma’s changes to iCloud Drive run deep, and so far (on limited testing) appear to increase reliability and make the choking of actions far less likely. The FileProvider API in macOS 14.1.1 doesn’t appear complete: I have seen at least one log entry referring to a feature which hasn’t yet been implemented. Although FileProvider should make any problems or errors more visible, the sheer volume of its log entries isn’t going to make their identification any easier. Once we’re more familiar with how it should work, it may well make diagnosing iCloud Drive problems more feasible, but the absence of technical documentation beyond that for third-party file providers doesn’t help.
The Eclectic Light Company 