Few parts of macOS are as undocumented as iCloud Drive. Even establishing its basic limits, such as the size of the largest file it supports, involves a lengthy trek around arcane support notes, now collated into a single article here. The moment you want to go a bit deeper, maybe to work through a problem you’re having, there’s just a void. I’ve also become concerned at some of your reports of troubleshooting advice provided by Apple Support, that have only deepened your problems.

When researching for my current series of articles looking inside iCloud Drive in Sonoma, I came across a new series of technical notes Apple has provided for developers of software that relies on CloudKit. While some of that is potentially relevant to iCloud Drive, there are no matching accounts covering that dark side of iCloud.

Tackling iCloud problems

Apple’s technical notes do repay reading, though, for some of their advice. For example, from recommendations about how to understand and debug problems in end-to-end syncing for apps in NSPersistentCloudKitContainer:
“Sysdiagnose analysis is the only means for peeking at the details inside the synchronization, and is typically for investigating a synchronization issue. To do that, capture a sysdiagnose, use your favorite tools to find the relevant logs, and then explore the messages. Focus on finding errors or suspicious activities that may be relevant to the issue you are investigating.”
“Use Console.app or log to find the relevant logs in the file and determine if the synchronization succeeds.”
“The logs are typically very overwhelming. Use the timestamps you noted when capturing the sysdiagnose to narrow the scope down to the time frame when the issue occurs, and use filters, such as process, subsystem, and message, to filter out the irrelevant logs.”

I can well imagine the rising feeling of panic those words must evoke the first time a developer reads them. Is there really no better way to do this? It seems the answer is no, there’s not. Those “typically very overwhelming” logs are all you’ve got, although there are ways to make their use less fraught.

sysdiagnose

First, when you’re trying to analyse an iCloud Drive problem (rather than for CloudKit), should you use a whole sysdiagnose containing a lot of irrelevant material as well as the desired logarchive? Do you even need a logarchive, or could the live log be good enough?

One of the most disorienting features of the unified log is its tendency to change with time. Traditional logs are written to static text files that get rolled once they’re a few days old. Because the unified log gets so many entries even in a quiescent Mac, there isn’t room for all those that it gathers. The first major weeding of entries starts a few seconds or minutes after each entry has been written, when those that aren’t going to enter disk storage get stripped out. Catch the entries when they’re still fresh and, in some cases, you’ll see five or ten times the number of entries. Just a few minutes later, most of them have gone forever. This still catches me on occasion, when I’m browsing one extract and go back to get a second, only to see the entries I’m most interested in have already vanished, and the record looks completely different.

That behaviour can of course be changed by altering log preferences, but they’re even more arcane than the logs themselves. If you’re quick and have your filter predicates set up, you should be able to get most of what you need from the live log.

It’s important to remember here that neither sysdiagnose nor a logarchive can save log entries that have already been weeded from the log. All they do is ensure that no further loss occurs in the time that you’re examining their log excerpt.

Tools

If you do want a logarchive, though, why not capture that, rather than a whole sysdiagnose? It has been a feature in Ulbow, but sadly because it requires elevated privileges, that doesn’t work in recent macOS, and awaits my writing a privileged helper app to do the job. You can create a logarchive using the log collect command, and that lets you specify how far back you want it to go, rather than sysdiagnose making that decision for you. Surprisingly, even Console’s documentation tells the user that the way to make a logarchive is by performing a sysdiagnose, and doesn’t mention the use of log collect.

Next we have the recommendation to use the Console app or log command to explore the entries in the logarchive. Of all the utilities bundled in macOS, the last I’d ever recommend anyone to use is Console. When it was introduced alongside the Unified log in Sierra, over seven years ago, it was the deepest disappointment, and since then it has struggled to reach version 1.1. That’s correct: in seven years of development, a period during which Xcode has gone from version 9.0 to 15.2, Console has advanced by one minor version number. Is there any other utility that is so neglected and unloved?

Very overwhelming

Finally comes the statement of the obvious, that “the logs are typically very overwhelming”. Since the introduction of Sierra, they have become progressively more very overwhelming to the point where, on this iMac Pro, there’s only the capacity to retain the last 20 hours or so of logs, and when the going gets tougher that can drop to just 4 hours, and there’s still no control in macOS that can extend that.

The log overwhelms because, even when making many of its entries ephemeral and not storing them at all, every single subsystem in macOS treats the log like it owns it. I wonder whether Apple offers a prize for the engineering team that puts the most entries into the log, or maybe it’s planning to sell advertising space there.

I’ve seen it suggested that Apple has its own in-house intelligent log tools, beyond libraries of scripts and predicates to use with the log command. If it has, that might explain its neglect of the Console app; if it hasn’t, then it manifestly isn’t investing sufficient to aid its engineers and support staff. Either way, the rest of us aren’t being provided with the log tools that we need.

Who’s the log for?

There are some who assert that the log isn’t for users, or even for third-party developers, but effectively private to Apple. The clue here is in its original name, the unified log. For the last seven years, it’s the only officially supported general-purpose log, and there are no separate log facilities for users, system administrators, third-party developers, security researchers, Apple engineers, or my aunt Mary. Apple explains this definitively in several places, and here writes:
“The unified logging system provides a comprehensive and performant API to capture telemetry across all levels of the system. This system centralizes the storage of log data in memory and on disk, rather than writing that data to a text-based log file.”

There’s even a box marked Important where it states:
“The unified logging system is available in iOS 10.0 and later, macOS 10.12 and later, tvOS 10.0 and later, and watchOS 3.0 and later. This system supersedes the Apple System Logger (ASL) and Syslog APIs.”

There’s a whole Console User Guide provided for users, where it states:
“Console collects log messages that are generated from your computer and connected devices, and you can use these messages to check on your computer’s performance and solve problems.”
“Use Console to view log messages collected by your computer and other connected devices. These log messages may deal with system events, dialogue text, errors, status and other communications. If a problem occurs, you may be able to find information about the cause of the problem by viewing either log messages or activities.”

I for one remain very overwhelmed. That won’t stop me trying to open access to the log and its contents for the rest of us, but it would be really helpful if Apple was heading in the same direction, instead of just making the log ever more very overwhelming.