Apple obsesses about privacy, which is commendable, but even the best-motivated obsession can at times become unhealthy. In the case of the unified log in macOS Catalina, Apple’s obsession threatens the health of its users. The problem is, in a word, <private>.

When Apple switched to its unified log in macOS Sierra three years ago, it wisely built in protection for the unintentional appearance of private and sensitive data in the message field of log entries. By default, certain types of data are censored using the now-dreaded mark <private>. Anyone with admin access to a Mac can’t therefore look through previous log entries to discover sensitive information about its users. This isn’t imposed after the fact by log access tools: those protected data aren’t written to the log in the first place.

There was an undocumented control, though, an option to the log config command which could disable this censorship altogether. This was potentially dangerous, in that the inattentive user could accidentally leave censorship disabled, letting their log steadily fill with private data. This could even have been used maliciously to monitor users, although if you’ve tried accessing the unified log using Console I’m sure you’d prefer one of many far easier ways.

This setting is quite easy to check for, and one of many features in my free app LockRattler. Apple has chosen to neither expose nor control this setting in the Privacy tab of the Security & Privacy pane, though.

It’s worth remembering that there has only been one albeit spectacular failure of protection in the unified log, and that wasn’t the result of this option. In certain circumstances, macOS High Sierra versions 10.13 to 10.13.3 leaked plaintext records of encryption passwords used for APFS volumes. This appeared to have been the result of incorrect formatting strings when composing the message field of corresponding log entries, which bypassed censorship completely.

Censorship of private data from log messages is a growing problem for those who access the unified log. A quick sample of 12 separate minute-long log excerpts taken throughout a day’s use of an iMac Pro shows the proportion of log entries censored with <private> ranged from 1-19%. In one period, 1636 log entries were censored from a total of 8638. Log entries occurred in these samples at frequencies ranging from 1150 to 27281 per minute, giving a good idea of one of the other problems of accessing the unified log.

Those figures are totals for all log entries, including signposts and activities. When you look at specific subsystems, the picture can be far worse. Messages from OpenDirectory, DiskArbitration and iCloud-related subsystems are particularly frequently censored. Hardly any log entries made by diskarbitrationd contain usable information in their message field. Trying to diagnose disk, iCloud and OpenDirectory problems from the unified log is almost impossible as a result. For those, we have generally relied on being able to disable log censorship.

In Catalina, Apple has removed that option to disable censorship. Saagar Jha has discovered that the only way to disable censorship now is to put macOS into a special diagnostic mode intended for use exclusively by Apple engineers. George Garside has packaged Saagar Jha’s code into a command tool which can be used to remove censorship in Catalina’s log. I’m very grateful to Moritz and Christoph for pointing these out to me.

I now face a problem with my free iCloud utility Cirrus, which provides a diagnostic log browser tailored for iCloud problems, with the ability in Mojave and earlier to turn log censorship off. That no longer works in Catalina. Should I incorporate Saagar Jha’s method of accomplishing this in 10.15?

I have regretfully decided that, as the Catalina workaround for disabling log censorship depends on an undocumented diagnostic mode whose side-effects are unknown, Cirrus will not attempt that. This makes diagnosing and addressing problems with iCloud much more difficult, and more users will have to resort to the well-worn Apple Support approach of just turning it off and back on again, and hoping for the best. With the spate of iCloud problems reported from those who have upgraded to Catalina, this is particularly ill-timed. It’s not as if Apple provides any alternative: iCloud is one service for which it appears to have completely forgotten to provide any diagnostics or utilities.

There’s another general issue here: many of those log entries containing <private> shouldn’t be appearing in release versions of macOS in the first place. The most obvious are entries whose entire message consists of <private>. These can now only contain useful information when collected from Macs which have been put into Apple’s internal diagnostic mode; the established convention is that such log entries are only made from debug versions of that software, or these meaningless stub entries just clutter up the log and make its use harder for everyone else.

The unified log is not Apple’s <private> playground. It’s a shared space, with users diagnosing problems, developers hunting bugs, support staff fixing glitches, and system administrators managing their networks. For us all to get benefit from our logs, Apple needs to provide a supported means of temporarily disabling this censorship in the unified log. If it won’t, then it’s time for Apple to admit openly that it doesn’t really want anyone else using the unified log.