From its introduction in macOS Sierra to Mojave, the unified log has undergone many changes in what it logs, how it analyses its logs, and most recently in the introduction of Signposts for performance measurement. Despite all the other momentous changes in Catalina, there are only subtle changes which have taken place in its logging. This article explains those that I have found so far.

Log file layout and internals

Unified log files continue to be stored in /var/db/diagnostics, which is now of course on the writeable Data volume, and the folder layout is unchanged from that in Mojave. Regular tracev3 log files are to be found in the Persist folder, those for Signposts in its folder, and Special continues to contain supplementary material. The HighVolume folder is still present, but I haven’t observed it being used to store anything yet. Often forgotten when considering the log are the ancillary files in /var/db/uuidtext, and time synchronisation data in /var/db/diagnostics/timesync, which log show also requires to function properly.

Apple’s log file format, tracev3, is proprietary and compressed, and remains completely undocumented. Although there don’t appear to be any changes in its format between Mojave and Catalina, as a matter of principle it remains wisest to access log archives on macOS which is at least the same version as that responsible for writing them, to eliminate any issues of compatibility. This is most important when trying to examine logs from Mojave and Catalina: you won’t be able to read them from High Sierra or earlier. To be able to read all versions of the unified log, it’s best to work in Catalina now.

Log content

Apple has progressively increased the number of fields available for use within log entries, from 16 in Sierra, to 22 in High Sierra 10.13.6, to 27 in Mojave and Catalina.

Sierra’s fields are:

1. timestamp, in full e.g. 2017-07-26 20:24:59.326229+0100
2. machTimestamp, in system ticks, e.g. 608403543041193
3. messageType, e.g. Default
4. category, e.g. security_exception
5. subsystem, e.g. com.apple.securityd
6. processUniqueID, e.g. 156
7. threadID, e.g. 868
8. traceID, e.g. 833721519476834308
9. senderProgramCounter, e.g. 193733726
10. processID, e.g. 156
11. eventMessage, e.g. MacOS error: -67062
12. processImagePath, e.g. /usr/libexec/taskgated
13. processImageUUID, e.g. 4F6F0B24-7A18-3AF9-853F-8F72F6C7D7C7
14. senderImagePath, e.g. /System/Library/Frameworks/Security.framework/Versions/A/Security
15. senderImageUUID, e.g. 005E8C96-40B6-35E3-B58B-888A5F5957C2
16. timezoneName, may be blank.

To those, High Sierra 10.13.6 added:

• 17. eventType, one of signpostEvent, activityCreateEvent, logEvent, or timesyncEvent (others such as traceEvent and activityTransitionEvent may be possible but are very rare)
• 18. signpostID, e.g. 14608112960288129006
• 19. activityIdentifier, e.g. 32688
• 20. parentActivityIdentifier, e.g. 0
• 21. creatorActivityID, e.g. 0
• 22. source, e.g. null. The only value I have ever seen for this is null.

Mojave 10.14 in turn added:

• 23. signpostName, a string name for a Signpost;
• 24. signpostType, the Signpost type – begin, end, or event;
• 25. signpostScope, a scope string, such as ‘process’;
• 26. formatString, giving the format string used to convert variable content into a string for output, e.g. “vm_page_bootstrap: %d free pages and %d wired pages\n” or “%{public}@”;
• 27. backtrace, consisting of imageOffset and imageUUID structured into a ‘frame’.

If you use the current version of my log browser Consolation, all the fields available for viewing are listed in a window accessed through the Help menu. The list provided depends on the version of macOS on which you are running Consolation.

Apple’s own processes now use subsystem names extensively, although this brings the problem that many of those use internal names which don’t correlate with developer documentation. A case in point is com.apple.runningboard, which refers to the RunningBoard sub-system in Catalina, and has been reported in iOS 13 as well.

There’s an interesting story behind what may appear a peculiar field to include in a log, formatString. You may recall that one of the many security gaffes which appeared in High Sierra was that, at one stage, FileVault passwords for encrypted external drives were recorded in plain text in the unified log. I understand that this was the result of using the wrong formatting string when writing that log entry, which released data which should have been marked as <private>. This field was added to ensure that no Apple engineer would use the wrong formatting string in the future. Otherwise, knowing the formatting string used to write that log entry is a fairly purposeless thing to do, and not the sort of content that you’d want to clutter up your log with.

For the last year and more, all versions of my free log browser Consolation 3 have given full access to all those fields except for the last, backtrace. I have so far not included that field, as it doesn’t appear useful to anyone outside Apple, and when present delivers a multi-line string such as
{
"frames" : [
{
"imageOffset" : 39623,
"imageUUID" : "100013AE-0443-3CF0-AC17-5D934608B60B"
}
]
}
This would occupy a lot of lines in a log extract.

Those fields are available in log show predicates, together with some additional derived fields such as composedMessage and timeToLive. A full list of the fields supported in predicates is given by typing log help predicates in Terminal, which also details the types used by type, signpostType, signpostScope and logType. However, there are discrepancies between those field names and those used in JSON-format log output which can cause confusion. None of these appears to have changed, and I list the fields and types at the end of this article for reference.

The log command

If you use the log command to work with logs, its verbs and options remain the same as in Mojave, and its man page hasn’t been updated for three years now. By default, Signposts are still not included in log show output. If you want them included, then you will need to provide the option
--signpost
in the command. Although Signposts are now quite profuse and at times may swamp other log entries, they can provide useful information and are worth considering in all log extracts made.

In Catalina, Apple has added code-level access to log entries using the OSLog framework, which was announced at WWDC in June 2019. However, although documentation stubs are now available, Apple hasn’t provided any further information, documentation, or example code, making its use a matter of inspired guesswork. As far as I can see at present, this only gives limited access to the contents of log entries, returning just two of the 27 fields now available using log show. It thus doesn’t appear suitable for use in apps like Consolation.

Writing to the log

Public interfaces for writing to the log don’t appear to have changed since Mojave. These are simple to use, for example in Swift 5.1:
self.myLog = OSLog(subsystem: "co.eclecticlight.Whither", category: "Signpost")
let mySpid = OSSignpostID(log: self.myLog!)
os_signpost(.begin, log: self.myLog!, name: "timingCheckMain", signpostID: mySpid, "%{public}@", "\(i)")

You can find full information about how to use Signposts from within Xcode and in scripting and other systems in my Signpost Kit, available from Downloads above and its product page.

Console

The biggest disappointment with the unified log remains Apple’s bundled GUI log browser, Console. Prior to the introduction of the unified log, Console was primarily used by a great many macOS users to browse recent log entries. This is only now possible through a circuitous route: you have to create a logarchive of the active log, open that in Console, then browse its contents. Most users also find the sheer volume of entries in the unified log forbidding, something which Console does little to address.

The version of Console in Catalina is the same as that in Mojave, 1.1, so if you were hoping that Apple might improve this utility, dream on for 10.16.

Predicate fields available in Mojave and Catalina

• activityIdentifier (integer)
• bootUUID (uuid)
• category (string)
• composedMessage (string)
• continuousNanosecondsSinceBoot (integer)
• creatorActivityIdentifier (integer)
• creatorProcessUniqueIdentifier (integer)
• date (date)
• formatString (string)
• logType (log type)
• machContinuousTimestamp (integer)
• parentActivityIdentifier (integer)
• process (string)
• processIdentifier (integer)
• processImagePath (string)
• processImageUUID (uuid)
• sender (string)
• senderImageOffset (integer)
• senderImagePath (string)
• senderImageUUID (uuid)
• signpostIdentifier (integer)
• signpostScope (signpost scope)
• signpostType (signpost type)
• size (integer)
• subsystem (string)
• threadIdentifier (integer)
• timeToLive (integer)
• traceIdentifier (integer)
• transitionActivityIdentifier (integer)
• type (event type).

Valid event types: activityCreateEvent, activityTransitionEvent, userActionEvent, traceEvent, logEvent, timesyncEvent, signpostEvent, lossEvent, stateEvent.

Valid log types: default, release, info, debug, error, fault.

Valid signpost scopes: thread, process, system.

Valid signpost types: event, begin, end.