You either love Apple’s unified log, or hate it. It has either been a great enabler, or you have given up even trying to access it. As there isn’t really any way around it – at least, not when you need the information which used to be so readily available in Console – the latter may mean that you no longer use your log properly.

From its launch at WWDC 2016 until last week, Apple had said remarkably little about the unified log in Sierra and beyond. As it is essentially undocumented, apart from the log man page which is woefully out of date and at least partly fictional, the changes which have taken place are only apparent when looking back. Apple has maintained a degree of consistency and compatibility: you can take logarchives from a Mac running Sierra and one running High Sierra, and read both on any Mac running Sierra or later. But what you will see differs greatly according to which version wrote the log, and which is used to read it.

This article explains those differences, and provides advice on how to get the most from your logs (and those from other Macs, iOS devices, etc.).

The fundamental problem behind all this is that logs are saved in closed, proprietary tracev3 format. There are only two tools which can access entries within a tracev3 file: the log command tool, and Console. As we have no way of knowing what is actually saved in log files, we can only see what is served up by log and Console; as they have changed capabilities, so they have changed what we can retrieve from the log.

Sierra

log and Console can deliver 16 different fields from log files, wherever they have originated:

1. timestamp, in full e.g. 2017-07-26 20:24:59.326229+0100
2. machTimestamp, in system ticks, e.g. 608403543041193
3. messageType, e.g. Default
4. category, e.g. security_exception
5. subsystem, e.g. com.apple.securityd
6. processUniqueID, e.g. 156
7. threadID, e.g. 868
8. traceID, e.g. 833721519476834308
9. senderProgramCounter, e.g. 193733726
10. processID, e.g. 156
11. eventMessage, e.g. MacOS error: -67062
12. processImagePath, e.g. /usr/libexec/taskgated
13. processImageUUID, e.g. 4F6F0B24-7A18-3AF9-853F-8F72F6C7D7C7
14. senderImagePath, e.g. /System/Library/Frameworks/Security.framework/Versions/A/Security
15. senderImageUUID, e.g. 005E8C96-40B6-35E3-B58B-888A5F5957C2
16. timezoneName, may be blank.

A seventeenth field, eventType, can be used in filter predicates, so it exists within log records, but cannot be retrieved as a field. It is also peculiar in equating to a string, such as logEvent, but being encoded numerically, e.g. as 1024. The other field which behaves in this way is messageType, although that can be retrieved as a field, e.g. in JSON format output from log show.

Sierra logs also only appear to contain those 17 fields.

High Sierra

Apple didn’t document when the Sierra formats changed, but this had occurred before 29 January 2018, so may have been in the initial release of High Sierra the previous September, or perhaps as late as the 10.13.3 update, which shipped on 23 January 2018.

log and Console can now deliver a total of 22 different fields from log files, wherever they have originated. Those include the 16 listed above, plus:

• 17. eventType, one of signpostEvent, activityCreateEvent, logEvent, or timesyncEvent (others such as traceEvent and activityTransitionEvent may be possible but are very rare)
• 18. signpostID, e.g. 14608112960288129006
• 19. activityIdentifier, e.g. 32688
• 20. parentActivityIdentifier, e.g. 0
• 21. creatorActivityID, e.g. 0
• 22. source, e.g. null. In High Sierra, the only value seen is null, but this may change in Mojave.

Signposts

These were announced at WWDC 2018, a week ago. They consist of four components:

• a system call os_signpost() to write them to the log
• log support for the eventType of signpostEvent as a key value
• log reader (log and/or Console) support for extracting the fields used by Signposts
• Xcode instruments support for reading and analysing them.

Although it has been claimed that these have been developed from kdebug_signpost, they appear to be straightforward log entries based on os_log.

Apple system components have been making Signpost entries into High Sierra’s log since at least 29 January 2018, as demonstrated in the logarchive analysis using Consolation shown above. This has required at least the first three of those four being present from macOS 10.13.3 onwards, if not earlier. However, the os_signpost call in High Sierra has remained in Apple’s private frameworks and was not made publicly accessible.

Thus the actual changes which were announced for Mojave – other than further extensions to the log and log command – are:

• os_signpost() made public
• Xcode Instruments support for reading and analysis.

This is not to belittle their utility, but establishes that in this case Mojave actually brings little that is completely new. I have now started to analyse Mojave’s logs using Consolation, and will be further modifying the app to make best use of the new fields which 10.14 brings.

Which tools?

So far, Apple has maintained good compatibility across different versions of the log files, such as tracev3 files, and logarchives can be analysed on any system running Sierra or later.

Your aim should therefore be to capture the logs into logarchives, which appears to transfer tracev3 and other files without any further processing, so preserving content. Those should be moved across to a Mac running the latest release of High Sierra or Mojave, and analysed there, for maximum information yield.

My log analysis apps – Consolation, Woodpile, Cirrus, and T2M2 – each rely on the local copy of log to access log content. You should therefore use them on the most recent version of macOS to which you have access, currently either macOS 10.13.5 or 10.14 beta. That will ensure maximum yield of information from them. Browsing or analysing logs from High Sierra or Mojave on a Sierra system does lose potentially valuable log information, and should be avoided if at all possible.