macOS Unified log: 3 finding your way

Armed with an understanding of the content of log entries, and knowing how to limit and filter those that you are browsing, you have the basic tools to start exploring the unified log in Sierra and High Sierra. Like any explorer, you need waypoints to help you recognise where you are, and what is going on.

Waypoints

One of the most fundamental waypoints is the beginning of the boot process. If you know what to look for, this makes it possible to examine what went on during startup.

Prior to about macOS Sierra 10.12.4 or .5, the two waypoints marking system shutdown and bootup were SHUTDOWN_TIME and BOOT_TIME, in the eventMessage field. Those have now vanished.

Normally, the last event logged now has an eventMessage field containing
=== system wallclock time adjusted

The new startup banner is
=== system boot:
followed by the UUID of the startup event. Filtering or searching on part of that, such as system boot:, is therefore the most reliable way of locating the beginning of the boot process, at least until Apple changes it again. Make a note of the time of that log entry.

startup02

The next valuable waypoint is the announcement of the shutdown cause code, which should normally be 5. Negative codes are indicators of hardware problems, and positive of software issues – see this article. That is given in an eventMessage like
Previous shutdown cause: 5

The login process starts with an eventMessage from loginwindow of
Login Window Application Started
following which there is a series of entries from SecurityAgent which step through the login process. Once that is successful, accountsd will unlock and open keychains, although in recent versions of High Sierra the most useful information is now being censored with <private>. You can read more about startup in this article.

I have started to explore some of the differences in High Sierra in two articles here: this examines startup, and this more general matters. I haven’t encountered any problems examining High Sierra logarchives in Sierra, using tools which work via Sierra’s log command, such as Consolation and Woodpile.

System sleep and wake are rather harder to spot. Useful waypoints include the following entries from the kernel:
gIOLastWakeAbsTime:
with the system ticks immediately before sleep, and
PMRD: System Wake
marking the start of waking up. Another mark of the system becoming fully awake is when CTS logs
User Active
twice in succession.

You may also encounter
Wake reason:
which is valuable, as a reason should then be given for awakening. I have listed some that I have encountered in this article, which also covers hardware ACPI which you will encounter in the log.

Because the unified log was not designed for audit purposes, important events such as logging in and out are generally not particularly easy to spot. Although I have not yet looked at these in detail in High Sierra, the following waypoints from Sierra may help.

Successful login is marked by eventMessages containing
AuthenticationAllowed and
result: Success
and should contain the username. Once login is complete, look for
login proceeding and later
Changing status for uid 501 from logged out to logged in

Perhaps the most distinctive message after logging out is the foreboding
point of no return

Fast user switching is broadly similar, and it is worth watching for an eventMessage containing
AuthenticationAllowed completed:

Usernames are usually visible in Sierra, but I suspect in High Sierra these are now censored using the dreaded <private>. You can read more about logging in and out in Sierra in this article.

Tips

It is often very useful to be able to write a message from a shell script to the log, something which is still supported by the logger command tool, but in a very rudimentary way. Blowhole does that rather better, with a distinctive subsystem making its messages easy to identify, using a command like
blowhole -s "Script failed, error -2"
and it has several other useful options.

If you’re writing your own code in Swift and want to write to the log, I offer as an example the code used by Blowhole.

blowholesource

I have many articles here about the unified log and accessing it, which are listed in the Mac Problems item above. Among these are some useful filter terms for Sierra’s logs, most of which carry over to High Sierra. However, 10.13 has undergone some major internal changes which have dramatically altered log content in places.

Updates and installations

As I mentioned in the first article in this series, one of the few non-unified logs remaining is the installation log at /var/log/install.log, which still keeps a valuable record of softwareupdate and other installations. You can browse this using Console, and may find my GUI app SystHist (from Downloads above) a useful adjunct too.

Because this log remains in traditional format, Console provides regular string search, rather than the new system of predicates, which you may find welcome relief.

Security and audit

I have left until the end the topic which for many is the most important of all: how you can use the unified log as part of a network security and audit system. The plainest answer, even eighteen months after its release, is that you can’t, at least not in any obvious or straightforward way.

Unified log entries can only be collected on the Mac or iOS device itself. There is no sensible way that you can gather filtered log entries from a couple of dozen or more Macs on a server for monitoring and analysis. That anyone should ever want to do so doesn’t appear to have registered with Apple yet.

Using log stream with a complex predicate filter to cut it from being a high-volume firehose down to a steady trickle is possible, but to the best of my knowledge is not something that is practical at present. Given the design aims of the unified log, this is not really surprising. Trying to use it for security and audit purposes is a bit like fitting a Formula 1 engine to a domestic lawnmower: your grass is not going to get cut at all well.

I think the best way forward for security and audit systems, if you need to go beyond what is available in BSM (which still works), is to instrument macOS to generate quite separate logs for that specific purpose; if that was something that I wanted to do, I’d be talking to Patrick Wardle and the folk at Digita Security, for example.

The other Apple OSes

Another of the goals of the unified log is for commonality across all Apple’s operating systems – macOS, iOS, watchOS, and tvOS. As far as I am aware, the tools that I have discussed in this series should all work equally well with logarchives acquired from non-Mac devices. However I have not tested any of them myself.