If you’ve ever accessed the unified log, you might think that the last thing you’d ever want is more detail. It’s heaving with incessant chatter between many different parts of macOS. When I’m trying to look at logs, I often sneak down early in the morning in the hope that I might catch it unawares. No chance, though – the log is just as busy at 0500 as much later in the day.

While there’s never any shortage of log entries, the ones that you do want are either rendered meaningless with <private> or missing altogether – and that’s what this article is about: how to get the log messages that you want.

Removing <private>

I’m a great believer in privacy, and in Apple’s exemplary protection of it in the unified log. But there are times when we need to know what the log normally censors using <private>.

If you’re running macOS 10.12 to 10.14, you can disable this censorship using the command
sudo log config --mode 'private_data:on'
and
sudo log config --mode 'private_data:off'
turns it back on again.

This changed with Catalina. Provided that you’re running 10.15.3 or later, privacy can be removed by installing a profile provided by George Garside on his blog. This is simple to install: just double-click the profile. To remove it and restore normal protection, select the profile in the Profiles pane and click on the – tool below.

Because the log doesn’t store censored data while privacy protection is enabled, the only entries which will have such data exposed are those written to the log from the moment that the profile takes effect until you remove it.

Network diagnostic logging

You’ll have noticed that the unified log doesn’t contain much information about network traffic. If you want to look at that, Apple recommends using CFNetwork diagnostic logging, and details how to do that in your own code. This is particularly valuable, as it can contain decrypted TLS data, but as described there is only available within code which you control.

It was the prolific Jeff Johnson who described how to use that same diagnostic logging in macOS. To enable it, enter the following command in Terminal:
sudo launchctl setenv CFNETWORK_DIAGNOSTICS 3
then authenticate. Existing processes won’t provide diagnostic information, though. If you need one to do so, stop it in Activity Monitor, and when macOS restarts it, diagnostic logging should then be enabled. He has used this to observe traffic from syspolicyd, for example.

When you’ve finished, use the command
sudo launchctl setenv CFNETWORK_DIAGNOSTICS 0
to restore normal logging.

Logging profiles

Perhaps the least used feature in unified logging, these are property lists in either the Processes or Subsystems folders in /Library/Preferences/Logging, with system-level profiles stored in /System/Library/Preferences/Logging. Apple’s brief documentation of them is directed at developers wishing to control logging for software they’re developing.

System-level profiles vary widely across different subsystems, and you can modify those to your advantage to adjust the detail shown in the log. For example, to see current settings for log entries for the subsystem com.apple.TimeMachine, use the command
sudo log config --status --subsystem com.apple.TimeMachine
and you should see the response
DEBUG PERSIST_DEBUG
meaning that log entries are made at a debug level, the most comprehensive, and that all log entries, even at a debug level, are retained in the Persist tracev3 files rather than being stored ephemerally in memory.

You can change those system settings using commands of the form
sudo log config --mode "level:debug" --subsystem com.apple.TimeMachine
Further details of the options available are given in Apple’s article and in man log, under the config command. It’s also worth browsing the system profiles.

I will look in more detail at logging profiles in a future article.

Example

To see what effect removing privacy and enabling network diagnostic logging has on logs during the first run of an app, I launched quarantined versions of two of my own apps, one with normal log settings, the other with those details enabled. I didn’t force a restart of syspolicyd or any other system service, though.

Detailed logging revealed the full process of setting up the app’s translocation folder by diskarbitrationd, but that service’s other copious entries only added clutter.

Details of the notarization ticket check were revealed not by enabling network diagnostic logging, but by disabling privacy. For example, here are the entries for that check for one of the apps:
0.833908 syspolicyd Newer ticket (1589816974) present in db, ignoring ticket (1589816974)
0.834066 syspolicyd looking up ticket: {length = 20, bytes = 0xaeee02629dbe1c21fd2f9a0ef5ec1c8c20d2161a}, 2, 1
0.834095 cloudkit record fetch: https://api.apple-cloudkit.com/database/1/com.apple.gk.ticket-delivery/production/public/records/lookup, 2/2/aeee02629dbe1c21fd2f9a0ef5ec1c8c20d2161a
0.834128 cloudkit request cache info: PFT-com.apple.gk.ticket-delivery-production-2/2/aeee02629dbe1c21fd2f9a0ef5ec1c8c20d2161a, max-age=300
1.023246 CFNetwork Task <956401B7-A6A8-41C8-8DBD-A8BBFCF2B339>.<5427> received response, status 200 content K
1.023421 CFNetwork Task <956401B7-A6A8-41C8-8DBD-A8BBFCF2B339>.<5427> summary for task success {transaction_duration_ms=189, response_status=200, connection=81, protocol="http/1.1", domain_lookup_duration_ms=1, connect_duration_ms=61, secure_connection_duration_ms=48, request_start_ms=63, request_duration_ms=0, response_start_ms=188, response_duration_ms=0, request_bytes=618, response_bytes=2639, cache_hit=0}
Those match the request headers reported by Jeff Johnson using network diagnostic logging. If you want to check whether notarization checks are being performed, it therefore appears sufficient to remove privacy from the log.

Although not as detailed as the headers revealed using network diagnostic logging, removing privacy was also successful in giving good insight into the HTTPS connection made by the app to check whether it was up to date. For example:
3.488957 CFNetwork Task <5C5EEB52-9940-4EC8-AFB4-28132D9A37BE>.<1> {strength 1, tls 8, ct 0, sub 0, sig 0, ciphers 1, bundle 0, builtin 0}
3.490363 libnetwork.dylib nw_connection_create_with_id [C1] create connection to raw.githubusercontent.com:443
3.742066 CFNetwork Task <5C5EEB52-9940-4EC8-AFB4-28132D9A37BE>.<1> summary for task success {transaction_duration_ms=254, response_status=200, connection=1, protocol="http/1.1", domain_lookup_duration_ms=1, connect_duration_ms=81, secure_connection_duration_ms=68, request_start_ms=86, request_duration_ms=0, response_start_ms=254, response_duration_ms=0, request_bytes=338, response_bytes=1649, cache_hit=1}
3.742165 CFNetwork Task <5C5EEB52-9940-4EC8-AFB4-28132D9A37BE>.<1> finished successfully
which shows the check against my app version database on GitHub.

Removing privacy alone is therefore a good way of gaining deeper insight into exchanges which are normally obscured by <private>, including notarization checks and network exchanges. If that provides insufficient information, it may well be worth enabling network diagnostic logging.