When Apple introduced the Unified log in Sierra, I marvelled at how long it kept its records: at that time, the log maintenance service logd typically rolled log files every 10-12 hours, giving most users around 20 days of full log records at any time. By Monterey, so many macOS processes were writing so frequently that the duration of log records had fallen to around five days. In Ventura, it’s even shorter, as some have discovered the hard way.

Lost logs

Unless you’re running Ventura and use a third-party security product using Endpoint Security to support this feature, the only way to check XProtect Remediator scan reports is to inspect their entries in the log, for instance using my own free XProCheck. As those scans tend to occur once a day at present, checking back a couple of days in the log should catch at least one set of scan reports. But some have reported that they can’t find any, apparently because their logs don’t even go back for 24 hours.

When I first heard this, I found it hard to believe, after all the Unified log has been so good at retaining log records in the past. But not now: when I checked my own production iMac Pro, it only retains around 1.7 days of logs now it’s running Ventura 13.1. On a bad day, even my Mac might not have long enough log records available to catch its daily XProtect Remediator scans.

Log roll

Traditional text logs roll on a calendar basis, and normally retain at least five days of entries at any time. From the outset, the Unified log has behaved differently, as you can see if you look at its tracev3 files in /var/db/diagnostics/Persist. Although it does store log files in other folders there, particularly Special and Signpost, those in the Persist folder form the bulk of its records, and contain the most important entries, such as XProtect Remediator’s scan reports. Rather than removing expired tracev3 files by age, logd sensibly works to keep the space occupied by the log files in the Persist folder down to around 525 MB.

If your Mac is writing copious entries in its log, this means that the records in the Persist folder will cover a shorter period than those on a Mac making fewer entries.

Each complete tracev3 file in Persist is around 10.4 MB in size, but there’s no simple way to estimate how many log records that represents. This iMac Pro currently writes log entries at up to 12,000 per minute, and logd rolls the log file every 10-100 minutes. This suggests that each 10.4 MB log file contains at least 100,000 entries, usually considerably more, and in the current Persist folder are well over 5 million log entries, perhaps as many as 20 million in all.

The good news is that all these log entries take surprisingly little space on disk: Apple’s proprietary tracev3 format is binary and compressed. Even at these high rates of log usage, maintaining the Unified log involves writing less than 1 GB per day, and the whole /var/db/diagnostics folder occupies less than 1 GB.

Log length and XProCheck

Although you can rummage around checking when the first tracev3 file in your Mac’s /var/db/diagnostics/Persist folder was created, to work out how far your logs go back, this is a perfect task for code to do for you. I therefore have a new version of XProCheck which reports how far back log records go; if that’s less than a day, then you will know that you’ve got to be lucky to catch a set of XProtect Remediator scan reports when you check for them.

XProCheck version 1.3 is now available from here: xprocheck13
from Downloads above, from its Product Page, and via its auto-update mechanism.

Implications

Although the length of the log record has been falling steadily over the years since its introduction, Ventura appears to have brought a step change for the worse. This affects all utilities that rely on log records, including T2M2, Mints and of course Ulbow. Over the next week or two, I will be investigating this further, to discover whether this is worse on Intel Macs (as appears here), why it’s occurring, and what a user can do to reduce the rate at which unnecessary entries are written to the log. Thankfully, there is a mechanism provided to do that, but at the moment I don’t know how practical or effective it might be.

Even if you aren’t particularly interested in checking XProtect Remediator scan reports, you may find XProCheck’s new feature of value. I’d be interested to know of any log records shorter than a couple of days. Of course this applies most to Macs that are left running and not asleep all the time.

I will also be adding similar features to assess the length of the log record to T2M2, Mints and Ulbow.