There are only two ways that macOS can normally access the unified log: the active log in a Mac can be browsed using the
log show or
log stream commands or streamed in Console, or you can browse any macOS, iOS, watchOS or tvOS log collected in a special logarchive bundle. If you’ve got a collection of log files copied from another Mac or device but haven’t collected them into a logarchive, there’s been no supported method that I know of for turning them into a logarchive and accessing them using
log show, Console, or even my free browsers Ulbow and Consolation.
This is a serious omission which has been present in macOS for the last three and a half years, since the introduction of the unified log. It hinders forensic investigations, system administrators, developers trying to fix bugs, and I suspect even Apple’s own engineers. If it isn’t the active log or a logarchive, then it’s been largely inaccessible.
A month ago I explained how you can manuallly convert log files into a logarchive of sorts, but if you’ve tried that you’ll know that it doesn’t always work. It’s reliant on an undocumented property list, Info.plist, which can make or break your ability to access the log contents.
This new beta-release of Ulbow tries to address that: in addition to supporting the official way of turning your Mac’s active log into a logarchive, it can now create logarchives from copies of the two key folders in /var/db, which could come from a backup or a simple copy from another Mac, device or disk.
Reversing the logarchive format isn’t in itself difficult, but it has changed over the period since the unified log has been in use. In the absence of any documentatiom or even clues from Apple, this makes it tricky to get right for all versions of macOS from 10.12 to 10.15. I can’t guarantee that the logarchives created directly by Ulbow will be compatible with Console or the
log command, but they do seem to work reliably with both Ulbow and Consolation.
This new tool in Ulbow also appears resilient, to a degree at least, to missing and incomplete files, even whole folders at times. All it requires is the same structure that you’ll find in /var/db: one folder named
diagnostics, which is structured to contain .tracev3 and other log files, and the other named
uuidtext, which contains all the referenced UUIDs structured into folders with names from 00 to FF. Ulbow will then copy those files and folders into a new logarchive bundle, and add the Info.plist file which should allow it to access that logarchive as a whole, or individual .tracev3 files within it, another handy feature of both Ulbow and Consolation.
There’s one interesting side-effect. When Ulbow uses the supported methodd of
log collect, this has to be called through an AppleScript, which isn’t allowed to save the logarchive to a removable disk, in accordance with Catalina’s privacy protection. That’s even true if you give all components Full Disk Access. When Ulbow creates its own logarchives, no admin password is required to obtain elevated privileges, and you can save the logarchive to any folder for which you have regular permissions, including removable disks if you wish. That privacy protection doesn’t exist when an app uses its own code for this task, instead of relying on macOS.
Because this is the first beta, it features a verbose mode which explains which folders and files it has copied to where, and any errors which have occurred in the process.
This Logarchive Tool provides two additional buttons for analysing any logarchive file, however created. Catalogue lists all the main .tracev3 files found within it, giving the datestamp that each was opened and when it was closed, its size in bytes, and the period of collection.
The Analyse button looks in the statistics files maintained in High Sierra and later and provides summary figures for each of the main .tracev3 files within a logarchive. Most interesting within these is a breakdown by frequency of log entries within that individual log file, in terms of the processes responsible over that collection period. These are available in formatted text, as shown here, or in CSV format for easy import into spreadsheets and databases.