Want to access log files directly, or analyse logarchives? Try Ulbow 1.2b1

There are only two ways that macOS can normally access the unified log: the active log in a Mac can be browsed using the log show or log stream commands or streamed in Console, or you can browse any macOS, iOS, watchOS or tvOS log collected in a special logarchive bundle. If you’ve got a collection of log files copied from another Mac or device but haven’t collected them into a logarchive, there’s been no supported method that I know of for turning them into a logarchive and accessing them using log show, Console, or even my free browsers Ulbow and Consolation.

This is a serious omission which has been present in macOS for the last three and a half years, since the introduction of the unified log. It hinders forensic investigations, system administrators, developers trying to fix bugs, and I suspect even Apple’s own engineers. If it isn’t the active log or a logarchive, then it’s been largely inaccessible.

A month ago I explained how you can manuallly convert log files into a logarchive of sorts, but if you’ve tried that you’ll know that it doesn’t always work. It’s reliant on an undocumented property list, Info.plist, which can make or break your ability to access the log contents.

This new beta-release of Ulbow tries to address that: in addition to supporting the official way of turning your Mac’s active log into a logarchive, it can now create logarchives from copies of the two key folders in /var/db, which could come from a backup or a simple copy from another Mac, device or disk.

Reversing the logarchive format isn’t in itself difficult, but it has changed over the period since the unified log has been in use. In the absence of any documentatiom or even clues from Apple, this makes it tricky to get right for all versions of macOS from 10.12 to 10.15. I can’t guarantee that the logarchives created directly by Ulbow will be compatible with Console or the log command, but they do seem to work reliably with both Ulbow and Consolation.

This new tool in Ulbow also appears resilient, to a degree at least, to missing and incomplete files, even whole folders at times. All it requires is the same structure that you’ll find in /var/db: one folder named diagnostics, which is structured to contain .tracev3 and other log files, and the other named uuidtext, which contains all the referenced UUIDs structured into folders with names from 00 to FF. Ulbow will then copy those files and folders into a new logarchive bundle, and add the Info.plist file which should allow it to access that logarchive as a whole, or individual .tracev3 files within it, another handy feature of both Ulbow and Consolation.

ulbow12b104

There’s one interesting side-effect. When Ulbow uses the supported methodd of log collect, this has to be called through an AppleScript, which isn’t allowed to save the logarchive to a removable disk, in accordance with Catalina’s privacy protection. That’s even true if you give all components Full Disk Access. When Ulbow creates its own logarchives, no admin password is required to obtain elevated privileges, and you can save the logarchive to any folder for which you have regular permissions, including removable disks if you wish. That privacy protection doesn’t exist when an app uses its own code for this task, instead of relying on macOS.

Because this is the first beta, it features a verbose mode which explains which folders and files it has copied to where, and any errors which have occurred in the process.

This Logarchive Tool provides two additional buttons for analysing any logarchive file, however created. Catalogue lists all the main .tracev3 files found within it, giving the datestamp that each was opened and when it was closed, its size in bytes, and the period of collection.

ulbow12b105

The Analyse button looks in the statistics files maintained in High Sierra and later and provides summary figures for each of the main .tracev3 files within a logarchive. Most interesting within these is a breakdown by frequency of log entries within that individual log file, in terms of the processes responsible over that collection period. These are available in formatted text, as shown here, or in CSV format for easy import into spreadsheets and databases.

ulbow12b106

Ulbow version 1.2b1 is now available from here: ulbow12b1
from Downloads above, from its Product Page, and via its auto-update mechanism.