Another step forward in browsing the log with Woodpile 0.8a1

Doing anything blindfold with one arm tied behind your back is a challenge – something that we don’t enjoy at the time, but which greatly increases the feeling of satisfaction (and relief) afterwards. So it is that I can very proudly offer the next release of my new-generation log browser, Woodpile.

The problem that I have been battling with is the unified log in Sierra and High Sierra, abounding in data but so extensive and dense that smarter people go off and do something different. Its TraceV3-format log files are a rich source of log entries which go back much further than normal Unix logs, but they’re locked in an undocumented, proprietary and compressed format.

When performing analysis on a log file, for example looking at the frequency of occurrence of entries, you need to know the start and end time for that section of log. Without that, you can only guess on the basis of excerpts obtained. Yet the only means of accessing the unified log, the command tool log, has no direct way of providing a log file’s start and end.

I seriously thought about asking it to dump the entire file, so that I could inspect the very first and last entries, but even when compressed these log files are normally over 10 MB in size, so that would have been an act of desperation.

If we couldn’t know the actual start and finish times, it would be almost as good to let the user set two or more views of that file to a common time base. Say that one window, looking at the kernel process, had log entries covering the period from 1000 to 1200, and another looking at Safari perhaps covered 1030 to 1130. If the user can set both of them to 1000 to 1200, then they would at least be able to view comparable data, and the bars in the chart would line up properly.

I toyed with different ideas of how to make that accessible, such as copying and pasting the period between different windows, or using a tool to ‘link’ them. All were cumbersome, confusing, and quite unnatural.

In the end, I decided that the best answer was a popup menu, which offered the periods currently being used in other windows.

woodpile84

The concept is simple: each time that you set the period in one window, that should be added to the menu on all the others. In the example above, you’d have the kernel set automatically to 1000-1200, and Safari at 1030-1130. You could then decide whether to switch Safari to the broader view of 1000-1200, or look more closely with the kernel set to 1030-1130.

As I will explain in gory code next week, implementing that has taken a bit of ingenuity. Each window that you see in Woodpile is actually a different document. Those documents cannot ‘see’ one another directly, so it has been a fascinating task to ensure that when one window changes its period setting, that is added to the popup menu on every other window.

I have also taken the opportunity to fix some embarrassingly silly bugs in Woodpile’s various calculations, which could on occasion suffer errors such as division by zero. Previous versions can sometimes crash on those, although they seem to occur relatively infrequently. This new version should prove more robust in those and other internal respects.

This new release is here: woodpile10b1
and in Downloads above.

My attention is next turning to making the bar chart more informative, without cluttering it up. As Woodpile is an exploration of this new approach to looking at a new type of log, I am not sure whether there are many significant additional features to be added before it goes into beta-release. One which I do want to offer is a checkbox option to show all log entries. I am also wondering whether to switch to Consolation 3’s custom log styles, instead of the current plain syslog style.

Your comments and thoughts are most welcome.