It’s easy to acquire log data: a Mac running Sierra often stores thousands of entries every minute that it’s in use. The difficult task is discovering how to use all that information to best effect. After the last few weeks, I’m convinced that powerful search tools are not the answer.

Looking at Time Machine backups is a clear case in point. The log show command and Consolation provide straightforward tools to filter relevant entries in Sierra’s new unified log, so that you can read those messages. An expert eye cast over them should be able to spot errors and abnormal behaviour, and tell you what’s gone wrong.

You could, if sufficiently interested, export those and perform further analysis, for example looking at variation in the time intervals between each automatic backup, to find evidence of irregularities which might reflect a problem in the dispatching systems in macOS. You’d then have to go back to the original log and look at other subsystems (like DAS and CTS), and correlate those with Time Machine’s entries.

Even when you know what you are doing, and are used to doing it, this quickly becomes tedious. What we actually most need is a tool which performs ‘smart’ analysis for us, and that is exactly what The Time Machine Mechanic now does.

Cast your mind ahead a few years and imagine having a tool which could analyse your Mac’s log for a whole range of different issues. It could explain why you’re having to keep entering your password to unlock your login keychain, whether there’s evidence of malicious activity, and a whole load of other issues.

Currently, the only major user of the entries in our Macs’ logs is Apple. Whenever there’s a serious problem with an app or with macOS, unless we have disabled it, log extracts are sent to Apple. Report a bug through Apple’s Bug Reporter (a feature intended for developers) and you are likely to be asked to send hefty log extracts, often through features such as sysdiagnose.

I presume that Apple’s engineers must have better analytical tools than Console and the log command. If they haven’t, it’d be an extraordinary under-investment on Apple’s part; if they have, then it makes me wonder why Apple doesn’t make them available to others, particularly to developers, system administrators, support teams, and forensic specialists.

So long as we only had Console and the log command, those outside Apple were unable to do much of value with the new unified log. Although Consolation is a step forward in opening access, and its ability to convert log entries for analysis in spreadsheets and other apps is valuable, its approach remains quite traditional. You have to know what you’re doing, pick the right predicate filters, and proceed in the knowledge of what you’re looking for, and how to interpret the messages which are shown.

It has been the last couple of weeks which have been more of an epiphany. The Time Machine Mechanic (T2M2) has shown me that automated log analysis can transform the diagnosis of problems. You don’t have to know what a predicate filter is, have a clue what you’re looking for, or to have read a log message ever before. It provides information, such as the duration of each backup and the intervals between them, which would otherwise require elaborate manual analysis of the log.

In its latest version, it also performs ‘smart’ diagnosis, deciding whether or not to obtain additional log extracts, and the period which they should cover. If your automatic backups have been occurring at regular, hourly intervals, it does not delve any further. But if those intervals have been irregular and prolonged, T2M2 traces the chain by which backup activity should be dispatched.

T2M2 still has a long way to go. As more users gain experience with it, I will improve and refine its techniques. But it would be misleading to refer to them as any form of artificial intelligence (AI): its rules are coded in using simple conditionals.

It would also be misleading at this stage to think that AI has any significant role to play in the development of future ‘smart’ tools for log analysis. AI can certainly play an important part in the analysis of network, server, website, and similar ‘traffic’ logs. But in Sierra’s log the key to many issues is detailed analysis of the content of individual – and very specific – log messages. The importance is in their meaning.

Over the next few weeks, I’m going to be looking to the future of tools like T2M2, and how to make them easier to craft. Unless of course, by some miracle, Apple announces something even better at WWDC. Although I won’t be holding my breath, I will be watching very carefully.