Essential Reading: *OS Internals, Vol I, User Space, by Jonathan Levin

The TL;DR is simple: skip to the bottom, click on the link to Amazon.com, and buy this book. You need it.

In an ideal world, where [insert here your favourite Brexit/White House/free beer daydream], Apple would now have expanded the iBooks making up the Inside Macintosh series to more than fifty volumes. Back here in reality, we have only two people who have been brave/foolhardy enough to try to attempt to document Mac OS X / OS X / macOS / iOS in the broad: Amit Singh, and Jonathan Levin.

Sadly, Singh’s monumental book proved a one-off in 2006, so Levin almost started with a clean slate with his Mac OS X and iOS Internals, To the Apple’s Core when it was published in 2013. Despite running to over 800 pages, there were many topics which it did not even touch on. And in those four years, macOS and iOS have changed greatly too.

This book is one of three volumes which Levin intends to cover macOS and iOS; a second, perhaps inevitably volume III and covering security, was published a year ago, and already has quite extensive updates. It too is available from Amazon.

For most Mac users, this is the crucial volume, as it covers most of the topics that you need to understand in order to tackle the majority of macOS and iOS problems. You can browse its full contents at the book’s website; it starts with the architecture of macOS/iOS and system calls, and takes you right through to hangs and crashes, in almost 450 pages.

There are copious tables, listings, and many superbly-produced diagrams, although it may help if you’re a bit myopic as some of the print on them is a little on the small side even though the book is a sensibly larger format.

Given the scale and complexity of macOS and iOS, this single volume cannot provide more than a thorough overview. To give you an idea of the level of detail which Levin achieves, on extended attributes (xattrs) he explains how resource forks are implemented as xattrs, lists almost twenty of Apple’s most important xattrs and explains their use, and goes into further detail on the ‘invisible’ xattr marking transparent file compression.

In the great majority of cases, this should give the reader sufficient information for everyday purposes, and perhaps prompt more detailed exploration where necessary. At the end of each chapter Levin lists some excellent references, where appropriate in Apple’s developer guides.

If you’re trying to get your head around recurrent crashing of an mdworker process, for example, page 91 explains what they are and do, how they’re launchd, the four different worker types, the .mdimporter plugin system (which is detailed in property list form), differences in iOS, and how fsevents prompts automatic indexing.

This is a unique and completely up-to-date reference which every advanced user, developer, sysadmin, and all support staff, should have at their side, even when asleep.

It has two faults, though.

The first is one word: more. This book is the only place which I know that documents those few xattrs which it does, but it doesn’t, for example, detail how Finder tags work. The other day, someone emailed me asking how he could add Finder tags to files at the command line, and finding out exactly how to do that involved searching and experimenting for more than a little while. In the end, he came up with his own solution which works, but – unlike that now documented on this site – cannot add two tags to the same item.

This book is a superb start in the right direction, but it simply cannot provide sufficient detail to tackle many everyday problems. We need a whole team of Jonathan Levins.

The other issue is more serious, and results from this book not being published by a major publishing house. One of the few benefits which that should have brought (although publishers are lamentably cutting back on this now) is an index.

Indexing books well is a tedious and uninteresting task which few authors find time for. Although some publishing software has good support for the production of indexes, you really have to use a heavyweight platform such as TeX/LaTeX and invest in a lot of effort in order to make a good index, and of course the moment that you add more content to page 56, every subsequent page number and most items in the index have to be updated and re-issued.

Levin compensates quite well for this with a highly systematic layout of contents, and a detailed listing in the table of contents, although that only takes you to the start of each chapter. My example about the crashing mdworker, for example, could be frustrating for someone who doesn’t know that mdworker is part of Spotlight. Even when you do, the table of contents lists Spotlight as one topic within a chapter of over forty pages, so you will need to thumb around a bit before you hit the right page.

But if you don’t know much about the topic that you’re looking for, you could be reduced to leafing through dozens of pages in pursuit.

This is made worse by the fact that (again for very understandable reasons), Levin does not offer an electronic version of this book. Searching those can be difficult enough (try searching for how to copy in a book about developing for macOS!), but the reader doesn’t have that option with a printed book.

Perhaps the best solution would be to build an online index, something that I am sure others (including myself) would be more than happy to help with.

If you still haven’t ordered your copy, now’s the time: as I said at the beginning, buy this book. You need it.

*OS Internals, Volume I, User Space, is by Jonathan Levin, and published by TechnoloGeeks.com. Its ISBN is 978 0 9910 5556 2, and you can order it from Amazon US. Complete contents and other details are here. It costs $75.