How macOS manages tasks on cores: a draft

I am here going to try to bring together what I have seen, performed, and read about the various interlinked systems which macOS Sierra uses to manage the scheduling, dispatch, and execution of tasks on the available processor cores.

My aim is to produce the diagram which developers and advanced users have needed since the introduction of Grand Central Dispatch (GCD). The opinions and draft diagram which I offer here are just that: opinions and draft. Please engage in discussing them so that we can together evolve the ideas to come closer to reality.

Developers only see the APIs made public by Apple. Advanced users only see what happens in the logs, and in the apps and tools that they use. Much of what has been published is based on Apple’s original description of the GCD API, together with some later additions.

We know that Apple uses private APIs for some of its own tools, such as Time Machine backups controlled via backupd-auto, as they achieve things that are not available in the public APIs. We also know that there is a lot which is clearly present and active in Sierra – Duet Activity Scheduler, in particular, has interfaces to a very broad range of sensors, notifications, and features – but which is currently undocumented.

I see three broad ways in which apps and command tools can submit tasks for concurrent or asynchronous execution.

GCD

The traditional GCD approach is to package the task up in one of a range of different execution units, including NSBlockOperation. GCD then provides local queues and access to global queues to which those tasks can be added, for execution on available cores. This is performed at low level, thus none of the structures or actions of GCD are reflected in the unified log.

So, for example,
DispatchQueue.global().async {
let task = Process()
task.launchPath = "/usr/local/bin/blowhole"
task.arguments = []
let outPipe = Pipe()
task.standardOutput = outPipe
task.launch()
task.waitUntilExit()
}

will put that closure as an asynchronous task into the global dispatch queue, for scheduling and dispatch by GCD. The only log entry seen will be that resulting from executing the tool blowhole. When that is called by another task in an app, the caller is likely to complete before that task does.

This was originally documented in the Concurrency Programming Guide, which is now hopelessly out of date but is presumably a fairly accurate reflection of GCD back in 2011-2012.

XPC Activity

Another public route to a similar result is for the app to use XPC Activity directly to add a task to Centralized Task Scheduling (CTS). I haven’t yet attempted this, but have observed some macOS tasks which appear to work in this way. It was originally documented in the Daemons and Services Programming Guide, which was last revised substantially in 2011-12.

That guide states that XPC services are managed by launchd, which is interesting, as activities which are run by launchd appear to be scheduled very precisely in time, unlike activities which are scheduled by other systems under discussion. I perhaps need to explore NSXPCProxyCreating, methods for creating new XPC proxy objects.

Apple’s other documentation, notably its most recent Energy Efficiency Guide for Mac Apps, refers to activities created through XPC Activity as being managed by CTS. For the moment, I assume that is true, at least for tasks which are deferred using XPC Activity.

CTS & DAS

The third route is to use the relatively new NSBackgroundActivityScheduler, which has a basic public API, appears to be used by macOS tools such as Time Machine’s automated backups and others, and involves not just CTS but Duet Activity Scheduler (DAS), which is not documented at all. I have traced the exchanges reflected in the unified log for the management and dispatch of these activities in a series of three articles here.

In code terms, this is simplest to achieve using a completion, such as
let activity = NSBackgroundActivityScheduler(identifier: "co.eclecticlight.GCDlaunch.check")
activity.repeats = true
activity.interval = 60 * 60
activity.schedule() { (completion: NSBackgroundActivityScheduler.CompletionHandler) in
let task = Process()
task.launchPath = "/usr/local/bin/blowhole"
task.arguments = []
let outPipe = Pipe()
task.standardOutput = outPipe
task.launch()
task.waitUntilExit()
completion(NSBackgroundActivityScheduler.Result.finished)
}

While direct use of XPC Activity and CTS can happily handle activities which are run frequently, NSBackgroundActivityScheduler with DAS and CTS work best with one-off activities, and those which repeat at intervals of thirty minutes or more. CTS and DAS are loquacious in the unified log, making it easy to keep a watch on them. However, most of the useful information is censored from entries by DAS, making it necessary to cross-correlate its entries with the far more informative entries by CTS.

My first draft of the diagram illustrating these systems and their interactions is here:

GCD4

And in PDF from here: GCD4

Names

Finally comes the controversial issue of terminology. Apple has, as yet, not released any name for this part of macOS. Although ‘strict’ GCD is only a part of it, it appears to be the final common system. It is also the only part of this which has any significant exposure to most advanced users. It makes sense to me to refer to the whole system, embracing GCD, CTS, and DAS and using (as a non-exclusive service) XPC, as GCD in a broad sense. If you know of a better term – preferably Apple’s – please let me know. We can then perhaps synchronise our activities and writing rather than wrangling over names.