In the first article, I showed how a third-party background activity is run using Centralized Task Scheduling (CTS) and Duet Activity Scheduler (DAS). In the previous article, I repeated that analysis for the most obvious scheduled background task in macOS, Time Machine backups. Amazingly, the systems were used in an identical manner, although there are likely to be differences in the extent of access provided through the public interface open to third-party developers.

The next big question is how those systems behave when things go wrong: pathology often gives insights into normal behaviour which we might never otherwise gain.

Earlier this year, I was (un)fortunate enough to experience problems in which Time Machine backups would suddenly become highly irregular. I was able to examine what had happened using Consolation to browse my log, and here explain what I found.

On this occasion, regular backups had stopped occurring during the night. Prior to this, they had run normally, with the same sequence of events occurring in DAS and CTS as shown in the last article. The last backup to be made using that was initiated at 02:14:27, following DAS rescoring its activities. The backupd-auto task came out of that with a DecisionToRun of 1, which was handed over to CTS to deal with in the normal way.

That backup completed very quickly, at 02:15:06. CTS then submitted the next DASActivity entry for another backup to take place between 03:05 and 03:35, and DAS continued running its activities until it made a final log entry at 02:15:10.

From that entry until 06:59:56, a period of over four hours, although my Mac did not sleep (that is blocked), there was not a single log entry from DAS, CTS, or any component of Time Machine.

The next log entry, at 06:59:56, was from DAS, which appeared to have been experiencing a problem, as it did not perform a normal rescoring of activities, but reported that it was “evaluating 72 activities” based on triggers, rather than the normal scores. It then confirmed that backupd-auto should run, and CTS started its normal sequence of entries which resulted in the first backup being made.

I have once again put this into graphical form, thanks to Scapple:

and a downloadable PDF is here: GCD3

This failure had apparently occurred in DAS, not CTS. For some reason, almost certainly a bug, and quite probably a memory leak or other issue which occurs after long periods of heavy use, DAS had lost its ability to schedule and dispatch normally. To me, this is fairly conclusive evidence that background and other activities which are being managed by macOS using these systems are primarily being controlled and dispatched by DAS, the Duet Activity Scheduler.

A problem of nomenclature

The conventional stance, which is supported by Apple’s out-of-date documentation, is that Grand Central Dispatch is neither grand nor particularly central, in consisting entirely of one relatively small system, libdispatch, which provides certain features to support concurrency in apps. However DAS is not mentioned anywhere in Apple’s documentation, and CTS is described fairly sketchily in one of the three main reference documents which Apple provides to cover this whole area.

libdispatch (‘old’ GCD) provides facilities which include the management and dispatch of code threads or Operation Objects, which can be distributed optimally across the processor cores available in a Mac. In particular it manages multiple dispatch queues.

Centralized Task Scheduling (CTS) allows a developer to assign criteria for when a task should be performed, so that scheduled and other tasks can be deferred when necessary, for example when a laptop is running from its battery rather than mains power.

I hope that I have shown that Duet Activity Scheduler (DAS) manages heuristically a pool or queue of tasks or activities, and determines when to dispatch them for execution.

We have three systems which work very closely together, managing overlapping tasks in remarkably similar ways. Although libdispatch is normally seen as working with ‘small’ activities, and CTS and DAS with those which are more substantial, it makes good sense to embrace these within a functional unit, and under a common name which reflects their grand control over so much of macOS, their centralised facilities for all different types of apps and software, and their primary role of dispatching tasks/Operations/more to the processor cores.

Why, a name like Grand Central Dispatch might be very appropriate in the circumstances.