hoakley May 13, 2017 Macs, Technology

How macOS runs background activities: 1 From within an app

Every few minutes, or even more often, your Mac runs tasks in the background. The most obvious for many users is the making of Time Machine backups every hour. But that is one of dozens – typically more than seventy – different background activities which take place, some every few minutes, others only once every now and again.

Because macOS is something of a mongrel, with various flavours of Unix at its heart, and has roots in both the original Mac operating system and NeXT, it has several different mechanisms for running and managing concurrent, background, and scheduled tasks. Some are quite well-known and described in many different sources, such as LaunchAgents and LaunchDaemons, which rely on launchd, the master task launcher in macOS. But most of the background activities which are run by macOS itself, and closely-fused apps like Safari, use different mechanisms.

This article, and its subsequent parts, attempt to explain on the basis of evidence how macOS is running and managing its own concurrent, background, and scheduled tasks.

Creating and scheduling background activity

In NSBackgroundActivityScheduler, Apple provides a mechanism by which any app can create an activity which it then hands over to macOS to manage in the background. These include one-off, on-demand, and repeated activities. This is how my own app DispatchRider works: it packages up the code to run a command tool in a completion, and hands it over to macOS to deal with.

This package is then registered in Centralized Task Scheduling (CTS) as what it terms a DASActivity, and is added to the list of activities which is managed by Duet Activity Scheduler (DAS).

Running background activity

DAS maintains a scored list of background activities which usually consists of more than seventy items. Periodically, it rescores each item in its list, according to various criteria such as whether it is now due to be performed, i.e. clock time is now within the time period in which CTS calculated it should next be run.

If its CurrentScore exceeds the ThresholdScore, and is closer to 1.0, then DAS sets the DecisionToRun for that activity to 1, which is the signal to CTS to run the activity. If the CurrentScore is below the ThresholdScore, then its DecisionToRun is set to 0, and the activity is left to the next rescoring.

With DAS giving an activity the instruction to run, that action is passed to CTS, which writes in the log that “DAS told us to run” the activity. It changes the activity’s state from 1 to 2, and initiates inter-process communication by XPC to the running activity.

When the activity is complete, its state is changed from 2 to 5, XPC activity finished, and CTS then works out the next time period in which that activity should be run, if it is due to repeat. This is then passed back to DAS for its list of activities and their scores.

Removing background activity

When the app which registered the activity in the first place decides that the activity is no longer required, it signals this through activity.Invalidate(), back to CTS. There it is unregistered, its state changed from 1 to -1, and DAS is told to remove it from its list of activities.

I have summarised these events and processes in the following diagram, drawn using Scapple:

If you want a copy in PDF, you can obtain it from here: gcd01a

Is this Grand Central Dispatch?

To understand what is going on, in the absence of any coherent account from Apple, we have to examine its other descriptions of related parts of macOS. As that documentation is aimed at developers, it primarily addresses how they can build concurrency into their apps.

Concurrency within applications allows them to make the best use of multiple processor cores, to maintain responsiveness to the user, and to cope with jobs which take noticeable lengths of time. Apple’s Concurrency Programming Guide (last updated in December 2012) details recommended approaches. These have now converged on the use of Operation Objects and Queues, implemented in NSOperation and its relatives, which Apple describes as being part of GCD, “a technology for executing asynchronous tasks concurrently”.

Apple provides more recent insights into these matters in its Energy Efficiency Guide for Mac Apps (last updated in September 2016 or March 2017 perhaps). This covers the prioritisation of work at app and task levels using NSOperation and relatives, as part of GCD.

Centralized Task Scheduling (CTS) and GCD are then advocated as means of managing background activity. CTS is described as a set of interfaces which “allow you to designate criteria for when a task should be performed, such as when a user plugs the computer into power or when the system is not performing higher-priority tasks.”

Later, Apple explains another programming interface, XPC Activity, which “can be used to request centralized scheduling of discretionary tasks in your app.” This is part of a much more general lightweight mechanism for inter-process communication known as XPC, which for once doesn’t seem to be a simple acronym. In its Daemons and Services Programming Guide, Apple explains how XPC Services are “integrated with Grand Central Dispatch (GCD) and launchd“, but makes no mention of CTS.

As far as I can see, no Apple developer document uses any abbreviation or term which might refer to Duet, Duet Activity Scheduler, or DAS. Equally, although Apple mentions GCD in many different sections of documentation, there is no named part, service, etc., within macOS which identifies with GCD.

I have shown how an app running background activities, which fall within the scope of the facilities of GCD, uses two named systems, DAS and CTS, in a close and highly interconnected manner. Until such time as Apple makes its terminology clearer, it seems logical to assume that DAS and CTS are at least functionally part of GCD. If you can think of a better suggestion, please make it.

6Comments

Add yours

1

Tony on May 13, 2017 at 8:58 pm

At the risk of oversimplifying, I had assumed that GCD and CTS/XPC were largely orthogonal.

I think GCD is an abstracted way of creating threads to allow multi-processing (and hence efficient use of multiple cores, amongst other things). Its model allows “worker threads” that can persist but are idle until tasked through a queue. The types of work that are put into that queue, and when they are put there, are the concern of application/system design.

One type of work is a simple division of an immediate task across several threads to allow several cores to be brought to bear on it simultaneously (imagine a bitmap image being sharpened, divide it into several pieces then several threads, and hence cores, can work on it in parallel to complete the work more quickly). This type of work is generally sent to its executing threads ASAP since the user is waiting for the results.

Another type of work is things that can be done in the background and may even be discretionary. Apple quotes an app downloading updates as an example of such a task that can be deferred until convenient. The question of when this work should actually be performed is more complex: it shouldn’t take cores from the user’s foreground processing since, by definition, it is less urgent (e.g. than sharpening his bitmap). It may even be best left until the device has mains power available.

This second type of work requires informed scheduling and I think this is where CTS will fit (the OS knows what cores are free, how long jobs have been deferred, whether there’s mains power etc). The work can still be done by threads controlled via GCD. Similarly, threads from GCD can communicate using mechanisms provided by XPC.

There is clearly some cross-over between roles in actual implementation but this seems reasonable as an architecture. Food for thought?

LikeLike
- 2
  
  hoakley on May 13, 2017 at 9:20 pm
  
  That’s a neat summary of what Apple’s documentation says, only CTS remains very poorly defined. There is, though, considerable overlap, and common technologies in queue management, scheduling, and dispatch to cores to achieve optimum performance, etc.
  We used to have an OS which had clearly-defined functional blocks: things like QuickDraw, QuickTime, and so on, with names. And Apple used to actually tell us what these were, and provide us with diagrams to help.
  We now get a long list of APIs, and a handful of names which are ill-defined, and at best incompletely documented.
  Are DAS and CTS part of something, or just casual acquaintances? All that I see of them indicates that they work together, very closely, implying that they are part of something. As Apple doesn’t even tell us of the existence of DAS, we are left guessing what else it may or may not do, and so on.
  Having a large bag of all these little bits – ‘old’ GCD (which appears neither grand nor central at all, according to the published API), CTS, DAS, and NSBackgroundActivity, which is another API which does not refer to DAS or CTS but which is wholly reliant on them, and overlaps again with GCD – is not conducive to synthetic thought or communication.
  Interestingly this is looking at CTS and DAS working with repeated tasks: NSBackgroundActivity and CTS also appear to work very nicely with one-off tasks, where they are even closer to ‘old’ GCD in their behaviour. Perhaps when I look at that overlap we’ll just run out of terminology!
  Howard.
  
  LikeLike
3

Tony on May 13, 2017 at 9:38 pm

GCD (I assumed that only ‘Despatch’ was serious and Grand Central was a playful allusion) was, IIRC, an attempt to supersede several existing, more complex mechanisms. The CTS, DAS and NSBackgroundActivity mechanisms that you have documented feel like a new set of related mechanisms devoted to tasks using the facilities offered by GCD. They may have a grand design or just be ad hoc implementation. Perhaps they will be next to be consolidated into something simpler.

NSBackgroundActivity is new to me. I didn’I think of looking at Energy Efficiency for scheduling control, silly me.

LikeLike
- 4
  
  hoakley on May 13, 2017 at 9:48 pm
  
  That’s the problem in a nutshell: ‘old’ GCD in Concurrency, XPC in Daemons and Services, more dribs and drabs in Energy Efficiency.
  And all this is actually about APIs. Users don’t do APIs. If DNS is in trouble, you tell a user what DNS is and how to deal with the problem. If it doesn’t have an API, Apple assumes that we don’t need to know about it, nor where it fits in. Then when it goes wrong, we’re all puzzled and lost for names.
  NSBackgroundActivity is really easy to use with Swift too – see my example in the source of DispatchRider here. I can’t understand why it isn’t being much more widely used for one-off and scheduled background tasks, instead of launchd.
  Howard.
  
  LikeLike
5

Tony on May 13, 2017 at 10:21 pm

Looks like NSBackgroundActivity only appeared in 10.10, which would explain why it is not widely used. More so because it doesn’t appear in the 10.10 Foundation Release Notes.

Intriguingly, the (Obj-C) header file for NSBackgroundActivity says “This class provides a Cocoa-level interface to the XPC Activity API (see xpc/activity.h)”. That implies that NSBackgroundActivity is a wrapper for XPC so they’re actually the same thing. If so, that’s a helpful simplification.

Please forgive me for going to the code to understand the system design. I would shoot a software team for doing that.

LikeLike
- 6
  
  hoakley on May 13, 2017 at 10:41 pm
  
  Ah – now that I do understand. XPC is just a bit of glue in all this. It’s lightweight inter-process communications, which are used by lots of other things, including CTS (which communicates with all the activities which it manages using XPC, even if the activities themselves don’t have XPC support), GCD (which uses XPC for all its inter-process communications), and more. XPC is documented in the Daemons and Services Programming Guide, which hasn’t been properly updated for 6 years.
  When you use NSBackgroundActivity, it’s all orchestrated using XPC, but you’re actually talking to DAS and CTS, as shown in the log excerpts above. I hope that’s what that quotation is intended to mean, because that’s what actually happens. Of course because you’re using XPC and CTS, the log entries cite both, but the subsystem is com.apple.cts.
  So it’s as clear as mud really!
  Howard.
  
  LikeLike

·Comments are closed.

Share this:

Like this:

Related