Last Week on My Mac: Updates and open loops

When you use Macs or any of Apple’s devices you’re heavily dependent on its servers, not least for the many updates which they deliver. Most of us are keen to install updates, particularly those fixing bugs and closing security vulnerabilities, as soon as we can. As our homes and small businesses now commonly include several Macs and devices, it makes good sense to run a local Content Caching server.

Like many relatively complex features in macOS, its Content Caching server is simple to use but quite opaque. There’s normally little to configure or control, and it’s mostly a matter of turning it on or off, or very rarely emptying its caches to start again. For many of us, it has just worked, providing multiple updates from just a single download from Apple’s software update servers. At least, that’s what it did until early June.

Since then, many of us running local Content Caching servers have had problems. Our Macs report a security data update is available, but it simply doesn’t work. A glance at the error message in an app like SilentKnight, or when using the softwareupdate command, reveals that the update was found, downloaded to the client Mac, but installation failed. Some have then emptied the caches of their local Content Caching server, only to find the same problem recurs. Eventually, out of frustration, we turn the local server off, and updating completes without any further problem.

Apple’s engineers are currently looking at the why and how of this problem, but that’s not my concern here, as I’m sure they’ll discover what’s going wrong and fix it. What concerns me is that, until I reported this, Apple was oblivious to the problem. That’s because the Content Caching server, indeed Apple’s software update service as a whole, appears completely oblivious to such problems. Updates are served but no one ever cares to check whether they work, as clients are functionally black holes, and the servers, whether Apple’s or local, are dumb.

At present, when a local Mac asks the local Content Caching server for an update, it’s fetched, either from Apple’s servers or its cache, delivered to the client, and that’s the job done. If the client fails to install that update successfully, neither the Content Caching server nor Apple’s servers seem to be informed of that failure. Even if they were, there’s nowhere the user can discover what went wrong.

As I have explained, the Content Caching server does make copious entries in the Unified log. But looking through those, none records any errors or problems when software updates fail to install on any client. Nor is there any record on the client of installation errors being reported to Apple. Instead, in the GUI they simply fail silently, leaving that Mac to try again another time, until the update finally fails to fail any more.

At the moment, Apple’s software update and the local Content Caching services are open loops. Updates are delivered to clients that function as black holes. What is needed is a way for softwareupdate to inform any caching server and Apple when an update doesn’t install correctly. Without that information, failed updates continue to fail, and Macs are left without the benefit of the protection delivered by that update.

It’s time for Apple to close this open loop, so it gets reliable feedback on the success of rolling out these important updates and any problems that are arising. Then it won’t take a month of sending sysdiagnoses and Feedback reports to Apple before updates become reliable again.

I’d like to thank Apple’s engineers who have been working on this problem. We do appreciate your work, thank you.