High Sierra: security files, internals and some bugs

At first sight, High Sierra is little different. If you use Photos much, you should notice some significant improvements, but otherwise everything looks pretty much the same. But the major changes which have taken place are largely unseen – like the new file system. They should just work, and you’ll only see the differences when you look at what’s going on inside.

These are some initial notes on the glimpses that I have had into different aspects of High Sierra’s internals, so far.

Some very old bugs remain

Also touted as fixing bugs, High Sierra quickly revealed that it still suffers one very familiar bug: the ability sometimes to set wildly incorrect column widths in the Finder. This bug goes back to Mavericks or earlier, and is almost an old friend now. I find it strange that such an obvious bug has been left in such a prominent place in macOS. It’s a bit like misspelling your name on an exam paper, and doesn’t bode well for attention to detail.

Or maybe I have got it all wrong, and it has persisted for so long that it is now officially a feature.

Experience using High Sierra should soon show whether it still has Bluetooth disconnection problems, key repeats when using wireless keyboards, and the rest of the bugs which have plagued Sierra, and in many cases El Capitan before it.

Security data file versions

Note: in a pushed update on 29 September, XProtect and MRT data have changed. I have amended the listing below to reflect those.

The only change in the standard data files is for the KEXT block version, which rises from 12.5.0 in Sierra to 13.0.0 in High Sierra. I presume this is a result of the new third-party extension blocking.

I have not yet looked for additional security settings files, but will be doing so in the coming days.

The relevant sections of the LockRattler report currently read:
System Integrity Protection status: enabled.
XProtect blacklist assessments enabled
XProtect version (/System/Library/CoreServices/XProtect.bundle) 2095
Gatekeeper version (/private/var/db/gkopaque.bundle) 130
Gatekeeper disk version (/private/var/db/gke.bundle) 7.2
KEXT block version (/System/Library/Extensions/AppleKextExcludeList.kext) 13.0.0
MRT version (/System/Library/CoreServices/MRT.app) 1.23

Backup dispatching bug

By this, I mean the bug in DAS and CTS dispatching systems which causes Time Machine backups and other background activities to fail when a Mac has been running for around a week or so. This has remained throughout releases of Sierra, despite being reported to Apple in February 2017.

It is too early yet to say whether this bug has been fixed – it will take until early October before we may start to know that. However, log entries made by DAS have changed considerably, implying that it has undergone extensive work since Sierra. First, DAS now declares itself to be the daemon dasd, instead of DuetHeuristic-BM as shown in Sierra’s logs.

Apple has changed logging policy too. In Sierra, most of the useful information about activities being scheduled and dispatched by DAS was censored, using the entry <private> Now, DAS writes more log entries like
com.apple.CFNetwork-cc-517-340-Task <31D7823C-DDFB-47FF-ACDF-79C1CC775CE4>.<com.apple.cloudd>.<outOfProcess=T:allowExpensive=T:powerNap=F:app=com.apple.bird:2app=4/com.apple.clouddocs.Z2R2HMF4E2.com.acrylic.Times:2,%:disc=F:pool=com.apple.cloudkit.BackgroundConnectionPool:uuid:82FF8FBB-8841-40FA-BA79-9A76C996D4DE>.<5>:20594B:[{name: ApplicationPolicy, policyWeight: 5.000, response: {Decision: Can Proceed, Score: 0.50}}] sumScores:32.310000, denominator:34.910000, FinalDecision: Can Proceed FinalScore: 0.925523}

There are still quite a few useless entries, though, such as
Running activities : <private>
making DAS entries rather hit or miss.

There seem to have been architectural changes in DAS. Previously, its log entries consisted of two types, DuetActivitySchedulerDaemon with DuetHeuristic-BM, and DuetActivityScheduler with UserEventAgent. The first seemed concerned with managing the list of activities, the second with telling CTS to start an activity.

In High Sierra, the log entries show the players have been renamed: the first pair, managing the list of activities, is now DuetActivitySchedulerDaemon with dasd, and the second DuetActivityScheduler with UserEventAgent and nsurlsessiond.

CTS log messages seem more similar to those seen in Sierra, although I see several references now to the creation and reaping of zombie activities which appear new to High Sierra.

In short, DAS and CTS have clearly changed, and there is a real prospect that these crucial dispatching systems will not cease working after about seven days of continuous running. We’ll see if that works out in practice.

Log contents

High Sierra’s logs frequently look very different from those found in Sierra. One of the most immediately apparent differences is the volume of entries made now by com.apple.SkyLight, which I believe is Metal 2 graphics acceleration, here primarily for WindowServer.

Sierra does contain some messages from this sub-system, typically error messages for failed operations on windows. Browse High Sierra’s logs and they are full of a very distinctive pattern of message, which is spread onto several lines, of the form
com.apple.SkyLight performance_instrumentation WindowServer SkyLight CompositeLoop
com.apple.SkyLight performance_instrumentation WindowServer SkyLight ContributingPIDs PID: 711
PID: 0
PID: 0
PID: 0
PID: 0
PID: 0
PID: 0
PID: 0

This implies that this first release of High Sierra is instrumented to monitor the performance of SkyLight functions, although the precise meaning of these frequently-repeated log entries eludes me.

APFS in the log

Given that the version of APFS shipped in High Sierra 10.13 release is far from finished, log entries referring to it are relatively infrequent. There’s the occasional glimpse of it at work on one of the new features, snapshots:
kernel: (apfs) find_next_non_dataless_snapshot:9713: apfs: next = 18446744073709551615 err = -1
for example.

Here’s an APFS volume being unmounted:
kernel: (apfs) apfs_vfsop_unmount:1444: unmounting devvp <private>
kernel: (apfs) apfs_vfsop_unmount:1590: flushed all txn's!
kernel: (apfs) apfs: total mem allocated: 165422648 (158 mb);
kernel: (apfs) apfs_vfsop_unmount:1692: all done. going home. (numMountedAPFSVolumes 3)

I hope that has given you some help in recognising what you are looking at when you start viewing logs from High Sierra systems. As far as I can tell, Consolation 3 and DispatchView work identically on High Sierra as on Sierra, so at least you should have the tools you need.

Updated 29 September 2017 to reflect latest security updates.