If you were unlucky enough to encrypt your APFS volume using High Sierra 10.13 or 10.13.1, we now know that the passphrase was written to the log on your Mac, thanks to Sarah Edwards’ recent discovery. Yes, this is the unified log which Apple touted as having privacy designed “into the system” (WWDC 2016). The same unified log which has many entries in which all the useful information is censored with <private>. So how good is its privacy? How easy is it to break?

One fascinating revelation is Craig Newell’s comment to one of my earlier articles about the unified log, that there is an undocumented switch which allows censored data in log entries to appear in the log. In case you missed it, it’s the command
sudo log config --mode 'private_data:on'

Armed with a new version of my command tool Blowhole, I set off in search of answers to these questions.

According to Apple, scalars and static strings are assumed to be public, and by default are written to the log; dynamic strings, collections (arrays etc.) and objects are assumed to be private and are censored. These can be overridden by formatting an individual item as public or private.

Some examples:

• an integer formatted with "%d" will write the integer to the log
• a static string like "error 21" will write that string to the log
• a dynamic string like the variable myString will be replaced by <private>
• a dynamic string like the variable myString formatted with "%{public}@" will write that string to the log
• an array of strings like the variable myStringArray will be replaced by <private>
• an array of strings like the variable myStringArray formatted with "%{public}@" will write that string array to the log.

In normal circumstances, that is exactly what happens in macOS Sierra and High Sierra.

Enter the command
sudo log config --mode 'private_data:on'
into Terminal and authenticate as an admin user, and dynamic strings, collections, and anything made private by formatting it with "%{private}@" is then inserted into the log as if it were public. There is still some censoring using <private>, though, and it isn’t clear how this is occurring. As this configuration mode is undocumented, Apple doesn’t explain why the log appears to have two different levels of privacy protection.

Although the log config command is documented, its option to turn privacy off and on isn’t mentioned in log’s man page, nor when you ask it for detailed help using log help config. It also begs an important question: does it control the representation of private data in the log files, or the manner in which log show presents them?

One implementation of privacy in the log would be for the censoring of protected information to occur when each log entry is written, so that the log files don’t contain any protected data under the current private_data policy. That means that such private data are only captured in the log when private_data is turned on using log config.

An alternative implementation would be to capture all the private data, and determine whether to release them on the basis of the private_data setting at the time that log show is called.

It is easy to demonstrate that the determinant of whether private data is stored is the private_data setting at the time the log entry is made. When private_data is off, the default setting, private data is not captured to the log, and changing the private_data setting cannot reveal any private data captured when the setting was off.

That is the good news, but Sarah Edwards’ bug brings more bad news, I’m afraid.

To understand where this particular bug lies, we have to look carefully at the log entries which revealed the plaintext passphrase. In both cases shown, diskmanagementd, which is here supporting Disk Utility, is calling execve(2) to run a new process file, the newfs_apfs tool to initialise an encrypted APFS volume. The log entry recording that includes the full list of parameters passed to newfs_apfs, including the plaintext passphrase.

Aside from the rights and wrongs of passing the passphrase in plain text in this call, for whatever reason, this log entry records a dynamic string as public. To do that, according to Apple’s rules for the unified log, the full list of parameters passed must be formatted with "%{public}@", i.e. this is an intentional disclosure in the log entry.

It’s not clear from this example alone whether this is an error in diskmanagementd, or perhaps in the execve mechanism itself. That it has gone away in macOS 10.13.2 is therefore no indication of whether that bug has been fixed.

Prior to 10.13.2, formatting an encrypted APFS volume resulted in three Fault level log entries of diskmanagementd calling execve, to newfs_hfs, initialising the container through newfs_apfs, then initialising the encrypted volume through newfs_apfs.

From 10.13.2, only the first two of those calls are recorded in the log. The third, in which the encrypted volume is created and its passphrase leaked, no longer appears in the log. It would appear that diskmanagementd is using a different, and more secure, mechanism to create the encrypted volume, but that diskmanagementd‘s execve calls are still being logged in explicit and public detail.

In the worst case, this security bug may have gone away not because Apple recognised and fixed it, but because the way in which Disk Utility initialises encrypted APFS volumes has changed. The leak of potentially private data from execve() calls into the unified log may remain in High Sierra 10.3.3. Without seeing Apple’s source code it is impossible to tell.

It also raises further questions about the security and privacy of the unified log. When developers use its public and private protection properly, the log’s privacy mechanisms do indeed prevent the leakage of private data into the log. However, that protection is easily removed by programming errors, which may still lurk in macOS 10.13.3 High Sierra. Don’t be surprised if you stumble across other sensitive data in your logs.