Apple had a bright idea with Catalina’s extended privacy protection, to stop apps and processes quietly accessing files in your Documents folder and other places without your agreement. This is a good idea, but the more that you think about it, and experience its implementation, the more you realise that it causes some true absurdities too.

Both my main log browsers, Consolation 3 and the new Ulbow (in its next beta release) let you create a logarchive of your current log. Although in the past I’ve partially reverse-engineered the logarchive format, the most reliable way to do this is using the log command and its verb collect. A command such as
log collect --output "testlog.logarchive" --last 2m
will write a special logarchive bundle containing the last 2 minutes of log entries to the file testlog.logarchive. However, to do that requires root privileges, so in practice you have to sudo that command.

From within an app like Consolation or Ulbow, that presents a problem. There’s no easy way of calling a command with root privileges from the main code of an app: you’re supposed to write a privileged helper, which quickly becomes non-trivial. But there is a sneaky way round this using a line of AppleScript instead. So the app merely runs something like
do shell script "log collect --output 'testlog.logarchive' --last 2m" with administrator privileges
macOS then displays the authentication dialog and executes that line of AppleScript, which in turn runs the single command with elevated privileges. It’s all secure, just a bit quick and dirty.

Having built this feature into Consolation 3 already, I came to add it to Ulbow, but couldn’t get it to write out a logarchive. It went through all the motions, including the authentication dialog, but no logarchive was generated and saved. I then tried the current version of Consolation 3, which behaved identically, and failed to save the expected logarchive.

When I repeated these tests but chose to save the logarchive in the ~/Documents folder, rather than to an external drive, they worked fine.

Suspecting this was an issue with privacy protection, I took to the log which, when you’re testing a log browser, is doubly convenient, as I used the build of Ulbow I was testing to inspect what was going on in the log when the logarchive wasn’t being saved. I also used the special log browser feature built into my free utility Taccy, which looks specifically at privacy entries in the log. I traced each step of the process, and found just a single entry from the kernel reporting the error, in a message similar to
Sandbox: log(54797) System Policy: deny(1) file-write-create /Volumes/External1/Documents/0newDownloads/Untitled.logarchive
Consolation and Ulbow don’t run in the Sandbox, and folder permissions are identical between locations where Ulbow can save correctly, and those where it can’t.

Most interesting of all, when Ulbow does write a logarchive successfully, Catalina attaches one of the new com.apple.macl extended attributes to it, and protects that using SIP. As I have suggested before, this is a marker of Catalina’s new per-document privacy enforcement system. It’s not possible to decipher the UUIDs attached in this case, but there are two, maybe one for the log tool and the other for the app which called it, here Ulbow.

Running an identical command from Terminal encounters no such problems with writing to the same external drive which Ulbow and Consolation can’t use, and there’s no com.apple.macl extended attribute attached to the logarchive created directly by the log collect command.

I have now played around with several other privacy controls. I’ve given Ulbow all the special ‘reason’ strings it might need, enabled it to control other apps, added it and the log command to the Full Disk Access list in the Privacy tab of the Security & Privacy pane, but nothing has enabled a logarchive to be written to my external drive, only to my (internal) startup volume.

I think the problem is that my app is running an AppleScript, which is running a shell script, which in turn actually needs to be entitled to write to this removable storage. Although the user selects the location in which to save the logarchive – which surely establishes user intent, Apple’s reason for permitting the action – that intent isn’t communicated through the AppleScript and shell script chain to give the log command the ability to save the logarchive where the user wants.

So now I’m going to have to try to explain to those using Consolation and Ulbow that Catalina’s new protection of the Documents folder permits them only to save logarchives in that protected folder (or elsewhere on their startup volume), not on removable storage. But we are getting used to privacy through perversity, I suspect.