Preparing to upgrade to Mojave: Scripting, commands, and development

In my previous checklist, I suggested ways in which every user who is intending to upgrade to Mojave should prepare. This article considers what you should do if you venture beyond using standard off-the-shelf apps – whether you’re scripting, building tools from source, or even just working in Terminal’s command line.

Setting your tools up

As soon as you have got your preferences set, switch to the Privacy tab of the Security & Privacy pane. In the list at the left, select Full Disk Access, then click the + tool to add all those apps like Terminal which need to access the newly protected folders, or which will run command tools to do the same. If you don’t do that now, when you first come to use those apps and tools, you’ll get pestered to give your consent to more specific access, which can quickly grow tiresome.

moprivprobs08

There’s nothing wrong with adding an app or tool now, and unchecking it for the time being, until you need to use it. Indeed, that is generally a good strategy to ensure that your privacy is best protected as much of the time as possible.

The new privacy system uses an Attribution Chain for command tools when they are launched. If you have tools which run automatically as LaunchAgents or LaunchDaemons, for example, you can also add those to the Full Disk Access list if they need to access protected folders. If you don’t, they probably won’t be able to access anything within those folders, which could affect the integrity of backups, etc. Apps like Carbon Copy Cloner already address this issue, provided that you’re running a recent version which is designed to work properly with Mojave. If you’re not sure, check the tool’s website.

Then, open those apps and check that they are configured to work well in Dark Mode. During your early days with Mojave, you may keep switching between Light and Dark Mode, as you work out which suits you best. If an app which you’re using doesn’t work properly in Dark Mode, now is the time discover that, and take any remedial action in its preferences, for example.

As Apple has only just released the Golden Master of Xcode 10, Homebrew, Ruby on Rails, and similar tools don’t yet officially support Mojave. However, they have been used throughout Mojave’s beta-testing and should work fine with suitable Xcode tools.

For AppleScript, I recommend that you take a good look at Mark Alldritt’s superb Script Debugger version 7.0.4 or later. He has written about the issues here. Even if you can’t justify its price, it runs in free ‘Lite’ mode which you may find an improvement on Apple’s bundled Script Editor. If you have already have version 6, the upgrade is only $49.

Xcode

If you use Apple’s Xcode or its command tools, the minimum version that you’ll need is 9.4.1, which has been used widely to build versions compatible with Mojave. That targets macOS 10.13 still; if you want to target Mojave then you’ll need Xcode 10, which can also be run on 10.13.6 and later. That new version should ship from the App Store around the time of Mojave’s release.

There is one distinct advantage in sticking with Xcode 9.4.1: Mojave doesn’t expect apps built against the 10.13 SDK to follow all the new rules for accessing protected data. In particular, the app doesn’t have to provide usage information strings in its Info.plist file. Xcode 9.4.1 does support them, though, if you do want to add them.

If you build against the 10.14 SDK in Xcode 10, though, Mojave requires those usage information strings to give your app access to the specific protected folders, location services, camera or microphone. Users can still add your app to the Full Data Access list, but otherwise your app will not be allowed to access those protected data.

Xcode 10 with its support for Mojave brings another important benefit, beyond support for the new features in Mojave: tools to make designing for Dark Mode much easier. Interface Builder, for example, can be switched between Light and Dark Mode previews, which is invaluable when designing an interface. You can also add graphics and icons which are to be used only in a given mode, and more. So if you are targeting Mojave, Xcode 10 is the way to go.

privacyxttred10

Xcode 10 lacks support for Subversion, and it cannot build with libstdc++, only libc++. If you’re using Swift, it offers version 4.2, and as usual the automatic source conversion makes that a breeze. There didn’t seem to be a great deal to convert in my apps, and what there was appeared to be the usual subtle changes in a few function calls.

The Golden Master has addressed one ‘feature’ of Xcode 10 betas which was quite infuriating: the Library browser no longer appears in the lower right corner of the main window, but pops up as a floating overlay window. That was not in itself a problem, but that window closed automatically whenever you dragged an item out of it. This made a long session building an interface extremely annoying. In the final release, if you hold the Option key when clicking on the Library tool to open that window, the window should remain open until you close it again.

Dark Mode

There’s actually a great deal to Dark Mode: it’s not just a matter of swapping black for white, and white for black. In the first instance, you’ll want to ensure that anything you have written works equally well in each mode, before going on to fine tune the use of different dark and light shades to make an app look simply gorgeous.

Although the two WWDC presentations are very interesting, I found little of help in them to sort out the problems which I experienced with Dark Mode. In essence, you must ensure that content in your interface uses standard system colours (or custom ones where you specify Light and Dark Mode variants) which macOS will render appropriately to the mode being used.

The instance which troubled many of my apps was NSScrollView scrolling text views. I used to append text into them using
let myString = NSAttributedString(string: string)
outputText.textStorage?.append(myString)

but that doesn’t set them in the right foreground colour, which is fixed at black, and becomes invisible in Dark Mode. Changing the code to set the colour explicitly as
let myString = NSAttributedString.init(string: string, attributes: [.foregroundColor : NSColor.textColor])
outputText.textStorage?.append(myString)

does the trick.

That said, Xcode 10 ensures that almost all controls are set to default colours which work well in both Light and Dark Modes.

There are some features which still don’t seem to work particularly well in Dark Mode, such as rendered HTML pages using WebKit and Help books, which usually appear as black text on white background even when everything else is in Dark Mode. These are complicated issues which could expend a lot of effort in their fixing, so for the moment I have left them well alone.

Data privacy

You should by now be very familiar with the fact that Mojave introduces new privacy controls which affect many apps and even simple command tools. Before you start tackling these, you should familiarise yourself with the content of Session 702 of WWDC 2018, and possibly Sessions 705, 709 and 715. It’s good to watch the presentations through in video, but to save time I also download the slides and transcripts, which at present are the best documentation.

I have written more detailed articles about this with illustrations taken from beta releases:
Working with Mojave’s Privacy Protection
Mojave’s privacy protection, command tools, and scripts
Mojave’s privacy protection: informing users
What can that app do? Entitlements, authorisation, and privacy

Apple is not going to back off, and I for one am very grateful. You only have to look at the recent issues surrounding the secret exfiltration of browser histories to realise that software development is no longer performed only by gentlepersons, and that users – including us – need robust protection against that sort of abuse.

My personal lesson in this area has been that reporting bugs is very important. The Security Engineering team at Apple is very keen to get this right, but they have a responsibility to all users to keep the system robust and secure. They respond very quickly to bug reports – in my case, within a couple of hours – they do fix them, and those fixes are implemented in updates as quickly as possible. Please ensure that you report any bugs you come across, as it does make a big difference.

Script Editor

No apps or tools are exempt from the new privacy protection policies. Apple’s apps, signed with its special certificate and safe within the umbrella of SIP, have special private entitlements which give them access to what they need: it would be crazy if the user had to give consent to Photos to open its own libraries, or Disk Utility to access the whole of a volume.

That includes all those neat little scripted apps which you have accumulated over the years. Just because there’s not much to them, they don’t get special treatment.

aeprivacy05

I have written about the provisions made in Mojave’s Script Editor to help your apps negotiate the new controls. Expect them to continue to evolve, and don’t forget to look at Script Debugger too.

Automator has options to sign the apps created with it, and like Script Editor adds all the popular usage information strings to the Info.plist of the built app. If you fancy a fun exploration of how Automator can almost make privacy easier, get it to record your actions in adding an app to the Full Disk Access list in the Privacy tab of Security & Privacy. You’ll discover that you can automate that perfectly – only the resulting app has first to be added to the Accessibility list before you can run it, which rather defeats the purpose!

Scripting apps and privacy

Developers working with privacy control in the betas have discovered that it gets most complicated when your app uses AppleScript or AppleEvents to control another app. As this is a well-established purpose of scripting, some see these controls as bringing the death of such automation another step closer.

aeprivacy01

The fundamental problem is that each relationship between a controlling app and a target needs individual consent. That is inescapable, as the user really can’t be asked to let any app control any other app in any way that it likes. If you have a script which controls a dozen other apps, then that means that the user will have to give their consent to each one of the ten relationships individually, which is a real interface mess. Furthermore, many of the targets are not fully-fledged apps which a user will recognise, but hidden apps like System Events, which might seem meaningless or even malware.

aeprivacy02

This makes the provision of meaningful usage information all the more important, with careful explanation to the user in clear documentation. I suspect that this will get easier in future releases of Mojave, but for the moment we will have to bear with it. I provide more detail here, and information about command tools and others here.

Helper tools present another set of problems, which I examined here.

Signposts

If you were hoping to see a revised log and Console app which returned to simpler Unix standards, then Mojave will disappoint. But it does introduce an extension to the unified log which makes it much easier to access, particularly for performance tuning: Signposts. These are described in Session 405 of WWDC 2018, and now replace dtrace-based instrumentation in Xcode’s Instruments.

The particular strength of using Signposts over regular log entries is that it is very easy to inspect or ‘harvest’ Signposts alone, separating them out from the incessant chatter which fills the log. Apple’s tool, in Xcode’s Instruments, is good, but restrictive: you can’t use it from a shell script, AppleScript, or anything which you don’t build and debug in Xcode 10.

I have articles here which explain how to write Signposts from anything which can call a command tool, and RouteMap, the start of a standalone app to harvest and analyse Signposts. They really are simple to use and retrieve.
Signposts for performance: 1 introduction
Signposts for performance: 2 Instruments
Quantum mechanics and Mojave – performance measurement, Signposts, and the log

I will also be writing a great deal more about Signposts and the new features in Mojave’s log once the beta NDA is lifted with the pubic release.

Additional documentation

The documentation in Xcode 10 ranges from adequate to absent, but maybe this time Apple will put that right over the coming months. For the time being, the best sources of information are the WWDC sessions, particularly those cited above. Tracking articles by developers about Mojave can be very difficult. My favourite resource is Michael Tsai’s blog which draws together brief excerpts from most of the best macOS developers around, and is remarkably well focussed.

Nearer the final release I will be posting an article explaining which of my free tools are best suited to Mojave, which will examine issues such as how to browse its improved log. If you have any questions, or wish to pass on your own tips, please don’t hesitate to add them as comments. Developing for Mojave is really good: sometimes it is like groping around in the dark, but there are many amazing new features like linguistic analysis and machine learning which make the effort well worthwhile.