Controlling processes and environments

When you run a process, such as a command tool either from a shell such as zsh or bash, or from within an app, there are two ways in which you control what that process does: the arguments passed to it when it’s launched, and the environment in which it’s run. Recent discussions here about a language problem I encountered when running system_profiler within an app revealed that environments aren’t well understood. This article attempts to clarify, hopefully without making any glaring errors in the process (or environment!).

Launching a Process within an app

Compiled apps which need the services of a command tool or other Mach-O binary may do so using a Process (or NSProcess). Typical code for setting up and launching might run:
// create a new Process
let task = Process()
// get the default environment
var theEnvDict = ProcessInfo.processInfo.environment
// you can add or change values in that environment as you wish,
// then set the Process’s environment
task.environment = theEnvDict
// specify the full path of the command tool to be launched
task.launchPath = "/usr/sbin/system_profiler"
// set its argument as an array of strings
task.arguments = ["SPiBridgeDataType"]
// set up a Pipe for stdout
let outPipe = Pipe()
task.standardOutput = outPipe

// launch the task
// read stdout as data
let fileHandle = outPipe.fileHandleForReading
let data = fileHandle.readDataToEndOfFile()

// wait until the Process completes
// check its returned status, handling errors, etc.
let status = task.terminationStatus
which runs what in Terminal would have been entered as
system_profiler SPiBridgeDataType

We have to specify the arguments in accordance with that tool’s man page, but how does the environment work?

As elsewhere, the environment is a collection of settings, most easily viewed as a dictionary of key-value pairs. This is determined by the context in which that app is being run. For example, when you run an app in Xcode’s debug environment, that includes more than 30 environmental variables, which might fall to a dozen when that app is run outside Xcode. In the latter case, these might be
"__CF_USER_TEXT_ENCODING": "0x1F5:0x0:0x2"
"SSH_AUTH_SOCK": "/private/tmp/"
"SHELL": "/bin/zsh"
"TMPDIR": "/var/folders/sz/qb23fyss56v96vmh60p8ft7r0000gn/T/"
"XPC_FLAGS": "0x0"
"HOME": "/Users/hoakley"
"USER": "hoakley"
"PATH": "/usr/bin:/bin:/usr/sbin:/sbin"
"LOGNAME": "hoakley"
"COMMAND_MODE": "unix2003"
"__CFBundleIdentifier": "co.eclecticlight.Mints"

In each case, the pair is given with the string key first, followed by a colon, and its value is another string. For example,
"HOME": "/Users/hoakley"
would be the same as setting the following in a normal shell environment:

The command tool can access the variables set in its own environment at launch. For example, a command which wants to know the name of the current user can retrieve that from the USER setting in its environment, and the Mac SDK provides convenience methods for obtaining such information. However, Mach-O binaries are usually quite simple beasts, and if they don’t make use of a variable there’s nothing that macOS will do about it.

A good example here is setting languages or locales. You can set the well-known environmental variable LANG to en_GB.UTF-8, but there’s nothing that automatically happens as a result. A command tool can ignore that, or it can respect it by converting any strings which it returns into those selected from collections of localised strings.

One variable which is normally set is the User Text Encoding, in __CF_USER_TEXT_ENCODING, which I examine next.


At the top level of every user’s Home folder is a hidden file .CFUserTextEncoding containing a format string whose purpose is explained in this Tech Note. This contains a string of the form M:0:N, where each of the three numbers is usually given in hexadecimal, for example as 0x1F5:0x0:0x2, which specifies that user number 501 (hex 1F5) uses encoding number 2. Normally, in the .CFUserTextEncoding file the user number is left as 0x0 for the system to convert into that user’s UID.

This doesn’t set the language or locale for that user, but the encoding used by CFString, and the full list of supported encodings is provided in Apple’s reference documentation for CFString. However, this value now appears to be largely ignored, and you’ll encounter users whose assigned encodings might be 0x2, which is supposed to be macChineseTrad, or 0x5 for macHebrew, which bear no relation to their language setting. As UTF-8 is the string encoding used throughout macOS, it’s hard to see what purpose this setting now serves, but you’ll still see it in environments as __CF_USER_TEXT_ENCODING.

Running a command tool in Terminal

When you run a command tool in a shell, such as zsh, the default in Terminal from macOS 10.15 onwards, the same principles apply with the addition of the environment settings for the shell. You can inspect those using the command env, which should return a list similar to my current environment:

An important addition here are the paths set in PATH, which save you from having to type the full path to well-known commands, and setting the PATH in your environment is one of the commonest reasons for changing it. You can also inspect language settings using the locale command, which lists each of the language-related variables, such as

As with other environmental variables, these are accessible to command tools, but don’t in themselves change anything. It’s up to the command tool to look up its environment and make any changes it deems necessary in response to a setting such as the language.

Running an app

Running an app from the command line is quite different from a command tool. Apps don’t simply parse the arguments passed from the command line, as they invariably rely on frameworks like AppKit and other code to switch to using translated strings, for example. Accordingly, there are two relevant options which the user can invoke, -AppleLanguages and -AppleLocale. For example, to launch an app in German regardless of the system language setting, use a command such as
/Applications/ -AppleLanguages "(de)"

Of course, this only works if that specific app has the appropriate localisation requested, and is either handled by its frameworks or by custom code in the app itself.

When you launch an app from the Finder, the only way you have of setting it to use different behaviours and variables is through its preferences, otherwise it simply uses those set by the system. Some apps allow you to set their language and other features using a command such as
defaults write AppleLanguages '("en-US")'
Unfortunately, as with most preference settings, this may well not be documented, so you’ll need to experiment to discover what works. However, for users this provides a convenient way of running an app using a language other than that set by the system, when it’s supported.

Call chains

When you run a simple command tool directly, there’s little to go wrong: you pass it an environment, and that determines how it behaves. This becomes more complex when the tool calls on other code to handle your command. I’ve been looking at how this works with system_profiler in particular.

system_profiler is surprisingly complex. The command tool in /usr/sbin/ turns out to be a small stub which relies on calling helper tools stored in the /System/Library/SystemProfiler folder as .spreporter bundles. Each of those contains another Mach-O executable complete with its own localised strings, and in some cases such as SPiBridgeReporter.spreporter there are also XPC services, which in turn have their localised strings.

You can normally trace a call chain by browsing the log from the moment that the command is invoked. Here is an excerpt which traces the call chain for the system_profiler command tool, either launched from within an app using Process() or run from Terminal’s command line, on an M1 Mac.

The system_profiler command tool next dispatches a request for the helper tool in /System/Library/SystemProfiler:
0.544816 Reporting system_profiler SPSupport -[SPDocument reportForDataType:] -- Dispatching helperTool request for dataType SPiBridgeDataType.
0.545010 Reporting system_profiler SPSupport -[SPDocument _reportFromHelperToolForDataType:completionHandler:]_block_invoke -- Launching task to collect SPiBridgeDataType
0.551195 Reporting system_profiler SPSupport -[SPDocument _reportFromBundlesForDataType:] -- Starting task to collect SPiBridgeDataType

That helper tool in turn calls the XPC service iBridgeDiscovery, which is handled first by opendirectoryd:
0.552426 session opendirectoryd opendirectoryd UID: 501, EUID: 501, GID: 20, EGID: 20, PID: 1103, PROC: system_profiler RPC: getpwuid, Module: SystemCache, rpc_version: 2, uid: 501

The XPC service goes to work:
0.623138 client iBridgeDiscovery CoreAnalytics Daemon status changed from 0 to 1
0.623262 xpc analyticsd analyticsd Setting new client connection handler. 169 active connections
0.623390 client iBridgeDiscovery CoreAnalytics Received configuration update from daemon and there was no CoreAnalyticsFramework external config.
0.623390 client iBridgeDiscovery CoreAnalytics Received configuration update from daemon (initial)
0.626248 RemoteServiceDiscovery iBridgeDiscovery RemoteServiceDiscovery Starting browsing: <private>
0.626371 RemoteServiceDiscovery iBridgeDiscovery RemoteServiceDiscovery Started browsing.
0.626380 RemoteServiceDiscovery iBridgeDiscovery RemoteServiceDiscovery Connection invalidated
0.626391 RemoteServiceDiscovery iBridgeDiscovery RemoteServiceDiscovery Connection invalidated

With the result back from the XPC service, that service is exited:
0.626472 Reporting system_profiler SPSupport -[SPDocument _reportFromBundlesForDataType:] -- Completed task to collect SPiBridgeDataType.
0.633413 session opendirectoryd opendirectoryd PID: 1103, Client: 'system_profiler', exited with 0 session(s), 0 node(s) and 0 active request(s)
0.633719 session opendirectoryd opendirectoryd PID: 1104, Client: 'AppleSiliconDiscovery', exited with 0 session(s), 0 node(s) and 0 active request(s)

Finally, the command tool has got the output data, returns it, and exits:
0.697375 Reporting system_profiler SPSupport -[SPDocument _reportFromHelperToolForDataType:completionHandler:]_block_invoke_2 -- Completed task to collect SPiBridgeDataType. terminationStatus is 0
0.733831 session opendirectoryd opendirectoryd PID: 1102, Client: 'system_profiler', exited with 0 session(s), 0 node(s) and 0 active request(s)

The calling app, in this case a special test version of Mints, has its own environment. When it calls system_profiler, it passes an environment which the app controls. system_profiler runs in that environment, but then invokes its helper tool, which has its own environment, and that in turn uses an XPC service executing in its own environment, which isn’t controlled by system_profiler, to gather and return the information. As that service has its own localised strings, it can perform conversion of the text in its response without any knowledge of the environment in which system_profiler is running.

Complex call chains like this make control of the environment far more difficult.