Last Week on My Mac: Big Sur’s broken clock

Part of the excitement about not being told what has changed in new versions of macOS is that you never know what’s going to break or misbehave next. To give Apple its due, it’s very seldom that any operating system manages to break its clock in the way that Big Sur has. The underlying reason is that macOS 11.2 no longer distinguishes between the illusions it creates for the user, and harsh realities required by the system. It has become psychopathic.

At the heart of this is a bug or feature in which macOS no longer formats time according to explicit rules, but does what it fancies.

Internally, dates and times are stored in an opaque data format called a Date. Setting a Date from text or turning one into text is performed with one of two classes of conversion: there are those intended for use with the user interface, which normally comply with a user’s settings in the Language & Region pane, and those which need to be more rigorous, for example for compatibility with commands, which are specified using a formatting string.

As an example, the Date representing the 20th of April in 2021 at three minutes and fifteen seconds past eight o’clock in the evening local time might be converted to the string
April 20, 2021, 8:03:15 pm
for display in a window or dialog, but
2021-04-20 20:03:15
when it’s going to be used as an argument for the log show command. Note the clear distinction here between what appears in the human interface, which can vary considerably according to language and regional settings and personal preferences, and what is fixed for internal system use.

Apple doesn’t define the formatting strings used internally, but relies instead on a series of reports provided by the Unicode Consortium referred to as Report TR35. These are minutely detailed specifications, and for my purposes here I’ll look at just one element, the formatting of numbers representing hours. Here are two main options: HH sets the hour to digits ranging from 00 to 23 to represent the 24-hour clock padded with a leading zero when necessary, and a single h sets them as unpadded digits ranging from 0 to 11, or perhaps even 1 and 12. Thus, when software wants to format a Date for use with the log show command, the formatting string should be
yyyy-MM-dd HH:mm:ss

No matter what the user might have set in their interface, even if they have elected for the hours to be shown using Roman numerals in a 12-hour clock, that should never change. But in macOS 11.2 it does.

Prior to Big Sur, all software had to do to obtain such a rigorously formatted string was to set the date format using that formatting string. Then, apparently without any warning to anyone, Apple changed this behaviour in about macOS 11.2, so that the formatting string is largely ignored, and when a user’s interface settings determine, that rigorously formatted string might become
2021-04-20 8:03:15 pm
As you can imagine, feed that as an argument to a command like log show, and all you get is an error.

This also applies to strings representing Date values obtained from the human interface, such as the standard Date Picker control. If the app asks for the date and time set with the formatting string of
yyyy-MM-dd HH:mm:ss.SSSSSSZZZ
instead of being provided a compliant result of
2021-04-20 20:03:15.006976+0100
when the user’s settings are such, the result could be
2021-04-20 8:03:15.006976 pm+0100
which is a bizarre hodgepodge that can cause no end of secondary problems.

The effects extend beyond the internal too. Dates and times are commonly used in text reports derived from apps, and incorporated into systematic document names. Where someone might have months-worth of files with names like
Output 2020-11-23 20:44:21.text
all of a sudden they’ll start seeing names like
Output 2020-11-23 8:44:21 pm.text
which destroy the sorting order, and any automated processing they might rely on.

Yet if you use the date command in Terminal, that has no such problems with formatting strings and user settings. For example, the command
date "+%Y-%m-%d %H:%M:%S"
will always generate a date and time of the form
2021-04-20 20:03:15
whether or not the user has set their clock to a 12-hour display.

Apple does provide a solution, for which it tries to excuse itself in an archived Technical Q&A concerning Internet Dates which was last updated seven years ago. According to that “There are many places where format strings behave in unexpected ways”. One of the two examples which it cites is in iOS:
“the user can override the default AM/PM versus 24-hour time setting (via Settings > General > Date & Time > 24-Hour Time), which causes NSDateFormatter to rewrite the format string you set, which can cause your time parsing to fail.”
Even in macOS 10.15, changing between 12- and 24-clock formats doesn’t affect what happens in macOS, though, nor should it.

The advice given is vague and raises more questions:
“if you’re working with fixed-format dates, you should first set the locale of the date formatter to something appropriate for your fixed format. In most cases the best locale to choose is “en_US_POSIX”, a locale that’s specifically designed to yield US English results regardless of both user and system preferences.”
How do we tell if our specific case is one of those “most cases”? What other effects might that setting have? If someone has set their Mac to run in Danish, should we be using something other than a US English setting here? But above all, why on earth would Apple change “in unexpected ways” a fixed format to something unspecified of its own invention which flies in the face of Report TR35? Just why is it impossible to follow the prescribed format string?

Of course, if you didn’t heed that archived note about Internet Dates, but tested your software against common language and clock settings, on macOS it worked fine until Big Sur, probably 11.2, when without warning it suddenly broke, without so much as a release note to explain why. Apple has lost the distinction between the illusions it presents in the human interface, and the consistent and clear rules by which internal processes must operate. For a consumer device like an iPhone, that might be excusable. For a computer like a Mac it’s a grave error.