Universal Binaries: inside Fat Headers

With the return of Universal Apps and Binaries, there will be occasions when you’ll need to check whether a Mach-O file, such as a dylib or command tool, can run natively on Apple Silicon. As I’ve explained, you can do this using my free utility ArchiChect, or with lipo or file at the command line. But what if you need to check a lot of binaries, as Mints does when it scans folders containing tens or hundreds of thousands? Calling lipo every time isn’t going to be such a good idea.

The best solution, if you need speed, is to inspect the Mach-O file itself, and read its Fat Header. This article explains how you can do that, providing code fragments in Swift.

Checking a bundle

Before looking at the problems posed by Mach-O files, here’s an example of how simple it is to check a bundle, using Bundle.executableArchitectures. If you want to run this on macOS 10.x, you’ll need to provide your own constant to represent the ARM64 architecture, as in
let theAppIsARM: UInt32 = 16777228

Then simply get the list of executable architectures for the bundle, and iterate through that to find if any matches the constant for ARM64:
if let theBundle = Bundle.init(url: theSourceURL) {
if let theArchs = theBundle.executableArchitectures {
if theArchs.count > 0 {
for item in theArchs {
if item.intValue == theAppIsARM {
// this supports ARM64
}}}}}

Mach-O Fat Header

Mach-O files which support multiple architectures start with a Fat Header, which contains the ‘Magic’ to declare that it’s a Fat Header rather than a single-architecture Mach-O file, and the number of Fat Architecture slices, which enumerate and identify each Mach-O binary to follow in the rest of the file. When you try to read these first eight bytes you immediately discover one pervasive problem with reading Fat Headers: their endianness. Most, if not all, of their fields are stored in big-endian format. These are detailed in Jonathan Levin’s authoritative account in the first volume of his *OS Internals book.

The Fat Header (struct fat_header) contains just two fields:

  1. ‘Magic’ of 0xCAFEBABE (magic, 4 bytes), in big-endian order, which will normally be read to match the constant FAT_CIGAM rather than FAT_MAGIC,
  2. an unsigned 32-bit integer (nfat_arch, 4 bytes) giving the number of Fat Architecture headers to follow; this is stored in big-endian format,

followed by a series of 20-byte structures containing each of the Fat Architecture slices in turn.

Reading this therefore gets a little twisted:
let theFHandle = try FileHandle.init(forReadingFrom: theSourceURL)
let theMagic = theFHandle.readData(ofLength: 8)
if theMagic.count > 7 {
let theFatHeader = theMagic.withUnsafeBytes { $0.load(as: fat_header.self) }
if theFatHeader.magic == FAT_CIGAM {
let theCount = theFatHeader.nfat_arch.bigEndian
// iterate through theCount number of Fat Architecture Headers
}}

which copes correctly with endianness and returns in theCount a small integer, rather than a very large one.

Each Fat Architecture slice (struct fat_arch) then consists of:

  1. an unsigned 32-bit integer (cputype, 4 bytes) giving the broad processor architecture, e.g. ARM64
  2. a single byte possibly giving subtype capability (cpu_subtype, 1 byte), which also functions for alignment
  3. a 24-bit value giving the CPU subtype (cpusubtype, 3 bytes), which can be combined with the single byte and treated as one unsigned 32-bit integer
  4. the offset to the Mach-O data (mach-o, 4 bytes)
  5. the size of the Mach-O data (size, 4 bytes)
  6. a further alignment field of 4 bytes.

Each of these is stored in big-endian format.

You can discover whether the cputype is ARM64 using:
let theMachoIsARM: UInt32 = 201326593
for _ in 1...theCount {
let theFA = theFHandle.readData(ofLength: 20)
if theFA.count > 19 {
let theFatArch = theFA.withUnsafeBytes { $0.load(as: fat_arch.self) }
let theCPUtype = theFatArch.cputype
if theCPUtype == self.theMachoIsARM {
// this fat_arch supports the ARM64 cputype
}}}

Here are three examples shown using Synalyze It!

fatheader01

This dylib contains 3 fat_arch slices, each of which is for an Intel processor: two are for X86_64, with cputype values of 0x01000007, and the third for 0x00000007.

fatheader02

This app binary contains 2 fat_arch slices, the first for the Intel cputype of 0x01000007, and the second for ARM64 with a cputype of 0x0100000C.

fatheader03

This command tool contains 3 fat_arch slices, the first two for the Intel cputype of 0x01000007, and the third for ARM64 with a cputype of 0x0100000C.

Tomorrow I hope to release a new version of Mints which includes these changes, so that it doesn’t have to keep calling lipo. At present, I don’t intend making this change to ArchiChect, which isn’t intended to examine tens of thousands of Mach-O binaries. I therefore welcome corrections and improvements to any of the above.