In the weeks prior to Apple’s announcement of the first Apple silicon Macs in June 2020, there was intense speculation over how they would run existing macOS software. While most correctly recognised that this would be implemented using translation of x86 code as had been performed by Rosetta during the transition from PowerPC to Intel in 2006, some also saw a role for virtualisation. At the time I wrote that “Apple could provide a complete virtualisation layer which let users run most Intel Mac apps, and could even install and run Windows.”

A few days later Craig Federighi described three pillars to support a diversity of apps on Apple silicon: Universal apps containing code for both architectures, Rosetta 2 to translate x86 code, and virtualisation. At the time, the latter was promoted for hosting Linux and Docker, and Andreas Wendker demonstrated a pre-release version of Parallels Desktop running Linux as a guest.

Hypervisor

Apple had originally built support for the central requirement in virtualisation, the hypervisor, back in OS X 10.10 Yosemite in 2014. That provides C APIs to virtualise in user space without the need for kernel extensions. On Intel Macs this provides hardware support in the VT-x feature set including Extended Page Tables (EPT) and Unrestricted Mode. This API doesn’t extend to providing support for drivers in the form of VirtIO devices, though.

VirtIO

Meanwhile, preparations for Apple silicon Macs were proceeding apace. Two new kernel extensions appeared around the time of Mojave, in 2018-19, apparently to provide initial support for VirtIO in AppleVirtIO and AppleVirtualGraphics, which reached version 2.1.3 with the update to 10.14.6 in July 2019, a year before the first Developer Transition Kit was to be released. Those developed rapidly in Catalina, reaching version 16.140.6 with 10.15.6 shortly after Apple’s announcement of Apple silicon.

Big Sur brought further additions to macOS that made it possible to demonstrate Linux running in a virtual machine. Parallels made no secret of its collaboration with Apple. The number of kernel extensions grew from the original two to five, with the addition of AppleParavirtGPU, AppleParavirtGPUMetal and AppleVirtualMCA. VirtIO, AppleVirtIO and AppleVirtualGraphics also advanced to version 74.50.1. Two frameworks joined those kexts: ParavirtualizedGraphics and Virtualization.

Virtualization Extensions

More fundamentally, Big Sur brought hardware support in what Apple termed its “Virtualization Extensions” presumably AArch64 virtualization as documented by Arm. This features an additional ‘exception level’, EL2 hypervisor, offering stage 2 translation, EL1/0 instruction and register access trapping, and virtual exception generation. Stage 2 translation allows a hypervisor to control which memory-mapped resources a VM can access, and where those appear in the VM’s address space, augmenting stage 1 translation controlled by operating systems. Trapping allows a hypervisor to trap operations, such as those configuring low level controls, and emulate them where necessary.

While Big Sur brought sufficient support for that brief demonstration by Andreas Wendker, it fell short of what was required for release, even following additional support by 11.5.2. At that stage, there were ten kernel extensions

• AppleParavirtGPU
• AppleParavirtGPUIOGPUFamily
• AppleParavirtGPUMetal
• AppleParavirtGPUMetalIOGPUFamily
• AppleParavirtIOSurface
• AppleVirtIO
• AppleVirtIOStorage
• AppleVirtualGraphics
• AppleVirtualMCA
• AppleVirtualPlatform

several of which were still in their first version. Apple had also added an audio plug-in, AppleVirtIOSound.

Launch

Lightweight virtualisation of macOS and Linux on Apple silicon was launched with limited support for Monterey at WWDC 2022 in Benjamin Poulain’s presentation explaining how simple it is to implement. He concluded by saying how the virtualisation team “cannot wait to see what you will do next with this technology”. Notable by its absence, though, was support for folders shared between macOS guest and host, that awaited the addition of the virtio file system to Ventura, and support of the built-in neural engine (ANE), which gained its own kernel extension AppleVirtIONeuralEngineDevice at the same time. Sonoma has since added two more kernel extensions, AppleVideoToolboxParavirtualization and AppleVirtIOBiometrics, as well as XPC services for the Virtualization framework for EventTap, Installation, LinuxRosetta, and VirtualMachine.

In addition to the hypervisor and VirtIO support, there were other hurdles to overcome for Apple silicon Macs to run a guest macOS. A standard macOS Installer app is insufficient to build a working VM, as it doesn’t contain most of the ‘firmware’ components required. Fortunately, those already come in the IPSW images used to perform a full restore to an Apple silicon Mac in DFU mode, and provide all the ingredients for installing a macOS VM.

As of Sonoma, there remain several outstanding issues that Apple has yet to address. Most significant is the thorny problem of Apple ID support, about which Apple has been silent. For those with ample Performance cores on their Ultra chips, Apple’s licensing limit of a maximum of two concurrent macOS VMs on any Mac is unnecessarily restrictive when they can virtualise as many instances of Linux as they want. Fewer, though, want to be able to nest virtualisation by running a VM inside another VM. Finally, those in Europe and many countries outside the US are keen for Apple to support ISO keyboard layouts in macOS VMs, which remains a surprising shortcoming. I’m sure there’s plenty more to come in the future.

Milestones

• 2014 OS X 10.10: Hypervisor APIs (Intel).
• 2018 macOS 10.14: VirtIO kernel extensions.
• 2020 macOS 11 Big Sur: Linux VM demonstrated, Virtualization Extensions for Apple silicon.
• 2021 macOS 12 Monterey: first public support.
• 2022 macOS 13 Ventura: full support including virtiofs sharing.
• 2023 macOS 14 Sonoma: virtual display and other enhancements.