What stops your Mac running incompatible extensions?

Kernel extensions are dangerous. Because they run in kernel space – with the same special privileges as the kernel itself – an old or rogue KEXT can readily cause a kernel panic. If you’re unlucky, this could even prevent successful startup, causing startup cycling in which all your Mac can do is try to start up, panic, then repeat the cycle. More probable are apparently random freezes in which you have to force your Mac to shut down or restart. These can even occur with KEXTs that are part of an official release of macOS. So what’s provided to try to prevent that from happening?

If you keep tabs on macOS security files, you’ll no doubt be aware of the KextExcludeList, which is inside one of the only three macOS KEXTs which are kept on the Data volume in Catalina and Big Sur, ready for updating. In macOS 10.15 and 11, you’ll find that at /Library/Apple/System/Library/Extensions/AppleKextExcludeList.kext, with the property list specifying known bad KEXTs in Contents/Resources/ExceptionLists.plist. In earlier versions of macOS, it’s in /System/Library/Extensions/AppleKextExcludeList.kext.

This property list has changed version number from Mojave (14.5.1) to Catalina (15.6.1) and Big Sur (16.0.4 on both Intel and M1 systems). In Mojave, it was accompanied by another property list KnownPanics.plist which contained further definitions of known baddies, including KEXTs and some other components installed by third parties. What you might find surprising is that all three versions of the ExceptionLists.plist file, spanning three major releases of macOS and over two years, are identical, and list very old KEXTs which you’re almost certainly not going to come across any more.

Big Sur does introduce Kernel Integrity Protection (KIP), which on Apple Silicon Macs even has support in the processor. This aims to prevent modifications of both kernel and KEXT code once your Mac has completed the boot phase when starting up. But KIP doesn’t appear to protect from bugs and other problems in installed KEXTs.

One thing you will see both Catalina and Big Sur doing increasingly is warning you about KEXTs which you should aim to remove or replace before they get ignored by the system completely. But other than being warned, macOS doesn’t do a great deal to help you address the problem. You can start up in Safe Mode (holding the Shift key) which should disable all third-party KEXTs, and even disables some of Apple’s, but that in itself doesn’t seem to make them any easier to remove.

M1 Macs here have two great advantages: as they’re brand new systems, even if you’ve migrated from an Intel Mac, KEXTs shouldn’t have been transferred across. If any has, then unless it’s recent enough to be Universal, Big Sur running on an M1 model simply won’t load a KEXT which only contains Intel code. As Universal binaries weren’t possible until last summer, that rules out almost every possible third-party KEXT which could cause trouble on an Intel Mac.

The best solution to removing any old third-party KEXTs is using their uninstaller, sometimes offered within its installer utility or the app which relies on the KEXT. If you can’t locate that, or it’s 32-bit and won’t run any more, then manual removal is unfortunately not trivial in Catalina, and largely uncharted territory in Big Sur.

These are compounded by the use of the StagedExtensions folder, and the fact that, prior to Big Sur, different command tools are used to manage KEXTs, which have now been functionally united in the new kmutil command, introduced in macOS 11.

Users, and uninstallers, only have access to the /Library/Extensions folder; the system manages the contents of /Library/StagedExtensions, and protects them using SIP. Although it’s tempting to disable SIP and manipulate the contents of that latter folder, that’s not advisable. To remove a third-party KEXT in /Library/Extensions that’s all that you should try to do, and leave the system to do what it needs from there. Before trying to manually remove a third-party KEXT from Catalina, you may wish to unload it using a command of the form
sudo kextunload /Library/Extensions/MyKext.kext

It shouldn’t then be necessary to use the commands to rebuild the KEXT cache, but in case you should ever need to, they go something like
touch /Library/Extensions
kextcache -i /​

following which you should restart your Mac.

In Big Sur, there’s very limited experience but a similar approach should succeed; note that kextunload and kextcache have been replaced by kmutil, which is exceedingly well documented but also very complex.

If you need to manually remove a new system extension, then this article explains how to use the new systemextensionsctl command. However, as system extensions are more recent, you shouldn’t have a problem in identifying them and using their app to uninstall them. It’s also worth noting that if both a KEXT and a system extension are installed by an app, macOS now loads the system extension and not the KEXT.

The answer to my original question is that you’re largely responsible for ensuring that your Mac doesn’t try loading incompatible extensions. Don’t leave it to the KextExcludeList, which probably doesn’t know.