hoakley November 12, 2022 Macs, Technology

What makes an external disk bootable?

Once upon a time, I’m sure all you needed to make a disk bootable was a System file and the ‘blessing’ of that disk. Then Apple decided we had to have System Folders, and OS X made it even more complicated. This article tries to outline what an external disk now needs for it to be bootable using a modern Mac.

Basics

Making a disk bootable is something we normally leave a macOS installer to fix for us. If you use a utility like Carbon Copy Cloner or SuperDuper! to clone an existing bootable volume group to a disk, that too should spare you the hassle. But when either goes wrong and you discover that Startup Disk won’t let your Mac start up from that volume group, it’s important to discover what might have gone wrong.

To understand this, consider how an Intel Mac boots from macOS Mojave or earlier, and where that can fail. That’s summarised for older Intel Macs and those with T2 chips in the following diagram.

BootDiskProblems

One fundamental requirement is that the hardware, the Mac’s interface, cable and drive enclosure, all must support booting that Mac. Some older enclosures offer connections that can’t be used for booting, in which case you’re onto a loser from the start. For T2 Macs, it also requires that booting from an external disk has been enabled using Startup Security Utility in Recovery mode. Although you’ll see some claim that’s also necessary for Apple silicon Macs, it isn’t, and their Startup Security Utility doesn’t offer it as an option, because it’s always possible to boot an Apple silicon Mac from an external disk when everything else is right.

Normally, on a non-T2 Mac booting from an HFS+ disk, it needs a boot loader at /System/Library/CoreServices/boot.efi. A helper partition is normally used when a disk is encrypted using HFS+ FileVault 2, or the storage device requires additional drivers, such as an external RAID array. On a non-T2 Mac booting from an APFS disk, the boot loader should be in the path [UUID]/System/Library/CoreServices/boot.efi on the special Preboot volume /dev/disk1s2.

SSV

Starting partially in Catalina, and in full from Big Sur onwards, the required container and volume structure on an external bootable disk is essentially the same as that on the internal storage of an Intel Mac, as shown in the diagram below.

BootDiskStructureIntelMonty

There’s the traditional hidden EFI partition, even for Apple silicon Macs that don’t do EFI at all, and an APFS container with the bootable system, consisting of five volumes:

the SSV, a mounted snapshot of the unmounted read-only System volume named by default Macintosh HD, which forms the root of the boot file system. The snapshot is named com.apple.os-update- followed by its UUID, and the volume (hence its snapshot) is typically about 15 GB in size;
the writable Data volume, by default named Macintosh HD – Data, which is normally hidden from view at /System/Volumes and accessed via firmlinks;
Preboot, a small volume;
Recovery, the paired Recovery Volume;
VM, containing virtual memory caches.

Note that the Seal on the unmounted read-only System volume is normally broken, but it’s the snapshot that’s important: that should be sealed, unless you have broken its seal intentionally.

On Apple silicon Macs running Big Sur, but not Monterey or later, the paired Recovery volume isn’t used, but Recovery mode always boots from its hidden container on the internal SSD.

Apple silicon Macs

Unlike Intel Macs, M-series Macs always start their boot process from their internal SSD, even when they’re going to complete it from an external bootable disk. This is to ensure that their boot process is secure throughout, using a verified chain of trust through each step in the process.

This starts with the Boot ROM, which validates the Low Level Bootloader (LLB), stage 1 of the boot process. The LLB in turn validates other firmware to be used in Stage 2, the LocalPolicy to be applied to the external startup disk, and iBoot (Stage 2) itself, in accordance with the requirements of the applicable LocalPolicy.

The user controls LocalPolicy through Startup Security Utility, which is only accessible in Recovery Mode, and requires user authentication. There is no LocalPolicy which applies to all users and all disks, though: each LocalPolicy is specific to a System Volume Group and authorised user. For example, these can allow:

a single bootable external disk to be used to start up two or more Macs;
one Mac to be started up from any of several System Volume Groups, which can be running older versions of macOS, or load third-party kernel extensions.

Default LocalPolicy created for each bootable external disk provides Full Security, which among other things blocks the loading of third-party kernel extensions and requires SIP to be fully enabled. iBoot (Stage 2) validates kernel collections, signed System volumes, and other components to ensure their integrity, and that the kernel, extensions and macOS to be loaded have an acceptable version number.

LocalPolicy is created when installing macOS to an external disk, when the boot volume group on that disk is assigned its Owner. It can also be created when selecting the boot volume group on a bootable external disk to be the startup disk, in the Startup Disk pane (or in Settings), if it doesn’t already have a valid LocalPolicy, for example when you want to boot from an external disk previously created using another Mac.

LocalPolicy can be changed, in particular changing Secure Boot settings using Startup Security Utility, by first booting from that boot volume group, shutting down, and starting up in Recovery mode. That sequence is essential to ensure that the Mac boots into the Recovery volume paired with the System volume whose security settings are to be changed (Monterey and later).

The current startup disk setting is stored in NVRAM, which on Apple silicon Macs can’t readily be reset by the user. In the event that disk can’t be used to boot that Mac, it will normally fall back to starting up from the internal SSD.

This is a more convoluted process in the event that the expected startup disk can’t be found at all: in that case, there will be a long delay in startup, during which the power light will remain on but the display will be black. Eventually, when all hope of finding the missing boot disk is abandoned, the Mac will boot from the hidden Recovery container on the internal SSD, normally used as Fallback Recovery, enabling the user to choose another startup disk and restart from that.

Summary

There’s a great deal more required for an external disk to be bootable than just having a valid macOS installed.

Intel Macs without T2 chips are simplest, but even they now require a complete boot volume group and EFI container/partition. T2 models also require their boot security changed using Startup Security Utility, to allow them to boot from external disks.

Apple silicon Macs are the most complex, in requiring LocalPolicy with an Owner. If that isn’t configured correctly, they will be unable to boot from that external disk. However, they don’t require to have their Secure Boot settings changed: unlike T2 models, by default they can boot from external disks in Full Security.

46Comments

Add yours

1

EcleX on November 12, 2022 at 10:06 am

Awesome information! Many thanks! I understand that:

i) It is not possible to remove the macOS system from the internal disk of Apple silicon Macs.

ii) It is not possible to completely erase-format such internal disk.

iii) If such internal disk fails (hardware failure, or similar), it is not possible to boot from an external disk.

iv) In such latter case, Apple Configurator is require to fix it:

Revive or restore a Mac with Apple silicon using Apple Configurator
https://support.apple.com/guide/apple-configurator-mac/revive-or-restore-a-mac-with-apple-silicon-apdd5f3c75ad/mac

LikeLiked by 1 person
- 2
  
  hoakley on November 12, 2022 at 10:44 am
  
  Thank you.
  1. What do you mean by “remove the macOS system”? Apple silicon Macs (and T2 models) are designed to boot from their internal SSDs. If you don’t like that, then don’t use a modern Mac. You pay a lot for an internal SSD which is wickedly fast, faster than anything you can attach externally. It has inline hardware encryption, the Secure Enclave, and more features that work with that internal SSD. You’d be nuts to choose never to boot from it.
  2. What do you mean by “erase-format”? Do mean completely erase, including the two hidden containers? If so, you can, but that makes your Mac unbootable, and you then have to perform a full restore in DFU mode before you can use it again. However, you can, in Recovery, erase the System and Data volumes from the macOS container. You then have to install macOS again, and migrate from a backup or copy. If instead you want to prepare that Mac for another user, you use Erase All Contents and Settings, which doesn’t actually erase anything but destroys the encryption keys, which is more effective.
  3. No. As I have explained repeatedly, Apple silicon Macs always start their boot process from the internal SSD. That’s the way they’re designed.
  4. If the internal disk has failed, then Apple Configurator 2 can’t fix it, as all that can do is restore to the internal SSD. If the latter has failed, there’s no other option.
  It’s perhaps worth noting that these SSDs are generally soldered onto the logic board (there are exceptions, like the Mac Studio, though, which has them socketed). For the vast majority of users, they’re expected to last ten years without failure. They’re not like hard disks which often struggle to last longer than 3-5 years, as that’s all they’re designed to do.
  Howard.
  
  LikeLike
  - 3
    
    EcleX on November 12, 2022 at 3:04 pm
    
    Thanks. For 1, I meant to delete macOS (leaving no information in the disk, as when was just manufactured and purchased by Apple, before installing anything). For 2, I meant to delete all information (likewise, leaving no information in the disk).
    
    I understand that neither is possible because 1 and 2 can be done, but only if macOS is re-installed again, not leaving the disk without information inside (macOS or any other data). I asked just to know.
    
    LikeLiked by 1 person
    - 4
      
      hoakley on November 12, 2022 at 7:42 pm
      
      I’m sorry, you’ve lost me again.
      When you buy any Mac, it comes with macOS pre-installed, all ready for you personalise and configure it. Customers never get completely blank SSDs, as that would be barking mad for any computer manufacturer.
      So why would anyone want to completely wipe an SSD? For the sheer hell of it? A completely empty SSD is utterly useless, except maybe to put into a paperweight. And when you might have paid $2,000 for Apple to provide that SSD in your Mac, it would be a colossal waste of money, wouldn’t it?
      The only reason that you’d want to ‘remove’ the data on an SSD was to render it inaccessible to others. As I carefully explained, there are two ways to do that, neither of which involves deleting any data as such:
      – to prepare a Mac to hand on to someone else. In that case, use Erase All Contents and Settings, which does the job in less than a minute. What it does is destroy the encryption key to the user data on the Data volume (and all other personal data, including LocalPolicy). That makes all the data inaccessible, while leaving macOS intact, ready for the new user to personalise and set up.
      – to put the Mac back to ‘factory, as new’ condition, the same as it was when it was unboxed. To do that, you put the Mac into DFU mode and restore it from the appropriate IPSW image. When that Mac starts up, it’s exactly the same as it was when it was first started up new, with no user data at all, and your chosen version of macOS ready to set up. Restoring like that normally takes about 15 minutes.
      Why ever would you want to do anything else?
      Howard.
      
      LikeLike
    - 5
      
      EcleX on November 12, 2022 at 9:58 pm
      
      Thanks. But I did not mean it. I did not mean to sanitize the disk for whatever reason. As I said ” I asked just to know”. Just to know if that is possible. Simply to know. Because that is possible with Intel x86 Macs. You could wipe out all contents of the internal disk. I understand that is not possible with Apple silicon Macs, which must always have macOS inside. But as said, my question just just for curiosity. Just to know. I like to know. Just that.
      
      LikeLiked by 1 person
    - 6
      
      hoakley on November 12, 2022 at 10:18 pm
      
      Ah right. It was just a vexatious question, then.
      I won’t bother answering them in future. If it’s just for idle curiosity, you can find out your own answer rather than using up my time on a Saturday when I have much better things to do.
      And technically you can’t “wipe out all content of the internal disk” of an Intel Mac anyway. I’m not going to explain any further, as you can discover that for yourself.
      Howard.
      
      LikeLike
    - 7
      
      EcleX on November 13, 2022 at 8:04 am
      
      Thanks. As far as I know, it boils down to the following, as summary, when trying to boot different Macs from the very same external disk (just for the sake of knowledge; the joy of knowledge):
      
      1. In the old days, you could boot different Macs from the very same external disk. You could even use the internal disk for Time Machine backup only. No booting macOS inside. That was great to work from such disk at work and home, for instance. All your work in your pocket. Much convenient than carrying a MacBook (larger screen, more powerful desktop Macs, smaller and lighter external disk, etc). No problem at all, except that you might need to update the involved Macs firmware from time to time. But it was easy, since it was available as a standalone installer from Apple, and it was not a frequent update, anyway.
      
      2. Then, that was also possible except for the firmware, which required to install macOS in any (internal or external) disk connected to such Macs, since the firmware standalone installer was no longer available. That is the current situation with Intel x86 Macs. It is a waste of time and energy to have o re-install macOS when not needed (since the external booting disk was already updated) just to update the firmware, but you could still use the internal disk only for Time Machine backups if needed.
      
      3. And now with Apple silicon Macs, that is not possible as previously done. You can still use an external disk to “boot” (sic) such Macs, but that is not really like before. Because the internal disk must have macOS installed to boot now. And of course, you must now install macOS in the internal disks of the involved Macs, which means that you cannot use it only for Time Machine backups. Besides, “booting” different Apple silicon Macs from the very same external disk and working from it all day long, for instance at work and home, is no longer recommended and problems might arise in the future with new macOS updates or upgrades.
      
      I know that it could be faster to boot from the internal disk, but current disks and Macs are so fast, that you do not notice that (at least with Intel Macs) except for longer booting time (for instance, 20 vs 40 seconds), which is irrelevant when done once or twice a day. Even more amazing, you do not notice difference in the working workflow speed after booting from an external SSD with USB 3.1 Gen 2 with ~540 Mbps interface (like Samsung Portable T5) versus an Apple internal 3,000 Mbps disk if you connect the former to the Thunderbolt 3 (40 Gbps) port of the Mac. You do notice difference when connecting to the slower USB 3.1 Gen 1 (5 Gbps) port of the Mac or when using a slower (~450 Mbps) external disk (like Samsung Portable T3) . That is for standard workflows. Of course, faster disks and connections make a difference when working with large monolithic files like huge movies in which sequential read/write is important beyond the ~540 Mbps threshold indicated above. Finally, no concern about security. All disk contents can be public. For people with personal private data, encryption and using internal disks may be useful.
      
      That is my conclusion from the available information and my experience, and I think that it is correct. I asked to know if that was correct. As said, all that is for the joy of knowledge; just that. I like to know. That is why I read this great site. And sorry if I did something wrong, which was not my intention.
      
      LikeLiked by 1 person
    - 8
      
      hoakley on November 13, 2022 at 9:02 am
      
      Thank you for your essay. Unfortunately, much of the latter part of it is flawed by statements that are blatantly incorrect or misleading at best. I’m not going to waste any more of my time, other than highlighting what you clearly haven’t understood correctly. The following:
      “And now with Apple silicon Macs, that is not possible as previously done.”
      “you must now install macOS in the internal disks of the involved Macs”
      “you cannot use it only for Time Machine backups”
      “problems might arise in the future with new macOS updates or upgrades”
      “it could be faster to boot from the internal disk, but current disks and Macs are so fast, that you do not notice that”
      “you do not notice difference in the working workflow speed after booting from an external SSD with USB 3.1 Gen 2 with ~540 Mbps interface”
      “All disk contents can be public. For people with personal private data, encryption and using internal disks may be useful.”
      I have assured you several times that your current archaic and perverse scheme for using Macs remains perfectly possible with Apple silicon Macs. The user doesn’t have to install macOS on their internal SSDs – that comes pre-installed, as with all Macs. That’s a BENEFIT to the user, even if they are going to be crazy enough to run their Mac entirely booted from external disks. That system and other overhead including Recovery (which you need on the internal SSD, whatever you’re going to do) impose an overheard of around 20-25 GB maximum, and if you really want to, you can use the whole of the rest of that SSD for Time Machine backups. So you could continue doing what you’re doing if you really want.
      Pause for a moment, though, and ask yourself whether it’s really sensible to use an SSD with around 7-8 GB/s read and write speed for making backups, which are deliberately throttled by macOS, to a speed slower than a hard disk? That’s what macOS does when making TM backups.
      As for encryption, security, protection against loss/theft, and so on, I’m really happy that you have no need to protect any of your data, such as you calendar, address book, passwords, email and so on, that would allow someone to steal you whole identity and impersonate you. Maybe if someone were to steal the SSD you carry between Macs, or one of those Macs, it would prove more than a minor inconvenience?
      I don’t think that you have ever used an Apple silicon Mac, nor assessed the features it provides. Nor do you have any inkling of how any of this works, or what its implications for the user are. I have a lot of articles about Apple silicon Macs here, explaining such matters. You might like to spend a little time reading them carefully in a bid to improve your understanding.
      Or maybe Apple and everyone else has got it wrong, and you’re the only person who has got it right?
      I’m perfectly happy that you’ve convinced yourself that Apple silicon Macs offer you nothing except trouble and disruption. You’re welcome to that opinion. But please stop pretending that you understand them when you manifestly don’t have a clue.
      Howard.
      
      LikeLike
    - 9
      
      hoakley on November 13, 2022 at 10:07 am
      
      Rather than carting external storage around with you, why not consider the following solution.
      One desktop Mac at work, one Mac (a laptop perhaps, even a MacBook Air if you want) at home or travelling with you. Each has your key working apps installed, and its own local Home folder.
      Common data such as address book, calendars, and keychain all shared in iCloud. You will still need to manually copy any custom certificates, though, as they’re not shared by Keychain in iCloud, but that’s hardly onerous.
      Around 80% of your common documents will be static. Those can be set up initially by copying from one Mac to the other, and thereafter by copying via iCloud Drive.
      For the 20% or so which are active and changing, put them into a folder in iCloud Drive, which you use as you main working folder on both Macs.
      You could put the whole of your Desktop & Documents into iCloud, but I’ve never liked that and it’s less efficient with significant quantities of static documents.
      Each Mac then backs up using Time Machine to a local SATA/USB-C SSD.
      This is the sort of setup used very widely and with great success. It makes best use of the capabilities of modern Macs and macOS, rather than trying to fight against them. It’s a round peg that fits nicely into a round hole.
      Howard.
      
      LikeLike
    - 10
      
      EcleX on November 13, 2022 at 12:20 pm
      
      Thanks for the information. I do really appreciate it. There are several possibilities, as you said. I personally do not like the iCloud one very much. I prefer to have all locally. Among other issues, iCloud or Internet may fail sometimes, or may not be available in some sites.
      
      LikeLiked by 1 person
    - 11
      
      hoakley on November 13, 2022 at 9:54 pm
      
      Well, even in Third World countries like the UK, reliable Internet connections are almost general, except in the most remote and unpopulated places like the Scottish Highlands. Here, in rural southern England, our Internet connection is more reliable than the mains power supply. Our daughter, who lives on the northern tip of the Outer Hebrides, has better Internet access than we do – it’s essential there as so many work remotely. Very, very remotely!
      iCloud use is also not dependent on continuous Internet connection: the files you’re working on are downloaded for you to edit them, and synced up to iCloud when the connection is available. For example, if you worked on a document in the evening, it would then have overnight to sync up for it to be available fully updated when you got into work in the morning.
      Millions of Mac and iPad users rely on iCloud, with very few problems. And when you do need to work in other sites, all you need is an Internet connection and you have full access to everything in your iCloud Drive.
      Howard.
      
      LikeLike
12

Simon Simpson on November 12, 2022 at 11:33 am

Blimey!

I have, in the past, found it invaluable to be able to have an external bootable disk. It does seem awfully complicated now.

Sometimes I long for the good old days when everything was simple and Macs were powered by steam.

LikeLiked by 1 person
- 13
  
  hoakley on November 12, 2022 at 7:27 pm
  
  Thanks.
  As I say, normally all this is handled by the macOS installer when you set a bootable external disk up. I can’t recall when I last had to do anything to correct or fix that. However, there are also more points of failure, so dealing with problems has become much more complex.
  The best answer is to ask someone else to fix it for you!
  Howard.
  
  LikeLike
14

A. Klinck on January 6, 2023 at 9:48 pm

Your site here is an amazing repository of information, so thank you for that. It’s unfortunate that simply booting from an external drive has become such a complex problem now that there’s a need for such expertise!

I had started typing up a bunch of details about our needs, but I’m going to backtrack and try to keep it short (re-reading now, I failed at that, sorry!). Bottom line is that we need to have a bunch of “different” machines, running different versions of macOS in different configurations, and they run on different networks with different firewalls. Being able to easily boot from different external drives and simply not allowing the internal drive to mount saves us from the financially unmanageable alternative of buying and maintaining a slew of physical devices. VMs could kinda-sorta work for some of this, but not for most of it. For several years we have been able to manage all of this with only 3-4 physical devices, and it’s as easy as swapping a thumb drive and rebooting. This new boot process and “security” that blocks us from even being able to override for our own purposes feels dystopian, but I digress…

So after reading many, many articles, including a number of yours here, it seems that our basic mode of operation may just simply off the table at this point on new machines – or at least I can’t get it to work and from what I’ve been reading I’m not sure it even *can* work. We have several master bootable thumb drives with different versions of the OS and some generic utilities installed. We clone those drives (with dd), install other custom stuff and tweak, and now we essentially have a whole array of “different machines”. Many of these are never allowed to touch the open internet, and Apple’s always-tethered-to-the-mother-ship mindset now is becoming a drag as well. We’ve been doing this for years, starting, I think maybe as far back as El Capitan, and most recently as new as Catalina, but now we’re in a pinch with a new test M1 laptop with Monterey, and looking to the future.

So the whole aspect of using APFS and slower drives is unfortunate, but tolerable. It doesn’t matter a huge amount whether it takes 30 seconds or 5 minutes to boot the machine, as long as it can function reliably once it’s up and running. But, after spending about a *week* now, I have not been able to get a single bootable external drive, whether thumb drive or large-capacity rotating drive, to even boot. For the first time in memory (I’ve been a dev for well over 30 years) I’m basically at a loss of how to simply build a bootable drive, and it’s infuriating.

Setup:
– MacBook Pro 13, M2 processor.
– Monterey (internal is 12.5, downloaded installer is 12.6.2)
– Several different external drives, various thumb drives and a 2TB seagate external.
– External drives are mounted via a Satechi adapter, thanks to zero USB-A slots, sigh. Actually, some dual-interface thumb drives were installed directly.

What I’ve done:

– A ton of reading and setup of our own stuff, in preparation.
– Used SuperDuper! to attempt to clone the internal drive, including the OS and our core tools, that failed. Various iterations on this that I probably don’t need to explain.
– Used Carbon Copy Cloner to attempt to clone the internal drive, including the OS and data, that failed. Also various iterations that I probably don’t need to explain.
– After the above failures, I hugely reduced expectations and used a freshly downloaded Monterey Installer to attempt to install just the vanilla OS without our work environment in the hopes that a data-only copy with SuperDuper! or CCC would work to (less efficiently) build our boot drives after the OS was installed. But even the vanilla OS doesn’t work.

I’ve changed the Boot Security mode to “reduced” (not “permissive”). I fought through many, many iterations of “authorizing” the external drive based on the internal drive’s admin account. I’ve erased the drives and formatted them at different times with both APFS and Extended/Journaled (I’ve read different recs in different places). TBH I’ve lost track of all the iterations of everything I’ve tried at this point.

So what’s happening now, is that after attempting to install a fresh copy of Monterey 12.6.2, the machine reboots from the external drive, the thermometer gets to the end and then reboots. Then it tries again and gets about 1/3 of the way and reboots again, and repeats that over and over, never getting farther than about 1/3 of the way. I’ve let it repeat up to about 10x, just to be sure it wasn’t trying to install something more and reboot. This happens the same whether on smaller test thumb drives or on a 2TB seagate external.

Is there something grossly obvious I’m missing? The Monterey installer is drop-dead simple, it’s not like I have a ton of options to mess with. If I install to the external drive, then boot from the internal drive and mount the external to look at it, there’s a System folder there, but it doesn’t look complete, and the Applications/Library/Users folders are just empty, so it looks like the install failed, but it’s not even consistent in how much gets installed – I’ve seen /Users populated at times. I’ve booted into Startup Options so many times at this point, and I’ve looked at log files, but it’s really hard to discern anything meaningful from them. If there are specific things to look for in the logs I can try, but there are two drives involved with the boot process, I’m not even sure what is being logged where anymore.

I realize that this mess isn’t your responsibility, but I’ve seen you answer so many questions, sometimes with the patience of a saint, lol, and I’m at my wit’s end. Is there something I’m obviously doing wrong, and/or is there some other info about our setup or logs that would be helpful? We simply want to be able to continue to boot reliably from a bunch of different external drives, and Apple has made this extremely difficult. Thanks!

LikeLiked by 1 person
- 15
  
  hoakley on January 6, 2023 at 10:22 pm
  
  I’m sorry to hear of your problems, but much of what you are trying to do is making success near impossible.
  Don’t try to make a thumb drive bootable. You can make bootable hard disks, but I don’t recommend them except in an emergency.
  When installing a bootable disk, it’s usually best if it’s connected direct to one of the USB-C/TB ports on the Mac. Yes, you can succeed with a disk connected to a USB-A port or via a dock, but again that can reduce the chance of success.
  Never try cloning a boot volume group. You can get them to work, but again they make failure more likely. The best way to create a boot disk is to install macOS onto it and migrate or otherwise set up that boot volume group in the normal way.
  If you want to be able to clone configurations, then the best way is to use lightweight virtualisation, as it’s really simple then. Provided you don’t need Apple ID, virtualisation works really well.
  Downgrading secure boot on the internal SSD does nothing to help install a bootable external disk – Apple silicon security is quite different from that of the T2, where booting from external disks has to be enabled in Startup Security Utility. All Apple silicon Macs running at Full Security are just as capable of booting from an external disk, which is all about ownership, not secure boot.
  In the first instance, I suggest that you upgrade the M2 to 13.1 internally, then create an external bootable SSD with 13.1 on it. That’s a good starting point from which you then work out how to install an older macOS onto an external disk (you’ll need to do that from an external bootable installer, as Ventura normally blocks the Monterey installer).
  With care, it’s perfectly possible to have a single external SSD with half a dozen different bootable systems on it, as I have done myself.
  Sadly, with an M2, you won’t be able to boot from anything older than 12.4, but you should be able to boot it into everything since.
  I wish you success!
  Howard.
  
  LikeLike
16

A. Klinck on January 7, 2023 at 8:35 pm

Thanks for your quick response!

Yes, this is likely to be imperfect moving forward, at this point we’re trying to understand just how imperfect it’s going to be after things working so cleanly and easily for several years.

There are so many things to consider…

– Other than speed, is there a reason to not use thumb drives? Reliability compared with spinning platters? Speed is almost always less of an issue for us than ease of use and portability, and with thumb drives we can afford to keep multiple backups handy for all configurations. Lugging around a live, spinning disk together with a laptop always feels dangerous. External larger-capacity SSDs would be another option, but they’re prohibitively expensive when you’re talking about a bunch of them. TBH, I do the bulk of my daily computing these days booted from a thumb drive. I love ‘em.

– Is a dock adapter really more likely to cause installation problems than a simple cable adapter?? If so, that’s disconcerting all on its own! The lack of even a single USB-A port anymore is already infuriating, but it seems very unlikely that Apple is going to give us that option again. Obviously there are thumb drives with USB-C now, but they stick out from the machines, and many interfere with the neighboring port, where low-profile USB-A fit safely embedded. But USB-C thumb drives are still better than a corded external drive for portability. Everything about these M1/M2 machines other than speed and power consumption feels like a downgrade for us 😦 And yet, to be able to continue testing and using modern machines, we have no choice but to figure out how to use these new machines.

And of course the Seagate drives come with their funky Micro-B USB connectors. I will buy a new cable for the Seagate to try it out, but is this really a likely culprit to not being able to install the vanilla OS at all? Or just the fact that it spins? I guess we’ll see soon.

– re: cloning a boot volume group. Yeah, our lives have been easy, but no more.

– re: lightweight virtualization. I just started reading about what you’re doing with that, and it sounds really cool, but I’m way behind the curve understanding how that works on a Mac, and whether/how we can build multple fingerprint-resistant virtualized machines with different OS versions, either on a single or multiple external drives. We don’t need AppleID or iCloud at all, so no problem with that! I need to read more and perhaps experiment with this. I read in your post from last July that you were recommending VirtualBuddy over Viable; how’s that coming along? Current thoughts? Perhaps the most important thing for us (other than general robustness) is isolation. Without getting into detail here, each instance *cannot* have any opportunity to break out of its virtualized environment and expose the host or other clients. Malware is an everyday concern. This is where separate thumb drives are perfect; zero risk of that.

– re: Secure Boot / Startup Security. Okay, thanks. I wasn’t totally sure about that.

– re: upgrading to 13.1 first. I’m a little surprised, but willing to consider doing this. I’d recently read about M1+ machines requiring any external boot drive have an OS that was at *least* as new as the internal SSD. I’m okay with this new machine only being able to boot 12.4 (or 12.5) as it’s base/oldest OS, whether internal or external, but I don’t want to lose the ability to run 12.x. If your experience is that I can set up a 12.4 or 12.5 external bootable drives from this M2 after updating the internal to 13.1 we will definitely consider doing this, pending some of the questions here. However, given that I can’t get a single external install to work, even just a vanilla OS, I’m reluctant to update the internal SSD yet, if that makes sense.

My plan right now is to get a new USB micro-B to C cable and try another vanilla install to the 2TB external. If that works, I can start experimenting with other installs and/or using tools like SD! or CCC to copy our setups over the top of a vanilla install rather than doing it in one clone operation.

That said, if the installations still keep failing, I’ll have to see what information I can cull from logs or whatever to see if it’s possible to understand what’s happening. We’re completely dead in the water with this new machine right now. I could see where thumb drives or spinning media might have runtime issues due to speed, but I’m having a hard time imagining how it would prevent a proper installation.

Sorry, about the length again, I really like to spell out as much as possible when asking for help; IMO too much information is almost always better then too little. Thanks so much.

LikeLiked by 1 person
- 17
  
  hoakley on January 7, 2023 at 10:44 pm
  
  There are basically two starting points you can choose between:
  – you can start with a well-known and standard method of creating an external boot disk, demonstrate that works, then try the variations to that to see which break it.
  – you can keep trying different non-standard methods until, by chance, you might stumble on one that works.
  For example, thumb drives. I don’t of anyone who has successfully booted an M-series from a thumb drive. Indeed, when you create a bootable macOS installer for an Apple silicon Mac, it doesn’t actually boot from it in the way that an Intel Mac does. ISTR hearing that thumb drives and similar media can’t have ownership, for security reasons, thus can’t be boot disks, although that might be wrong.
  Given that your boot disk contains the Data volume, to which a great many cache and other files are written, a thumb drive isn’t a suitable boot disk anyway.
  Hard disks aren’t too bad until you start using them. The problem is that APFS doesn’t know anything about fragmentation, or care about it. Not only do the data blocks get horribly fragmented, but so does the file system metadata. Run for any length of time from a hard disk it steadily grinds to a halt as it explodes into fragments. So, OK for a few emergency boots, but you really don’t want to do that for long.
  For some years now, Macs (like the rest of the industry) have been moving to standardise on USB-C connections, running USB 3.1 Gen 2 and Thunderbolt 3 or 4. Most modern external storage now follows that for simplicity and performance. The further that you move from those, the less likely your external disk is going to boot. Yes, I’ve booted from external disks connected to USB-A ports, and on Thunderbolt 4 hubs. But that’s not through choice.
  You are extremely unlikely to get any useful information from the Unified log about failure to boot from an external disk. It’s not designed to do that.
  I don’t know who has told you that your M2 won’t boot from an older macOS on an external disk (that’s older than the version installed on its internal SSD), but that’s simply untrue. The difficulty is installing that older version of macOS when running a newer one, as macOS blocks the installer app from running directly. You’ll find at least a couple of articles here explaining how to work round that, using an external bootable installer (which can be on a thumb drive, because Apple silicon Macs don’t actually boot from them). And you can certainly run macOS right back to 12.4 in a VM.
  Finally, if you need isolation, then you shouldn’t boot from an external disk at all, as it’s not isolated from the internal SSD. The only way to isolate macOS on Apple silicon is to run it in a sandboxed and locked-down VM, as I did recently to assess malware.
  Howard.
  
  LikeLike
  - 18
    
    A. Klinck on January 11, 2023 at 7:42 am
    
    Indeed, those two methods are exactly what I was fighting with at first. But after a quick handful of failures at method #2 I did switch to method #1 fairly quickly. However, even that has been a dismal failure thus far.
    
    Right now, I have the existing setup described above, with a fresh Monterey Installer for 12.6.2. I long since gave up on thumb drives for testing with this machine, and bought a direct USB micro-B to USB-C cable to bypass the adapter after you said that might cause problems. Erased the disk with Disk Utility, to a clean APFS. Ran the Monterey installer and it failed again, just like before. After installation, the thermometer runs to 100%, reboots, then goes to about 1/3 and reboots again and repeats that cycle forever (or at least maybe 10x before I killed it).
    
    So now, after reading a lot of discussion and your advice, I went and bought a brand new Samsung T7 SSD. That has to work, right? Erased it to APFS and ran the downloaded installer again and the same thing happens.
    
    Is it possible that some type of security issue is preventing a basic installation from completing? If I look on the SSD now, the System folder exists and contains 6 folders, but the Applications, Library and Users folders are empty.
    
    The machine is actually a loaner that I was able to snag before it went into production elsewhere, to test our workflow on a machine with an M1 (M2) processor, and I’ve kind of over-kept it already. But that’s why I’m reluctant to arbitrarily upgrade someone else’s machine to Ventura. Still, I don’t see that being the problem here.
    
    How can I check to see why a basic installation is failing? Thanks.
    
    LikeLiked by 1 person
    - 19
      
      hoakley on January 11, 2023 at 7:51 am
      
      The normal reason for this failure is that creating LocalPolicy fails, normally because ownership can’t be established. Yes, you can look through LocalPolicy on the internal SSD, but I don’t think that will provide any reason for failure.
      Although a few users report problems with the T7, I have set one up without any trouble at all, and booted fine from it. That was in Ventura, though.
      Howard.
      
      LikeLike
20

A. Klinck on January 7, 2023 at 9:33 pm

Ah, I just found the ViableS article. Clearly I have much reading to do on this new-to-me world of lightweight virtualization on macOS! I will probably still have questions about how (or if) this option might work in our situation, but I will continue reading first. Great stuff.

LikeLiked by 1 person
21

A. Klinck on January 8, 2023 at 9:22 pm

Again, great stuff. I will write more later tonight, but I have a quick question. It doesn’t look like Viable/ViableS has a way to choose between bridged and NAT networking, is that right?

In our case, “isolation” means not just from the disk-access perspective, but also the network. Now, each of our bootable thumb drives is configured to access different networks (which in turn run different firewalls), and at no time do we want the host and client to be sending packets out the same network. Connecting host/client to 2 separate WiFi networks is one option, but also if we could simply shut down networking entirely on the host and only allow on the client, that would work as well. Even if it’s not implemented yet, is this something that would be *possible* through lightweight virtualization?

This stuff is really exciting, and we’re seriously trying to consider whether it might be possible to do much of what we’ve been doing via bootable thumb drives via virtualization, just not yet sure if it’s possible.

LikeLiked by 1 person
- 22
  
  hoakley on January 8, 2023 at 9:35 pm
  
  Bridged networking is only available if Apple grants a special entitlement to use it. I haven’t applied for the entitlement, and don’t know whether I would be granted it.
  Even then, I’m not sure if would do what you want. Although it does allow the user to choose which network interface to use for the VM, Apple explains: “The host computer and your virtual machine share access to the physical network interface, but communicate over it using distinct network layers.”
  Although it might be possible to run a VM from a thumb drive, you really don’t want to – write performance would be dreadful. I recommend a decent Thunderbolt 3 NVMe SSD for hosting VMs. It does make a big difference.
  Howard.
  
  LikeLike
  - 23
    
    A. Klinck on January 11, 2023 at 7:27 am
    
    I’ll add a small note here before moving to the bigger question.
    
    I’ve found something quite interesting while experimenting with Viable/S. It turns out that the client VM can bypass host firewalls like Little Snitch. I thought LS was installed as a kernel extension, but I’m probably way out of date on that, because the client VM completely sidesteps LS. The client VM is indeed restricted by /etc/hosts, so the world is still sane, but the possibility of completely blocking all outbound network requests on the host and then setting up different client firewall restrictions than the host might allow for some interesting setups. I’m not sure what types of system processes might not be blocked by LS on the host, so that may still be an issue.
    
    LikeLiked by 1 person
    - 24
      
      hoakley on January 11, 2023 at 7:47 am
      
      Thank you. I presume that’s the way that NAT works through the host.
      Recent versions of LS don’t use kernel extensions, but system extensions instead, because of the limitations on loading kexts on Apple silicon Macs.
      Howard.
      
      LikeLike
25

A. Klinck on January 11, 2023 at 9:06 am

Looks like maybe too deeply nested to do a proper Reply.

re: “The normal reason for this failure is that creating LocalPolicy fails, normally because ownership can’t be established. Yes, you can look through LocalPolicy on the internal SSD, but I don’t think that will provide any reason for failure.
Although a few users report problems with the T7, I have set one up without any trouble at all, and booted fine from it. That was in Ventura, though.”

Sorry, but how do I look through the LocalPolicy? I’m kind of at the end of the road right now, and will probably have to give up this machine tomorrow.

LikeLiked by 1 person
- 26
  
  hoakley on January 11, 2023 at 9:11 am
  
  I have two articles which step through this in detail:
  Booting macOS on Apple silicon: LocalPolicy
  Booting macOS on Apple silicon: Multiple boot disks
  If you want to search this blog, there’s a magnifying glass at the top right of each page which gives excellent search access to every article here.
  Howard.
  
  LikeLike
  - 27
    
    A. Klinck on January 11, 2023 at 9:14 am
    
    Yes, as soon as I clicked send I regained my senses and found one of those and started reading. Then ran into the roadblock below.
    
    LikeLiked by 1 person
28

A. Klinck on January 11, 2023 at 9:13 am

Sorry again, it’s very late (my time zone), and I’m quite tired and fried. If you mean the Startup Security when you hold the power button during boot, I try to load it and it says:
“Security settings cannot be changed. In order to change security settings, please power off your Mac and then hold the power button to startup macOS Recovery”. And yet, that’s what I did to get to the dialog to attempt to open Startup Security Utility.

LikeLiked by 1 person
- 29
  
  hoakley on January 11, 2023 at 9:18 am
  
  In that case, your Mac is currently trying to start up in Fallback Recovery, or from the wrong paired Recovery mode.
  Restart it from the system you want to change startup security on. Then shut it down. After 10 seconds, start it up again, ensuring you engage primary Recovery by a single long press on the Power button.
  That should take you into paired Recovery, which gives you full access to Startup Security Utility for that system volume group.
  I’m not sure what you want Startup Security Utility for?
  Howard.
  
  LikeLike
- 30
  
  hoakley on January 11, 2023 at 9:20 am
  
  Do you realise that Startup Security Utility isn’t the problem? That’s the utility that lets you change Secure Boot on that boot volume group. Although it does that through the group’s LocalPolicy, that’s not where you access or check LocalPolicy.
  Howard.
  
  LikeLike
  - 31
    
    A Klinck on January 11, 2023 at 9:30 am
    
    But if I can’t even open the Startup Security Utility, doesn’t that imply something is wrong?
    
    LikeLiked by 1 person
    - 32
      
      hoakley on January 11, 2023 at 9:35 am
      
      Yup
      Howard
      
      LikeLike
    - 33
      
      A. Klinck on January 11, 2023 at 9:37 am
      
      I’m now reading the (I hope) proper page (first link you gave just above). I was on a different page earlier. You have so much information on your site it’s overwhelming! And much of it, like this page, is dense material, I need to read carefully, and reaching the end of my ability to do so tonight, unfortunately. I do really appreciate your help. Rebooting again now to go look in ISCPreboot…
      
      LikeLiked by 1 person
34

A. Klinck on January 11, 2023 at 9:29 am

Shut down.
Physically disconnected the external.
Rebooted on internal.
Shut down again.
Long press reboot into recovery.
Selected from the menu: Utilities -> Startup Security Utility

Got the same error message above.

I guess I can go into Terminal and use bputil. I just wanted to check the security policy, but it seems like something is awry. (?)

There’s a lot of info in bputil -d, but possible items of interest:
– OS Pairing Status: Not Paired
– Pairing integrity : Valid
– Security Mode: Full (I reduced this a couple days ago, but restored to full)
– SSV Status: Enabled

Anything else of merit?

LikeLiked by 1 person
- 35
  
  hoakley on January 11, 2023 at 9:35 am
  
  If you can’t use Startup Security Utility, then that Recovery is broken. No wonder it can’t create a LocalPolicy.
  I’d do a full restore in DFU to current Ventura, but that’s not something to do with a loan Mac.
  Howard
  
  LikeLike
  - 36
    
    A. Klinck on January 11, 2023 at 9:45 am
    
    re: loaner; I know, right? But on the other hand, can I return it like this? I’m in a bind. Plus, I spent a couple days setting up their system, installing all kinds of apps and pre-configuring stuff for them.
    
    Looks I actually have 12 (!) UUIDs in iSCPreboot. I guess that’s how many drives I attempted to install an external boot drive onto over the past week.
    
    That said, if I use Disk Utility, select the (internal) Macintosh HD volume and click Info to find the UUID for that volume, I find a UUID that has no match amongst all the folders in iSCPreboot. That seems like a problem, right? How could that happen, and is there a way to fix this short of DFU?
    
    LikeLiked by 1 person
    - 37
      
      hoakley on January 11, 2023 at 9:55 am
      
      I’m sorry, I don’t know.
      Howard
      
      LikeLike
38

A. Klinck on January 11, 2023 at 10:19 am

Darn.

Well, I’m continuing to feed from this firehose, read your 2nd article on LocalPolicy, and I see I was looking at the main “Macintosh HD” volume’s UUID, and not the Data volume embedded there. When I looked at that UUID it does indeed match one of the UUIDs in the iSCPreboot folder. So hopefully that’s a good thing, but that said, I’m not sure what I can discern from it.

Believe it or not, I’ve never actually done a recovery/reinstall. I even had to run off to look up whether that would leave the user data untouched. It looks like that’s a relatively safe option to try now, rather that resorting to a DFU, I think/hope. But I don’t have a clue if it has any chance of being able to fix this problem.

On the other hand, as long as it’s not likely to make things worse, I could give it a try. If it doesn’t touch the /Applications and /User folders I think there would be very little rework for me. Any thoughts before I do this?

LikeLiked by 1 person
- 39
  
  hoakley on January 11, 2023 at 10:26 am
  
  There are two quite different procedures.
  You can reinstall macOS in Recovery, for which you will need a good recent backup. That may well require you to migrate from that backup, although it can sometimes leave the Data volume intact. While that should refresh paired Recovery and install fresh System volume and SSV, it probably won’t touch the hidden containers on the internal SSD.
  If you want to restore everything back to as-new, factory fresh, and restore all ‘firmware’ and hidden containers, then a restore in DFU is required, but that will wipe the internal SSD completely, so you’d then need to personalise the Mac from scratch, and migrate from a backup to its empty Data volume.
  Howard.
  
  LikeLike
  - 40
    
    A. Klinck on January 11, 2023 at 10:40 am
    
    I don’t have a problem personalizing from scratch (I actually got it new in the box before they touched anything), but I really don’t want to redo the multi-days of work to get it all set up like it is.
    
    Before anything I need to get a fresh backup via CCC or SD!, just to feel semi-safe; that’s a good reminder. I managed to purge the one good one I had while trying to do one of these failed OS installs. Then I think I’m going to try a Recovery Reinstall and see what happens. Before I even think about doing a DFU I’d need to read up on that. It’s amazing how much of this stuff has changed in the past few years and I feel completely powerless (and inept) now. I do see some benefits to these changes, but overall I’m not a fan of how complex this has become for the amount of benefit. It feels incredibly fragile now, due to the complexity.
    
    Thank you so much for your help. I will update you tomorrow, but I need to sleep before I do anything else for fear of making some kind of silly mistake.
    
    LikeLiked by 1 person
41

A. Klinck on January 12, 2023 at 8:18 am

I promised an update. Unfortunately it’s not a fun one.

First, I updated our data backup on a spinning media drive; all good. Then went into Recovery and reinstalled macOS to the internal drive. This took something like 4 hours! And required the machine to be online, which is so obnoxious, but I can’t digress now… Fortunately, all the data/apps seem to be intact after the reinstall, and when booting from the internal drive’s recovery I can now open Startup Security Utility, so it would appear that the internal drive’s System is repaired. Yay. Maybe. Since I was there, I set it to “reduced security”, given that it *said* that would allow an offline install, but I don’t believe that part anymore.

So now that the internal drive’s System is (presumably) back at a known state (quietly updates from 12.5 to 12.6.2 behind my back without telling me), I tried to do a basic install to the external SSD (a T7-shield with supplied, solid, USB-C to USB-C cable) via the downloaded Install macOS Monterey app (12.6.2) downloaded directly from apple. Erased the SSD to APFS/GUID. Ran the Installer, but it failed after like 20 minutes when offline (ugh), so I erased the drive again, put the machine online, and ran again and it got through what looked like an installation, but the drive failed to boot again. It creates 4 top-level folders: Applications, Library, System and Users, but the only folder with any files is /System.

So, I decided to the the “tried and true” method of using the createinstallermedia app embedded in that installer. Everyone seems to say this works, and you have it on your 2021/08/26/how-to-create-a-bootable-external-disk-in-macos-11-or-12/ page as well. Erased a 16g USB-C thumb drive to HFS+J, ran createinstallermedia (with the —downloadassets switch this time, to build a proper offline installer) and it did its thing and all seemed to go well. Restarted into Options, was able to select the installer thumb drive, and was able to boot. Great so far.

Erased the SSD again to APFS. Attempted to run the installer offline, which of course was a failure that wasn’t reported until after another 20 minutes. Erased the SSD to APFS yet again, put the machine online, ran the installer again (entered password twice) and after an hour or so it finished. Got the “Restart Your Mac, ready to install” screen, the system did it’s reboot-cycle a couple times, all looking great, but ultimately failed to boot again. Once again, the SSD has 4 folders, 3 of which are empty.

I’m really trying to bring this down to the most simple version of creating just a single external boot drive on SSD, just to get anything to succeed! Ultimately I want to have several bootable volumes on this drive that I can move among different machines, but I’ve completely failed to make a single, simple bootable drive!

It feels like something very basic and obvious is missing, because the end result is consistently wrong through a variety of different types of installation attempts. And yet the internal boots and operates just great. I’ve asked to hold onto this machine for another day or two, but it’s getting to be a tough ask, and I’m virtually out of other ideas at this point, given the basic install that I’m attempting now. I’d be grateful for any thoughts of how I can debug this. Thank you.

LikeLiked by 1 person
- 42
  
  hoakley on January 12, 2023 at 10:00 am
  
  Well done for recovering Recovery. However, I don’t understand why you have reduced Boot Security. As I haven’t tried installing an external bootable disk in anything other than Full Security, you’re now outside my experience.
  I have no suggestions as to what is going wrong, other than Ownership failing. Again, that’s something I’ve not experienced since macOS 11.4. Some users appear to suffer this repeatedly, and I have absolutely not idea why, if they follow the procedure completely.
  Howard.
  
  LikeLike
  - 43
    
    A. Klinck on January 12, 2023 at 10:34 am
    
    I reduced security because I’m strongly averse to any machine being *forced* to talk to Apple’s servers to simply install an OS. I’ve opened up and put this machine back online to do this because it has zero personal, business, or testing data on it, but once that happens with our own machines, they get locked down so they’re never allowed to talk to Apple’s servers, and whitelisted for almost all other (limited) internet activity. There are layers of reasons for this, but not really relevant for this discussion.
    
    FWIW, I’ve been an Apple dev “official/certified” all the way back in the mid-late 80s, and an Apple owner since the late 70s! I bleed in 6 colors, and I’ve only allowed one Windows machine on any of my personal networks (years ago), and that was very carefully controlled and short-term. But this absolute lack of being able to be in any control of Apple machines that we (sort of) own has me strongly considering a switch to Linux for new machines – at least any new ones where we can do that. We are perfectly comfortable with them for a lot of purposes. This has taken almost 2 weeks of my time on this problem that shouldn’t even exist, and I’m no further than I was when I started. 😦
    
    I think I have only a couple more things to try. I can increase the security setting back to full, and repeat, although I did try that previously, before the reinstall. And if that doesn’t change anything, then I can go ahead and update the OS to 13.1 and basically try the same stuff I’ve been trying. I got permission to do that, and I don’t think it will be detrimental in any way to the owner. I do like the virtualization improvements that come with 13, and I’m actually kind of excited about that, but I’m fed up beyond belief with all the complexities that this so-called boot security has brought into macOS. I know Apple has been moving closer toward the iOS model of them controlling everything in macOS also, but they are very different beasts, and they’re playing with fire IMO if they continue to pretend like they own our computers. These are not appliances.
    
    Sorry if I sound extremely frustrated, but I am. You are amazing. Your site here is an amazing resource. I’m incredibly glad to have found it, and I will continue reading through a lot of it over time. So this is no gripe with you At All.
    
    You say you’ve heard of others suffering this, which is both hopeful and frustrating also, because even you don’t know why. But have you heard about any of them recovering later? Maybe by upgrading their OS? I’m not really super hopeful about that, but will try tomorrow barring any other ideas that pop into your or my head. Again, thank you.
    
    LikeLiked by 1 person
    - 44
      
      hoakley on January 12, 2023 at 6:30 pm
      
      “But have you heard about any of them recovering later?” One or two, yes. But as far as I can recall, it just happened. Most never report back, so I don’t know whether they simply give up or not.
      I’ve been an Apple developer since 1989, and had (for example) one of the first apps to use TrueType font internals, to drive laser cutters. One think I have learned is that, while you can sometimes get Apple to listen and to change for the better, you can’t determine its direction. Apple has invested a great deal in security measures and engineering that make booting from the internal SSD far superior to any external boot disk. It also relies on connections with Apple’s servers. The compromises you have to make to avoid those simply aren’t worth the effort, and only make life harder for yourself in the end. With Apple, it’s either go with the flow, or go to something else. As the alternatives are far worse, I just stick with it, and the rewards are worth it.
      Howard.
      
      LikeLike
45

A. Klinck on January 21, 2023 at 11:51 pm

I promised an update, so here we go, and though I have a ton of information and questions, and pages and pages of notes, I think I’ll try to post things here in smaller, digestible chunks instead of multi-page long descriptions. [ edit: I may have failed again with length, but at least I think the question is fairly constrained ]

A little update on what’s happened though. I’ve read many of your pages on this site (as well as many others) some repeatedly, but you had one paragraph — that actually happens to be on this page — that seemed especially helpful, because without understanding the details of the authentication process, on first read it seemed completely unnecessary.

Your comment: “…first booting from that boot volume group, shutting down, and starting up in Recovery mode. That sequence is essential to ensure that the Mac boots into the Recovery volume paired with the System volume whose security settings are to be changed…” led me to better understand the need for multiple reboots to even get to a “known state” as far as LocalPolicy, and I think I may still not be fully wrapped around everything, but that was quite helpful, so thanks.

At the end of the day, what we need is simply a setup that can boot several different OSs/configurations from one or more external drives. Also relevant, many of these setups are fully offline. And the ones that will be booted with an available network are tightly whitelisted firewall, i.e. they will not be able to communicate with Apple servers. Right out of the gate, this presents huge problems with Apple requiring themselves to be in the middle of us booting our own machines in the confines of our own lab environment(s), but after careful consideration and experimentation, I was able to build multiple *generic* bootable APFS volumes on a single external SSD, by reluctantly (with some disgust I might add) temporarily attaching these systems to an open network where they were able to communicate with Apple servers one time *before* any custom tools and proprietary data are added. In theory, as long as we have a set of simple, isolated, externally-bootable drives, we can add our software on after Apple’s authorization and move forward, never touching the open internet again (or in some cases with very specific whitelisting).

I did this with both SuperDuper! “cloned” installs and also one install directly from a Ventura Installer image.

Unfortunately, as soon as I took this machine offline I found very strange behavior. I was often not able to “cross-boot”, i.e. running from external drive ExtA, select ExtB in Startup Disk. Somehow this required a live connection to Apple to simply boot from one authorized volume to another – but not always.

What I *thought* the issue was (and maybe is?) is that each bootable volume has its own separate list of other authorized volumes. That seemed to more or less be the case, so I went through the incredibly tiresome process of attaching to an open network, booting into each drive, and doing an orderly authentication process for each of the other drives. This is a *ton* of booting, and typing a long password up to 4 (and even 5) times per cycle. This had to happen all online during the first pass, then all offline (multiple times) to verify that we could move forward with this SSD and several installed OSs.

This seemed to work at first. So I started extending the process to more volumes, and you can do the math to see how much work this is once you start to get to even just 6 or 7 volumes. But even doing all this, partway through the process I started to have problems again, where sometimes the drives wouldn’t boot without being online and communicating to Apple’s servers again! At this point I was keeping track with careful notes, and I am absolutely certain that volume pairings that worked at one point, did not work later. It feels random at times, and many, many cuss words have been uttered!

So based on the above quote from your article here, I rewound back to the possibility of annoyingly limiting ourselves to always reboot from the internal drive before rebooting from any external drive. In theory this would make the process of doing all the authorizations at least linear with the number of external drives. During this process I also reduced security just one level on each of the external drives (while booted into the *internal* drive’s recovery tools; and again, your words above were instrumental in understanding that, so thanks). I believe the reduced security helps with being able to do boot offline. This actually *seems* to work a bit more reliably — as long as we always boot from the internal and initiate a Startup Options to select the external drive (not simply using Startup Disk selection). All of this is inconvenient at best, but if it worked consistently it might be feasible. But after doing a bunch of testing once things were set up, I still keep getting the occasional problem where the machine says it need to have an internet connection to boot. 😦

It’s totally possible I’m still missing some basic facet of how this works. But, it *feels* like after making the conciliatory gesture of allowing Apple to see a given boot drive one time and authorize it, that we should be done, never to be bothered again. One location where I often work simply has no internet, and if I can no longer boot from an external drive when I’m there, I’m back to wondering if I can continue to use new Macs moving forward; and I do not say that lightly! Constantly being required to be tethered to Apple’s servers to simply boot a machine smells a lot like we don’t own our machines anymore.

So, my questions right now are:

1) Should my method of booting on the internal and allowing a one-time authorization for each external drive work? We can install all of our custom tools and data after doing that, but not if at some point later Apple says we need to get on an open network.

2) Is there something notable that I’m still missing about this process? This is certainly possible.

Thanks for reading.

LikeLiked by 1 person
- 46
  
  hoakley on January 22, 2023 at 8:41 am
  
  Thank you.
  I think you have a couple of fundamental misunderstandings that are making things confusing for you.
  
  1. Paired recovery. Since Monterey, Apple silicon Macs have booted into the Recovery system that is part of their last boot volume group. In Big Sur, that didn’t happen: whenever you booted into Recovery, the Mac booted from the Recovery system installed in a hidden container on the internal SSD. From Monterey onwards, the Recovery system used is that paired with the last boot volume group.
  So if you’re currently booted in macOS 13.0 from an external disk, when you next boot into Recovery, it will be the Recovery volume that is part of that group, i.e. on the external disk and for 13.0.
  This is most important for Startup Security Utility, as the settings in that apply only to that boot volume group. If you boot from that external disk and 13.0 there, then changes to Boot Security apply to that boot volume group, no other.
  Otherwise, you should generally prefer the most recent Recovery, which would normally be that on the internal SSD. But I see absolutely no point in going through a ritual of booting in internal Recovery before booting in something else.
  
  2. LocalPolicy. Each Apple silicon Mac has its own library of LocalPolicy, controlling which boot volume groups it can boot from. These are stored on the internal SSD. Any LocalPolicy that appears to be on an external disk is either your misunderstanding as to where it is, or superfluous. LocalPolicy is accessed early in the boot process, when the Mac is still running from the internal SSD, and can’t be accessed on any external disk at that time. Indeed, it can’t be accessed because until that LocalPolicy has been validated and accepted, your Mac can’t even access any external boot system.
  
  I have absolutely no idea what you’re referring to when you talk about “one-time authorization”. The only way an external boot disk can work is with a valid LocalPolicy, and that can’t be “one-time”: it either works or it doesn’t, and isn’t time-limited.
  
  I’m afraid I have no knowledge at all of running Apple silicon Macs offline, so can’t provide information on that. The only time I understand they have to be online and able to contact Apple is when performing initial personalisation. Beyond that, you’ll need to check with Apple.
  Howard.
  
  LikeLike

·Comments are closed.

Share this:

Related