Recovering from a hard crash

Crashes don’t come any harder than when your Mac just sits there, your startup disk missing or unusable.

These only ever happen when you are shattered, up against a deadline, and haven’t got time to spare messing around with your Mac.

My hardest crash was in Copenhagen. I had been there, at a customer’s, for a few days, and was booked to fly back the following morning. The customer’s final request of my time, in the late afternoon of the day before I returned, was to give his Mac a good tidy up and check, so as to leave it in fine fettle. I trusted a disk utility to do some of this, and it promptly crashed his Mac and rendered the startup drive inaccessible to Mac OS. I spent the rest of the night manually recovering his system before flying back very tired and jaded. At least I left his Mac in better shape than me.

Modern Macs and OS X from Lion Onwards: the Recovery System

Recent Macs running recent versions of OS X are both simpler and more worrying. Once you have got the Mac restarted, if they cannot find a bootable system, they should automatically offer your the Recovery System. If they don’t, restart with Command and R keys held to enter that.

The Recovery System can run from your startup drive, using a hidden partition created there when it was last initialised, from an external drive, notably a USB stick or ‘thumb drive’ created previously, or over the Internet, provided that your Mac has a suitable connection. This depends on the age of your Mac and your prior preparations. If you have repair tools such as Drive Genius 4 or DiskWarrior 5, then you should prepare the supplied USB stick in advance, so that you can also use those tools during recovery.

Following a hard crash, in which your startup drive does not operate normally, the first thing that you should do is run Disk Utility and/or other utilities (such as Drive Genius) to check and repair your normal startup drive. If the crash has damaged the partition map of your regular startup drive, you may be unable to use the hidden Recovery Disk on that drive, which will limit the possibilities: hence a prepared USB stick can be very helpful. Certainly if you care for more than one Mac, keeping a USB stick ready for recovery is a wise move. Apple details how to install OS X on an external drive here, and how to create a bootable OS X installer for a USB stick or similar here.

The Recovery System also provides Safari should you need to refer to Apple’s online documentation on the Recovery process. The key article describing the process is here.

You should, with any luck, have two or more options for recovery. If you can, try local recovery, in which the process runs from the Recovery Partition on your startup drive, or an external drive (including USB stick). This may offer the options to recover from your Time Machine backup, when available, which usually gives the most complete result and should minimise the amount of additional work you need to do, or to re-install your most recent version of OS X.

Internet Recovery will proceed more slowly, as determined by the bandwidth of your Internet connection, and will install the version of OS X which originally shipped with your Mac, leaving you to bring it up to date using updates from the App Store. You could thus have several hours of further work to do, including installation of key apps and other software.

After Recovery: Rehab

Once you have completed recovery, you should be able to restart from your regular startup drive and work normally again. If you cannot, it indicates that there is a remaining problem which you still need to fix, or that the recovery itself went wrong. Unfortunately there is no quick or simple answer to those, and you will need to work through standard diagnostics to distinguish them.

Persisting problems should make you suspicious of a hardware problem, and you should try to rule that out. For Macs released before June 2013, you can run Apple Hardware Test, which is detailed here. More recent models use Apple Diagnostics, which is detailed here. Either way, you start them up by holding the D key during startup.

If your hardware appears OK and you have restored from a Time Machine backup, suspect that restoration. You may find regular diagnostic routines a help, such as inspecting the logs, and even the old panacea of repairing permissions, but this can be a long a troublesome task. Sometimes it is simpler to start again, re-install OS X, and hope that runs better.

One problem which can arise after recovery or any other re-install of OS X concerns permissions. This is because OS X (like any other form of Unix) does not store user names for file and folder permissions, but uses a number to identify the user; for the primary admin user this is normally 501. When you re-install OS X the primary admin user should remain set at 501, and that should associate with that account. Sometimes though the new primary account is created with the ID 502, which is not associated with those files and folders whose owner is still set to user ID 501. The best way to deal with this is internal surgery to the user ID list, which requires a bit of Unix wizardry. Thankfully it rarely occurs now.

Older Macs: Back to Basics

With older Macs and older versions of OS X, prior to Lion, you will need to use more traditional methods, as detailed in the other articles here. It is important to remove as many peripherals as possible before you try to get your Mac up and running again, so that they do not complicate matters. If necessary, revert to vanilla USB wired mouse and keyboard.

You then need to choose whether to restart from an external drive, from which you can diagnose and repair your regular startup drive, or in Single User Mode, if that is possible.

Single User Mode allows you to attempt to repair your startup volume using the standard shell command
/sbin/fsck -fy
Repeat until it runs without any errors or repairs. You then restart using the command
reboot
with your fingers crossed, lucky rabbit’s foot ready, and so on, hoping that all will work properly again.

Once you have got your regular startup drive functioning again, and can start up from it, do so in Safe mode, with the Shift key held down. This forces another directory check of the startup volume, allows only required and standard kernel extensions (KEXTs) to load, and disables all startup and login items. All fonts, other than those in /System/Library/Fonts, are disabled, and font caches flushed. It also flushes the dyld dynamic loader shared cache, which sometimes caused ‘blue screen’ freezes following failed OS X updates. You will find that many features are disabled in Safe mode, but it gives you a chance to clean up after the crash, and restore order to your startup disk before going fully live again.

If you cannot get your Mac to start up from a hard drive, the next best option is likely to be your most recent OS X Install disk, or an AppleCare diagnostic disk.

Recovery Without Restarting

If the crashed Mac is a server, or you need to keep it running to work out what caused the crash, you may still be able to recover it without a restart. If you have enabled the sshd service (in the Sharing pane) and set up its authentication keys beforehand, you may be able to try this. Open Terminal on another networked Mac and type
ssh username@machineaddress
where username@machineaddress is your user name on that Mac and its resolvable network address, such as the IP address. Provided that you have set this up before, and the private key system stored in ~/.ssh works, you can then control the Mac remotely from Terminal’s command line. This allows you to gracefully shut down services running on a server, and sometimes even bring it back up to full functional capacity. Typing
logout
closes that ssh connection. This is documented in the Command Shell Tools manual for your version of OS X, and all good Unix books.