Should you enable defragmentation on APFS hard drives?

Like it or not, many of us are still using hard drives after we’ve upgraded to High Sierra or later. For bulk storage of more than a terabyte of files, they’re still the only affordable option for most users. If you’re using your hard drive(s) for Time Machine backups, then you have no option as to their format, as those backups must go onto HFS+ volumes. But backups made using other software, and simple file repositories, could be in either HFS+ or APFS.

Choosing which to use can be difficult. APFS was designed primarily for SSDs, and has quite a major problem when used with traditional rotating hard disks: fragmentation. For an SSD, fragmentation of the file system metadata and file data has no cost, but for hard disks it can kill performance, as each attempt to access data on the disk turns into a long series of seeks to find all the blocks required.

In HFS+, the solution has been to defragment the used and free space on the disk, something which macOS does to a degree, and several third-party tools have long supported.

In APFS, one of its most important features, copy on write, is almost designed to cause fragmentation. It’s essential for the file system metadata, as there’s no journalling, which has proved a saving grace in HFS+. The end result is that file system metadata in APFS get hopelessly fragmented, which in turn causes performance problems on hard drives.

It has been quite difficult to demonstrate relative performance between HFS+ and APFS on hard drives, as conventional disk benchmarking doesn’t produce a standard degree of fragmentation for the purposes of comparison. Take a freshly-formatted hard disk and run normal benchmarks on it, and you’ll probably find little difference between the two file systems.

A little while ago, though, Mike Bombich, developer of Carbon Copy Cloner, came up with a much better way of comparing performance. He enumerated a million files in the file system on a hard disk over a period of simulated modifications to the file system, under both HFS+ and APFS. In case a million files seems excessive, with files nested three folders deep, this is only 111,000 folders, and a maximum file size of 20 GB – hardly excessive by real world standards.

The time taken to enumerate the million files was consistently shortest (best performance) on HFS+, and changed little there over twenty repetitions of the test. APFS, though, was much slower to start with, and got progressively worse, regardless of whether defragmentation was enabled, and largely irrespective of snapshots.

Most importantly, he concluded that enabling defragmentation in APFS doesn’t cause file system metadata to be consolidated into storage areas which reduce or eliminate seek time on a hard disk.

You can try this yourself, although the documentation of defragmentation is minimal. The diskutil apfs command allows you to enable and disable defragmentation at Container or Volume level, with a command such as
diskutil apfs defragment volumeDevice enable
enabling it on the specified volume. But I wouldn’t expect it to have any significant impact on the poor performance that you’re experiencing on APFS-formatted hard drives. I hate to say it, but if you possibly can, it would make good sense to keep those in HFS+ format until you replace them with SSDs.

Problems come with more recent versions of macOS and Macs which boot from a hard disk or Fusion Drive. Some users report that they have managed to get Mojave to boot from HFS+, but it’s unsupported, and you (almost) certainly can’t boot Catalina from HFS+, because of its requirement for APFS-specific features for its System and Data volumes and their Volume Group. Use of the SSD component in a Fusion Drive as a cache should cushion some impairment of performance, but booting from a hard drive definitely isn’t a good choice. If you possibly can, before upgrading to Catalina, ensure that your Mac boots from an SSD, even if it’s only small and most of your files will still be kept on a hard disk.