Disk images and their sub-species sparse bundles are among the oldest of Mac file formats, but they have changed in the last couple of years and can still catch the unwary. In this article I look at some of their features that may no longer work as you might expect.

There’s no shortage of different disk image formats. Among the most common are:

• plain disk image, known as a UDIF read/write image, abbreviated to UDRW;
• sparse image, the UDIF variant that has variable size, UDSP;
• sparse bundle, a bundle that has variable size, UDSB;
• CD/DVD master for export to optical media, UDTO;
• plain UDIF read-only disk image, UDRO;
• various compressed read/write formats.

Here I concentrate on the first and third, which are among the most commonly used.

Sparse bundle size is limited by band size

The maximum size for the whole sparse bundle is the size of its virtual volume. It’s best to set this close to the largest capacity you need in the virtual volume, as that keeps the size of the sparse bundle itself to a minimum. For a bundle containing backups, though, you know that it’s likely to continue increasing in size, and it’s often simplest to start off with its maximum size set to the largest that you want it to grow to during the life of that backup set.

Band size is the maximum size of each band file, and determines two things: the number of band files, and how efficiently the whole sparse bundle can change in size. In most cases, the default is 8.4 MB, which generally works well for all but the largest of sparse bundles. There’s one important limit to bear in mind when setting band size: all the bands of a sparse bundle are stored inside a single folder. If the number of bands reaches the maximum for a single folder for the host file system, then it will start to fail, and you could lose part or all of its contents. Currently, in macOS with HFS+ or APFS, that critical number is believed to be 100,000 (an empirical guesstimate). So whatever you do, ensure that your sparse bundle will never need 100,000 or more band files, as that could spell disaster.

The simplest way to approach this is with a critical band size. Calculate that by dividing the absolute maximum size that your sparse bundle might reach (even after resizing) by a number smaller than 100,000, such as 50,000. It’s best to do this in consistent units such as MB: if your sparse bundle could reach 1.1 TB, that’s 1,100 GB, or 1,100,000 MB. Divide that by 50,000 and you should get a critical band size of 22 MB. That’s the smallest band size that will keep the total number of bands below 50,000.

You can then choose any band size from the critical upwards. The smaller you make the bands, the more efficient the sparse bundle’s use of storage space will be, as it can add (or remove) bands to accommodate changes in size. For a 1.1 TB maximum size sparse bundle, you could therefore choose a band size of perhaps between 22-200 MB, as giving a good compromise without approaching the maximum number of bands in a single folder. Note, though, that setting too small a band size may limit the maximum size for the whole sparse bundle. When creating very large sparse bundles, macOS may restrict their size if the chosen band size is too small.

Encrypted sparse bundles can’t change passwords

Until Ventura 13.2.1, encrypted sparse bundles seem to have worked as advertised. Although Disk Utility has never offered an easy way to change their passwords, that could be accomplished interactively using hdiutil in Terminal. A command of the form
hdiutil chpass sparsebundle
where sparsebundle is the path to the sparse bundle, starts this process. The user is first prompted at the command line to enter the current password, then the new password twice, thus
hdiutil chpass /Users/hoakley/Documents/0test1.sparsebundle
Enter password to access "0test1.sparsebundle":
Enter a new password to secure "0test1.sparsebundle":
Re-enter new password:

This stopped working by macOS Ventura 13.3.1, since when hdiutil still goes through the same sequence but the password remains unchanged. As of Sonoma 14.1.1 this remains broken, as Apple has still not fixed this bug. As there’s no other means of changing a sparse bundle’s password, this severely limits the usefulness of encrypted sparse bundles. The only workaround seems to be to make the change using macOS 13.2.1 running in a VM. Without that, no encrypted sparse bundle can now have its password changed.

Sparse bundle won’t compact

Compaction of sparse bundles to recover space when their contents have shrunk has never been perfect, and over time sparse bundles accumulate lost free space that can’t be recovered. However, there are times when you try to compact a sparse bundle and this doesn’t work as expected. One common cause is that the Mac is running off its battery. Although it’s clearly documented in man hdiutil, you too may not be aware of this unusual limitation. As the man page explains: “SPARSE images could be damaged if power is lost during a compact operation. The default is not allowed.” But there is the option of -batteryallowed to override this behaviour.

If you use my free utility Spundle, it now has a Battery checkbox. If you try compacting a sparse bundle on your notebook when it’s running on battery, you’ll see the error reported, and Spundle’s advice to either connect the notebook to mains power, or tick the box, and try again.

Sure enough, with the Battery box ticked, Spundle now takes your sparse bundle’s life into its hands and compacts it regardless.

UDRW read/write images are now trimmed as sparse files

Over two years ago, Apple changed the way that UDRW disk images work on APFS. Whenever they’re mounted, APFS performs a TRIM to compact their free space, so reducing the size of the disk image on disk. This turns the disk image into a sparse file,* which is extremely efficient in the use it makes of storage space, as it only takes up the space required to store real data, and not blank space within the disk image. This transforms the use of UDRW disk images, which were previously inefficient in the way they made use of storage space.

However, in tests, sparse bundles remain the most efficient of the common disk image formats, despite their tendency to accumulate lost free space over time. UDRW disk images come a close second, though.

Performance

Storing files in any of the disk image formats isn’t as fast as keeping them directly on an APFS volume, and some formats have dreadfully slow write speeds. Sparse bundles again normally come out fastest, with read speeds only a little below those of a native volume, but their write speeds are likely to be around 70% of native. UDRW disk images again come second, with good read performance, but write at around 30% of native speed. On all modern Macs, using encryption should make little difference.

Summary

• When creating larger sparse bundles, don’t leave the band size to default, but calculate what’s appropriate for the total size.
• Currently, encrypted sparse bundles should be avoided, as macOS is unable to change their passwords.
• When compacting a sparse bundle on a notebook, power it from the mains supply, or enable compaction to take place on battery.
• Read/write disk images (UDRW) are now stored as sparse files, and far more space-efficient.
• Sparse bundles are still slightly more space-efficient than UDRW.
• Write speeds are significantly reduced on all disk image formats, although better on sparse bundles, and UDRW.

* Note: a sparse bundle is a folder containing a disk image stored in many band files. A sparse file is a single APFS file in a special format that economises on the space it requires on disk by not storing empty data. They are very different.