Fitting a quart into a pint pot with sparse bundles

One of the really neat features of sparse bundles is that you can keep a quart sparse bundle on a volume with only a pint of free space. That’s so long as you don’t actually try to use all its capacity. For no matter how ingenious macOS, it can’t actually magic storage from nowhere. This article explains how this flexible storage works, so that you can make use of it without suffering errors.

Permissions

Spundle has to assume that you have permissions to create files wherever you try to create a new sparse bundle, and some other operations may also expect you to have write permission.

If you don’t have write permission to the folder in which you want to create a new sparse bundle, the best workaround is to create a folder there instead, change its permissions so that you can write inside it, create the sparse bundle within that, then move it up one level to the folder where you want it to remain.

I’ve looked at getting Spundle to save you the trouble, and sudo its hdiutil command to create the new sparse bundle. Unfortunately, this turns out to require a helper app, greatly increasing Spundle’s complexity. I’m going to be looking at doing this for other apps, and will get back to add this feature to Spundle in time, but for the moment please bear with me: it’s not simple by any means.

If you do try to create a sparse bundle where macOS won’t let you, you’re most likely to encounter an error 1. Try adding Spundle to the Full Disk Access list in the Privacy lists, and check permissions on the folder into which you are trying to save the sparse bundle.

Free space

Having ensured that you have permissions which allow Spundle to create the sparse bundle where you want it to be, you should be able to adjust its settings and give it the maximum and band sizes that you want. So what happens in a sparse bundle when you set it to have a maximum size greater than the amount of free space on the volume on which it’s stored?

The maximum size set for a sparse bundle is what it says: a maximum. When you mount a sparse bundle stored on a volume whose free space is less than that maximum, the available free space is slightly less than the total free space for that volume. Let’s say you’ve made a sparse bundle with a maximum size of 400 GB, and that is stored on a volume with just over 200 GB of free space at present. When you mount that sparse bundle it will offer you just over 200 GB of free space in that virtual disk.

This is because, when you add items to the virtual disk, they are copied into the bands of the sparse bundle. As there’s only room for just over 200 GB on that volume, that’s all the free space available on the virtual disk.

How this is reported varies according to the file system used in the sparse bundle (not the file system of the storage on which the sparse bundle is stored).

spundle61

If the sparse bundle uses HFS+J, for example for use to store Time Machine backups, then Disk Utility reports the size of the virtual disk as its set maximum size, the size of that virtual volume as that same maximum size, but the shortfall between the maximum size and the available free space in the volume is marked as being used, as shown above.

spundle62

If the sparse bundle uses APFS, the Disk Utility reports free space differently. The disk and container size are shown as the set maximum size, as is the volume size. The shortfall between the maximum size and available free space in the volume is left unaccounted for, as if it were being taken by another volume, which it isn’t. Hence the sum of the volume free and used space is much less than the volume capacity.

Either way, any software querying the amount of free space in the virtual volume of your sparse bundle should be returned the same amount.

Eject the virtual disk and move or copy the sparse bundle to a volume which does have sufficient free space to support the whole of the maximum size of your sparse bundle, and when it’s mounted from there, the virtual volume will be shown at that maximum size. You don’t need to change its maximum size, compact it, or do anything else – that’s automatic, provided that it’s stored on a file system which understands sparse bundles.

I will add an explanation of this to Spundle’s Help file in due course.