One of the main reasons for Apple adding Time Machine to Mac OS X was for its Time Capsules. Over the next ten years, it must have sold many millions of what it termed ‘backup appliances’. Essentially combined Wi-Fi routers and network storage (NAS) systems, internally they ran NetBSD as their operating system on ARM processors, and had rotating hard disks of up to 3 TB capacity.

Although Time Capsules made a name for themselves for convenience, they were never known for their speed. First full backups from the modest internal storage of MacBook Pros of the day usually required to be run overnight, even when performed over a wired Ethernet connection. But that use case – of someone returning to base with a MacBook Pro and leaving it to back up the day’s work over Wi-Fi – proved popular and generally successful.

A great deal has changed in the thirteen years since so many took to using Time Machine and Time Capsules. M1 MacBook Air and Pro models are far faster and more capable than the Intel Core 2 Duo processors typical of the better laptops in 2008. What we store on today’s laptops has also grown both in size and complexity. Apps like Xcode are now not only almost 30 GB in size, but contain more than half a million individual items (even though the Finder claims there are just under 100,000). Users happily let their Photos libraries grow to contain more than 100,000 images, and there are plenty of other items that we go around with which have become increasingly hard to back up.

Time Machine has also changed to keep pace with these increasing demands. In its latest incarnation, it has finally done away with the millions of files and hard links which it needed to store backups on HFS+. Instead, it pulls every trick in the book of APFS, building off-volume snapshots complete with their clone files. At last we have a bundled backup system which performs well and shouldn’t corrupt its file system every few weeks. But try running your backups over a network, perhaps to a desktop M1 Mac, and little seems to have changed since 2008.

The reason for poor performance of Time Machine backups over a network becomes obvious when you look at the rates of data transfer and copying of items during a first full backup.

This first graph shows data transfer rates in MB/s during the backup. These peaked twice at around 50 MB/s during the copying of two files of around 12 GB each, and remained almost zero between 14:30 and about 17:20, a period of almost three hours when Time Machine was copying 660,000 files with a total size of less than 10 MB.

The second graph shows the other side of the coin: the rate of transfer of items (predominantly files) over time. When the single very large files were being transferred, that fell to zero. During the copying of many small items, when the data transfer rate was almost zero, items were being copied at a rate of around 60 per second, reaching a maximum of just over 140 per second.

Overall, it took over 5 hours to copy just under 80 GB in over a million items. Scale these figures up to those likely to be required for a decent desktop system, and it wouldn’t be hard for that to take more than a day.

I happen to have a comparable first full backup performed to local storage, in this case an external SATA SSD which typically writes at around 500 MB/s, also using Time Machine to APFS in Big Sur 11.2.3. That required about 26 minutes to back up 143 GB in just over a million items, which included a copy of Xcode. The highest data transfer rate recorded during that was around 238 MB/s, almost half the write speed of the storage, and the highest number of items transferred per second exceeded 725.

Extremely slow transfer of items like Xcode which contain huge numbers of files occurs when simply copying them across a network shared volume, from Mac to Mac, although transferring the same amount of data compressed into a single AAR archive is considerably quicker even when you allow for the time required to compress and decompress the files.

Thus the rate-limiting step in backing up items such as Xcode to a network share isn’t so much the data transfer rate as the rate at which individual items (of negligible size) can be copied. Although this might be exacerbated by copying items to a sparse bundle, the major part of that slow performance is down to SMB. If that were able to create new files and copy the small amounts of data in them at ten times the speed (as seen when backing up to local storage), then the time taken to back up Xcode would fall from almost three hours to around 17 minutes.

There’s now a compelling case that what’s slowing both file transfer with network shares, and backing up to them, is SMB, which must be able to copy files ten times faster for Time Machine to be a good choice for backing up over a local network. It’s wonderful that Apple’s engineers have come up with the much improved version of Time Machine in Big Sur, but without improvements in its infrastructure, over a network it’s still dead in the water.

Postscript

Thank you for all your comments below: I value them, and agree with much that you say. I don’t have any comparison with another implementation of SMB which supports TMA in this way on M1 Macs, so I can’t say whether this is a failing of Apple’s SMB, nor whether an alternative network file system would solve the problem. I suspect not, because in any file system there’s going to be significant overhead when transferring hundreds of thousands of files in this way. To fix this properly will, I think, require a different approach which makes the most from data transfer rates, much as combining all the small files into a single archive does. It’s inherently quicker to create new files at the destination rather than by secure negotiation between client and server. There’s an irony here in Time Machine transferring a snapshot, which appears to be quick, and the far slower transfer of the files required to reconstitute that in the backup.

I think this is very important: the promise of simple and quick backups over a network is of great value to many users. NAS are a good solution for many backup needs, but currently are too far from perfect to recommend to ordinary users. No wonder Apple got out of the market when it did. I hope that Apple’s engineers will come up with a better solution: we need it.