If you’ve tried backing up over a local network or simply transferring files to or from a shared volume, you’ll be well aware of how difficult SMB can be. When I was setting up Time Machine backups over a network, my example Data disk contained just under 80 GB in just over a million files. Although hardly excessive as a first full backup, that took over 5 hours to complete, under macOS 11.2.3. This article shows what happened, and explains what you need to do to avoid similar transfers.

At approximately five minute intervals during each backup, Time Machine writes an entry in the log which records progress in its transfer. For example:
2021-04-15 14:23:16.609140+0100 Copied 60.21 GB of 79.08 GB, 146004 of 1041141 items (~50.94 MB/s, 8.28 items/s). Last path seen: /Volumes/Backups of Howard’s MacBook Pro/2021-04-15-124839.inprogress/Data/Applications/XcodeTools/Dashcode.app/Contents/PlugIns/TemplateWebUtility.wdgtTemplate/Contents/version.plist

Having recently performed a first full backup over a local network, I extracted the transfer rates reported by Time Machine during that to establish why performance was so abysmal.

The first graph of performance during the backup shows how the cumulative total transferred to the backup storage increases with time.

For the first hour or so, until just before 14:00, transfer rates were as high as 10 MB/s as around 20 items were copied to the backup store every second. Time Machine then encountered a couple of very large files of more than 12 GB each, and the transfer rate increased until just before 14:00. There were a couple of step changes until nearly 14:30, after which there was a period of three hours during which hardly any data was transferred at all.

Looking at the rate of transfer of items (files), that was flat during the backing up of the very large files, but otherwise showed a more continuous increase throughout the whole period.

During the three hours in which very little data was transferred, there was no reduction in the rate of transfer of files. Some 660,000 files totalling less than 10 MB were being transferred at a rate of about 60 items/s.

Those phases are seen even more clearly in the final two graphs. The next shows the reported transfer rate in MB/s, which dropped to almost zero during the three hours in which backup appeared to choke.

Viewed in terms of the rate of transfer of files, in items/s, that three hour period saw greatest transfers.

Although when backing up to local storage, Time Machine does slow slightly when it has to copy very large numbers of small files, its performance remains far better than when copying via SMB to shared storage. On the other hand, performing a manual copy of Xcode to or from shared storage is as painfully slow as Time Machine backups.

The common factor is SMB, which is clearly incapable of achieving the performance necessary to support Time Machine backups to networked storage. To underline that, the incremental backup which followed that first full backup only needed to copy 215 MB in 1,786 files, yet it took 20.6 minutes.

If you do need to back up using Time Machine to shared storage, you’ll need to exclude all apps and folders which contain very many small files, or they too are likely to choke your backups so that they barely complete in time for the next backup to be made. When you need to back up similar items, try to copy them once they’ve been converted into an archive format such as AAR, which allows them to be transferred in a single file.

Apple needs to address this problem very soon. Until it does, many users will simply not back up to shared or networked storage. And I’ll be one of them. A total of 79 GB over 5 hours works out at an average transfer rate of 4 MB/s, which is considered poor for an Internet connection. AirDrop is far superior.