Copying files is one of the most basic tasks for a computer: a core function of the Finder, at the heart of many tools such as Time Machine and Migration Assistant, and the sole purpose of others such as Installer.

Copying a few files every now and again is easily done manually using the Finder, but repetitive tasks such as performing hourly backups, creating a mirror or clone copy of a whole volume, and maintaining synchronisation between large and complex folders, is not the sort of thing that you want to perform yourself. This article looks at some of the ways – other than the likes of Time Machine – which are readily available.

Take an apparently simple chore of keeping a folder full of working documents in sync between your startup volume and network storage (NAS). If at the end of each day you have to work out which files have been updated and copy those across individually, you will waste a lot of time and make mistakes. If you instead opt just to copy the whole folder across, overwriting the existing files on the NAS, you will waste a lot of time copying files which have not changed. So what you want is a tool which selects those which have changed, and only copies those across.

Now consider how to deal with a similar situation, but where two or more people might be sharing that ‘master’ folder on the NAS. How would an app decide which updated files to replace? Or how would the app cope if you wanted to merge together the contents of several working folders on your startup volume, to create a consolidated copy on the NAS?

You can now see how an apparently simple and basic task can become very complicated. Anyone writing a utility to perform such copying needs to build in a lot of flexibility, and with that, of course, comes complexity.

Command basics

The most basic tool of them all is the Terminal command cp, which simply copies the contents of files. cp operates in two modes: it can simply copy the contents of one named file to another named file, or it can copy from source files in one location to place copies (with the same names) in another location. So
cp thisfile.ext thatfile.ext
simply makes the file named thatfile.ext a copy of that named thisfile.ext, and
cp thisfile.ext ~/Documents
simply copies thisfile.ext into your Home Document folder, using the same name. It does have options which allow it to copy folder hierarchies, but is relatively inflexible when doing so.

At its simplest, ditto copies one or more source files or folders to a destination folder, overwriting any existing files. It can also create and extract archives, and copy files on the basis of a ‘bill of materials’ (BOM), which you may be familiar with from Apple’s Installer and its BOM files. ditto is by default careful to preserve forks and metadata, including permissions, Access Control Lists (ACLs), and quarantine information, and because of this and its support for HFS+ and PKZip compression, it is popular for installers. An alternative, which has slightly different strengths and weaknesses, is pax.

rsync is the mother of all file copying tools, with more options than can be adequately documented in its man page. It can be run as a standard command-line tool, or set to work as a daemon, making it ideal when copying to or from a remote host. However it cannot be used to copy directly from a remote host to another remote system. It supports the copying of links, devices, owners, permissions, and more, and is the tool of choice when working with remote storage. With its batch mode, it is also invaluable for applying the same set of updates to many identical systems, for labs or classrooms, for instance.

Annoyingly, in El Capitan, rsync remains stuck at version 2.6.9 which was released in November 2006; it is disgraceful that any major release of an operating system should contain key tools which are almost eight years out of date (rsync 3.0.0 was released in March 2008). In spite of its age, it does support 64-bit (vast) files, hard links, symlinks, batchfiles, IPv6 and some other ‘modern’ options, and is something of a standard. If you want to use rsync seriously, consider upgrading to the latest version from here. However, with SIP active in El Capitan, you cannot replace the old version, making updates and subsequent use messy.

If you want to use rsync, or to put it at the heart of an AppleScript, Automator, or other script, then its website contains many additional resources which may help you to get your head round its complexities.

As always, you can read the full documentation for these commands at man cp, man ditto, and man rsync, although the last is seriously long and cryptic.

Enhancements

Unison is a sync utility which runs on OS X, Windows, and Unix systems, which among other things allows you to store two replicas of a source collection of folders and files, on different hosts. Those replicas can then be maintained separately, and when you wish they can be brought up to date with one another by propagating changes between them. It is donationware, and detailed here. At its heart is its command shell tool, but the OS X binaries come with a friendly wrapper which make them easier to use.

Friendly apps

Several products provide a relatively user-friendly front end to rsync or related engines, and there is now a wide choice on the Mac App Store, for example.

Folder Sync costs only £6.99 there, and has worked brilliantly for syncing folders, but has not been updated for nearly 2 years. Despite that it still appears to work fine, and offers a wide range of options to support almost all the sync features available in rsync.

DropSync 3 is more expensive at £11.99 (Mac App Store), but appears to be better-maintained at present, with a fairly similar interface.

The longest-established is Mike Bombich’s Carbon Copy Cloner, which has been maturing since the very earliest days of OS X. Aimed primarily at producing, and maintaining, a complete mirror or clone, it now does a great deal more than that. Anyone wanting to copy files in bulk or automatically should start by downloading this and evaluating it in demo mode. It is the benchmark, and uses rsync as its engine.

Acrosync is a relative newcomer with high ambitions, and is again based on rsync. Available across platforms including Windows and Linux, the OS X version costs $49.99 direct, or £39.99 in the Mac App Store. An iOS port is offered in the iTunes App Store for £3.99, a tenth of the price of the computer version, and could be invaluable for those who need to transfer files between OS X and iOS. However, its interface is limited and feels kludgy in use.

iBackup (donationware) uses a combination of rsync and ditto primarily to make backups rather than simply syncing folders. Although it provides an easy-to-use interface, it has many advanced controls and settings which should accommodate the most demanding of users. If you want to use a method of backing up other than Time Machine – or in addition to it – then you should look at this in the first instance.

Conclusion

Tools available for copying files in sophisticated ways – syncing folders locally or with a remote system, backing up, and more – should cater for every taste and level of complexity. Most are based on command line tools, particularly rsync, which is always there if you need it, and are prepared to learn how to use it. You should never have to attempt any tedious or repetitive file copying by hand: there is always a good choice of tools to make the task simpler, quicker, and more reliable.