Last Week on My Mac: Enduring file references in macOS

I don’t move folders around too often, or rename them, but it worries me that when I do, some apps (and some users!) won’t remember where they’ve gone. By a quirk of Unix and MS-DOS, most software including operating systems locates files by their directory path. When we innocently move a folder from ~/Documents to ~/Pictures, their directory paths change, breaking anything which has used them to track the files’ location.

This is an extraordinary shortcoming. Imagine losing track of a stack of books when you move them from one room to another!

Those who know Unix will know of its solution: the operating system assigns each file (and directory) on any given file system a unique number in its inode data. Then all that applications and users have to do is remember two integers, one to specify the file system (volume) containing the item, the other to specify the item itself. It’s a shame that human brains can’t memorise dozens of integers.

So the inode number is a feature most useful at the command line, where a script might chug through a long list of such numbers without complaint.

In macOS, this has been implemented through a file system, centred on a hidden folder at the top level of the boot volume, named .vol. Instead of using a regular directory path like /Users/hoakley/Documents/0newDownloads/apfs/0precize/IntroducingPrecize1.0b1.rtfd, apps and users at the command line can refer to that same file as /.vol/16777224/360517297, its volfs path.

When you move the folder 0precize from /Users/hoakley/Documents/0newDownloads/apfs/ to /Users/hoakley/Documents/, changing all the directory paths, the volfs path of /.vol/16777224/360517297 remains the same.

Many Mac apps have used this feature: the files listed in their Open Recent… menu command work across restarts, and are robust in the face of reordering your folders. So long as the file stays on the same volume, macOS will find it. That is a bold leap forward from MS-DOS.

Although there’s no sign that inodes are about to vanish from macOS, and Apple’s new file system, APFS, works fine with them, they are changing. For a start, inode numbers have gone from 32-bit integers in Sierra/HFS+ to 64-bit in High Sierra/APFS, to accommodate its higher specification. Apple has also removed or deprecated most features which use inode numbers, including a key command tool, GetFileInfo, which is still available in High Sierra, but probably not for much longer.

Apple clearly doesn’t want us, whether developer, user or sysadmin, to use inode numbers, volfs paths, or the simpler data types based on them. Instead, macOS both provides and uses Bookmarks, which are opaque chunks of around 1 kilobyte of binary data. Now, when an app uses standard support to handle its Open Recent… command, it is leaving macOS Launch Services to do so using Bookmarks.

Using a Bookmark is a bit like sealing my stacks of books in boxes, and putting barcodes on the outside. I can’t tell what’s in the box without a barcode reader linked to a database listing the contents of all the boxes. That may be great for operating a warehouse, but it doesn’t get me very far when I haven’t got a barcode reader or access to the database.

Bookmarks are fine for app developers, who can make a macOS call which returns the current directory path to any given Bookmark. But users and sysadmins lack any tools for working with them, and kilobyte slabs of binary data are not good to have floating around in Terminal. Just as happened when Apple replaced traditional logs with the unified log, its fine engineering solution has proved a stumbling block for everyone else.

But why should users – or, indeed, anything beyond the Open Recent… command – want Bookmarks or any other form of enduring file reference?

What we currently do in the Finder remains firmly rooted in the path-based confines of MS-DOS, and that results in inefficiency and constraints. By coincidence last week, I published a short listing of document packages commonly encountered in macOS. When I write the documentation for an app, I work in RTFD format. I drag and drop the screenshots into the document, and copies are made of those PNG files in the RTFD package. But I also keep copies of the original screenshots.

I do the same with Keynote presentations and their still image and movie content. Add all these duplications together, and quite a lot of my storage is taken up with unnecessary and identical copies. This is simply because RTFD and Keynote documents cannot rely on soft links to the original content files, as they break so easily, being path-based.

This is reflected in the structure of our Home folders. We tend to put still images in Pictures, video in Movies, mail in Mail, then in Documents we collate content from those folders by duplicating it into finished documents.

So long as we remain wedded to directory paths as a means of referencing files and folders, much of our storage will contain duplicated content. Bookmarks are a central part of a more advanced approach to structuring and organising content which can at last move on from the primitive constraints of MS-DOS.

Their precursor is, of course, the Finder alias, and it is perhaps best to think of a Bookmark as an alias for the future. Finder aliases have never been fully documented by Apple, but have been found to contain within their opaque data structures inode numbers. Their problem is that they were never designed for the sophisticated features of modern macOS, such as the Sandbox. Bookmarks are designed to work properly with apps running in their Sandbox, as every app shipped by Apple’s App Store does.

I think that we will see more, not less, use of Bookmarks as enduring file references in future versions of macOS. They don’t spell the end of inodes, aliases, links, or directory paths, but they need to be made much more accessible to users and sysadmins.

Postscript:

More details about the recent convergence of Finder aliases with Bookmarks are here. Although developers have access to both, and can write Finder aliases within apps, Apple has not improved access to Finder aliases from the command line, for example.