Metadata are often as important as the data they relate to, and may be as difficult to recreate. You might recognise everyone in that photo shortly after you’ve taken it. In ten years time, though, identifying the faces might be well nigh impossible. Anything which fails to preserve the metadata you attach to that image is seriously bad news.
Most image formats allow the embedding of a lot of structured metadata in the image file itself. That isn’t true, though, of many other file formats, particularly if they’re less widely used, or even app specific. My example here comes from a popular format used for ‘comic’ books, Comic Book Archive, which has been used by the popular macOS app ComicBookLover. It comes in two main variants, CBZ and CBR. Both consist of compressed folders containing page images: the CBZ version is compressed using Zip, and CBR using RAR. Other less common variants use different compression formats such as 7z.
These often include extensive metadata about the image archive, which can be stored as a comment to the Zip archive, in an XML file included in the folder of sequential images or, on Macs at least, as an extended attribute (xattr) attached to the CBZ file itself – and that’s how ComicBookLover likes to store them.
These metadata are stored in JSON format, and when in a xattr it is a custom type named com.bitcartel.comicbooklover.xattr.metadata. ComicBookLover makes more extensive use of xattrs too, adding a thumbnail preview in com.apple.ResourceFork, a star rating in com.apple.metadata:kMDItemStarRating (intended for use by Apple apps), another in org.openmetadata.time:kMDItemStarRating, and a third in org.openmetadata:kMDItemStarRating.
Unfortunately, most if not all of those xattrs will be stripped when the CBZ file is moved, for example to iCloud Drive or DropBox. This is a common problem with apps which use xattrs to store metadata, and results from incomplete understanding of how xattrs are handled, which in turn reflects Apple’s failure to document them fully for developers. The same issue also severely limits the usability of other apps like Skim, which uses xattrs to store PDF annotations.
Xattrs – extended attributes – are parcels of data which are attached to a specific file or folder. They’re not stored with the rest of the file’s data, but in the file system metadata. They’re used for a wide range of purposes: some are clearly ephemeral and shouldn’t be preserved when making a copy of that file, whilst others are more permanent. macOS has a complex system for determining which xattrs are preserved and which are lost in various situations, but that is known to very few developers or users.
As far as custom xattrs like com.bitcartel.comicbooklover.xattr.metadata are concerned, they will be treated by macOS as being ephemeral unless a handling flag such as #S is added to their name, as in com.bitcartel.comicbooklover.xattr.metadata#S. As that isn’t done by ComicBookLover, its precious metadata stored in that xattr will be all too easily lost. And that is generally the case for all custom xattrs used by third party apps to store such metadata.
Salvage
The immediate task for anyone using metadata in custom xattrs is to salvage their contents before those xattrs become stripped. If you don’t do that, just passing the file through iCloud Drive or copying it off your Mac could lose those metadata.
One simple way to salvage what you have is to open each item bearing metadata in xattrs using my free xattred, select the xattr concerned in the upper list, then select all its contents in the view below, copy and paste that into a text document. In this case, com.bitcartel.comicbooklover.xattr.metadata content in JSON format will be preserved for the future.
Protection
Once you have salvaged and preserved the existing metadata, you should consider how best to protect it from removal.
The only way that I know of to accomplish this is to use the existing feature of macOS, and add a handling flag of #S to the name of the xattr. Although you can do this in xattred, it’s a little messy, as there’s no simple call to change the name of a xattr. Instead you select the xattr you wish to preserve, click on the Edit button, and in the xattr’s name field at the top append #S. When you then click on Save, you’ll have a new xattr with the new name, and the old one without the #S. Select the old xattr with its original name and click the Cut button to remove it.
The problem with doing this is that accessing xattrs with such handling flags isn’t transparent to apps: this even catches the Finder out, which fails to recognise the few xattrs which it would normally display in its Get Info dialog. So the chances are the app which uses that xattr now won’t be able to locate it.
I also have a test app, SaveTheMetadata, which appends #S handling flags to most xattrs. This is still in development, and shouldn’t be used on original files, but is available from here: STM10b2
Conversion
The only permanent solution is to discover a file format which doesn’t rely on storing metadata in xattrs. In the case of CBZ files, this should be possible by writing the metadata from the com.bitcartel.comicbooklover.xattr.metadata xattr to the Zip file comment, or to an XML file stored within the CBZ itself.
Although it may be possible to perform this using AppleScript, or even a shell script, in general working with xattrs requires access fairly deep into macOS, and is best performed using a compiled language such as Swift or Objective-C. I’m currently looking at the best solution for CBZ files, and welcome your comments.
Lesson
As a friend used to say, there ain’t no such thing as a free lunch. Xattrs are marvellous features of macOS, but unless a developer is aware of their lability and how to preserve them, they aren’t good places to store metadata of any importance.
Thanks to Clément, who raised this issue with CBZ files, and had already solved quite a lot of it before coming to me.
