You won’t find any mention of it in Big Sur’s marketing features, nor can I find it in any of the presentations given at WWDC this year, but macOS 11.0 brings AppleArchive, new system-level support for compressing and decompressing files. It’s well-documented, with example code that actually works straight from the Clipboard, and I’ve been exploring what it has to offer.

AppleArchive is described in Apple’s developer documentation, and parts of it have already been implemented in Catalina. What’s different about Big Sur is that compression methods which were previously only available to Apple, and have been used in its software updates, are now accessible to third-party developers.

It supports four built-in methods: LZ4, LZMA, zlib and Apple’s proprietary LZFSE. The latter is preferred as it normally compresses as well as zlib level 5, but both encoding and decoding are more energy-efficient and considerably quicker. As it’s proprietary, it shouldn’t be used for archives which are intended for use on non-Apple platforms.

Support in Apple’s bundled Archive Utility is incomplete. That app happily decompresses LZFSE .aar archives, and I suspect the version shipped with Catalina also does, but doesn’t currently offer compression using LZFSE. So I built a test app, Cormorant, which enables you to try it out.

One task for which I depend on compression, is moving copies of Apple’s Xcode between my Macs. When I download pre-release versions from Apple, they come compressed into a .xip archive, a painful reminder of that previous Apple compression format which was once supported for users, but was taken back in-house and lives on with Xcode delivery. But when Apple publishes release versions to the App Store, downloading it separately for each Mac is wasteful.

Because Xcode contains a huge number of files, even moving it between Macs on external storage isn’t efficient. So here’s an opportunity to move it around in a single compressed archive, which can then be copied over AirDrop.

Using my new beta-test utility Cormorant, it took just 2 minutes 34 seconds to create a compressed .aar archive, and 6 minutes 40 seconds to decompress. That’s 14.6 GB containing an original 28.7 GB and maybe over 400,000 items. Compressing 17.7 MB of PDF documents to 13 MB took just over a quarter of a second, and decompressing them was equally quick.

If you use AirDrop to distribute this type of archive, you’ll need to remove the quarantine flag added on transfer. For the moment this requires a separate utility such as my free xattred, but could easily be incorporated into Cormorant, if there’s demand.

Cormorant is simple to use: click on its Open button to open a file, folder or archive. If you open a file with the extension .aar, it tries to decompress it into a folder; with the extension .lzfse, it treats it as a file to be decompressed into a single file. Opening a file without an extension, or any other extension, it compresses it to an .lzfse archive; opening any folder will cause that folder to be compressed into an .aar archive.

Currently, it won’t compress a bare app (with the extension .app), so I put Xcode inside a folder for compression.

My M1 Macs have yet to arrive (should be any day now), but I understand that AppleArchive is particularly quick on them, and has already been optimised for their new architecture.

You can download an early beta-test version of Cormorant from here: cormorant10b2
As its compression and decompression relies entirely on AppleArchive, I believe that to be free from bugs. If you want to decompress any .aar archive, you can do that using Archive Utility instead if you prefer, but that doesn’t currently recognise the .lzfse extension used for compressed single files. Cormorant requires Big Sur, but I believe that Archive Utility in Catalina also supports decompression of .aar archives.

Where does Cormorant go next? That’s up to you, at least in part. I’m already working on the next version, which incorporates timing so that I can use it for one benchmark for my M1 Macs. I’m probably going to add a feature which will automatically strip quarantine flags before decompression, so that I can use it more easily with AirDrop transfers. It needs a more convenient interface, perhaps as a Finder Extension, although those still appear poorly documented. I’m also wondering whether to incorporate integrity checks for Fintch/Dintch. Please let me know what interests you.

Postscript:

Thanks to @jimmyjamesuk123 for pointing out that AppleArchive was covered briefly at WWDC in the What’s new in Swift session, and best of all that it has excellent support in the new command tool aa, which has been added to Big Sur.