hoakley June 5, 2018 Macs, Technology

APFS comes to Fusion Drives, but there’s no sign of Time Machine 2

In my preview of WWDC 2018 last Sunday, I identified two key features which Apple needed to include in the next major version of macOS, now named Mojave: APFS support for Fusion Drives, and Time Machine 2 which is no longer reliant on Apple’s old file system, HFS+.

Slipped in at the end of the last major section of the Keynote yesterday, Craig Federighi mentioned in passing that Mojave will at last bring APFS support for Fusion Drives (he also added hard drives, but APFS has officially been supported on hard disks for some months now). But neither he, nor the list of new features behind him, mentioned Time Machine changes.

The prime mover behind APFS was always iOS. Apple had the chance to adopt a modern file system suitable for macOS more than a decade ago, when it tested the water with ZFS. But at the same time, Apple was bringing iOS to market in its new iPhone, for which ZFS was less than ideal. Apple stuck to its own file system, HFS+, for its new products, and so started the quest for a new file system, which led to APFS.

As developers got to learn more about what was coming in APFS, their concerns started to grow. The most immediate was its handling of Unicode equivalent code points in folder and file names. HFS+ dealt with many Unicode characters which are visually indistinguishable by the established process of normalisation: converting many equivalents into common code points. But Apple’s specification for APFS stated that it didn’t normalise, as with some other newer file systems.

Apple found itself having to bolt on workarounds to effectively perform normalisation for APFS after it had been released in iOS 10.3, and it took until iOS 11 before the issues provoked by the original intention had settled down. Ironically, it was some of Apple’s own software which was worst affected by this.

Being designed for SSD/Flash storage, many have questioned whether APFS would ever run satisfactorily on rotating hard disks, or Apple’s hybrid Fusion Drives, which it has continued to sell as standard in many of the most popular models of Mac. A year ago, at WWDC 2017, the APFS engineers asserted that their new file system already ran on both plain hard disks and Fusion Drives, although there were remaining performance issues which they were addressing.

Beta-releases of High Sierra did support APFS on hard disks and Fusion Drives, but when macOS 10.13 was first released, those were suddenly unsupported; when questioned, users were reassured that their support would be added in a later release. Since then, APFS has become supported on plain hard disks, but anyone, like me, foolish enough to have trusted Apple to bring it to Fusion Drives has been waiting almost nine months. Mojave will make that a full year.

Of course there’s nothing intrinsically wrong in having an operating system founded on two different file systems. But there are features, such as Mobile Time Machine, which are now heavily reliant on running on APFS, and others like Time Machine itself which remain totally reliant on features only available in HFS+. Forcing many users to mix and match file systems like this can only be a temporary solution, at best.

For some Mac users, with recent laptop models which don’t use desktop or network storage much, APFS as it stands may seem almost ideal. So as of High Sierra 10.13.5 and iOS 11, Apple’s users are divided into those for whom APFS is just fine, because they’re primarily concerned with iOS systems and SSD-based Macs, and those whose Macs can’t boot from it and can’t use it for their Time Machine backups either.

The announcement yesterday is a big step forward, but Apple’s silence over Time Machine is disquieting. As it’s not a technology of relevance to iOS, I suspect its development is proceeding slowly and with limited resources. As is so often the case, you shouldn’t listen to what Apple wants you to hear and enthuse about – does it really envisage the average MacBook user bolting on an eGPU? – but you should beware of its sins of omission.

14Comments

Add yours

1

Gord on June 5, 2018 at 11:58 pm

I don’t see the urgency. It makes sense that of all the APFS-related issues, this one would get the lowest priority. Any external volume that’s used for a Time Machine backup, is going to be solely dedicated to that function, so why should the user care what disk format is on that volume? It’s abstracted away and invisible. When Time Machine (on external volumes) is revamped to use APFS’s snapshots instead of HFS+’s hard links, the user interface and functionality will remain the same as it is now for end users, with the bonus of extra space efficiency.

And for end users who use Time Machine on Time Capsules, the filesystem is NFS (I think?), so they’re not really affected by this issue at all.

I have no doubt Apple is working on bringing APFS to Time Machine external volumes. Until that work has been completed, I’m glad they focused instead on all of APFS’s (many) other issues. Lotsa kinks to iron out. Time Machine can wait.

LikeLike
- 2
  
  hoakley on June 6, 2018 at 5:18 am
  
  Thank you.
  If Time Machine was working well for the great majority of users, I’d agree. But if you have a modern desktop system with, say, 2 TB of storage and run TM backups for much longer than a couple of years, you’ll find that TM degrades to the point where backups take much too long, and your backup becomes progressively unreliable.
  The clever trick of using hard links worked well with smaller backups over shorter periods. Once the backup grows large and extends well back into the past, it starts to fail. As a result, a lot of users are migrating away from TM. What was a backup solution which just works has been turning into a disaster for many – when they turn to their TM backup, it lets them down. Any backup system which does this on such a large scale is no longer fit for purpose.
  APFS would also bring a major improvement to volume management: its flexibility with volume sizing. It should also make encrypted backups more feasible.
  The fact that Apple is not intending TM 2 for Mojave means that the earliest it is likely to appear is in late 2019. I doubt that I will still be running Time Machine 1 by then, and will probably – like so many other users – have abandoned it for something which does cope reliably.
  Of course, as TM is not in iOS, Apple won’t give the work the resources it really needs. So it will, like so much else in macOS, fester and die quietly. Apple has no excuse that it couldn’t give it the resources that it needs: it is a deliberate choice not to invest in TM 2. Does that put macOS users first, as claimed in the Keynote?
  Howard.
  
  LikeLike
3

Gord on June 6, 2018 at 12:25 am

Who knows, maybe it’s taking Apple a while to refine the procedure for converting HFS+ Time Machine archives to APFS. Presumably that will be an option.

LikeLike
- 4
  
  hoakley on June 6, 2018 at 5:20 am
  
  Translation: maybe Apple chose not to devote sufficient investment to this project, because it has no impact on iOS or its users, therefore is low priority.
  Howard.
  
  LikeLike
5

Rock on June 6, 2018 at 10:06 am

I agree TM1 is not fit for purpose. I’m sick of the number of times I have seen the backup failed message and have had to erase and do over.

LikeLiked by 1 person
6

Sebastian on June 6, 2018 at 11:31 am

I cannot comment much on Time Machine’s current state, other than that I’ve abandonded it years ago (in 2012, to be exact), for the simple fact that even then it didn’t serve its most basic purpose, namely reliably backing up my files. (Some were quietly and inexplicably omitted from the backup—important files—and if that happens even just once, there is simply no way I can entrust the software with this fundamentally important task.)

Technically, you are most certainly quite right: a modern, properly working Time Machine 2 is desperately needed. But if Apple had announced now that Time Machine 2 is coming, I’d have zero confidence that it would even remotely perform the way I’d need it to—or even the way the average Mac user would need it to. Just looking at all the software Apple has put out over the last years paints a very clear picture: functionality is getting reduced, reliability is declining rapidly, and virtually every piece of software is shipped with a multitude of bugs, some of which are so severe that no one in his right mind could deem the software fit for general use. And what’s worst, many bugs just never get fixed. It very much appears now that if something is—or gets—broken but doesn’t make a buzz and doesn’t impact Apple’s bottom line, it is not even considered for repair anymore. It will just linger on in that state. And that’s what I would expect a Time Machine 2 to look like.

The best example for this kind of disregard for software quality is, of course, APFS itself. Ask an average Mac user what benefits it brings for him, and I doubt he will be able to name any. Instead, APFS’s history is already littered with security issues, performances issues, and compatibility issues—and severe ones at that, in each category. Basically, as of now, it is a clear step down from HFS+ for almost every practical purpose. (There are minor advances over HFS+, like local snapshots, but none come to my mind that are worth all the trouble APFS brings with it in general use.)

Basically, my idea of the next major version of macOS is—and has been for a long time—one that adds ZERO new features and instead just fixes the hundreds or thousands of little and big issues that have accumulated since… well, more or less since macOS reached a relatively stable and performant state in version 10.7. Actually, I am moderately happy now to see that Mojave will indeed only bring very minor front-end additions—though I wouldn’t count on that meaning Apple will actually do some of the heavy bugfixing (and design-fixing) work that macOS so desperately needs. But at least we might be spared one round of breaking and degrading various parts of macOS.

And just one other thought concerning Time Machine, though this is of course just speculation on my part: Apple’s silence on this may also be an indicator that it is comtemplating abandonding local backups altogether, in favor of a cloud-based solution. Already, users are pushed to save many of their files remotely, which in Apple’s view seems to mean that those files are safe anyway and not in need of (user-end) backups anymore. (I even remember reading about a comment by an Apple engineer when APFS was originally announced, to the point that SSDs are so reliable anyway that backups are hardly needed. Wouldn’t be surprised at all if other, higher-ranking Apple employees thought much the same way…) And anyway, Apple has a history of ditching local data in favor of remote storage (think Movie rentals, think OS distribution, think early pushes towards the cloud with MobileMe, think online versions of iWork, etc. pp.). If the current implementation of Time Machine can be made to hold out for a few years more (well, “hold out” as in “not explode violently and destroy millions of users’ backups”), it is quite conceivable that Apple’s favored successor will not be another local backup solution but rather something like Time Machine in the Cloud—or even no Time Machine at all but simply pushing every piece of user data into the cloud. (It sounds dystopic even to myself, but look at how iCloud drive is already working: originals in the cloud, and local copies only for improved performance. (And look how swell that’s going.))

Anyway, I’ll stop, but I’d argue that in many respects, a missing Time Machine 2 might really just be the tip of the iceberg.

LikeLiked by 1 person
- 7
  
  hoakley on June 6, 2018 at 11:40 am
  
  Thank you.
  Hmm – local backups no longer needed? You can back everything up to iCloud? Those are scary thoughts indeed.
  I think you’re being a bit unfair to APFS: it has been very successful for what it was designed for – iOS. I also think that snapshots and sparse storage are more advantageous than you credit them with – Mobile Time Machine was rubbish before APFS, but now it’s something many users choose to use.
  Howard.
  
  LikeLike
  - 8
    
    Sebastian on June 6, 2018 at 1:21 pm
    
    I cannot comment at all on iOS matters since I do not own an appropriate device. I was referring solely to APFS on the Mac.
    
    Why do you think APFS was designed “for iOS”? That was not my impression at all—to the point that I rather thought the forced conversion of iOS devices was Apple’s substitute for a beta test before throwing the file system at Mac users.
    
    In what way has APFS improved the experience for iOS users? (“Very successful” would imply that it had notable benefits.)
    
    On the Mac side, I have no intention of quibbling over specific features. No one will dispute that HFS+ was long outdated and in need of replacement. However, a filesystem is such a basic system component that simply having some cool new stuff (even if it’s needed for other future cool stuff) doesn’t suffice; it must be reliable and performant from the day it is forced (!) upon the user, at least to the extent that the old filesystem was. And that is simply not the case. We’ve had several severe APFS-related security issues (as I need not tell you); we’ve had a possible data loss scenario (apparently still unfixed); we’ve had inferior performance even on SSDs, for which APFS was explicitly designed; we’ve had neglect of HDDs and Fusion drives; and I don’t even want to start about everyday usage cases, some of which have suffered dramatically. And the list goes on. All that is just not acceptable, and the few useful features APFS does introduce don’t even remotely make up for it.
    
    LikeLiked by 1 person
    - 9
      
      hoakley on June 6, 2018 at 1:40 pm
      
      APFS could not have been designed for macOS, as it omitted key features which, on a computer system, were going to cause problems. First, it was from the outset incompatible with Time Machine in not supporting directory hard links, which guaranteed that it could never host TM backups, past or future. Second, its whole design is aimed at SSD/Flash storage, which is the only form of local storage in iOS, but for the foreseeable future only one of several types of storage used by macOS. Third, its lack of journalling or equivalent – again, fine for the small and simple local storage in iOS, but unsuitable for more complex desktop systems with long-term hard disk requirements. Fourth, its complete lack of accommodation for Unicode normalisation, which really only proved a problem with iOS when it worked with external storage (syncing).
      HFS+ was never a good fit for iOS, as would have been expected. Apart from the normalisation issue, the transition on iOS devices was swift and painless. They now have features like sparse storage, snapshots, and so on, which make better use of their local storage than HFS+ ever could.
      As I wrote above, if Apple had been looking for a file system for macOS, it would have gone for ZFS, with features like spanning physical storage devices. But for iOS, ZFS would have been crazy – all its biggest benefits would have been wasted. So ZFS was rejected, and Apple specified APFS primarily for iOS storage instead.
      As for bugs in APFS, Apple should have expected them. Many people (myself included) warned that switching to a new file system would take time and a lot of engineering investment. Instead Apple tried to rush it through in a matter of months – something they got away with in iOS, but didn’t in macOS.
      Howard.
      
      LikeLike
- 10
  
  ThePrimeDirective (@ThePrimeDir) on June 23, 2018 at 2:26 am
  
  A cloud-based backup solution is no solution at all. A 1 TB per month data limit is becoming standard, as ISPs strive to eliminate 4K video use from anyone other than themselves (they exempt themselves from the limit to establish a TV monopoly). Of course they also intend the data cap to block cloud backup. I would quickly bust that limit just doing incremental backups on my 4 computers, never mind trying to do a restore. Then there are those times when internet service is down (sometimes for days) and anything cloud based ceases to function. Mission critical functions should operate without an internet connection.
  
  LikeLike
11

Eric on June 8, 2018 at 9:47 pm

I suspect that someone would expect the average user to notice the benefits of a file system is showing a lack of understanding how file systems work. My only problem with APFS to this point has been not being able to make everything run on it. For various reasons of course. Spinning disks because of performance issues, Time Machine because of hard links, etc. HFS+ is ancient. Like FAT32 with warts. Adding journaling made it even more problematic for long term viability of one’s files.

ZFS was the “it” file system, until Oracle (Borg-acle?) took it over and left it with instant licensing issues. But it had other issues as well. As does BTRFS (pronounced Butter Fuss. Synology loves it.) It too has issues that made it not ideal for an ecysystem like Apple has. There is no way Apple designed it for iOS and then forced it onto their other devices. But each system takes its own time to get right. And Apple has been holding back and getting it right. (Remember ZFS on Mac? No more repeats of that kind of thing please!)

I can imagine, as we pointed out up above that converting a Time Machine Backup to a Time Machine 2 Backup would be very complex and messy, and takes a lot of time to accomplish without errors. Consider many Time Machine volumes are darn near full, making such conversion a real challenge, let alone converting the hard links to something usable.

I love APFS on my Mac, iPhone, iPad, and Apple TV. Why? Because I never, ever notice it unless I’m running some kind of utility that deals with volumes and formatting and such. And that’s the best part about it. So there is no user satisfaction to look for when it comes to average users. You might as well ask how did quantum mechanics influence your trip to New York? If you noticed, that’s when it’s time to worry.

LikeLiked by 1 person
12

Tony on June 12, 2018 at 4:27 pm

I agree that APFS has many issues and that its introduction without Unicode normalisation was laughably naive. I also agree that it was designed for SSD, not rotating HD, and we will have to see how well it ages before we know whether Apple has reinvented the need for defrag. However, I think some of the system-level problems may be being overstated.

The TM issues centre on the format of the back-up drive. From a system perspective, I’m with Gord on this: when regarded as a black-box data store, I don’t care what format the drive uses. The “wear-out” problems of backup drives dating back many years are regrettable but may just represent asking too much of a consumer-quality solution. I have had problems with TM drives too but most of them seem to be related to interface problems in Yosemite-era versions of Mac OS X where so much was broken in the various comms stacks that it’s a miracle anything worked. Things have been better of late (FX: sound of fingers crossing)

I still feel (with no rigorous analysis, it’s true) that APFS can provide an equivalent back-up drive to HFS+ (that is one that can present a snapshot of the filing system from a point in the past without storing multiple copies of unchanged files); for hard links, substitute clones. From the system perspective, the difference could/should be invisible.

It is not unreasonable, IMHO, for Apple to suggest that HFS+ TM drives be archived at some convenient time and new drives used for the transition to APFS. Indeed, periodic archiving of drives rather than trusting TM’s thinning algorithms over too many years might be prudent anyway for any long-term back-up process.

As for lack of journalling, when APFS was announced Apple claimed that it had “a novel copy-on-write metadata scheme to ensure that updates to the file system are crash protected, without the write-twice overhead of journaling” (so, one presumes, avoiding the need for roll-back/replay of the journal with its consequent start-up delays). I presume that this coalesces file operations and then makes a single metadata change from the self-consistent old file to the self-consistent new file. I have read nothing more on the subject but have trusted that this is indeed true and that it works acceptably.

LikeLiked by 1 person
- 13
  
  hoakley on June 12, 2018 at 4:35 pm
  
  Thanks.
  It’s my understanding that copy-on-write is a technique best suited to SSDs, whereas journalling seems still to be a favourite for HDs. But that could have been coloured by reading some of the debates about the APFS specification.
  Howard.
  
  LikeLike
14

Week 23 – 2018 – This Week In 4n6 on June 17, 2018 at 8:24 am

[…] APFS comes to Fusion Drives, but there’s no sign of Time Machine 2 […]

LikeLike