hoakley February 17, 2018 Macs, Technology

Is APFS fully supported yet?

Yesterday’s article about selecting hard disks raised questions about whether APFS is now officially supported on them. Although I stated that it is not, that is neither completely correct, nor competely wrong. This article tries to clarify APFS support beyond SSDs.

APFS has worked on non-SSD storage since May-June 2017. In his presentation to WWDC in June 2017, Pavel Sokolov, a File System Manager, stated unequivocally that APFS was the official replacement for HFS+, Fusion Drives were fully supported as boot volumes, APFS stored all its metadata on the SSD part of a Fusion Drive, APFS automatically defragments on hard disks, hard disks were not automatically converted from HFS+ to APFS during High Sierra installation, and performance improvements were still being sought.

There was no doubt at that time that APFS was intended to replace HFS+ as the standard file system across hard disks, Fusion Drives, and SSDs.

However, when High Sierra 10.13 was released on 25 September 2017, not only did it not convert Fusion Drives to APFS, but Apple made it clear that APFS was not then supported on Fusion Drives. There were reports at the time that Apple staff stated that such support would be provided in a future update to High Sierra, but I did not see any written release from Apple making that clear, nor was there any promised timescale.

APFS has been updated several times since 25 September, the last being in the 10.13.3 update of 23 January 2018. I have not come across any release notes which have claimed any change in the storage supported by APFS. However, on 5 February 2018, Apple updated its developer documentation to state that:
Apple File System is optimized for Flash/SSD storage, but can also be used with traditional hard disk drives (HDD) and external, direct-attached storage. and
Can I boot macOS High Sierra from an APFS-formatted hard disk?
Yes. macOS High Sierra supports Apple File System for both bootable and data volumes.

No mention is now made of Fusion Drives in that FAQ.

There are other Apple support documents which cover APFS for users. Among them, this article provides advice on choosing the most appropriate format for disks. It states that the installation of High Sierra results in the automatic conversion of SSDs to APFS, and that Fusion Drives and hard disks aren’t converted. It gives no advice as to which to choose when formatting a hard disk, although it hints heavily that APFS is intended mainly for SSDs.

Help for High Sierra’s Disk Utility mirrors the developer FAQ, advising that
While APFS is optimized for the Flash/SSD storage used in recent Mac computers, it can also be used with older systems with traditional hard disk drives (HDD) and external, direct-attached storage.
It does not mention Fusion Drives.

So, as of High Sierra 10.13.3, APFS is the standard file system for SSDs which are only used by High Sierra systems, “can” be used on hard disks which are only used by High Sierra systems, but remains unsupported on Fusion Drives.

There are four major limitations to the use of APFS.

Backups

Time Machine backup drives cannot be in APFS format, but must either be in Journalled HFS+, or networked via SMB. As far as I can tell, Apple has given no indication as to whether the current version of Time Machine will ever support APFS, or what its solution will eventually be. Whatever the type of drive, if it is to be used for Time Machine backups, it must be in HFS+ format for the foreseeable future.

Shared storage is an interesting case. At present, AFP is deprecated but still works for drives which are not formatted in APFS. Networked drives which are formatted in APFS are not accessible by AFP, though, and can only be accessed using SMB or NFS. They cannot be used for Time Machine backups anyway.

If you prefer a volume cloning and backup tool, then SuperDuper! version 3.1.4 claims full support for APFS, as does Carbon Copy Cloner 5. Both should work fully with non-SSD APFS drives.

RAID

AppleRAID doesn’t support APFS, although Apple claims that you can combine APFS volumes with Apple RAID volumes to support RAID levels 0 and 1. For most RAID users, that is woefully inadequate. SoftRAID will support APFS in version 6.0, which is currently in test. So at present there is essentially no support for software RAID on APFS volumes.

Access from Sierra

Although macOS Sierra’s APFS support was updated in the recent security update, you shouldn’t ever expect a Sierra system to be able to mount and access an APFS volume. APFS support in Sierra was only ever pre-beta, and that in High Sierra has continued to diverge ever since its release. You may be lucky enough to get an unencrypted APFS disk to mount on a Sierra system, and may even get some files off it, but Sierra currently cannot even see an encrypted APFS volume, let alone try to access it.

One potentially good reason for wanting to use APFS for an external disk is its superior encryption. Because APFS encrypted volumes cannot be accessed from Sierra, you should only consider using them in situations where there will never be any need to attempt that.

Repair and data recovery

Bundled Disk Utility in High Sierra has full support for APFS, with supporting tools such as fsck_apfs too.

Support from third-party tools remains more patchy. For example, DiskWarrior 5.0 does not support any operations on APFS, but Prosoft claims its Drive Genius supports all features apart from defragmentation and repartitioning of APFS containers/volumes. iBoysoft Data Recovery for Mac is claimed to work fully with APFS, and the same team in Chengdu, China, has implemented APFS support for Windows. I don’t know how well either works, and welcome your experiences, please.

If you were to send your damaged 16 TB APFS drive off to a data recovery centre today, they would probably not get results comparable to those from a damaged HFS+ disk. It may be many more months before their service for APFS matches that currently achieved with HFS+ recoveries.

So yes, it appears that APFS is now supported on hard disks, provided that you don’t want to use them to store Time Machine backups, don’t use them in a RAID array, don’t need to access them from Sierra, and won’t need to recover data from them in the event of damage. You may also find that they performed better when formatted in HFS+, which seems to be the consensus from experience.

As for Fusion Drives, which I fancy far outnumber internal hard disks in relatively recent Macs, the answer still seems to be no, across the board.

Postscript:

I have now located a clear and very official statement that macOS High Sierra 10.13 didn’t support Fusion Drives or hard disks – in the launch press release of 25 September 2017:
APFS currently supports every Mac with all‑flash internal storage — support for Fusion and HDD Mac systems will be available in a future update.
There has been no more recent press release which has updated that statement.

19Comments

Add yours

1

Miles Wolbe on February 17, 2018 at 4:52 pm

APFS still needs some work.

From Mike Bombich, creator of Carbon Copy Cloner:

macOS may lose data on APFS-formatted disk images

LikeLiked by 1 person
- 2
  
  hoakley on February 17, 2018 at 5:26 pm
  
  Thank you, Miles – yes, another indicator that AFPS is far from complete, although in this case I think it is an issue which should affect quite few users.
  I didn’t consider disk images above, and I’m not really sure what advantage APFS might have for a disk image.
  Howard.
  
  LikeLike
3

Tony on February 17, 2018 at 6:58 pm

If I was running Apple’s software architectures, I would cut my losses. APFS is designed and optimised for SSD and SSD is the future. HFS+ contains years of optimisation effort for HDDs and is inherently faster than APFS on them (we have previously discussed the video at http://docs.macsysadmin.se/2017/video/Day3Session4.mp4 that explains this).

Why not retain HFS+ where it is superior and develop APFS for the day when all Macs have internal SSDs? This allows new macOS features that depend upon APFS to become discriminators for the latest hardware, something that marketing at least will appreciate. It also admits the simple answer (well, simpler anyway) for Fusion drives of using the existing Core Storage architecture to present two drives, each using their optimal formats, to the user as a single drive – all the logic of putting the right data in the fastest location already exists.

There has been a disappointing lack of system thinking around APFS from the beginning. APFS’s inability to match Unicode file names in iOS 10 and Apple’s advice that all apps would need to be changed to compensate for this, followed by an about-turn and a fix to APFS in 10.3.3, is a glaring example. I trust that lessons have been learned internally!

LikeLiked by 1 person
- 4
  
  hoakley on February 17, 2018 at 7:09 pm
  
  Thank you. That is certainly one of Apple’s options, and one which would not necessarily be bad for users.
  There is one significant problem, though: Time Machine. As SSD RAID arrays become more common, TM has to support their use for backups (I’d love to be able to do that here!). Re-engineering TM to support APFS would either make it much more complex (to support the current scheme and APFS) or would make it work very differently on HFS+, probably to the detriment of those using it to hard drives.
  Above all, though, Apple needs to communicate. It’s one of the reasons that Apple has kept falling flat in corporate markets. It is capricious (which is great for consumer products), and usually makes long-term planning impossible.
  Is it now worth replacing my old Promise Pegasus RAID HD unit with a new RAID HD unit for TM backups? I’ll just nip off and inspect the goat entrails and tea leaves…
  Howard.
  
  LikeLike
5

Tony on February 18, 2018 at 4:35 pm

I think the temptation from APFS Snapshots as an improved way of storing TM backups could lead to a similar divide as with internal drives. This could well be your ‘Time Machine 2’.

Interesting points about the corporate market. In this case, I think that rather than being capricious, there either wasn’t a coherent longer-term plan or it fell apart in the face of implementation difficulties.

LikeLiked by 1 person
- 6
  
  hoakley on February 18, 2018 at 4:49 pm
  
  APFS snapshots are ideal for Mobile TM, where they are simply made in the volume metadata and no data copying takes place.
  Backing up to a different volume using a snapshot system would be a very different matter, and is not something which I can recall Apple even suggesting. The difficulty would there appear to be in making the data part of the backup incremental, something which the current TM does very elegantly and efficiently.
  The capriciousness in APFS for macOS was in thinking shallow, that any file system which was good enough for iOS (which was clearly the main driving force) could be made to work on Macs. I don’t think for a moment that it was ever intended that they should have different case-sensitivity, for example. TM and RAID weren’t significant issues in design, because they are irrelevant to iOS. It just seemed like it would all be fine.
  Howard.
  
  LikeLike
7

Tony on February 19, 2018 at 5:50 pm

I haven’t thought deeply about this but snapshots would seem to be a neat structure for storing the time dimension on the back-up drive.

The data part (if I understand your comment correctly) would likely work as in the current TM: FS events would store changes made so that TM could periodically copy changed files to the back-up drive (where APFS could apply its delta-storage to save space). The back-up drive could then use periodic snapshots (the same period) to replace the bespoke links used by TM in HFS+.

This all presupposes that the APFS mechanisms scale appropriately but I guess we’re dependent upon that anyway. Have I missed something fundamental?

LikeLiked by 1 person
- 8
  
  hoakley on February 19, 2018 at 6:32 pm
  
  I’m not sure that would work.
  The brilliant think about snapshots is that they are simply a copy of the volume metadata at that instant. So they point to all the currently-allocated file data. So long as none of those extents get written over, all the file system has to do to roll back to the snapshot is just swap out the current metadata, and swap in the old.
  Those metadata are meaningless on another volume, such as a backup. So what would have to happen is that the FSEvents data would be used to determine the changed files. They would then have to be copied over to the backup. Something else would then have to make the connections with the previous versions of those files, and the ‘new’ structure of the backup built from scratch.
  There’s no difference from a regular incremental backup system, unless TM were to spend a lot of time with the backup piecing each backup together to the illusion of a regular volume, as is currently accomplished in the file system using hard links.
  But, as we know, the current scheme works well for a while, and as the backups get bigger and older, it grinds painfully slow, and prone to failure. So Apple needs to come up with a better solution.
  Howard.
  
  LikeLike
9

Tony on February 19, 2018 at 6:52 pm

I agree. I was assuming that some process similar to the current TM would construct a snapshot ‘as-if’ the back-up drive had actually been the original drive. So same logic as now but construct the data structure for a snapshot rather than one using HFS+ links. The end result uses the same high-level logic to produce an APFS equivalent to the current back-up structure.

I’m now unsure whether this is nonsense or obvious 😫

LikeLiked by 1 person
- 10
  
  hoakley on February 19, 2018 at 7:48 pm
  
  I don’t know: in the continuing absence of any meaningful documentation on APFS, I think that we’re all bluffing a bit!
  Howard.
  
  LikeLike
11

Sebastian on February 20, 2018 at 2:10 pm

Howard, you write:

“One potentially good reason for wanting to use APFS for an external disk is its superior encryption.”

Could you point me to some resources on this topic? I couldn’t find much of value about how FileVault is implemented on APFS, let alone in which way this might be superior to the HFS+ implementation (only on external volumes, or generally?).

Sebastian

LikeLiked by 1 person
- 12
  
  hoakley on February 20, 2018 at 2:23 pm
  
  Sadly, as I’m sure you known, Apple has not released any good documentation yet on APFS.
  However, HFS+ was not designed for encryption, nor was encryption designed into it, but FileVault 2 has been bolted on afterwards.
  APFS is designed from the outset to support encryption, which Apple terms “native”. It offers AES-XTS or AES-CBC encryption modes, depending on hardware, and different encryption models, of whole-disk, single key for the whole volume, or per-file keys with a separate key for sensitive metadata.
  That provides much greater flexibility than HFS+, I believe.
  Howard.
  
  LikeLike
  - 13
    
    Sebastian on February 20, 2018 at 3:05 pm
    
    Thank you. I see the general benefits of native encryption, though this does sound like the benefits are currently more of a potential than an actual nature. No CoreStorage construct needed, good. Maybe a smaller performance penalty. But in the end what really counts is the safety of the encrypted data, and in that regard, may I presume that disks formatted with APFS—as currently treated by macOS, Disk Utility et al.—are pretty much the same as those formatted with HFS+? Or are there options to, say, actually use per-file keys, or even use a longer encryption key, etc.? (I can’t check, I don’t have a 10.13 installation.)
    
    LikeLiked by 1 person
    - 14
      
      hoakley on February 20, 2018 at 3:24 pm
      
      I don’t know what differences there are between HFS+ and APFS whole disk encryption, but I don’t think that HFS+ offers per-volume or finer grain at all except in disk images.
      I think you need to ask Apple for these issues really.
      Howard.
      
      LikeLike
    - 15
      
      Sebastian on February 20, 2018 at 3:40 pm
      
      “I think you need to ask Apple for these issues really.”
      
      I’m sure they’d answer all my questions readily and in meticulous technical detail. 😉
      
      For the record, though, per-volume (as in per-partition, if that’s what you meant) encryption is available with HFS+. Internally, this gets you a CoreStorage contraption with several virtual disks, but for most practical purposes, those behave as you would expect them to.
      
      LikeLiked by 1 person
16

Michael Tsai - Blog - Is APFS Fully Supported Yet? on May 1, 2018 at 6:34 pm

[…] Howard Oakley: […]

LikeLike
17

Peter Sauermann on May 30, 2018 at 10:10 pm

I took a chance and formatted my iMac Late 2013 with the standard 1TB HD to APSF and after an initial hick-up, it worked. With hick-up I mean, that the colour wheel was busy for some time. Not supported, does not mean, that it does not work. However, it is something for the bold with a computer that can be missed. It worked with all Apple Apps, like Final Cut X and Adobe CC applications and Microsoft Office 360.
Reformatting to HFS was not possible with High Sierra. Using an USB-Stick with Sierra did the trick.
One has to use the stick with OSX Sierra to re-format, then shut down and then start up with another stick with High Sierra to re-install.

LikeLiked by 1 person
- 18
  
  hoakley on May 30, 2018 at 10:23 pm
  
  As I state in the article, currently APFS “can” be used on conventional hard drives. It is only unsupported on Fusion Drives.
  I’m surprised that you couldn’t format the hard drive in HFS+, though – High Sierra is certainly capable of that, so maybe you were stopped by a bug in Disk Utility.
  Whether anyone would choose to use APFS on a hard drive is another question which I’ll leave to others to answer. The reports that I have heard would suggest not, but could be misleading.
  Howard.
  
  LikeLike
19

Chris Hand on June 27, 2018 at 11:37 pm

hi – I’m a Mac specialist in UK & I’ve only just come across what I’ve been expecting but dreading since High Sierra was released: a customer with a corrupted, not backed up APFS volume containing significant data…Disk Utility recognises the corruption but can’t fix it, Techtool pro same, Drive Genius doesn’t do much, Data Rescue 5 doesn’t even read the apfs directory with a quick scan(!) & Diswarrior 6 is an indeterminate time from release & I wonder if it will actually happen … iBoysoft actually looks promising from a data recovery angle but after years of relying on the Mac filesystem magic bullet (Diskwarrior) I’m wondering even whether a volume rebuild is going to be feasible with APFS…so sadly, for now its back to the bad old days of finding a corrupted unrepairable (APFS now) volume and having to recover data, wipe the drive & copy back on. Misery me

LikeLiked by 1 person

·Comments are closed.

Share this:

Related