Skip to content

The Eclectic Light Company

Macs, painting, and more
Main navigation
  • Downloads
  • M1 & M2 Macs
  • Mac Problems
  • Mac articles
  • Art
  • Macs
  • Painting
hoakley August 7, 2021 Macs, Technology

Explainer: File systems

Stored data, whether it’s on a physical disk connected to your Mac or accessed remotely over a network or the cloud, is usually managed in chunks. On physical disks, these are normally known as blocks. A file system is responsible for turning those blocks into individual files, and organising those into directories and other structural systems such as volumes.

Inevitably, there are many different ways of doing this, but it’s common ground that the file system has metadata which not only tells it which blocks make up each file, and the arrangement and hierarchy of directories, but provide information about each file, such as its name, the date and time it was created, who has permission to read and write to it, and so on. This metadata – the file system metadata – is itself kept in the storage.

To enable software to access the files in that storage, each type of file system supports standard features such as calls to create a new file, read and write to it, and more. Different file systems also support additional features: HFS+, the older of the two Mac file systems in common use, allows ‘hard links’ to be made to its directories, which is very unusual, but doesn’t support ‘sparse files’ which are supported by APFS, its newer replacement.

macOS currently has built-in support for around 15 different file systems, which notably include:

  • APFS, Apple’s modern file system of choice.
  • HFS+, Apple’s previous file system of choice.
  • exFAT, Microsoft’s file system introduced in 2006, and commonly used on Flash storage.
  • NFS, a distributed file system developed by Sun in 1984, still used on some networked storage.
  • NTFS, Microsoft’s main file system from Windows NT 3.1.
  • AFP, Apple’s older network file system protocol, now being phased out.
  • SMB, a cross-platform network file system protocol, which has largely replaced AFP.

You can see the main list of supported file systems in /System/Library/Filesystems, and some are also supported by kernel extensions and other components. You can also add third-party support for other file systems including ZFS.

In a perfect world, these would all be transparent to the user. All you would have to do is connect to that storage system, and the Finder and your software would be able to see and work with that storage like any other. However, because of the differences between their features, there are plenty of traps into which you can fall.

Some are simple but surprising. You’re unlikely to know when a file stored on APFS is kept in very compact sparse format. But copy that 5 MB to an HFS+ volume and it could suddenly expand to full size, taking several GB, and may take minutes to copy to far more space than you had anticipated. Two of the more common frustrations which Mac users can experience when working across different file systems are naming problems resulting from case-sensitivity and Unicode normalisation, and differences in handling extended attributes.

Older file systems like MS-DOS may not handle case at all, and can have severe constraints on the naming of files and directories. More modern file systems may be quite the opposite, and allow the two files This.text and this.text in the same directory. When these come into collision, problems are likely, for example preventing you from copying a folder containing those two files to an APFS volume on your Mac, as by default macOS APFS preserves case but isn’t case-sensitive, unlike the flavour of APFS used by iOS.

Unicode normalisation is another trap which catches even the most wary. Because some Unicode characters which are visibly identical can have two or more different encodings, some file systems normalise these into one standard form, either Form C or Form D. Given a choice of two, both are in use, and other file systems don’t normalise at all. For a great many writing systems, this makes little difference, but when using accented Roman characters or Korean it can cause havoc.

Extended attributes are additional file metadata which are stored in the file system metadata of HFS+ and APFS. Although they’re not peculiar to Apple’s operating systems, they are used more extensively in macOS than elsewhere. They can contain important supplementary information about a file, such as whether it’s in quarantine, where it was downloaded from, or copyright details. There are schemes to retain at least some metadata on exFAT and related file systems, but generally extended attributes get stripped when files are moved to non-Mac file systems.

Finally, cloud file systems are different again, as they’re normally built on huge distributed databases, which work quite differently. You’re no doubt already familiar with iCloud’s illusion that it’s just an extension of your Mac’s file system, and may have come across the Amazon Elastic File System used on AWS Cloud services. In most cases you’ll never deal with the underlying cloud file system, just its front end when working with Dropbox, iCloud and others.

Share this:

  • Twitter
  • Facebook
  • Reddit
  • Pinterest
  • Email
  • Print

Like this:

Like Loading...

Related

Posted in Macs, Technology and tagged AFP, APFS, Cloud, disk, exFAT, FAT, file system, HFS+, iCloud, metadata, NFS, NTFS, SMB. Bookmark the permalink.

2Comments

Add yours
  1. 1
    Week 32 – 2021 – This Week In 4n6 on August 8, 2021 at 6:34 am

    […] Howard Oakley at ‘The Eclectic Light Company’Explainer: File systems […]

    LikeLiked by 1 person

  2. 2
    Justin McMurtry on November 29, 2021 at 1:36 am

    Hi Howard! ~ It’s been 18 months since you last mentioned RAID systems in this blog, pointing out that at that time, the only non-DIY options for Mac users were AppleRAID and SoftRAID (which I gather remains true today). I’m trying to set up a four-disk RAID array, and I feel skeptical of the reliability and effectiveness of both of those two options. As you mention in the post above, ZFS is a third-party filesystem option available for macOS, and I’ve read elsewhere that BTRFS is another—and it seems both of these are reputed to have outstanding RAID-like functionality designed into the filesystems themselves: redundancy, vigilant automated error detection & correction, even “self-healing” abilities. Could you please share your viewpoint on the practical feasibility of using either of these filesystems as the basis of a solution to integrate multiple external SSDs (or perhaps even HDDs) attached to a Mac system, as an alternative to AppleRAID and SoftRAID, for the purposes of improved read/write performance and reliability? ~ Thanks, Justin

    LikeLiked by 1 person

·Comments are closed.

Quick Links

  • Downloads
  • Mac Troubleshooting Summary
  • M1 & M2 Macs
  • Mac problem-solving
  • Painting topics
  • Painting
  • Long Reads

Search

Monthly archives

  • February 2023 (2)
  • January 2023 (74)
  • December 2022 (74)
  • November 2022 (72)
  • October 2022 (76)
  • September 2022 (72)
  • August 2022 (75)
  • July 2022 (76)
  • June 2022 (73)
  • May 2022 (76)
  • April 2022 (71)
  • March 2022 (77)
  • February 2022 (68)
  • January 2022 (77)
  • December 2021 (75)
  • November 2021 (72)
  • October 2021 (75)
  • September 2021 (76)
  • August 2021 (75)
  • July 2021 (75)
  • June 2021 (71)
  • May 2021 (80)
  • April 2021 (79)
  • March 2021 (77)
  • February 2021 (75)
  • January 2021 (75)
  • December 2020 (77)
  • November 2020 (84)
  • October 2020 (81)
  • September 2020 (79)
  • August 2020 (103)
  • July 2020 (81)
  • June 2020 (78)
  • May 2020 (78)
  • April 2020 (81)
  • March 2020 (86)
  • February 2020 (77)
  • January 2020 (86)
  • December 2019 (82)
  • November 2019 (74)
  • October 2019 (89)
  • September 2019 (80)
  • August 2019 (91)
  • July 2019 (95)
  • June 2019 (88)
  • May 2019 (91)
  • April 2019 (79)
  • March 2019 (78)
  • February 2019 (71)
  • January 2019 (69)
  • December 2018 (79)
  • November 2018 (71)
  • October 2018 (78)
  • September 2018 (76)
  • August 2018 (78)
  • July 2018 (76)
  • June 2018 (77)
  • May 2018 (71)
  • April 2018 (67)
  • March 2018 (73)
  • February 2018 (67)
  • January 2018 (83)
  • December 2017 (94)
  • November 2017 (73)
  • October 2017 (86)
  • September 2017 (92)
  • August 2017 (69)
  • July 2017 (81)
  • June 2017 (76)
  • May 2017 (90)
  • April 2017 (76)
  • March 2017 (79)
  • February 2017 (65)
  • January 2017 (76)
  • December 2016 (75)
  • November 2016 (68)
  • October 2016 (76)
  • September 2016 (78)
  • August 2016 (70)
  • July 2016 (74)
  • June 2016 (66)
  • May 2016 (71)
  • April 2016 (67)
  • March 2016 (71)
  • February 2016 (68)
  • January 2016 (90)
  • December 2015 (96)
  • November 2015 (103)
  • October 2015 (119)
  • September 2015 (115)
  • August 2015 (117)
  • July 2015 (117)
  • June 2015 (105)
  • May 2015 (111)
  • April 2015 (119)
  • March 2015 (69)
  • February 2015 (54)
  • January 2015 (39)

Tags

APFS Apple AppleScript Apple silicon backup Big Sur Blake bug Catalina Consolation Console diagnosis Disk Utility Doré El Capitan extended attributes Finder firmware Gatekeeper Gérôme HFS+ High Sierra history of painting iCloud Impressionism iOS landscape LockRattler log logs M1 Mac Mac history macOS macOS 10.12 macOS 10.13 macOS 10.14 macOS 10.15 macOS 11 macOS 12 macOS 13 malware Mojave Monet Monterey Moreau MRT myth narrative OS X Ovid painting Pissarro Poussin privacy realism Renoir riddle Rubens Sargent scripting security Sierra SilentKnight SSD Swift symbolism Time Machine Turner update upgrade Ventura xattr Xcode XProtect

Statistics

  • 13,770,102 hits
Blog at WordPress.com.
Footer navigation
  • About & Contact
  • Macs
  • Painting
  • Language
  • Tech
  • Life
  • General
  • Downloads
  • Mac problem-solving
  • Extended attributes (xattrs)
  • Painting topics
  • Hieronymus Bosch
  • English language
  • LockRattler: 10.12 Sierra
  • LockRattler: 10.13 High Sierra
  • LockRattler: 10.11 El Capitan
  • Updates: El Capitan
  • Updates: Sierra, High Sierra, Mojave, Catalina, Big Sur
  • LockRattler: 10.14 Mojave
  • SilentKnight, silnite, LockRattler, SystHist & Scrub
  • DelightEd & Podofyllin
  • xattred, Metamer, Sandstrip & xattr tools
  • 32-bitCheck & ArchiChect
  • T2M2, Ulbow, Consolation and log utilities
  • Cirrus & Bailiff
  • Taccy, Signet, Precize, Alifix, UTIutility, Sparsity, alisma
  • Revisionist & DeepTools
  • Text Utilities: Nalaprop, Dystextia and others
  • PDF
  • Keychains & Permissions
  • LockRattler: 10.15 Catalina
  • Updates
  • Spundle, Cormorant, Stibium, Dintch, Fintch and cintch
  • Long Reads
  • Mac Troubleshooting Summary
  • LockRattler: 11.0 Big Sur
  • M1 & M2 Macs
  • Mints: a multifunction utility
  • LockRattler: 12.x Monterey
  • VisualLookUpTest
  • Virtualisation on Apple silicon
  • LockRattler: 13.x Ventura
Secondary navigation
  • Search

Post navigation

Impressionist painting in Britain: 3 The New English Art Club
Saturday Mac riddles 111

Begin typing your search above and press return to search. Press Esc to cancel.

  • Follow Following
    • The Eclectic Light Company
    • Join 3,130 other followers
    • Already have a WordPress.com account? Log in now.
    • The Eclectic Light Company
    • Customize
    • Follow Following
    • Sign up
    • Log in
    • Copy shortlink
    • Report this content
    • View post in Reader
    • Manage subscriptions
    • Collapse this bar
%d bloggers like this: