hoakley December 24, 2022 Macs, Technology, Updates

How does macOS tell the type of a file?

The most distinctive and revolutionary feature of the Mac is the Finder, and one of its fundamental skills is to know what type each file is. This enables it to open that file with your preferred app, to preview its contents using QuickLook, and to index its contents for Spotlight search. Those and many other features we take for granted depend on every file having a type, a designator to associate that file with a file format and apps that can open it.

UTI

The standard system in macOS is based on a Uniform Type Indicator, or UTI, like public.plain-text for a plain text file, and public.jpeg for a JPEG image.

UTIs use a structured hierarchical taxonomy forming a vast interconnected tree. For example, when I write a file of Swift source code, the .swift file has the type public.swift-source, which is a specialised type of public.source-code, which is both public.data and public.plain-text, which is public.text. When I open that file, LaunchServices first looks for an editor for public.swift-source files, but can ascend the UTI tree as necessary and use an app designed to open text files of public.text more generally. Some of the more frequently encountered UTIs are shown in the diagram below, which you’ll probably need to expand to full screen to read clearly.

Determining the UTI

It’s often claimed that macOS depends on filename extensions to determine different types of file, but that’s incorrect: macOS is more capable, and uses MIME types when downloading from the Internet, can still use old Classic Mac OS four-character type codes, and ultimately relies on UTIs.

This is easily demonstrated in Terminal. Type the following command
touch notypefile
to create a new file in the current directory without any extension or other clue as to what it is. Then look in the Finder’s Get Info dialog, and you should see that macOS has already assigned it a default type associating it with a default editor. Inspect that file using my free utility Precize, and you’ll see that it has a UTI (listed in the Type entry) of public.data.

Now give it an extension unknown to macOS, such as .xyz, and inspect it again with Precize: its type has changed to something more cryptic like dyn.ah62d4rv4ge81u8p4. That’s a dynamic UTI, created on the fly by macOS to distinguish it as having a unique type, described in Get Info simply as a Document, but still with its default associated editor.

Every item in your Mac’s file system has a UTI to tell Launch Services what to do when you try to open it, for instance by double-clicking its icon. You may find an exception to this, from a longstanding bug dating back to OS X 10.5 in 2007: some files may not return a UTI, but a NULL instead. This seems to be confined to sockets, which might appear to be files but aren’t really.

Discovering the UTI

Unfortunately, although discovering UTIs is key to dealing with documents which are treated as having the wrong type, there’s no easy way to find a file’s UTI in macOS. Thankfully you don’t need long reference lists to find out key information such as what a filename extension or MIME type represents in terms of a UTI: it’s all contained within macOS, if you know how to look using one of the tools listed below.

My own utility for working with UTIs is UTIutility, which I have just updated to version 1.2, with improvements in its crawler to scan folders and list the UTIs it discovers there. It’s now available from here: utiutil12
from Downloads above, and from its Product Page.

UTIutility’s main window lets you enter an extension like xls, and tells you all macOS knows about that and its corresponding UTI. Alternatively, you can open its Crawler window and get it to list all the UTIs it comes across in the selected folder. That can take a long time to work through large folders, or those loaded with UTIs like /Applications, but its results are revealing.

The correct answer to the question of what determines a file’s type is therefore a whole list:

UTI, e.g. com.adobe.pdf,
filename extension, e.g pdf,
OSType, e.g. PDF,
MIME type, e.g. application/pdf, or
Pasteboard type, e.g. Apple PDF pasteboard type.

While you can change a file’s type directly by giving it a different UTI, it’s far simpler to do that indirectly by changing its extension to one correct for the type, such as txt or text for a text file. The OSType refers back to Classic Mac OS, as it’s the four-character type system used before UTIs. MIME types are mainly used for Internet file transfers, and Pasteboards are used when copying chunks of data using the Clipboard, which relies on the same basic system.

LaunchServices

UTIs are one of the many pieces of information stored by LaunchServices in its registry. If you’ve got a few spare MB of disk space and want to see what it gathers, you dump that to a text file using a command like
/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -dump -all system > lsdump.txt
Further information about the hidden lsregistry command is here, and Thomas Tempelmann’s utility is invaluable.

Tools

Thomas Tempelmann’s free Launch Services features an excellent UTI browser.
Precize and UTIutility are free from their Product Page.

Apple documentation

UTI overview (archived)
Framework doc (current)
System-declared UTIs (current).

20Comments

Add yours

1

piattj on December 24, 2022 at 9:27 am

Howard, thanks as ever for wonderful Mac content every day! (& paintings natch!). Enjoy an Xmas break. JP

LikeLiked by 1 person
- 2
  
  hoakley on December 24, 2022 at 10:25 am
  
  Thank you.
  Merry Christmas!
  Howard.
  
  LikeLike
3

EcleX on December 24, 2022 at 10:12 am

Thanks. It was previously possible to find the extension of a file now showing such extension in its name, using applications like File Buddy. Is it possible now to determine the extension of a file that does not show in its name? How?

LikeLiked by 1 person
- 4
  
  hoakley on December 24, 2022 at 10:32 am
  
  Let’s step through that.
  If a filename doesn’t have an extension to it, then there’s no extension. Nothing can show you an extension that doesn’t exist. Try duplicating a file with an extension, perhaps .txt, and on the duplicate removing the extension. What happens to its QuickLook thumbnail, and the default app to open it in Finder’s Get Info? It’s now not a text file, as it doesn’t have the right extension, but is just a generic document.
  It, and every other file, still has a file type. You can inspect that by opening with my app Precize, which tells you a great deal about files, including the official name of their file type. For a generic document, you’ll find that is public.data.
  That’s why using the correct extension is important. Give a text file the extension .jpeg and it won’t open as text any more, and as it’s not really a JPEG image, it won’t open as that either.
  Howard.
  
  LikeLike
  - 5
    
    EcleX on December 24, 2022 at 12:21 pm
    
    Thanks. I meant when somehow a file with an extension in its name looses its extension, for whatever reason. That happens sometimes, for instance, when downloading some files from Internet, like images showing in web pages, files received by e-mal, etc.
    
    Searching Internet with Google I have found an application that reveals the missing extension (File Juicer) most of the times, but I wonder if there are other tools capable of doing it as well now (as said, File Buddy did it in the past).
    
    LikeLiked by 1 person
    - 6
      
      hoakley on December 24, 2022 at 3:02 pm
      
      OK – that’s a completely different question, as the file may well not have had an extension at all. As I explained above, macOS has several ways of determining the file type.
      In this case, when you download a file, the server should give it a MIME type. Any better browser or app downloading that file will assign the downloaded file that MIME type. As that’s one of the ways that macOS works out what UTI to assign it, when it’s downloaded it should be assigned the correct file type depending on that MIME type. That doesn’t require any extension at all, and unless the filename already has an extension, it won’t be given one.
      So long as that file remains on a volume whose file system supports file type metadata, then nothing will change its assigned file type, unless you for example give it an extension which conflicts with that. Problems can arise, though, if that file is copied/moved to a file system which doesn’t preserve the metadata. If you’re going to do that, then giving the filename the appropriate extension could help ensure that its file type is preserved.
      So the extension isn’t missing, it wasn’t there in the first place, nor does it need to be.
      Howard.
      
      LikeLike
    - 7
      
      EcleX on December 24, 2022 at 4:55 pm
      
      Thanks. In some cases, the file has extension (for instance, png) but does not show it for some reason. Because of that, it shows a generic icon in the Mac Finder.
      
      When I drag and drop such file with generic icon over File Juicer, it extracts several items, and among them is the very same file with the png (in this example) extension (or whatever it is such extension).
      
      That happens for instance when downloading an icon from a web page on Internet, received by e-mail, old Mac classic files, etc.
      
      What happens using File Juicer as described above, shows that the file extension was indeed inside the file (albeit it did not show in its name), and File Juicer is capable of finding such extension and regenerate the file name with its correct extension.
      
      What I wonder is if there are other tools capable of that (as said File Buddy did also that in the past). It would be great if Precize had such feature. Just a suggestion for your consideration.
      
      LikeLiked by 1 person
    - 8
      
      hoakley on December 24, 2022 at 5:35 pm
      
      I’m sorry, I don’t think you understand how this works.
      The extension to a filename is part of that filename. When you have the Finder configured to display extensions, that’s exactly what it does. Similarly, Precize takes the filename, extension and other details from the file system. They’re not playing games with you: if you don’t see an extension, then there isn’t one. And that’s why those files are shown with a generic icon.
      File Juicer doesn’t find any ‘hidden’ extension. What it does is look inside the file’s data, trying to recognise different types of data within it. When it finds what looks like a PNG image, then it tries to write that out as a PNG file, with the .png extension, which it deliberately adds to the filename so that you can view it as a PNG image. It can do the same with images embedded in PDF files, for instance.
      It appears very good at its job. If that’s the tool you need, then use it. But it’s not discovering some magically hidden extension that nothing else knows about.
      It’s way beyond Precize’s remit: as I wrote, if you find it does what you need, then use File Juicer.
      Howard.
      
      LikeLike
9

Tristan Hubsch on December 24, 2022 at 6:14 pm

Thank you very much for illuminating this evolving bit of bl@ck ma9ic!

For my own part, the few times I did run into such problems was if and when I needed the “source” from a particular AI (artful irritation, authentic idiocy,…) site, for example, as available from the “Download source” link at https://arxiv.org/format/2212.06086 (all dB entries there are logged are such dot-separated numbers). What turns up in the Downloads folder is a file called “2212.06086”, which the webpage description says is any one of tar.gs, .gz, .dvi.gz, .ps.gz or .html.gz types — but none of which ever shows up as an extension. In the past, macOS would report to not know how to open a file with the numerical extension (such as “.06086” above), and one would have to try one of possible extensions from the list at a time, until the file “unpacked.”

When I tried it today (with the above URL, to test the showcasing example), by Murphy’s law or out of sheer macOS’ spite, double-clicking magically and correctly unpacked it via ArchiveUtility.app. It remains unreadable to BetterZip.app, which I’d have preferred, so as to give a preview before I decide to unpack it whole, or extract a single file out of the package. (And, yes, I did know that the downloaded material was tar-gZipped, so that the “.tar.gz” is in fact the correct extension in this case.) Lastly, your Precize (thank you!) does report the rather useless “NSFileTypeRegular: dyn.ah62d4rv4ge8xarvuha5a”, and I see nothing further that would indicate warranting the “.tar.gz” as the correct extension.

So, while I’m grateful that macOS today knew how to “unpack” the showcasing sample (Xmas miracle?), I remain mystified as ever. Given the stated mission of the the afore(un)mentioned AI site, I should like to hope that its programmers are not spitefully hiding the “unpacking” info, but …where is it? What am I not seeing?

BTW, ever so rarely, the “Source” is in fact an apparently inscrutable file, such as the one that downloads from https://arxiv.org/abs/2211.07061v3 (upper right-hand corner “Source”), …which stubbornly refuses to unpack… until one realizes that it weighs in at a mere 12 bytes, and in fact does open with any text editor to reveal its one-line content, “%auto-ignore”. Brilliant. [/rant …sorry]

LikeLiked by 1 person
- 10
  
  hoakley on December 24, 2022 at 6:38 pm
  
  Thank you.
  This is all rather sad. Web servers can readily provide the MIME type of downloaded files, and most do, either that or retain standard extensions. Instead of doing that, to help all users handle downloads, that server chooses do neither. That leaves you and your Mac completely in the dark as to which of the several formats its has actually delivered.
  The reason you see that UTI in Precize is that macOS is at a loss to know what type the file is, so gives it a unique and dynamic UTI.
  I can’t think of anything more unfriendly than a server refusing to provide any information about the format of the files it delivers, forcing users to guess.
  Ugh.
  Howard.
  
  LikeLiked by 1 person
  - 11
    
    Tristan Hubsch on December 27, 2022 at 3:09 pm
    
    I concur with the sadness of the macOS-unfriendly opacity. …but, there must be more to this! (…and mystifyingly baffling to me.)
    
    I just downloaded (a fresh example of) a “source” from https://arxiv.org/format/2212.11906, and on a whim tried opening the downloaded file “2212.11906” with ‘BBEdit’. Lo and behold: ‘BBEdit’ opens it instantly, as a folder, with all the “source” contents listed in the side-bar. So…
    • How does ‘BBEdit’ know that this is in fact a folder of separate files? (Presumably, there is something in that “2212.11906” download that instructs ‘BBEdit’, but is invisible to macOS.)
    • Why does macOS not see what BBEdit does see? (BTW, TextEdit also opens the “2212.11906,” but displays gibberish, interspersed with legible content.)
    • Or… does the numerical “extension” of the item distract macOS? (I did try manually changing “.” in the item name to “·” then to “p”, to no avail: macOS remains in the dark, while BBEdit is unfazed.)
    • Furthermore, opening “2212.11906” with ‘Archive Utility’ instantly “unpacks” it into a macOS folder. (So, what does ‘Archive Utility’ see that macOS doesn’t? Wasn’t ‘Archive Utility’-post-processing supposed to be triggered upon download? …yes, I know, this ties in with the aforementioned macOS-unfriendly opacity… Does this work (& how) on other OS’s? …other than iOS, which is as dumbstruck.)
    • Might one be able to build a post-processing applet that converts the downloaded “2212.11906” into a macOS-friendly folder? (For this to be automagically triggered at the end of a successful download, it would presumably have to be a browser extension? …to Safari? …or a ‘folder action’ trained on ~/Downloads/ ?) (…one can dream, can one not?)
    
    I know that this particular frustration is a scratch within a crease within a niche…
    
    LikeLiked by 1 person
    - 12
      
      Tristan Hubsch on December 27, 2022 at 3:20 pm
      
      PS. Reading Milo’s comment and your response, and under the impression that the AI of the source of my “source”-frustrations is most likely UNIX-based, …could it be that the “2212.11906” → macOS folder bl@ck ma9ic requires/uses exactly that CLI-file function? That is, the downloaded packet “2212.11906” has no additional information, but ‘BBEdit’ and ‘Archive Utility’ peek into the contents and guess the conversion?
      
      LikeLiked by 1 person
    - 13
      
      hoakley on December 27, 2022 at 3:32 pm
      
      This all comes down to what is trying to discover the format of the file.
      BBEdit is one of the most mature apps on the Mac, and full of really ingenious features. It’s here opening a single file. If it takes a second longer to check what’s really inside that file, you won’t complain, and when it works this magic you’ll be impressed, and rightly so.
      UTI file types aren’t intended for that purpose. They’re primarily there for the Finder and related features such as QuickLook. When you open a folder in the Finder, you expect to see the correct icons instantly, even if that window has to show dozens of them. The Finder doesn’t have the time to look inside each file, check its ‘magic’ and work out what format it’s in. It needs an instant answer, so it can call the right qlgenerator to create a thumbnail or the appropriate icon. Only UTIs can serve that purpose, and many others, such as working out which app to open a file you’ve just double-clicked.
      Downloads have a widely-used system based on MIME types to make this all easier, whatever the operating system involved: a server that supplies the MIME type with each download saves the client computers from having to work out what format a file is, and they can process it appropriately. The additional work involved in providing a MIME type is trivial, and it annoys me that some services simply won’t make life any easier for their users, forcing them to play guessing games like this. MIME types are architecture-independent and quite well standardised, as are filename extensions. So why not use them?
      Howard.
      
      LikeLiked by 1 person
    - 14
      
      Tristan Hubsch on December 27, 2022 at 4:14 pm
      
      @ hoakley, “So why not use them?” — Couldn’t agree with you more. …but the afore(un)named AI doesn’t listen so as to ask it the question. Oh, well.
      
      LikeLiked by 1 person
15

_flex on December 25, 2022 at 8:07 am

You might also want to mention the following CLI command:

mdls -name kMDItemContentType -name kMDItemContentTypeTree

which may also be useful discovering the UTIs.

LikeLiked by 1 person
- 16
  
  hoakley on December 25, 2022 at 8:57 am
  
  Thank you.
  Merry Christmas!
  Howard.
  
  LikeLiked by 1 person
17

Milo on December 27, 2022 at 12:42 pm

Thank you for this article. There is one aspect that is not clear to me though. Does macOS store the UTI for specific files somewhere, and if yes, where?

One more tip for readers of your blog who know how to use the terminal. There is a very useful command called ‘file’, which is able to determine the file type from the content of the file, even if the extension is missing or wrong.

LikeLiked by 1 person
- 18
  
  hoakley on December 27, 2022 at 1:36 pm
  
  Thank you. UTIs are stored in file system metadata – every file and other object in the file system has one assigned, even if it’s just a dynamic UTI. They’re widely used too – obvious examples are by QuickLook to generate thumbnails and previews, in the Finder of course, and Spotlight for indexing.
  Although the file command tool has its uses, these days they are quite limited. I don’t think it’s capable of returning the UTI, which is a major shortcoming, and AFAIK it’s fooled by many modern file types, such as Zipped XML types. But where it uses ‘magic’ to tell apart traditional Unix files, it is invaluable.
  Howard.
  
  LikeLike
  - 19
    
    Milo on December 27, 2022 at 3:54 pm
    
    Thank you for this clarification. The UTIs are rather well hidden. I don’t remeber every “seeing” them while inspecting the file system with common tools.
    
    I agree, the ‘files’ command is no panacea. But it’s quite good in detecting archival file formats in my experience. Also, I think the version shipped with macOS is somewhat dated. It seems to be much more useful on Linux.
    
    LikeLiked by 2 people
    - 20
      
      hoakley on December 27, 2022 at 4:41 pm
      
      “It seems to be much more useful on Linux.”
      I’m sure, but then Linux has nothing like UTIs in file system metadata, so it’s the only way to discover a file’s format. In macOS, all an app has to do is inspect its UTI, which is far simpler and quicker. It’s used in AppKit, for example, in File Open and File Save dialogs.
      Howard.
      
      LikeLike

·Comments are closed.