Here’s a bonus riddle for this weekend: what’s so invisible you can never see it in the Finder, is in many of the folders in your Home folder, and can break your backups? The answer is a .DS_Store file, officially a Desktop Services Store. Although they might appear more ancient, they originated in Mac OS X when its Finder was being rewritten from scratch in 1999.

It had been intended that Desktop Services would eventually gain a public API, but somewhere along the line Apple decided to keep it private, and their format and function have never been officially documented. Its name starts with a dot/stop/period to make it invisible in the Finder, and since macOS Sierra it has been made invisible even when the Finder reveals other invisible files. Currently the best way to see it is in Terminal, where the -a option to ls should include .DS_Store files.

They can be confused with another annoying but more useful hidden file: shadow files whose names start with ._ that are used to carry extended attribute data as part of the AppleDouble file format used on some FAT file systems. They too are invisible in the Finder even when hidden files are supposed to be displayed, but are associated with individual files rather than folders.

Function

The Finder will normally create a .DS_Store file in a folder that you have write access to, when some change is made to it in the Finder, such as creating or copying a file into that folder.

.DS_Store files contain a folder’s custom attributes, data like icon positions, and in more recent versions of macOS custom settings for the display of file metadata.

Among the most important of their contents for some users are Finder or Spotlight Comments, which are normally displayed in the Comments section of the Get Info dialog for a file. Those comments may also be duplicated in the com.apple.metadata:kMDItemFinderComment extended attribute (xattr) of that file, but that’s a secondary copy that can fall out of sync with what’s stored in the .DS_Store file, and the Finder ignores the xattr anyway. The reliance of Finder Comments on invisible .DS_Store files can lead to their unreliability compared with other forms of metadata.

Problems

You’re more likely to come across .DS_Store files when they make a nuisance of themselves by tripping something up. Send a folder from your Mac to a Windows or Linux system, for example, and it’s likely to confuse the recipient with that mysterious extra file that you can’t see at all. Send a folder to another Mac by AirDrop, and any .DS_Store file inside it will also accompany its visible contents. That in turn can cause problems with some backup utilities if it results in an older .DS_Store file being found in a folder that has already been backed up with a newer one.

Recent versions of macOS should no longer write .DS_Store files to computers connected to them over a network. If you want to stop them from being exposed in network volumes of older systems, use the command
defaults write com.apple.desktopservices DSDontWriteNetworkStores -bool true
to disable that. One place .DS_Store files can prove particularly troublesome is in Git repositories. Mikey @0xmachos has provided a simple solution for eradicating them.

At one stage Apple even recommended that they should be explicitly excluded from servers used for network backups or other storage. They can trip up revision control systems, baffle those who open archives created on a Mac, stop folder copying, and confound folder comparison. The simple solution to these, as with so many other problems with .DS_Stores, is to open the folder containing that hidden file, move some of its contents about to force it to be refreshed, and move on.

In the past, .DS_Store files have been suspected of leaking data, and were involved in at least one security vulnerability. Thankfully they now seem as puzzling and opaque to the developers of malware as they are to other users, but I’m sure that one day, someone else will try to do bad things with them again.

Removal

You can recursively delete .DS_Store files from a hierarchy using the command
find . -name .DS_Store -delete
and Ross Tulloch’s BlueHarvest can automatically remove them.