Inside iCloud Drive: Optimize Storage, and its bugs

In my explorations of iCloud Drive and its behaviour in macOS Sierra and High Sierra, I have not looked at what Apple calls Optimize Storage. Although I hadn’t realised it when I started, there is a very good reason: I can’t get it to work in Sierra 10.12.6. Furthermore, Apple uses the term for two quite different features, which have very different effects on iCloud Drive, so knowing which is being discussed is essential.

Look through iCloud Drive settings in Sierra and High Sierra, and you’ll find two different controls, or maybe Apple thought they were the same.

optimizestore01

In the iCloud pane, click on the Options… button next to iCloud Drive, at the top of the list, and at the foot of the segue which drops down there’s a checkbox labelled Optimize Mac Storage.

optimizestore02

Open the Storage Tab in About This Mac, and for your startup drive click on the Manage… button, which opens another window in which the second item in the body is titled Optimize Storage.

I’ll get the second of those out of the way before discussing the first.

In Storage, by Optimize Storage, Apple is referring to a feature which will strip watched movies and TV shows, and old email attachments, when additional storage space is needed. This doesn’t affect iCloud Drive (directly), so I will ignore this.

I’ll also note in passing here that the top item in Storage, Store in iCloud, duplicates in a different way the option in the iCloud pane to store Desktop and Documents Folders in iCloud, which confuses many users who think that these are different features. They’re not, they’re just Apple trying to bamboozle you into turning them on, and augmenting your iCloud storage to cope.

The rest of this article concentrates on the effects of the checkbox labelled Optimize Mac Storage in the iCloud Drive options of the iCloud pane.

On my iMac running macOS Sierra 10.12.6 in a fairly vanilla configuration, checking or unchecking Optimize Mac Storage does nothing: in effect, that feature is not available, and all iCloud Drive items are always synced down to that Mac. I don’t know whether this is a general bug in Sierra, or if it is a fault somewhere in my configuration, but I wouldn’t bank on Optimize Mac Storage working in Sierra.

On my MacBook Air running macOS High Sierra 10.13.4, Optimize Mac Storage works almost as advertised. When the checkbox is ticked, some files in my iCloud Drive are stored only in the cloud, and not locally on that Mac. When the checkbox is empty, all files in my iCloud Drive are copied to that Mac’s local storage.

Except that even that doesn’t actually work as expected: I currently have less than 720 MB of files in my iCloud Drive, and over 380 GB free space on my startup volume (APFS) on that MacBook Air. By macOS’s own admission, it should not be performing any optimisation by removing local copies of files held in iCloud Drive. Yet it returns files to the cloud regardless.

optimizestore03

From this evidence, this Optimize Mac Storage doesn’t work properly in either 10.12.6 or 10.13.4.

optimizestore04

When I turn Optimize Mac Storage off on the MacBook Air, in 10.13.4, the missing files are quickly downloaded to local storage, as would be expected.

With Optimize Mac Storage turned off, once the change to iCloud Drive has been synced down, neither Sierra nor High Sierra perform any check against a quota before deciding whether to download the whole of new or changed items from iCloud Drive. Instead of a sequence in CloudDocs such as:
[INFO] ┏2458 Apply Changes[1:2699]: attempting to apply creation server item: … n:"co.eclecticlight.Cirrus.data" … local item: (null)
[NOTICE] we will now download new documents automatically because the account contains less than 5.37 GB
[INFO] ┗2458

all you see is
[INFO] ┏1a346 Apply Changes[1:2728]: attempting to apply creation server item: … n:"co.eclecticlight.Cirrus.data" local item: … n:"co.eclecticlight.Cirrus.data" doc reserved … downloading{content+thumbnail}:f8u faulting thumb{greedy}
[INFO] ┗1a346
[NOTICE] downloading 1 documents in com.apple.CloudDocs

The new or changed items are then automatically downloaded to local storage.

Turning Optimize Mac Storage on in Sierra 10.12.6 ran through a full cycle of invalidating the connection to iCloud Drive, then obtaining a full listing and comparing it against the files held locally. However, it didn’t appear to set or enforce a quota.

Whilst I am looking at bugs, I have had a chance to examine a potential failure in syncing Macs running Sierra with iCloud Drive: the DAS bug. Throughout released versions of macOS Sierra, those of us who leave our Macs running constantly have suffered from this. After a few days of continuous running, automatic hourly Time Machine backups become highly irregular. This is because of a bug in the macOS dispatching service DAS (Duet Activity Scheduler) which Apple never fixed, but is believed to have fixed for High Sierra.

Transferring files up and down with iCloud Drive relies on DAS dispatching activities to perform the transfers, and I have been concerned that the bug in DAS could result in failure to sync properly with iCloud Drive. I have now had the chance to test this, when my local DAS had collapsed in a snotty heap, leaving Time Machine in the lurch.

optimizestore05

I then ran the test in Cirrus, and found that it performed perfectly normally. Examining the logs showed that DAS responded quite normally and dispatched the activities for iCloud, even though it was unable to dispatch its normal scheduled activities.

This also casts further light on the nature of that bug: it appears to disable the dispatching of scheduled activities, not tasks passed to DAS to perform for other processes. If it were possible to force DAS to rebuild its dispatch list, it might be possible to recover normal function without having to restart the whole Mac.