hoakley March 18, 2024 Macs, Technology

Serious bug in Sonoma 14.4 will destroy saved versions in iCloud Drive

Is your Mac running macOS Sonoma 14.4?
Do you save documents to iCloud Drive?
Do you have Optimise Mac Storage turned on?

If your answers to all of those three questions are yes, then be aware that files that you save to iCloud Drive are at risk of losing all their saved versions.

I’m very grateful to JK, who has alerted me to what I can only assume is a serious bug in 14.4. In previous versions of macOS, when a file is evicted (‘Remove downloads’) from local storage in iCloud Drive, all its saved versions have been preserved. Download that file again from iCloud Drive, and versions saved on that Mac (but not other Macs or devices) have remained fully accessible. Do that in 14.4, and all previous versions are now removed, and lost forever.

From my previous testing, that certainly doesn’t happen in Ventura, and when I tested earlier in 14.3, that wasn’t what I observed either.

If you value the saved versions of your documents, either:

don’t put them into iCloud Drive at all, or
turn Optimise Mac Storage off, so they can’t be evicted from local storage.

Please report this to Apple via Feedback, particularly if you have been affected by it.

I have now reported this to Apple as FB13691058, dated 18 March 2024.

I have now given a complete account of this bug and how to preserve file versions robustly in

Sonoma’s lost versions bug:what it is and how to safeguard versions

Postscript

This bug has now been fixed in Sonoma 14.4.1. Provided that files haven’t been evicted in 14.4, once your Mac is updated to 14.4.1, eviction shouldn’t result in the loss of local versions any more.

44Comments

Add yours

1

Michele Galvagno on March 18, 2024 at 9:30 pm
Reply

Oh my….

I’ve just updated V’s Mac mini (Intel) to 14.4 (btw, none of the troubles I had on M3 Max), and she has OptSto checked!

Was this announced anywhere? I don’t think this could ever be a “feature”…

LikeLiked by 1 person
- 2
  
  hoakley on March 18, 2024 at 9:43 pm
  Reply
  
  It certainly wasn’t announced (but what does Apple tell us about iCloud Drive?), and am certain this is a bug. So I have already sent FB13691058.
  Howard.
  
  LikeLiked by 3 people
3

Enzo Vincenzo on March 18, 2024 at 10:10 pm
Reply

And there you go… For many years now I have been IMPOSING myself to NEVER install any version and/or upgrade of macOS, iOS, iPadOS and TVOS that is not XX.X.1, i.e. with the final .1 and following; but then I always fall for it, can’t resist and rush to install, as I did this time as well 😡
Ironic note: if instead of releasing a new macOS every saint year Apple would release it every Holy Year, it would be much better 🤣🤣🤣

LikeLiked by 1 person
- 4
  
  hoakley on March 18, 2024 at 10:18 pm
  Reply
  
  I don’t follow your logic, Enzo.
  The final .1 means it’s the first patch release: Apple generally only issues those for urgent security updates, in between its scheduled minor releases. And for many minor updates (like 14.4) there may not be any intermediate patch release, or the patch release may only address security vulnerabilities and not general bugs.
  In the version numbering hierarchy, it’s the minor versions (14.4) that are more substantial, and likely to fix more general bugs, while the patch releases (14.3.1) should only be small changes, but often for important security benefits.
  Howard.
  
  LikeLiked by 1 person
  - 5
    
    Enzo Vincenzo on March 18, 2024 at 10:42 pm
    Reply
    
    I understood your logic, Howard, thank you!
    But I’m sure 14.4.1 will solve this issue, the bug of unrecognized USB devices, the problems with Java reported by Oracle…
    To remember one among other glaring incidents that have happened over the years with new macOS version or major upgrade (for example, I seem to remember an iTunes coming up to delete data), do I remember wrongly or with 14.0.1 Apple fixed the macOS 14.0 bug that turned some MacBooks into bricks?…
    
    LikeLiked by 1 person
6

ecamhi on March 18, 2024 at 10:16 pm
Reply

I just alerted Econ Technologies since they include backup of iCloud as a feature in Chronosync. Based on your description of the bug this would negatively impact Chronosync backup of iCloud.

LikeLiked by 2 people
- 7
  
  hoakley on March 18, 2024 at 10:21 pm
  Reply
  
  Thank you, but no backup can ever back up versions.
  I have an app (of course) named Revisionist that does allow you to back up versions, and even move them through Windows, Linux, etc., but that has to do sneaky things to preserve the versions. Backup apps can’t and don’t do that.
  Howard.
  
  LikeLiked by 2 people
8

Jon on March 18, 2024 at 10:19 pm
Reply

Excuse my ignorance, but what’s the difference between an updated file and a previous version of a file

LikeLiked by 1 person
- 9
  
  hoakley on March 18, 2024 at 10:26 pm
  Reply
  
  When you save a file using an app that supports macOS versions, each saved version is also retained in that volume’s hidden version database. That allows you to use the Revert To command in the File menu to access those previous versions of that document.
  What happens here is that, after eviction and download, all a document’s previous versions are lost, leaving only the last saved version.
  I have some documents with dozens of previous versions, all of which are accessible through the Revert To command, and a few that have more than 100.
  iCloud Drive can now blow every one of those away, without you even being warned of it.
  Howard.
  
  LikeLiked by 1 person
  - 10
    
    Ivan on March 20, 2024 at 8:03 am
    Reply
    
    But even before 14.4, were the versions kept in iCloud? Since the versions are stored locally in volume, I suspect the loss of versions after evicting a file inevitable and “as designed?”
    
    Thanks for great dive on apfs and iCloud 👍
    
    LikeLiked by 1 person
    - 11
      
      hoakley on March 20, 2024 at 8:15 am
      
      Thank you.
      There was a period when I believe that Apple experimented with saving versions to iCloud Drive, but the results were chaotic, and they seem to have abandoned that some time ago.
      But preserving versions doesn’t require them to be saved to iCloud at all. On the Mac that creates those versions, they’re saved in that volume’s version database, and preserved there. That should still work when the file is evicted from local storage, because that file still exists in local storage, it’s just dataless.
      That worked fine, without any loss of versions, in Ventura (I tested that extensively last June and July), and I checked again in Sonoma 14.3 and am sure that it was still working then. However, it has definitely stopped working with 14.4 – eviction now strips locally saved versions, even though the file is retained locally, with the same inode number. So there’s absolutely no reason that versions should be stripped in this way.
      If Apple deems that this behaves as intended, then it should remove its versions system from macOS, as it makes it unfit for purpose, and iCloud Drive is then a just a means of losing data, also unfit for purpose.
      Howard.
      
      LikeLike
    - 12
      
      ivwang on March 20, 2024 at 8:30 am
      
      Thank you for detailed reasoning, now I see the point you are making. Indeed, evicting can certainly leave the versions in local database intact with an dataless stub file.
      
      Let’s hope Apple addressing the issue and state of file versioning soon enough.
      
      LikeLiked by 1 person
- 13
  
  hoakley on March 18, 2024 at 10:29 pm
  Reply
  
  There’s an easy way to see this, using TextEdit.
  Open a new file and paste or type in a paragraph of text. Save that file.
  Now add another paragraph. Save it again. Repeat as many times as you wish.
  Now close that document. Open it again, and you see only the last version that you saved.
  Now go up to the Revert To command in the File menu, and Browse All Versions.
  That opens a Time Machine-like browser in which you will see every one of those previously saved versions, from the very first with just a single paragraph.
  And all that comes free with macOS, without you having to do anything special.
  Howard.
  
  LikeLiked by 2 people
  - 14
    
    Curtis Brown on March 18, 2024 at 11:40 pm
    Reply
    
    Howard,
    
    A. Thank you endlessly for your incredible support, generous work and insights on Mac OS. It can’t be said enough.
    
    B. That bug is terrifying to the long time iCloud user.
    
    C. I was unable to recreate the bug per your TextEdit test. I’m running 14.4 and tried your exact experiment saving versions to iCloud. I was able to revert to several different earlier saved (fewer paragraphs) versions without the later paragraph additions. All saved versions were made available and worked normally.
    
    I don’t know what this means but remain in your debt.
    
    Regards and thanks,
    
    Curtis
    
    LikeLiked by 2 people
    - 15
      
      hoakley on March 19, 2024 at 6:34 am
      
      Thank you.
      Did you try evicting the file from local storage, waiting a minute or so, then downloading it from iCloud Drive again, and checking its versions then?
      Howard.
      
      LikeLiked by 1 person
16

Alec Kinnear on March 19, 2024 at 12:37 am
Reply

Howard, when will you learn to stop counting on advanced OS features, and just manage your own versions?

Save as… when you think you have created a substantially different draft works a treat.

Thanks for the heads up though.

LikeLiked by 2 people
- 17
  
  hoakley on March 19, 2024 at 6:41 am
  Reply
  
  “Save as… when you think you have created a substantially different draft works a treat.”
  Does it? Can you explain how you do that in, say, Xcode, when editing source files?
  I’m also a little surprised that you assume that I count on this feature (which isn’t “advanced” anyway, it’s actually very old and basic). I don’t, but others do. If I want to ensure that any document’s versions are preserved robustly, the I use my own app Revisionist to make an archive of those versions. That will work whatever you then do with them, including transiting them through Windows or Linux, or on ExFAT format disks.
  Howard.
  
  LikeLiked by 1 person
  - 18
    
    JK on March 20, 2024 at 1:15 am
    Reply
    
    Howard thanks for pointing out Revisionist here. Had no idea it existed and will give it a whirl!
    
    LikeLiked by 1 person
    - 19
      
      hoakley on March 20, 2024 at 7:49 am
      
      Thank you.
      Howard.
      
      LikeLike
- 20
  
  JK on March 20, 2024 at 1:25 am
  Reply
  
  There is nothing advanced about revisions. It is simply broken at the moment in this particular scenario. And “save as…”, while not a bad idea, is hidden behind the option key in many Apple apps (TextEdit, Pages, etc). And to note, this feature works fine if you don’t have optimize mac storage turned on (and used to work even with it turned on). Most people would expect it to keep working.
  
  LikeLiked by 1 person
21

thymine02rising on March 19, 2024 at 4:37 am
Reply

ha! Regarding iCloud with or without current glitches, I can’t help but mentally add a couple of bullet points to the list “Do you… “ value your time… value your sanity. Thanks for the lesson on text edit!

LikeLiked by 2 people
- 22
  
  hoakley on March 19, 2024 at 6:42 am
  Reply
  
  Thank you.
  Howard.
  
  LikeLike
23

EcleX on March 19, 2024 at 8:08 am
Reply

Oops! My message did not get through. Here it is again:

Thanks. There are two ways to report to Apple. The first is more general and the second is more for bugs, but it could be useful to report to both:

Apple Product Feedback at https://www.apple.com/feedback

Apple Feedback Assistant https://feedbackassistant.apple.com

The latter is also available at /System/Library/CoreServices/Applications/Feedback Assistant.app

LikeLiked by 1 person
- 24
  
  hoakley on March 19, 2024 at 9:32 am
  Reply
  
  Thank you.
  Feedback Assistant isn’t more for bugs at all, but feature requests and more. I thought it was only available to developers, but if someone is prepared to go through its more thorough process, then fine.
  Howard.
  
  LikeLike
  - 25
    
    EcleX on March 20, 2024 at 8:31 am
    Reply
    
    Thanks. I have used Apple Feedback Assistant for years, they answered, followed up, and sometimes requested more information and fixed issues.
    
    One of the latest was when they fixed the serious bug generated in Ventura not identifying Panasonic televisions. Some Macs cannot be further upgraded to Sonoma, so they were left in the cold. Surprisingly, Apple fixed it in the last security update, even when Ventura does not receive more general updates outside the security ones.
    
    How to report bugs to Apple now?
    
    LikeLiked by 1 person
    - 26
      
      hoakley on March 20, 2024 at 9:13 am
      
      You have already provided links to the ways to report bugs to Apple. There’s no other way that I know.
      Howard.
      
      LikeLike
    - 27
      
      EcleX on March 20, 2024 at 2:06 pm
      
      Thanks. I have been searching and it seems that a third way to report bugs to Apple is in its Developer forum:
      
      https://developer.apple.com/forums
      
      LikeLiked by 1 person
    - 28
      
      hoakley on March 20, 2024 at 2:08 pm
      
      No. If you try to report a bug there, you’ll be politely asked to file a Feedback.
      Howard
      
      LikeLike
29

Raoul on March 19, 2024 at 8:14 am
Reply

mmm…

another reason to learn about and use ZFS.

LikeLiked by 2 people
- 30
  
  hoakley on March 19, 2024 at 8:16 am
  Reply
  
  So ZFS supports Apple’s version database and iCloud Drive?
  Howard
  
  LikeLiked by 1 person
31

Raoul on March 19, 2024 at 11:14 pm
Reply

Sad you asked that question.

No one is saying ZFS can be a direct like-for-like replacement that functions identically to how you expect it to which appears to be Apple’s way or nothing else.

That is why I used the word “learn” where the willingness to adapt to a change can be left up to the individual.

As you know, other options exist on macOS to guarantee consistency of one’s data that enable rollback to previous versions of files. This does not imply that the means to do so is identical as Apple’s, or, that users should expect it to be.

I’m able to rollback a file to previous versions on my Mac without the need of APFS/iCloud Drive. The file does not even have to be stored locally on my Mac and yet I can access it–and–previous versions anywhere in the world using Mac/PC/Linux. Not identical as you seemingly insist upon but it sure sounds familiar to me.

LikeLiked by 1 person
- 32
  
  hoakley on March 20, 2024 at 7:43 am
  Reply
  
  Thank you.
  Are you telling me that you have ZFS support for the versioning system built into macOS and many of its apps? Because that’s the feature at the centre of this bug.
  And, if you have that, does your ZFS storage host iCloud Drive local storage? Because that’s the other central issue here.
  Howard.
  
  LikeLike
  - 33
    
    RAoul on March 20, 2024 at 8:30 am
    Reply
    
    Don’t worry about it Howard.
    
    I can’t make the above any clearer. ZFS can snapshot a dataset every second if you want it to. Thus every single app installed that writes to the dataset receives the ability to rollback a file if you quick enough to manually make changes and save within a second!
    
    What you fail to see is that I don’t use any of Apple’s services to achieve this. nada… zilch… I don’t even use iCloud so I’m perplexed why you keep asking me questions about Apple’s services that I don’t use.
    
    Once again, I’m able to achieve versions of files using ZFS, just you like you do using Apple’s services. I don’t know what else to type–your questions don’t make sense because I don’t use any of Apple’s services. Anyway, I was pointing out that I’m achieving what Apple’s services offer (and so much more) without the bugs. I’m done explaining.
    
    LikeLiked by 1 person
    - 34
      
      hoakley on March 20, 2024 at 9:12 am
      
      Thank you.
      Strange as it might seem, I *want* to use Apple’s services. I find iCloud Drive valuable, it costs me just £1 per month, and works with all my Macs and Apple devices. ZFS doesn’t.
      I also like the macOS version system, and my apps are written to support it. ZFS isn’t.
      So, while it’s always nice to see you back here, I’m not sure why you keep proselytising something I and the vast majority of Mac users won’t ever use. Sorry.
      Howard.
      
      LikeLike
35

Abraham Brody on March 20, 2024 at 12:00 am
Reply

iCloud can sometimes be misleading as to whether or not data is backed up. Pay special attention to clues such as he solid Cloud icon versus the the dashed Cloud icon for iCloud. Unless you are keeping data both on a folder “On My Mac”, “On my iPad”, “On my iPhone” and iCloud you can only assume the file may be on iCloud itself. During backup to iCloud on a Mac, the file gets temporarily stored at the root of the hard drive. The same thing happens during a Time Machine backup. If you have insufficient space, this temporary file may be very limited what it is copying. With all backups, it is important to keep data in a minimum of two distinct places that are tested all the time for validity. Whether it is on the cloud, or Mac, or phone, or external hard drive or thumb drive, each individual distinct place will prevent you from being caught with your pants down due to bugs like above.

LikeLike
- 36
  
  hoakley on March 20, 2024 at 7:48 am
  Reply
  
  I’m really sorry, but I don’t understand what you have written, and much of what I think I understand there isn’t correct. For instance:
  “During backup to iCloud on a Mac, the file gets temporarily stored at the root of the hard drive.”
  Macs don’t back up to iCloud, and there’s no such temporary storage, and many Macs don’t have a hard drive either.
  No current method of backing up can or does back up saved local versions of files. So this isn’t about backups at all, is it?
  Maybe I’m answering an AI?
  Howard.
  
  LikeLike
37

Raoul. on March 20, 2024 at 9:51 pm
Reply

That’s great that you want to use Apple’s Services. I just don’t seem to have the bugs you endure. Never thought the word learn could be shoehorned into meaning convert either. Apology accepted.

LikeLiked by 1 person
- 38
  
  hoakley on March 20, 2024 at 9:57 pm
  Reply
  
  Well, you seem to have presumed that I’ve “endured” this bug. I haven’t, thank you, as I don’t rely on saved versions in documents saved to iCloud Drive. But if I only wrote about the few bugs that I “endure”, there wouldn’t be much to write about here.
  BTW, which cloud service do you sync your Macs and devices with?
  Howard.
  
  LikeLike
39

Raoul on March 20, 2024 at 11:10 pm
Reply

Zerotier & ZFS, thus all my devices appear in the Network section of the Finder window sidebar, even if the OS is macOS, Linux or Windows, and from wherever I am in the world instantly with no additional effort on my behalf. I move from location A to Location B, lift the lid of my laptop, TouchID to auth and the entries in the Network section are right there still ready for access.

Like I’ve mentioned before, very familiar in terms of the service iCloud drive offers and I’m able to retrieve previous versions of any file should I need it.

LikeLiked by 1 person
- 40
  
  hoakley on March 21, 2024 at 9:31 am
  Reply
  
  Thank you. Having taken a look at what you suggest, there are still big gaps that I don’t understand:
  1. These don’t cover iCloud shared databases, accessed using CloudKit, for instance, so I’d still need to pay my iCloud+ subscription.
  2. Zerotier presents itself as a network share, so doesn’t use FileProvider, and can’t use dataless files or eviction. How do you manage those?
  3. This says nothing about how to support built-in macOS versions, which can’t be shared over a network.
  So far, it would appear to significant additional cost, with no benefit.
  Perhaps you can explain how to address those requirements, please.
  Howard.
  
  LikeLike
41

Raoul on March 24, 2024 at 9:31 pm
Reply

Is h [dot] oakley [at] btconnect [dot] com still active that’s on your about this blog page? I got host or domain not found when I was emailing you to provide more info.

LikeLiked by 1 person
- 42
  
  hoakley on March 24, 2024 at 9:32 pm
  Reply
  
  Yes. I’m quite happy to read it here, though.
  Howard
  
  LikeLike
43

Mark Job on April 1, 2024 at 6:15 pm
Reply

I have lost the ability to copy and paste on my desktop and also have lost access to my Documents folder since I updated my Sonoma to 14.4.0 and now 14.4.1 !!!! I am pulling my hair out !!! I have tried EVERYTHING !!!! I literally do not know what the hell I am going to do ?!! I have been completely defeated by this issue !! I deeply regret updating my System to Sonoma PERIOD ! I want to Revert back about two OS versions, when I knew everything was working perfectly !

LikeLiked by 1 person
- 44
  
  hoakley on April 1, 2024 at 6:26 pm
  Reply
  
  Well, if you have really tried everything, that doesn’t leave us much we can suggest.
  I presume, therefore, that you have tried starting up Safe mode. As I’m not sure what you mean “lost access to” your Document folder, I can’t suggest whether this is a permissions problem, or whether ~/Documents has gone missing altogether. But if you can spare a few moments to be a bit more explicit, I’ll see if I can suggest what to do next.
  Howard.
  
  LikeLike