Where do you turn to when you need help with your Mac? The fact that you’re reading this suggests you might ask a question here, or send me an email, as many do. As I still find (a little) time to sleep, I suspect those are a tiny minority. Others ask their questions on forums such as Apple Support Communities or MacRumors, or contact Apple Support. Using online search like Google is fine if your problem can be expressed succinctly, but what if it’s a group of more ill-defined symptoms? How about asking ChatGPT?

Over the last couple of months, many have been amazed at how ChatGPT appears able to respond to questions intelligently, and now there’s a new version of Microsoft’s Bing that seems to be pushing the performance of artificial intelligence even further. Why not give them a try?

A little while ago I was contacted by a troubled reader who had done just that, and asked ChatGPT to diagnose the problems they were experiencing with their Mac. It came back and informed them that their Mac had a Rootkit installed, and was thus under the control of someone else. Could that be correct?

Before going any further, let’s evaluate that advisor.

Validate your advisor

To do this, the reader posed ChatGPT a reasonable request:
Please write the following in the style of a Computer Scientist explaining a concept to a group of graduate students. Explain the Secure Boot and SSV for M1 Macs.

To which came the response:

Secure boot and System Software Verification (SSV) are security features implemented on M1 Macs to protect the system from malicious software and unauthorized modifications.

Secure boot is a boot time verification mechanism that ensures the authenticity and integrity of the boot components and system software. It does so by verifying cryptographic signatures of the boot components and system software, starting from the firmware and extending to the bootloader, drivers, and finally the operating system. If any of these components fail to pass the signature verification process, the secure boot mechanism will prevent the system from booting and show an error message.

On the other hand, SSV is a runtime verification mechanism that ensures the authenticity and integrity of the system software and configuration files. The SSV process operates continuously and checks the cryptographic signatures of the system software and configuration files at regular intervals. If any of these components are modified or corrupted, SSV will detect the changes and prevent the system from booting, similar to the secure boot process.

Both secure boot and SSV use public-key cryptography to verify the authenticity and integrity of the system components. The public keys used for verification are stored in the firmware, which is considered to be a secure location on the system, and are used to verify the digital signatures of the components. This process helps to ensure that the system is not compromised by malware or other malicious software.

In conclusion, secure boot and SSV are critical security features on M1 Macs that help protect the system from malicious software and unauthorized modifications. These features help ensure the authenticity and integrity of the system components, making it more difficult for attackers to compromise the system and keep the system secure.

Before completing the first sentence of its reply, ChatGPT had revealed that it knew nothing about what it was so confidently writing about. From there on, what it responded with looks plausible to someone entirely unfamiliar with the subject, but is actually riddled with errors.

If ChatGPT were knowledge-based, it would have found Apple’s Platform Security Guide, where SSV is explained as meaning signed system volume, although in its early days it was also referred to as sealed system volume. Nowhere has Apple ever referred to it as meaning System Software Verification, and if you search on that term, you’ll discover that it’s very seldom used in this context. In other words, ChatGPT had invented the term: it’s a lie.

Another gaping error is referring to integrity-checking of the SSV as relying on public-key cryptography. I would have accepted a tree of cryptographic hashes, as the Platform Security Guide explains, but what ChatGPT has written is pure made-up nonsense. And nowhere has it mentioned the crucial fact that the SSV is in fact a read-only snapshot, and has been since its introduction with Big Sur.

Validate the advice

Even if the source of advice appears to be reliable, and their facts check out, the advice they give should remain suspect until you have validated it. Simple and practical tests you can apply include:

• Does it contain only generic solutions, like reinstalling macOS? In Big Sur and later, with the SSV, the value of reinstalling macOS is usually in far greater doubt than before. Besides, most solutions start with limited and specific actions, and should only fall back on generic solutions when those fail.
• Is there evidence of understanding, the how and why of the problem and its solution, not just the what? Any good advisor should be able to explain what’s going wrong, and why the action they’re recommending is a suitable solution.
• Can they answer questions to clarify what they have said, that are explanatory, or are they simply repeating a formula they’ve come across?
• Where any of the facts they assert appear wrong or doubtful, can they provide evidence to support them? Ask them for links and references, preferably to reputable sources.
• Do they show any evidence of experience? macOS changes every year, and in many cases the nature of and solution to problems also change. Does what they had said only apply to specific versions of macOS? If so, which? For many issues, Mojave and Catalina are watersheds, while Big Sur and later are more similar.
• Can they provide alternative solutions, and estimate the likelihood of particular diagnoses and outcomes? ChatGPT and Bing are notoriously assertive, and appear unable to consider alternatives, or the risk of failure, or data loss.

Not just for AI advice

What I’ve written here doesn’t just apply to AI advisors like ChatGPT, but information and advice given elsewhere, including those who answer questions in Apple Support Community Forums, by YouTubers, and bloggers like myself. Although it’s reassuring when someone is a Level 9, that’s no guarantee that anything they’ve written is reliable. Apple Support Community Forums has some restrictive rules about what can and cannot be mentioned that do affect the completeness of answers. YouTube suffers major flaws as a means of delivering technical information, and its methods of reward don’t encourage accuracy or reliability.

Please be careful how much you trust what you hear and read, and whatever you do, don’t believe a word of what ChatGPT and other AI systems tell you. They excel at plausible nonsense, not well-sourced fact.