Like it or loathe it, AI seems here to stay, and in the form of machine learning (ML) has already been changing our Macs. Use Spotlight, Siri, word completion, or any image processing tools, and you’ll be benefitting from them. Apple silicon chips contain sophisticated hardware support for both AI and ML in their GPUs and ANE, a dedicated neural engine. While the latter is probably the least-used part of the chips at present, that’s changing rapidly, and Apple is set to release more support in the near future. One clue has dropped in the appearance of a new Private Framework named DeepThought in Sonoma 14.2.

But not all M-series chips are equal in this respect: M1 chips have more limited support for recent AI/ML features, including what has become a near-universal format for floating-point numbers, bfloat16. Without that, Macs with M1 chips are likely to remain at a significant disadvantage when running AI and ML functions.

Representing integers (whole numbers) in binary formats used by computers is relatively straightforward: the more digits, the larger the numbers that can be represented. With one hex digit, you get 0-15 in decimal; double that to two hex digits, and the range goes from 0-255. If you want negative numbers, then just set a bit to indicate that, and the range can go from -128 to +127.

The most common way of representing floating-point numbers is to express them in a similar format to scientific or engineering format in decimal. The latter uses a sign (+ or -), a fraction, and an exponent. For example, the number -1,234,567.89 might be expressed as 1.23456789 x 10^6 (ten to the power of six) with the negative sign: that has a fraction of 1.23456789 and an exponent of 6. Being computers, rather than using powers of ten for the exponent, powers of 2 are used instead.

The most common floating-point formats are those of the IEEE 754 standard, where a single-precision 32-bit float has a sign bit, an 8-bit exponent, and 23 bits to contain the fraction. The size allowed for the exponent determines the range of floating-point numbers that can be represented in that format, while the size allowed for the fraction determines how precise any number can be.

With recent rapid developments in AI and ML, several new floating-point number formats have come into use, among them what’s known as bfloat16, with a sign bit, an 8-bit exponent just like the single-precision 32-bit float, but only 7 bits to contain the fraction. Compared with the 32-bit standard, in half the number of bits, bfloat16 numbers cover the same range at lower precision. That’s claimed to be ideal for AI, ML, and use with smart sensor technology.

bfloat16 was developed as part of Google Brain, and has been adopted quickly over the last couple of years across Intel, AMD and Arm processors, and is widely supported in the tools and libraries used for AI and ML. As far as Apple’s M-series chips go, M2 and M3 CPUs support the ARMv8.6A instruction set, which includes bfloat16 support, but the M1 only supports ARMv8.5A, which doesn’t. Support by GPUs and Apple’s neural engine (ANE) is less clear, although work on the M1 ANE suggests that it uses float16 (presumably IEEE half-precision 16-bit float) throughout. Given that the first M1 chips were being delivered in M1 Macs in late 2020, it seems most unlikely that Apple could have incorporated support for bfloat16 in their design.

If the use of bfloat16 is as advantageous as is generally claimed, it looks like M1 Macs will remain at a significant disadvantage compared with M2 and later models. As Apple and third-parties roll out more products with AI and ML at their heart, don’t be surprised if their performance on M1 Macs proves disappointing compared with their M2 and M3 successors.

This situation is starker with Intel Macs, though, as they lack any hardware support for AI and ML, and are already being left in the past.

Reference

Wikipedia