Are Thunderbolt SSDs more consistent than USB?

Last week, I drew attention to problems users have been experiencing with the performance of external SSDs claiming to be compliant with USB 3.x standards. After extensive wrangling and discussion here, I think it’s fair to conclude that the standards are a mess, and that there’s most probably one or more problems with the controllers or their firmware in Apple’s current M1 series Macs running Monterey 12.3.1. Under duress, I agreed to look at Thunderbolt 3 external SSDs. This article reports what I’ve found so far.

Test systems

I’ve been using three Macs for testing: my trusty iMac Pro, with its four Thunderbolt 3 ports, a MacBook Pro 16-inch with an M1 Pro chip with its three Thunderbolt 4 ports, and a new Mac Studio Max with its four Thunderbolt 4 ports. All are running Monterey 12.3.1 with fairly lean systems, although the iMac Pro does have Samsung’s kexts to support its X5 SSD installed.

Rather than perform single tests with lots of different cables and SSDs, I’m once again performing repeated tests with two devices:

  • a Samsung X5 500 GB external SSD, updated (painfully) to the current firmware;
  • a Samsung 980 PRO 1 TB PCIe 4.0 M.2 SSD installed in a Sabrent Thunderbolt 3 Certified M.2 NVMe SSD Solid Aluminum Enclosure (EC-T3NS).

Both have been connected using the same 0.8 m passive CalDigit Thunderbolt 4 cable (not the one I used in my previous tests, though).

Benchmarking software

My primary test utility is my own Stibium, in which I normally perform ‘gold standard’ testing without restarting between the write and read tests. This performs write tests first, with a total of 160 files ranging in size from 2 MB to 2 GB. Following that, it reads those 160 files in the read test, ranging again from 2 MB to 2 GB in size. I also report results from Blackmagic and Amorphous, the latter run with 5 repetitions at 1 GB size, as is popular.

Results

The Samsung-Sabrent device returned consistent read rates of 1.5-1.6 GB/s and write rates of 1.5-1.6 GB/s in all configurations and systems tested, with very little variation. These were consistent across:

  • one of the TB3 ports on my iMac Pro;
  • one of the TB4 ports on my Mac Studio Max;
  • the right TB3 port on my MacBook Pro M1 Pro with a SATA/USB SSD and Ethernet adaptor connected to the two TB3 ports on the left.

When writing test files, rates were slightly lower with file sizes under 60 MB, but remarkably even on 60 MB to 2 GB. Read rates showed a slight tendency to fall with increasing file size, but were normally steady at about 1.6 GB/s from 600 MB to 2 GB.

Blackmagic reported a write speed of 1.4 GB/s, and read of 1.6 GB/s. Amorphous results were similar:
SEQ1M QD8 read 1686 write 1479
SEQ1M QD1 read 1450 write 1292
RND4K QD64 read 1140 write 259
RND4K QD1 read 64 write 38

(all in MB/s).

tbssd980

An attempt to induce thermal throttling in the Samsung-Sabrent device by writing 320 consecutive files totalling over 100 GB in a period of 85 seconds failed to show any reduction in write rate.

Results for the Samsung X5 weren’t consistent for write rates, but were more consistent for reads. Read rates were typically around 2.6 GB/s, while write rates were around 0.7 GB/s. However, the method used to compute the overall write result had considerable effect: for example, a simple average gave higher results, of perhaps 1.3 GB/s, with a 20% trimmed mean similar. Linear and Theil-Sen regression resulted in lower write results. This is because the results changed markedly during testing, as demonstrated during a successful attempt to induce thermal throttling.

This followed a similar protocol, this time writing a total of 800 test files ranging in size between 2 MB and 2 GB. The graph below shows measured write rate against the cumulative total size of files written.

tbssdthermal

Once 23.9 GB of data had been written to the X5, write rates fell suddenly from about 2.3 GB/s to 0.6-0.7 GB/s, which is consistent with the onset of thermal throttling, from which it didn’t recover during the rest of that test.

Because other benchmarking software doesn’t normally approach such a high total of data written during tests, results given by Amorphous appeared impressive:
SEQ1M QD8 read 2870 write 1956
SEQ1M QD1 read 2428 write 1666
RND4K QD64 read 1132 write 212
RND4K QD1 read 58 write 41

(all in MB/s).

tbssdx5

Additional read testing of all three TB3 ports on the MacBook Pro failed to show any significant differences in speed between ports.

Discussion

Unlike my previous test results, Thunderbolt devices and ports were highly consistent, apart from write tests on the X5. There is no evidence here of any problem with Thunderbolt ports when run at the transfer rates used here, and in conjunction with other Thunderbolt devices which use little if any bandwidth, unlike external displays and competing storage systems.

Benchmarking write rates on the X5 isn’t easy, because of its repeated throttling. This could instead be the result of a buffer or cache (such as an SLC Write cache) becoming exhausted, but as it occurred after a relatively small amount of data had been written, that appears unlikely. Write rates before throttling occurred were considerably lower than seen when writing into fast memory, making a memory buffer very unlikely. If the total of SLC Write cache on an otherwise empty 500 GB SSD were to be less than 25 GB, then it would be readily exhausted in many other use cases.

This dramatic reduction in write rate is also the cause of differences observed in benchmark results for the X5. It highlights the danger of relying on writing less than 10 GB to the SSD under test, and of using simple arithmetic averaging to calculate the overall write speed. Averages are dangerously misleading estimates when used with what is essentially bimodal data.

But the biggest question, the elephant in the room, is why two high-performance SSDs could only attain read rates of around 1.5 and 2.6 GB/s over Thunderbolt 3, which is capable of 40 Gb/s, equating to 5 GB/s.

In the case of the Samsung-Sabrent device, its read and write rates are determined not by the bus or SSD, but by the interface in the case. Sabrent only claims these transfer rates, as do most manufacturers of most affordable Thunderbolt cases. Some claim up to 2.2 GB/s read and 1.8 GB/s write, but cases which claim much more than those generally cost more than the Samsung 980 PRO 1 TB SSD inside the case. None appears to offer anything approaching 5 GB/s.

While the unthrottled performance of the Samsung X5 was impressive by comparison, accurate benchmarking indicates that it’s only capable of reading at 2.6 GB/s and writing at 2.3 GB/s, half that which could be achieved over Thunderbolt 4.

However Apple’s internal SSDs might achieve them, they consistently do much better, between 6.5 and 7.5 GB/s, even over sustained writes totalling more than 100 GB. External Thunderbolt SSDs appear reliably quick, but invariably less than half the speed of those internal SSDs, and in many cases around a quarter. External Thunderbolt SSDs are only significantly faster when compared to run-of-the-mill SATA/USB-C SSDs, which remain significantly cheaper, in the case of the Samsung-Sabrent combination tested, around half the cost.

If you need better performance than you can get from a SATA/USB-C SSD, then Thunderbolt 3 is far simpler and more consistent than messing with USB 3.x standards. But it can’t compete with M1 internal storage.