Getting SSD performance data which make sense

I’m delighted to report that I think I’ve now found a way to nail jelly to a wall. Or, if you prefer an even more impossible task, providing real-world benchmarks for Mac storage. In the last episode here, I started to wrestle with exceptionally high ‘burst’ or possibly cached reads of file sizes below about 1 MB. With the unstinting help of Macintouch‘s Ric Ford, I think I can finally put those to bed, even if I can’t explain them.

High read speeds

What started this were reports, which I confirmed, that ATTO’s disk speed test was reporting extraordinarily high read speeds on the internal SSDs in M1 Macs. Here’s the chart it gave me:

attom1mini1

While writing rates stabilise at just under 4 GB/s, even at 64 MiB, reads are continuing to increase in speed, at just under 10 GB/s. Are these results real, a bug in the benchmarking code, or a trick pulled by macOS?

I can confirm that, when benchmarking the reading of files on an M1 Mac’s internal SSD, you can readily obtain such results. Here’s a set obtained using my app Stibium on my M1 Mac mini.

StibM1Zone

The fastest read speed here exceeeds 12 GB/s, which looks incredible because it is. Note the distinctive pattern, though: these very high speeds are confined to reading files between 50 KB and 1 MB, above which they suddenly drop to something which looks more reasonable.

StibiMacProZone

Here they are on my iMac Pro’s internal SSD, an almost identical pattern, this time peaking at just under 6 GB/s. Was I on the verge of discovering some type of ‘burst’ read mode, or what?

StibAshurZone

The answer lies here: these are the same tests, this time from a known and relatively slow fully encrypted external SSD connected via USB-C, not Thunderbolt. While almost all read and write transfers are performed at around 300 MB/s, in this special zone reading can reach almost 10 GB/s, which convincingly breaks the laws of physics.

Never being one to break a law, the best solution is to avoid trying to measure transfer rates in this zone, up to 1 MB file size. I return to my original purpose here: this isn’t to be an exercise in demonstrating that my SSD is faster than yours, but a meaningful comparison of what we should experience in real use.

stibium1041

So I have a new version of Stibium, 1.0b5, which does just that. It uses test files ranging in size between 2 MB and 2 GB, outside the envelope in which those strange results occur. It also fixes some issues in which files that couldn’t be read/written and took zero time won’t show up as NaNs, and a bit of tidying up. You can download Stibium 1.0b5 from here: stibium10b5

New results

I now have a new set of results from those two Macs, and the rather slower external encrypted SSD, in which I have greater confidence. See what you think.

StibiMacProTimes

Regression lines of time taken against file size for my iMac Pro look good, although there are a few outliers which took quite a lot longer, so have lower transfer rates, not higher. Apart from those points, each cluster is comfortingly tight. To get a more accurate gradient I removed those outliers and refitted the line.

StibiMacProRates

There’s also significant scatter on the plot of transfer rates against file size (on an exponential scale), but a clear tendency towards horizontal lines across larger file sizes. As has been seen consistently on benchmarks for internal SSDs controlled by the T2 chip, write rates (blue) are faster than read (red). The highest transfer rate is here about 3.2 GB/s, which seems more realistic at last.

StibM1Times

Graphs of the equivalent data for the internal SSD of my M1 Mac mini also cluster quite tightly, both for times (above) and rates (below). Maximum transfer rate is just below 3.9 GB/s.

StibM1Rates

StibAshurTimes

The slower external encrypted SSD appears even better, with very tight clustering at each point on the time graph above, and little scatter except at smaller file sizes on the rate graph below.

StibAshurRates

Transfer rates

A total of 160 files were written and read for each SSD tested. In each case, test files were written using Stibium a few minutes after a restart. Write tests were then performed, following which that Mac was restarted, and after a further waiting period those test files were read. No file was read more than once.

The iMac Pro internal SSD had a median read rate of 2.2 GB/s (range 0.3-2.5), with an overall rate by regression of 2.2 GB/s, and a latency of 0.00003 seconds. Its median write rate was higher at 2.8 GB/s (1.7-3.2), by regression 3.0 GB/s with a latency of 0.0009 seconds.

The M1 Mac mini internal SSD had a median read rate of 2.9 GB/s (1.7-3.3), by regression 2.8 GB/s, and a latency below zero. Its median write rate was similar at 2.8 GB/s (1.7-3.9), by regression 2.6 GB/s, latency below zero.

The external encrypted SSD (connected to the M1 Mac mini) had a median read rate of 300 MB/s (260-485), by regression 302 MB/s, latency 0.0009 seconds. Its median write rate was 296 MB/s (186-343), by regression 296 MB/s, latency 0.0004 seconds.

Returning, at last, to the original comparison that I wanted to make, the M1 SSD is similar in performance to that in the iMac Pro. Their write speeds are almost identical when comparing median values, although using linear regression the iMac Pro is faster at 3.0 versus 2.6 GB/s. However, the iMac Pro has poorer read performance, at 2.2 GB/s, compared with the M1 at 2.8 GB/s. Whether those will translate into any perceptible difference in performance isn’t clear.

The differences between the internal SSDs and the external encrypted SSD are stark: typically 300 MB/s read and 295 MB/s write, around 10% of the internal SSD transfer rates. This is abundantly clear when looking at the charts above with regressed lines. Transferring (read or write) the 1 GB test file on the external SSD took almost 3.5 seconds, while on the M1’s internal SSD that took only a tenth of that, less than 0.4 seconds.

Admittedly that external SSD writes and reads at a little more than half the speed of unencrypted high-performance SATA SSDs, but even they would take about 2 seconds to transfer that file. Any Mac user who is unable to notice the difference in performance clearly isn’t paying attention!

What next?

I’m happy now, as I hope you are, that Stibium is producing test results which appear consistent and reliable. Although later I want to open it to custom test regimes, my next priority in development is to improve its data analysis. Using means and conventional parametric methods isn’t going to be safe or reliable, though, as scatter in the results shown above demonstrates. It’s easy to get outliers, and the majority of them are reductions in transfer speed. I’m looking at using a robust regression method for estimating overall transfer rates which will provide the user with a single result, plus a measure of dispersion/confidence. I also want to provide a bar chart as is traditional in benchmarking apps.

Currently testing is more laborious than using existing apps. The primary nuisance here is restarting the Mac before running the write test, and again before the read test. I’m looking at alternatives which could be more convenient, although all the evidence that I have at present says that if you don’t restart in this way, then the quality of results will suffer.

As a reminder, here’s how I used it to obtain the measurements above:

  1. Restart, open Stibium, set it to No Cache, Verbose, with 10 repeats.
  2. When everything’s settled, click on Write… to write 160 test files to a folder on the disk to be assessed.
  3. On completion, copy and paste results into a text file.
  4. Set the Repeats in Stibium to 1 (very important to remember this now).
  5. Restart, open Stibium, allow the Mac to settle.
  6. Click on the Read… button and select the folder containing the 160 test files.
  7. On completion, copy and paste results into a text file.

Analysis involves preparing the text saved as a plain CSV file by trimming its top and tail, and importing that into Numbers for editing. There, sort the rows by the Size column, remove any spurious measurements of small files, and marry the Read and Write data into a single CSV file to import as data into DataGraph or the charting app of your choice.