Random Read

Random read speed is the most difficult performance metric for flash-based SSDs to improve on. There is very limited opportunity for a drive to do useful prefetching or caching, and parallelism from multiple dies and channels can only help at higher queue depths. The NVMe protocol reduces overhead slightly, but even a high-end enterprise PCIe SSDs can struggle to offer random read throughput that would saturate a SATA link.

Real-world random reads are often blocking operations for an application, such as when traversing the filesystem to look up which logical blocks store the contents of a file. Opening even an non-fragmented file can require the OS to perform a chain of several random reads, and since each is dependent on the result of the last, they cannot be queued.

These tests were conducted on the Optane Memory as a standalone SSD, not in any caching configuration.

Queue Depth 1

Our first test of random read performance looks at the dependence on transfer size. Most SSDs focus on 4kB random access as that is the most common page size for virtual memory systems and it is a common filesystem block size. For our test, each transfer size was tested for four minutes and the statistics exclude the first minute. The drives were preconditioned to steady state by filling them with 4kB random writes twice over.

Random Read
Vertical Axis scale: Linear Logarithmic

The Optane Memory module manages to provide slightly higher performance than even the P4800X for small random reads, though it levels out at about half the performance for larger transfers. The Samsung 960 EVO starts out about ten times slower than the Optane Memory but narrows the gap in the second half of the test. The Crucial MX300 is behind the Optane memory by more than a factor of ten through most of the test.

Queue Depth >1

Next, we consider 4kB random read performance at queue depths greater than one. A single-threaded process is not capable of saturating the Optane SSD DC P4800X with random reads so this test is conducted with up to four threads. The queue depths of each thread are adjusted so that the queue depth seen by the SSD varies from 1 to 16. The timing is the same as for the other tests: four minutes for each tested queue depth, with the first minute excluded from the statistics.

The SATA, flash NVMe and two Optane products are each clearly occupying different regimes of performance, though there is some overlap between the two Optane devices. Except at QD1, the Optane Memory offers lower throughput and higher latency than the P4800X. By QD16 the Samsung 960 EVO is able to exceed the throughput of the Optane Memory at QD1, but only with an order of magnitude more latency.

Random Read Throughput
Vertical Axis scale: Linear Logarithmic

Comparing random read throughput of the Optane SSDs against the flash SSDs at low queue depths requires plotting on a log scale. The Optane Memory's lead over the Samsung 960 EVO is much larger than the 960 EVO's lead over the Crucial MX300. Even at QD16 the Optane Memory holds on to a 2x advantage over the 960 EVO and a 6x advantage over the MX300. Over the course of the test from QD1 to QD16, the Optane Memory's random read throughput roughly triples.

Random Read Latency
Mean Median 99th Percentile 99.999th Percentile

For mean and median random read latency, the two Optane drives are relatively close at low queue depths and far faster than either flash SSD. The 99th and 99.999th percentile latencies of the Samsung 960 EVO are only about twice as high as the Optane Memory while the Crucial MX300 falls further behind with outliers in excess of 20ms.

Random Write

Flash memory write operations are far slower than read operations. This is not always reflected in the performance specifications of SSDs because writes can be deferred and combined, allowing the SSD to signal completion before the data has actually moved from the drive's cache to the flash memory. Consumer workloads consist of far more reads than writes, but there are enough sources of random writes that they also matter to everyday interactive use. These tests were conducted on the Optane Memory as a standalone SSD, not in any caching configuration.

Queue Depth 1

As with random reads, we first examine QD1 random write performance of different transfer sizes. 4kB is usually the most important size, but some applications will make smaller writes when the drive has a 512B sector size. Larger transfer sizes make the workload somewhat less random, reducing the amount of bookkeeping the SSD controller needs to do and generally allowing for increased performance.

Random Write
Vertical Axis scale: Linear Logarithmic

As with random reads, the Optane Memory holds a slight advantage over the P4800X for the smallest transfer sizes, but the enterprise Optane drive completely blows away the consumer Optane Memory for larger transfers. The consumer flash SSDs perform quite similarly in this steady-state test and are consistently about an order of magnitude slower than the Optane Memory.

Queue Depth >1

The test of 4kB random write throughput at different queue depths is structured identically to its counterpart random write test above. Queue depths from 1 to 16 are tested, with up to four threads used to generate this workload. Each tested queue depth is run for four minutes and the first minute is ignored when computing the statistics.

With the Optane SSD DC P4800X included on this graph, the two flash SSDs are have barely perceptible random write throughput, and the Optane Memory's throughput and latency both fall roughly in the middle of the gap between the P4800X and the flash SSDs. The random write latency of the Optane Memory is more than twice that of the P4800X at QD1 and is close to the latency of the Samsung 960 EVO, while the Crucial MX300 starts at about twice that latency.

Random Write Throughput
Vertical Axis scale: Linear Logarithmic

When testing across the range of queue depths and at steady state, the 525GB Crucial MX300 is always delivering higher throughput than the Samsung 960 EVO, but with substantial inconsistency at higher queue depths. The Optane Memory almost doubles in throughput from QD1 to QD2, and is completely flat thereafter while the P4800X continues to improve until QD8.

Random Write Latency
Mean Median 99th Percentile 99.999th Percentile

The Optane Memory and Samsung 960 EVO start out with the same median latency at QD1 and QD2 of about 20µs. The Optane Memory's latency increases linearly with queue depth after that due to its throughput being saturated, but the 960 EVO's latency stays lower until near the end of the test. The Samsung 960 EVO has relatively poor 99th percentile latency to begin with and is joined by the Crucial MX300 once it has saturated its throughput, while the Optane Memory's latency degrades gradually in the face of overwhelming queue depths. The 99.999th percentile latency of the flash-based consumer SSDs is about 300-400 times that of the Optane Memory.

SYSmark 2014 SE Sequential Access Performance
POST A COMMENT

110 Comments

View All Comments

  • romrunning - Monday, April 24, 2017 - link

    Speaking of real-world tests, I am waiting for SQL Server tests on an Optane SSD - like on that DC P4800X. The "enterprise" review of the 4800 was all synthetic benchmarks with some disclaimer that they can't simulate all enterprise loads. Sure, you can't simulate everything, but I'm very disappointed that -nothing- enterprise level was even tested. Reply
  • ddriver - Monday, April 24, 2017 - link

    I am sure it is just an unfortunate coincidence, and it is not like intel is trying to hide the actuality of real world performance :) Reply
  • darkfalz - Monday, April 24, 2017 - link

    SSD cache / Hybrid SSD drives work okay on certain workloads, mainly productivity stuff, but if you have a lot of games/media they tend to fill up really quickly and I don't think any of the companies that write the algorithms, Intel included, can really figure out how reliably and over long usage periods decide what should be in the cache and what shouldn't.

    I have a 24GB SSD cache (ExpressCache) in my Notebook and I partitioned the OS/Programmes for to one partition, and put all the media on the second partition, and set it to only cache the first partition. This setup works pretty well.

    I also have a Hybrid SSHD in another laptop (only 8GB I think) that I mostly use as a background downloading PC, and after a few days of doing this any useful boot / OS / Chrome stuff that was in the cache has been evicted and it's back to booting at the same speed as a regular HDD.

    Nice in theory, highly variable in practice. I never tried the Intel SRT out because larger SSD affordability improved a lot after it was released.
    Reply
  • satai - Monday, April 24, 2017 - link

    Can I just put it into a PCIe slot (via a reduction), boot linux from an other SSD drive and use it as any other block device? Reply
  • romrunning - Monday, April 24, 2017 - link

    Per the article: "However, the Optane Memory can also be treated as a small and fast NVMe SSD, because all of the work to enable its caching role is performed in software or by the PCH on the motherboard. 32GB is even (barely) enough to be used as a Windows boot drive, though doing so would not be useful for most consumers." Reply
  • DigitalFreak - Monday, April 24, 2017 - link

    Are you also going to test Intel SRT with a ~$77 SATA SSD and the same WD HDD? I bet it would perform about the same, and SRT works with non-boot drives. Reply
  • eddieobscurant - Monday, April 24, 2017 - link

    How about using the same test setup as with the other ssds and run the same benchmarks for comparison?

    I get you wanna please intel for giving you access to optane (which should be named remote preview by the way) , but come on !!!

    Also the new graphs ( probably suggested from intel , since tomshardware has something like these ) are not easy to understand with a quick look.
    Reply
  • Billy Tallis - Monday, April 24, 2017 - link

    These two Optane reviews interrupted my work on putting together a new 2017 consumer SSD test suite to replace our aging 2015 suite. When the new test suite is ready, you'll get comparisons against the broad range of SSDs that you're used to seeing and more polished presentation of the data. Reply
  • Shadowmaster625 - Monday, April 24, 2017 - link

    Can this be used as a boot drive? Reply
  • romrunning - Monday, April 24, 2017 - link

    Per the article: "However, the Optane Memory can also be treated as a small and fast NVMe SSD, because all of the work to enable its caching role is performed in software or by the PCH on the motherboard. 32GB is even (barely) enough to be used as a Windows boot drive, though doing so would not be useful for most consumers." Reply

Log in

Don't have an account? Sign up now