Direct-Attached Storage Benchmarks

Our evaluation routine for hard-drive based direct-attached storage devices borrows heavily from the testing methodology for flash-based direct-attached storage devices. The testbed hardware (the Thunderbolt 3 / USB 3.1 Gen 2 Type-C port enabled by the Alpine Ridge host controller in the Hades Canyon NUC) is reused. CrystalDiskMark is used for a quick performance overview. Real-world performance testing is done with our custom test suite involving robocopy bencharks and PCMark 8's storage bench.

CrystalDiskMark uses four different access traces for reads and writes over a configurable region size. Two of the traces are sequential accesses, while two are 4K random accesses. Internally, CrystalDiskMark uses the Microsoft DiskSpd storage testing tool. The 'Seq Q32T1' sequential traces use 128K block size with a queue depth of 32 from a single thread, while the '4K Q32T1' ones do random 4K accesses with the same queue and thread configurations. The plain 'Seq' traces use a 1MiB block size. The plain '4K' ones are similar to the '4K Q32T1' except that only a single queue and single thread are used.

Comparing the '4K Q32T1' and '4K' numbers can quickly tell us whether the storage device supports NCQ (native command queuing) / UASP (USB-attached SCSI protocol). If the numbers for the two access traces are in the same ballpark, NCQ / UASP is not supported. This assumes that the host port / drivers on the PC support UASP. We can see that the Seagate Backup Plus external storage drives do support NCQ and UASP. Performance numbers are typical of what one might expect from a 5400 RPM hard drive, with peak performance close to 150 MBps for the 5TB Backup Plus Portable and around 135 MBps for the 2TB Backup Plus Slim.

HDD-Based Direct-Attached Storage Benchmarks - CrystalDiskMark

Moving on to the real-world benchmarks, we first look at the results from our custom robocopy test. In this test, we transfer three folders with the following characteristics.

  • Photos: 15.6 GB collection of 4320 photos (RAW as well as JPEGs) in 61 sub-folders
  • Videos: 16.1 GB collection of 244 videos (MP4 as well as MOVs) in 6 sub-folders
  • BR: 10.7 GB Blu-ray folder structure of the IDT Benchmark Blu-ray (the same that we use in our robocopy tests for NAS systems)

The test starts off with the Photos folder in a RAM drive in the testbed. robocopy is used with default arguments to mirror it onto the storage drive under test. The content on the RAM drive is then deleted. robocopy is again used to transfer the content, but, from the storage drive under test to the RAM drive. The first segment gives the write speed, while the second one gives the read speed for the storage device. The segments end with the purge of the contents from the storage device. This process is repeated thrice and the average of all the runs is recorded as the performance number. The same procedure is adopted for the Videos and the BR folders.

Photos Read

The 5TB Backup Plus Portable comes out on top in a couple of workloads, as does the 2TB Backup Plus Slim. However, the relative positions across different workloads are not consistent. This indicates that performance consistency under sustained traffic is not predictable for these drives.

High-performance external storage devices can also be used for editing multimedia files directly off the unit. They can also be used as OS-to-go boot drives. Evaluation of this aspect is done using PCMark 8's storage bench. The storage workload involves games as well as multimedia editing applications. The command line version allows us to cherry-pick storage traces to run on a target drive. We chose the following traces.

  • Adobe Photoshop (Light)
  • Adobe Photoshop (Heavy)
  • Adobe After Effects
  • Adobe Illustrator

Usually, PC Mark 8 reports time to complete the trace, but the detailed log report has the read and write bandwidth figures which we present in our performance tables. Note that the bandwidth number reported in the results don't involve idle time compression. Results might appear low, but that is part of the workload characteristic. Note that the same CPU is being used for all configurations. Therefore, comparing the numbers for each trace should be possible across different DAS units.

Adobe Photoshop Light Read

The lack of performance consistency is more pronounced in these benchmarks. In fact, the two drives being reviewed today appear in the bottom half of the graphs more often than not. The reason for this requires deeper investigation into SMR characteristics, and this is presented in the next section.

Introduction and Product Impressions Investigating SMR for Consumer Workloads
Comments Locked

62 Comments

View All Comments

  • Sivar - Tuesday, June 18, 2019 - link

    This is an interesting way to look at it, but it makes sense.
    Benchmarks for small writes have not gone well for early SMR drives, though that may have improved since. I suspect they would make great storage drives for movies, music, images, or other large files.
  • abufrejoval - Wednesday, June 19, 2019 - link

    I wonder about the command set, too. Crystal disk info typically mentions TRIM support and doesn't show it the posted picture. Dunno if it asks HDDs for TRIM or if the command set is different.

    There must be some way to control that behavior otherwise the hyperscalers wouldn't have co-invented them with the HDD manufacturers.
  • Arbie - Tuesday, June 18, 2019 - link

    Thanks for the excellent article evaluating these drives and describing the state of SMR. It's good to see that the process has improved.

    Seagate nowadays makes it very difficult to determine before purchase if a drive is SMR or CMR. As a result I've gotten into the habit of just going for other brands first - not the best approach, certainly, but I'm only willing to do just so much research.
  • oRAirwolf - Tuesday, June 18, 2019 - link

    I use 7 of the 4TB Seagate drives shucked out of their enclosures in a Dell R620 SFF sitting in a data center. The drives have cold air constantly moving over them and see use as a RAID 5 storage array with the Dell H710P (LSI SAS2208) RAID controller handling everything. They are used for media storage and see very little write activity. These drives drop like flies. If I get 6 months out of them, I feel lucky. Of the 7 drives in the server over the past 1.5 years, I have had to replace 5 of them. I only use them because of their density and the fact I am kind of locked into them at this point. I would not recommend these things to my worst enemy, though.

    I would definitely spend the extra on a WD if it is going to be used with it's external enclosure. Sadly, the WD drives have the USB connector directly soldered to the drive, rather than using a SATA to USB adapter like the Seagate.

    Seeing as how bad their CMR drives are, though, I can't even imagine how bad the SMR drives are. Maybe it would help if they put an iron wolf sticker on them like their other drives. Then it would be strong like iron and loyal like wolf.
  • takeshi7 - Tuesday, June 18, 2019 - link

    The "cold air constantly moving over them" might be part of the problem. Hard drives like to operate at warm temperatures. They're designed that way
  • takeshi7 - Tuesday, June 18, 2019 - link

    And also running shucked drives in RAID just sounds like a terrible idea.
  • Qasar - Wednesday, June 19, 2019 - link

    sounds like oRAirwolf is using hdds that aren't made for the use case he is putting them in, and they are just wearing out, and as was mentioned.. the constant cold air.. cant be good. maybe went the inexpensive way vs the slightly more expensive way just to save a few bucks ?
  • BeethovensCat - Tuesday, June 18, 2019 - link

    Are there any rumors or reliable information about WD releasing a 5Tb or 6Tb version of their Passport/Elements series drives? I have a number of 3Tb drives and they are getting too small. Have been holding on to them in the hope that there will be a 5Tb version available sometime during 2019. Has anyone heard anything. Would like to avoid the large drives with power cables etc.
  • fmcjw - Tuesday, June 18, 2019 - link

    no innovation in this area. BTW, the article, being Anandtech, should've looked into background shuttling of data across shingles, which expend precious energy to gain performance (much like a TLC SSD), but also increase wear. And how about read performance compared to non-SMR drives?
  • fmcjw - Tuesday, June 18, 2019 - link

    no innovation for about 3 years, just enclosure changes.

Log in

Don't have an account? Sign up now