Testbed Setup and Testing Methodology

Thunderbolt's adoption, till now, has mainly been driven by the Mac ecosystem. Solutions for DIY PC enthusiasts are not widespread. GIGABYTE has a few motherboard models with Thunderbolt integrated, while Asus's Thunderbolt-capable motherboards need an add-on PCIe card. Our Windows-based testbed for direct-attached storage (DAS) systems is based on the latest Z97 chipset from Intel. The build originally started out as a multimedia editing machine. The requirements of a DAS testbed fit in nicely with the original purpose of the build. The following table details the configuration of our testbed.

AnandTech DAS Testbed Configuration
Motherboard Asus Z97-PRO Wi-Fi ac ATX
CPU Intel Core i7-4790
Memory Corsair Vengeance Pro CMY32GX3M4A2133C11
32 GB (4x 8GB)
DDR3-2133 @ 11-11-11-27
OS Drive Seagate 600 Pro 400 GB
Optical Drive Asus BW-16D1HT 16x Blu-ray Write (w/ M-Disc Support)
Add-on Card Asus Thunderbolt EX II
Chassis Corsair Air 540
PSU Corsair AX760i 760 W
OS Windows 8.1 Pro
Thanks to Asus and Corsair for the build components

We decided to prepare the testbed with 32 GB of memory in order to make a large enough RAM drive available for our file copy tests. Similar to our NAS testbed, we used OSFMount to allocate 20 GB of the DRAM for a RAM disk partition.

The internal layout of the Z97-Pro (Wi-Fi ac) board is presented below.

It should be noted that the Thunderbolt add-on card can only be connected to PCIe lanes hanging off the PCH (and can't be connected directly to the CPU). Apparently, this is a requirement for vendors to obtain Thunderbolt certification from Intel. In the above layout, we see that the add-on card can only go to the bottom PCIe slot (PCIEX16_3). In addition to the hardware restriction, the BIOS actually operates this slot in x1 mode, with the other three lanes being shared with the ASMedia USB 3.0 ports in the rear panel and the x1 PCIe slot (PCIE X1 SLOT4). The BIOS setting was altered to operate the x16 slot in x4 mode (maximum possible bandwidth for the card). The BIOS entry also has a PCIe cache line setting for Thunderbolt. It was left at 32 bytes (the default) for our evaluation.

Our testing methodology for DAS units takes into consideration the usual use-case for such devices. The most common usage scenario is transfer of large amounts of photos and videos to and from the unit. The minor usage scenario is importing files directly off the DAS into a multimedia editing program such as Adobe Photoshop.

In order to tackle the first use-case, we created three test folders with the following characteristics:

  • Photos: 15.6 GB collection of 4320 photos (RAW as well as JPEGs) in 61 sub-folders
  • Videos: 16.1 GB collection of 244 videos (MP4 as well as MOVs) in 6 sub-folders
  • BR: 10.7 GB Blu-ray folder structure of the IDT Benchmark Blu-ray (the same that we use in our robocopy tests for NAS systems)

For the second use-case, we take advantage of PC Mark 8's storage bench. The storage workload involves games as well as multimedia editing applications. The command line version allows us to cherry-pick storage traces to run on a target drive. We chose the following traces.

  • Adobe Photoshop (Light)
  • Adobe Photoshop (Heavy)
  • Adobe After Effects
  • Adobe Illustrator

Usually, PC Mark 8 reports time to complete the trace, but the detailed log report has the read and write bandwidth figures which we present in our performance tables. Note that the bandwidth number reported in the results don't involve idle time compression. Results might appear low, but that is part of the workload characteristic. Note that the same CPU is being used for all configurations / will be used for future DAS reviews. Therefore, comparing the numbers for each trace should be possible across different DAS units.

Introduction Performance Evaluation - 2big Thunderbolt 2
POST A COMMENT

42 Comments

View All Comments

  • ciparis - Wednesday, July 9, 2014 - link

    Looks like a terrible Thunderbolt implementation on the PC side of things. Too bad, but it renders this review a lot less informative about the drive side of things that it would be if you did a quick Mac comparison to illustrate how a more native implementation can take advantage of these drives. Reply
  • repoman27 - Wednesday, July 9, 2014 - link

    Ganesh, the ASM1156 is most likely a SATA 6Gb/s switch (2:1 or 1:2 mux/demux) similar to the ASM1456: http://www.asmedia.com.tw/eng/e_show_products.php?...

    There should also be a SATA 6Gb/s host controller on the board somewhere, but it doesn't appear to be in any of your photos. The original LaCie Thunderbolt designs used a Marvell 88SE9182. This should show up in Device Manager as well, and I'm curious as to what drivers it was using and what link speed it was reporting. The performance of the SATA host controller vs. the SATA to USB 3.0 bridge is the most likely culprit in terms of why your USB 3.0 speeds were consistently higher than with Thunderbolt. Although they were using a Mac and the Blackmagic Disk Speed Test, StorageReview showed much higher throughput over Thunderbolt in RAID 0 than you were able to achieve with your setup. http://www.storagereview.com/lacie_2big_thunderbol...

    If you're going to disassemble the device to take pictures of the internals, it would be much appreciated if you could provide photos of the PCBs in their entirety. Some folks, myself included, are quite interested in seeing the evolution of Thunderbolt device designs.

    Also, there's a minor typo in the introduction. The name of the 4C Thunderbolt 2 controller should be "DSL5520", not "DL5520".
    Reply
  • repoman27 - Wednesday, July 9, 2014 - link

    "Given the bus-powered nature and the cost, it is not a surprising to find the PM851 inside.the unit."

    Although SSDs with PCIe controllers apparently command a premium at the moment, don't forget that they can be connected directly to a Thunderbolt controller, whereas SATA SSDs require the inclusion of a SATA host controller.

    I think there are several more likely reasons for going with SATA. First, they also produce versions of the Rugged with HDDs in them, which aren't available with PCIe interfaces. Second, you can make a dual-interface device like this by using a 1:2 demux and cheap SATA 6Gb/s to USB 3.0 bridge chip.

    Although you didn't touch on it in the article, these new tethered cable Thunderbolt designs are actually quite interesting. They're similar to the Apple Thunderbolt to Gigabit Ethernet and Thunderbolt to FireWire Adapters. The DSL2210 Thunderbolt controller is embedded in the Thunderbolt connector, and the cable is actually a PCIe 2.0 x1 cable, not an active copper Thunderbolt cable. You can almost see the end where it connects to the device in one of your teardown photos. Eliminating the active cable saves a considerable amount of money and also uses way less power, leaving up to 9W of the 10W power budget for the device itself.
    Reply
  • MikhailT - Wednesday, July 9, 2014 - link

    The one thing I'm very curious about is the latency but I can't find any good reviews on this. I was hoping this article will cover it but alas, no.

    If I put my collection of my virtual machines on those devices, would it be more responsive on USB or TB?

    Logically, TB should be much responsive but by how much?

    I'm constantly running out of space on my laptop because of the massive VMs and I'd like to get the rugged device here, does anybody think this is a bad idea?
    Reply
  • Howard - Thursday, July 10, 2014 - link

    IP ratings are not for ruggedness, with arguably the sole exception of IP69K. Reply
  • torp - Thursday, July 10, 2014 - link

    Q1: Why nothing about CPU utilization? Back when it was Firewire vs USB 2.0, the difference was horrendously in favour of Firewire. I've personally observed 3% with Firewire vs 30% with USB 2.0 on the same enclosure, which had both options.
    Q2: (Which has been asked before) Why no OS X tests? I doubt anyone bothers to optimize Windows Thunderbolt drivers because almost no one will use it...
    Reply
  • SirPerro - Thursday, July 10, 2014 - link

    At this point it's pretty clear that Thunderbolt is a niche for Mac users who are willing to pay top dollar for a non-upgradeable cilinder for which they have to buy top dollar external storage because Apple decided so.

    For 99,9999% of the above users, USB3 and Thunderbolt performance differences are undetectable in real world cases. No matter how awesome the product is, or is exposed to the world by Apple.

    So quick question. Why the fuck the remaining us 99,99999% of the world should care about Thunderbolt and why on earth should the vast majority of OEMs build TB devices?

    I say that as a MBP owner which finds it POINTLESS to pay any premium for thunderbolt.

    In before the "convenience" comments. I find it pretty inconveninent to pay the $20 tax for Apple cables, and I couldn't care less about having one cable for the monitor and another for the external HDD.

    The "convenience" of using the same USB interface as 99,9999% of the world though... that DO is important.
    Reply
  • repoman27 - Thursday, July 10, 2014 - link

    Thunderbolt is for connecting external PCIe devices or displays. If you're not the type of user that opted to go with PCIe solutions for storage in the past, then you're not likely to be interested in Thunderbolt storage solutions.

    While the majority of PC usage may be for content consumption and recreation, there's a legitimate percentage of the population for whom PCs represent tools. Just as certain tools are more ideally suited for particular jobs, different PCs and peripherals are better tailored to particular workflows. Complaining about Thunderbolt serving the needs of a minority is like complaining about the existence of any other tool that you personally don't have a current use for. I don't need all the specialized tools that my car mechanic, carpenter, electrician and plumber have, but I do prefer them to have the best tool for the job if and when I require their services. You may not need a Thunderbolt peripheral, but you probably do enjoy the work done by professionals in the content creation, engineering and research fields, whose workflows can benefit tremendously from technologies like Thunderbolt.

    Or you can just go ahead and hate on the fastest external I/O interface to ever grace a consumer PC. All PCs already come with USB ports, and that isn't going to change. The whole point of USB is to be fast enough for most uses and as cheap as possible. Thunderbolt allows you to do PCIe 2.0 x4 over a miniDP port. If you don't need to add a PCIe device externally, just ignore the little lightning bolt icon next to the port and use it to connect a display instead.
    Reply
  • mschira - Friday, July 11, 2014 - link

    Oh give the Mac Pro a break. It may not be a machine for you - it's not my machine either, but I can see people wanting it. And contrary to popular believe it is simply not overpriced. Have you looked at the prices HP or DELL ask for Xeon based workstations? They are BEYOND ridiculous and make the Mac Pro look positively cheap. I am not making this up, go check for yourself.

    And frankly - who needs internal storage these days anyway?
    I don't. I did not long ago, but no more. Our lab has a 5 bay Qnap with RAID 6 for storage. Plenty of space, pretty small and plenty fast for our purpose. We have WD myBook (3 TB and 4TB models) connected via USB3 for archive. Awesome, we put them in the shelf for cold storage. Yes you can do this with plain harddrives, and we did do that with hot adapter boxes - but the boxes are cumbersome and they break. And what when in 5 years getting an SATA interface will be hard? How long will it take until we will have trouble connecting with USB3?
    M.
    Reply
  • HangFire - Thursday, July 10, 2014 - link

    Interesting article, but I'm dissappointed the reviewer didn't test the implementations for vulnerabilities. I have used previous LaCie NAS products and they were immediately flagged by IT's security scanning as having outdated and insecure SSL and SAMBA implementations. I raised these issues with LaCie only to hear crickets. Reply

Log in

Don't have an account? Sign up now