For the first time since the original Storage Pod, Backblaze is announcing a completely redesigned approach with the introduction of the first “direct wire” Storage Pod. This new Storage Pod performs four times faster, is simpler to assemble, and delivers our lowest cost per gigabyte of data storage yet. And, once again, it’s open source.
The Original Storage Pod
In order to provide unlimited online backup for $5 per month when Backblaze first started in 2007, we needed to figure out the least expensive way to “attach a hard drive to the Internet.” This meant cost-efficiently attaching as many drives as possible to a single motherboard. We tried USB hubs, daisy-chaining Firewire, and various other approaches. In the end, we found one that worked: port multiplier backplanes.
We used these port multipliers to design a system with 9 five-drive NAS backplanes that connected via 3 SATA cards to the motherboard. The results were incredibly dense storage, currently 180TB in a 4U rack, from off the shelf consumer commodity parts. This design has served us through Storage Pod versions 1.0, 2.0, and 3.0, and today stores 100 petabytes of customer data in the Backblaze cloud.
However, the port multiplier backplanes had three key issues:
- They were one of the least reliable hardware components,
- When they had an issue, they affected 5 drives at once, and
- They were not completely a commodity part, thus making them somewhat difficult to buy (especially for someone building a single Storage Pod on their own.)
The New Storage Pod Design
Fast forward nearly 5 years as we introduce Storage Pod 4.0 and a completely new way of attaching 45 drives to the motherboard. The 5-drive backplanes are gone, replaced by individual direct wire SATA/Power connectors. In concert with this we replaced the three 4-port SATA cards with two high performance 40-port HighPoint Rocket 750 cards delivering SATA 3 throughput. Simpler to build and maintain, and seriously fast.
The New Storage Cost – $0.051 per GB
When we designed Storage Pods 2.0 and 3.0 our goal was to reduce the total cost per GB to store data. Pod 2.0 was great, but when the Thailand Drive Crisis pushed up hard drive prices we struggled to lower the cost of Storage Pod 3.0. With Storage Pod 4.0 we are pleased to announce it delivers 180TB of storage for just $0.051 per GB — our lowest cost per GB to date.
Here’s a video highlighting some of the new features in Storage Pod 4.0.
We summarized all of the changes below and as always we’ve included a parts list at the end of the post.
Screaming Fast SATA Cards — Storage Pod 4.0 uses two HighPoint Rocket 750 cards replacing the previous 3 SATA cards. Each Rocket 750 supports 40 directly connected SATA device channels via ten 6 Gbps Mini-SAS ports. The Rocket 750 card connects to the motherboard via an 8x PCIe 2.0 connection that supports 8 parallel “lanes” to the motherboard. Currently we are seeing a 4 to 5 times performance improvement in throughput when using this set up versus Storage Pod 3.0.
Backblaze has never had a problem with data throughput speed based on how we use a Storage Pod in production, but the increased throughput does dramatically reduce the amount of time it takes to sync the RAID arrays and burn in the system. Typically, it takes 5-6 days to do these activities when we use 4TB disk drives. With Storage Pod 4.0, this drops down to 1-2 days, so we can get 4.0 pods into service much quicker.
Direct Wire Hard Drive Connections — In previous Storage Pods, there were 9 backplanes, each supporting 5 hard drives. There were 9 SATA cables from the SATA cards to the backplanes and the backplane multiplexed to 5 different drives. With Storage Pod 4.0 each of the 45 drives has its own direct wire connector that provides power and a SATA connection for each drive. Each SATA connection goes directly back to the SATA card, the Rocket 750. Fifteen of the connectors are affixed to one of three cross supports in the chassis.
The direct wire connectors have multiple benefits:
- They are fundamentally simpler in design than the 5-drive backplanes.
- By replacing the 5-drive backplanes, all of the set up and management of the 45 drives is done on the motherboard or in the SATA cards.
- The direct wire connectors are less likely to break or burn out than the previous backplanes.
- When a 5-drive backplane went down, we had 5 drives go offline. Now if a connector goes down, only 1 drive will go offline.
New SATA cables — Each of the Rocket 750 Cards has 10 ports with Mini-SAS connectors. The new SATA cables start with a Mini-SAS connector and fan out to 4 individual cables each of which terminates in the direct wire connector. Here is a link showing the mapping between the Mini-SAS ports on each Rocket 750 card to the individual direct wire connectors. The new cables are designed to provide both SATA and power to each hard drive. Power is provided via Molex connectors attached to a wiring harness that is connected to, and part of, the Power Supply wiring harness.
One Power Supply — Previous Storage Pod versions had two power supplies, which totaled nearly 25% of the cost of the Storage Pod chassis (not including the drives). The two power supplies were not for failover purposes, we just needed the power they created. For Storage Pod 4.0 there is only one power supply to power everything. It generates enough power to run all 45 drives, the motherboard and all other components. Having one power supply saves on the cost and complexity, reduces the weight, and increases reliability of Storage Pod 4.0.
Thinner Chassis and New Side Rails — One of the byproducts of replacing the 5-drive backplanes is the reduction of the width of the chassis by 1/2 inch from 17 5/8 to 17 1/8. This allows us to use side rails for mounting Storage Pods in our cabinets. The previous L-bracket rails were known to be sharp in all the wrong places and more than once drew blood from our Operations staff. For them, this is the most important feature of Storage Pod 4.0.
Faster CPU – We’ve upgrade the CPU to an i3-3240 (Ivy Bridge) from an i3-2100 (Sandy Bridge) CPU. This makes processing faster and when combined with the Rocket 750 cards provided the boost in throughput of 4 to 5 times we’ve experienced. The i3-2120 and the i3-3220 have also been tested in this environment and seem to work fine, but given they are currently either the same price or a little more expensive as the i3-3240 we are currently using that CPU.
Boot Drive – The boot drive was moved from the side to the rear of the chassis. This was done to allow for mounting holes for the new side rails on each side of the chassis. This also makes the cabling a bit easier. The new location can support either one or two 2.5” boot drives.
Storage Pod 4.0 uses forty-five 4 TB hard drives for a storage capacity of 180 TB, the same as Storage Pod 3.0. Why didn’t we use 5 TB or even the 6 TB drives in Storage Pod 4.0? In short, these higher capacity drives are not yet widely available and are not cost effective for Backblaze at the moment.
For example, as of March 17, 2014 an HGST 6 TB internal hard drive listed on Amazon was $745.67 or about $0.124 per GB with an availability of 5 drives.
5 TB drives are also hard to find, with only LaCie’s external 5TB drive showing up on Newegg on March 17, 2014. The list price for this external drive is $449.99, making the cost about $0.09 per GB, although that’s the cost of the entire unit, not just the hard drive.
Currently 4 TB internal hard drives that we use are readily available for $160.00 or less in many places, that’s about $0.04 per GB.
While we’re on the topic, here are the 4 TB hard drives we currently use in our Storage Pods. We have tested these extensively and are satisfied with their performance.
- HGST (formerly Hitachi) — HDS5C4040ALE630
- Seagate – ST4000DM000
Although we like the HGST drives, as of March 1st, the availability of this HGST model is becoming limited as its looks like they are being EOL’d. We currently have the Seagate 4 TB drives installed in about 50 storage pods and they are performing well. We expect to try out the Western Digital WD40EFX (RED) drives in the near future.
Over the next few months we expect the price of the 5 TB and 6TB drives to decrease. At some point we’ll buy 50 drives (45 plus spares) and install them in a Storage Pod to see how they perform in our environment. As we’ve seen previously, some drives do better than others in our Storage Pods. If we were able to use the 6 TB drives for example we’d have 270 TB of storage in a 4U rack. As always we’ll keep you posted.
Buying a Storage Pod
If you are interested in buying a Storage Pod we recommend you talk to the folks at 45 Drives, a division of Protocase. They are very capable of building you a Backblaze inspired Storage Pod that can be customized if desired. They also can sell you a Storinatorâ„¢ their 180TB storage solution. The Storinator is similar to Storage Pod 4.0 in many ways, and is readily available for $5,403 for the non-redundant version and $6,888 for the redundant version. This does not include the price of the drives, tax, or shipping. While the 45 Drives system does not include hard drives, they test each system they build with drives before they ship the unit to you. Then it’s up to you to buy and install the hard drives. Remember that while the 45 Drives products are inspired by Backblaze, we do not build, test, or warrant them. Since we don’t sell the Storage Pods and don’t make money off any ancillary services, we don’t provide support for people who buy them from 45 Drives or other manufacturers.
Assembling Your Own Storage Pod
For those of you who want to assemble your own Storage Pod there is a parts list at the end of this post. You can order the 4U metal box from 45 Drives and start from there. Their 4U metal box includes the 3 metal “cross supports” to mount the direct wire connectors (see the parts list) that you can also purchase from 45 Drives. Once you have the metal box and the direct wire connectors, the other parts can be sourced as noted in the parts list or you can add in your own motherboard, memory, SATA cards, etc. to meet your needs. If you go this route, here is a wiki page, where you can learn more about the assembly process. Backblaze does not provide support or warranty to anyone who wishes to use the design to build or assemble a Storage Pod.
The Price of Storage Pod 4.0
In truth, there are actually three different prices for a 180 TB Backblaze Storage Pod 4.0:
- The price Backblaze contractually pays to build about 25 Storage Pods a month.
- The price you would pay to purchase a Backblaze inspired Storage Pod or a Storinator from 45 Drives.
- The price you would pay to purchase the parts and build your own Storage Pod.
In our previous Storage Pod blog posts we published the Backblaze prices, but people soon learned that they had to pay more for many of the parts, sometimes a lot more. We buy parts in bulk and as such get better pricing on many of the parts. So the real cost of building your own Backblaze Storage Pod was higher than expected. So to be clear, here are the approximate costs for a 4.0 Storage Pod with 180 TB of storage i.e. forty-five 4TB hard drives;
- Backblaze – $9,305
- Make Your Own – $10,587
- 45 Drives – $12,603
The Backblaze price of Storage Pod 4.0 of $9,305 for the complete 180TB system is about 13% less than the cost of Storage Pod 3.0 at $10,700 for an 180TB system. Of course these prices are not guaranteed as things like hard drive shortages can change prices pretty quickly.
The Cost per Gigabyte — Onward and Downward
A Storage Pod costs Backblaze $9,305 for 180TB of storage which translates into $0.0517 per GB. This is our lowest price per GB ever as you can see below:
Over the next few months we’ll continue to look for ways to decrease the various costs and with the expected increase in storage capacities such as 5 and 6 TB drives we’re sure the cost will drop below $0.05 per GB.
Appendix A — Parts List
This is the parts list for Storage Pod 4.0, not including the 45 hard drives. The prices listed are from the sources noted and are as of the date of the publication of this blog post. Taxes and shipping are not included. You may find better prices with a little more research. Good luck.
Includes case, anti-vibration assemblies, cross supports, etc.
Zippy PSL-6850P Power Supply
Supermicro MBD-X9SCL-F (MicroATX)
Kingston 4GB – KVR16E11S8 DDR3 1600 (PC3 12800)
Intel Core i3 processor i3-3240
Dynatron 120x120x25mm 2600RPM Case Fan
Rocket 750 40 port PCI2.0 SATA III HBA CARD
Direct wire cable SATA/Power 4 connectors – 24″ length
Direct wire cable SATA/Power 4 connectors – 30″ length
Direct wire cable SATA/Power 1 connector – 24″ length
FrozenCPU ele-302 Bulgin Vandal Momentary LED Power Switch 12â€³ 2-pin
WD Blue WD5000LPVX 500GB 5400 RPM Internal
Supermicro MCP-290-00053-0N Side Rails – 1 pair
- Screws and other small parts are generally purchased in packages (e.g. 10 or 100), but the price list assumes you will only pay for the quantity you actually need.