Twitter turned 5 years old yesterday. The service now has 200 million users (including @backblaze) who post over 1 billion tweets per day.

One interesting tidbit they recently mentioned was that all of those billions and billions of tweets compiled over the course of half a decade took a mere 20 terabytes of storage space!

In other words, you could archive all of the last 5 years of Twitter in just 1/3rd of an original Backblaze Storage Pod.

Happy Birthday Twitter! We wish you an amazing next five years!

As we approached the end of 2010, our vp of engineering sent an email:

Backblaze crossed the 10 petabyte mark in data storage for our customers.

While l see us constantly adding storage for our online backup service, it was one of those moments that made me think, “Wow, that’s a lot of storage.” Ten petabytes is roughly double the entire archive of the Internet.

We tried to visualize what 10 petabytes of storage looks like. Here are three takes on it:

#1: 10 Petabytes in Backblaze Storage Pods

#2: 10 Petabytes in Pixels by Drive Size Used

#3: 10 Petabytes of Drives Stacked Vertically (my favorite)

How would you visualize ten petabytes of storage?

Wow. Seagate just announced the world’s first 3 TB drive. Called the Seagate FreeAgent GoFlex Desk, it’s big enough to hold about 1,000 full-length HD movies or 1 million digital photos or songs.

It is also twice the size of the Seagate drives we used in the Backblaze Storage Pod when we published the design in September. Switching to the 3 TB hard drives in our storage pods would mean one Backblaze Storage Pod would hold 135 TB of data: 45,000 full-length HD movies or 45 million digital photos or songs. That is one monster collection of media.

The Seagate 3 TB hard drives are only available for external use only at this point, so we can’t actually put them into our storage pods – but we’re keeping an eye out. If you want one one of these drives for yourself, they’re $250 at Seagate. (Note that you need to be using a 64-bit OS to take advantage of all the space.)

For my estimates, I’m assuming an HD movie is 3 GB (as Hot Tub Time Machine on iTunes is when I just looked at it a second ago) and a photo or song is 3 MB (as they are with a 5 megapixel camera and downloaded from iTunes respectively.)

In 2008 we started deploying our storage pods and in September 2009 we gave away the design on our blog in a post titled, “Petabytes on a budget – How to build cheap cloud storage.” This weekend we deployed our 100th Backblaze Storage Pod.

For those of you counting, that’s:
* 100 pods
* 4,500 spinning hard drives
* 6,668 terabytes (6.5 petabytes) of storage

Below is a picture (taken on a first generation iPhone this morning) of one set of racks in the data center. You can see the 100th pod at the very end of this row.

Onward to the next 100 storage pods

You may not be familiar with the name Crispin Porter + Bogusky, but you’re probably familiar with their work. The firm, which was named U.S. Agency of the Year by Adweek last year, created “The King” and “Whopper Freakout” campaigns for Burger King; the Windows Mojave, Jerry Seinfeld/Bill Gates and I’m a PC campaigns for Microsoft; as well as ads for Guitar Hero, Old Navy, Best Buy, Coke Zero, and others.

For the previous four years they have also been the official U.S. agency for Volkswagen and have created a lot of media during that time. So, when it came time to archive all of that media somewhere…they decided to build their own Backblaze Storage Pod.

Ryan Banham, Windows Evangelist at Crispin Porter + Bogusky took on the task:

Just as everyone is settling down for a big turkey dinner our first
Backblaze storage pod will be preparing to feast on terabytes of data.

He customized the Backblaze storage pod reference design with a different motherboard, more memory, Samsung instead of Seagate drives, a single power supply, and used Windows Server 2008 as the operating system. It’s great to see people making the design suit their particular purpose. Once he hones in on the final design for their purpose, he plans to deploy several racks of mirrored archive servers to support their storage needs.

Some of the feedback Ryan provided to us on his customized version included:
* The pod is uber-cool: Even under full load the drives stay under 72 F, so he also swapped our fans for quieter and lower power intake fans.
* No trampolines for the pod: Moving the pod around requires the RAID cards to be reseated (possibly because the bottoms of the RAID cards stick out of the case.)
* $20 gets you far. A pod running 50% – 75% of the month costs just $20 in electricity.

Ryan says:

Thanks for sharing the build and giving me something fun and
interesting to do over the last few weeks! I learned a lot.

Glad it was interesting and useful and thank you for sharing your learnings!

Photos Ryan sent us of his pod:

CrunchGear published that the National Security Agency (NSA) is forecasting it may need yottabytes of storage to keep all of its surveillance data by 2015.

What is a yottabyte?
1000 GB = 1 Terabyte (TB)
1000 TB = 1 Petabyte (PB)
1000 PB = 1 Exabyte (EB)
1000 EB = 1 Zettabyte (ZB)
1000 ZB = 1 Yottabyte (YB)
In other words, a Yottabyte = 1,000,000,000,000,000 GB.

How much will this cost by 2015
On the one hand, what makes this even tougher is that typical storage systems cost 10x the price of the raw hard drives. Thus, the likely actual cost of storage for the NSA:
* $1,000 trillion for a complete storage system

The NSA may need to partner with NASA to see if it can spin off about 15 more planet Earth’s so their combined GDP could pay for its storage requirements.

On the other hand, this is based on prices and storage technology in 2009. But the cost per GB has dropped consistently 4% per month for the last 30 years. Assume the trend continues for the next 5 years, by when the NSA needs their yottabyte of storage. The costs in 2015 then would be:
* $8 trillion for the raw drives
*$80 trillion for a storage system

Well, that’s getting closer – a bit less than today’s global GDP.

How much space will this take by 2015?
Per historical metrics, a drive should hold 10 TB by 2015. The NSA would require:
* 100 billion hard drives
* 2 billion Backblaze storage pods

And of course, they would probably want this data backed up.
That might really test our offer of $5 for unlimited storage.

To be fair, the original analysis states that the need for yottabytes of information may not be accurate because it assumes that data is collected in a way similar to today. Instead, it purports they may only need hundreds of petabytes of data storage by 2015; significant, but completely manageable. Hard drives are also the assumed technology – which has been a good assumption for 30 years and may continue to be a good one for the next 5 years. SSD and other technologies may provide some interesting options, but in the near term, the price and density winner will likely continue to be spinning platters.

Regardless of how much data the NSA ends up needing to store, could we perhaps recommend a storage design?

Don Honabach has the honor of being the first person to successfully build his own Backblaze storage pod. (At least the first we know about.)

With four servers running at home for media storage, Don, was using a fair bit of power (and probably generating a lot of heat and noise and taking up space.) For five years he was working to come up with an “Extreme Media Server” and after reading about the Backblaze storage pod, he decided this may be the way to go.

Having expertise in the space, Don customized a variety of items in the pod including:
* The operating system (switching to Microsoft Windows Server 2008 R2)
* Power supplies
* Motherboard
* and more…

In just a couple weeks Don had completed his “Extreme Media Server”. Combining all four servers into one, Don is saving 500 watts of power, and can run 16 independent movie streams across two monitors from a single storage pod.

Don created a blog that describes his experiences building his Extreme Media Server.

Congratulations Don and good luck watching all those movies at the same time!

Last month’s blog post about building our Backblaze storage pods generated a ton of interest and many people are building their own pods! Our post also generated a ton of questions so below we answer the common ones and provides more detail about where to get components.
Continue reading…