CrunchGear published that the National Security Agency (NSA) is forecasting it may need yottabytes of storage to keep all of its surveillance data by 2015.

What is a yottabyte?
1000 GB = 1 Terabyte (TB)
1000 TB = 1 Petabyte (PB)
1000 PB = 1 Exabyte (EB)
1000 EB = 1 Zettabyte (ZB)
1000 ZB = 1 Yottabyte (YB)
In other words, a Yottabyte = 1,000,000,000,000,000 GB.

How much will this cost by 2015
On the one hand, what makes this even tougher is that typical storage systems cost 10x the price of the raw hard drives. Thus, the likely actual cost of storage for the NSA:
* $1,000 trillion for a complete storage system

The NSA may need to partner with NASA to see if it can spin off about 15 more planet Earth’s so their combined GDP could pay for its storage requirements.

On the other hand, this is based on prices and storage technology in 2009. But the cost per GB has dropped consistently 4% per month for the last 30 years. Assume the trend continues for the next 5 years, by when the NSA needs their yottabyte of storage. The costs in 2015 then would be:
* $8 trillion for the raw drives
*$80 trillion for a storage system

Well, that’s getting closer - a bit less than today’s global GDP.

How much space will this take by 2015?
Per historical metrics, a drive should hold 10 TB by 2015. The NSA would require:
* 100 billion hard drives
* 2 billion Backblaze storage pods

And of course, they would probably want this data backed up.
That might really test our offer of $5 for unlimited storage.

To be fair, the original analysis states that the need for yottabytes of information may not be accurate because it assumes that data is collected in a way similar to today. Instead, it purports they may only need hundreds of petabytes of data storage by 2015; significant, but completely manageable. Hard drives are also the assumed technology - which has been a good assumption for 30 years and may continue to be a good one for the next 5 years. SSD and other technologies may provide some interesting options, but in the near term, the price and density winner will likely continue to be spinning platters.

Regardless of how much data the NSA ends up needing to store, could we perhaps recommend a storage design?

Have you upgraded to Snow Leopard? Users on Apple’s forums are reporting a bug that deletes all user data if someone mistakenly logs into the “Guest” account. This issue was originally reported about a month ago on Cnet’s MacFixIt but today is being mentioned across the web as more people are being bitten by the bug.

While not completely reproducible, it is happening frequently enough to take a basic action: turn off the guest account.

eHow explains how to turn off the guest account - it’s quick and takes just a minute.

Of course, you should also make sure your data is backed up, but I assume you already use Backblaze to do this.

Dan Tynan’s article “The 7 dirtiest jobs in IT” hit a nerve with many an IT worker, and Dan followed it up with, “Even dirtier IT jobs.”

Number 6 on the list of the dirtiest jobs? Data crisis counselor.

This is a person on the other end of the phone from “sobbing adults who’ve lost images or videos of their recently deceased parents” and dentists, and IT managers, and tons of others who lost data and were frantic, upset, and panicked. Kelly Chessen, who has this job for DriveSavers, a firm that tries to recover data from dead hard drives, says her five years on a suicide prevention line is what prepared her best for this position.

Have you been an unwitting data crisis counselor for a friend or family member?
Make sure they backup so you don’t have to be…

Cloud service are forecast to be a bright spot amidst dire economic times. Reaching $56.3 billion by end of 2009, cloud services are expected to generate 2x the revenue of Google (which is partially included) and a growth rate that exceeds the company’s 18% year-over-year rate. By 2013, Gartner expects cloud services to reach $150 billion in sales.

Revenues from cloud application services (such as Backblaze) “were almost twice as large as the market for systems infrastructure and will continue to show strong growth,” according to the Gartner report.

While there is a lot of discussion about terminology and taxonomy (”cloud services” vs “SaaS” etc.)…there is no doubt the market for infrastructure and applications provided over the Internet has arrived.

Online backup service provider Carbonite was all over the news this week for suing two of its suppliers, claiming the vendors setup the systems that were responsible for losing 7,500 customers’ data. First published in The Boston Globe, the story was picked up by TechCrunch, Computerworld, Cnet, and at least 20 other media outlets.

A large number of users losing their data certainly makes for good headlines and stories about the risks of cloud computing, but I think there is a different story here.

Carbonite’s CEO complained the media response was overblown. He claimed, correctly, that the majority of these reports were misleading. While 7,500 customers’ data was lost, only a small percentage of users (54 in total) actually lost data because this data was a backup and most people still had the the files on their computers.

Numerous articles also used this as an opportunity to raise doubts about using “the cloud” in general. Again, this seems to be a red herring. As a whole, cloud computing is generally more available and reliable than systems users put in place themselves. What’s more, backup is the ideal cloud computing application because it adds to the reliability by having data offsite that you already have locally.

What was missed in the news is that a company who’s core business is to provide online backup outsourced their online backup. Carbonite hired a system integrator, and now by suing this vendor, they are effectively denying responsibility for the backup systems.

This would be the equivalent of Google outsourcing search technology.

Building robust online backup technology is difficult. There are certainly lots of complexities involved to ensure data is backed up, redundant, and secure. It is the role of the online backup service provider to have the technical expertise and laser focus to work through these items. Pushing it off on an outside company just seems a bit …risky.

Update: After writing this post, David Friend, CEO of Carbonite, emailed me to state that while they purchased the Promise Technology boxes from system integrator, Interactive Digital Systems, they now write their own software.

It’s 6:30pm on Friday the 13th, so if you’re reading this, it probably wasn’t a completely unlucky day.

Did you avoid walking under a ladder today?
Stood a bit further back from large windows?
Drove a little slower?

According to the Stress Management Center and Phobia Institute, about 1 in 15 people in the U.S. are affected by a fear of this day. Many people acquire good luck charms - horseshoes, lockets, or feathers. However, as R. E. Shay said, “Depend on the rabbit’s foot if you will, but remember it didn’t work for the rabbit.”

At least if you have been backing up with Backblaze, you don’t have to worry if your computer was lucky today.

Ma.gnolia.com was a bookmark storing and sharing service (similar to Delicious.com) that has shut down due to data loss. The company started several years ago and built a small, but adoring user community that liked Ma.gnolia’s easy-to-use API and caching of linked pages. The service was the work of Larry Halff who nearly single-handedly built the service.

Alas, as for many companies without a good backup, data loss caused a mortal wound: Ma.gnolia completely shut down on February 17th, 2009.

Larry explained that when he started the company several years ago, no good cloud-based backup services existed, thus requiring him to develop his own backup. The backup was doing a file sync over Firewire to another computer. Unfortunately there was no integrity checking, no versioning, and the system was never tested to see if the backups worked. When it came time to restore, it turns out they did not.

People often assume simply setting up a copy or sync process is sufficient for backup, only to discover the issues with this when a restore is needed. At Backblaze, every file is compressed, encrypted, de-duplicated, and integrity-checked to ensure the backed up file exactly matches the original.

Larry intends to develop a new service and says when starting the new company, “My first priority is better backups.”

We wish Larry the best with his new service and hope others benefit from Larry’s pain.

HP is shutting down its Upline online backup service
No official announcement has been made, but HP has sent its users a notice (shown below) saying it will shut down the service on March 31st. TechCrunch has reported about previous trouble the service had and the current shutdown.

Backblaze is offering HP Upline users a special 20% one-time discount
With this discount, the Backblaze service is over 30% less than HP Upline. HP Upline users can switch to Backblaze by purchasing with coupon code “upline2backblaze” by March 31st, 2009. Sign-up now >

Start your backup now to ensure there is no interruption in your data backup.

Email sent to HP Upline customers:

Thank you for your interest in HP Upline.

HP continually evaluates product lines and has decided to discontinue the HP Upline service on March 31, 2009.

HP will no longer be backing up your files to the HP Upline servers as of Feb 26, 2009 at 8 am Pacific time. HP will keep the file restore feature of the Upline service operational through March 31, 2009 Pacific time in order for you to download any files you have backed up to Upline.

If you have a paid subscription to HP Upline, you will be refunded the full amount of the fees you paid for the service. That refund will be credited to the credit card account or PayPal account that you used to subscribe to the Upline service. If you do not receive the refund prior to March 31, 2009, please contact our customer service team at https://www.upline.com/support/email.aspx.

HP looks forward to offering you additional technology products and services in the future.

Thank you.

HP Upline team

JournalSpace, a 6 year old blog hosting service, closed shop on Tuesday after losing all of its users’ data. Details are sketchy, but the company claims the cause was either an OS failure or a disgruntled employee that deleted the data.

Regardless of the cause, since the JournalSpace had no backups, in addition to the death of the company, all of the bloggers who hosted their sites with the company lost their data. Many are trying to reassemble their years of blogging from a combination of Google cache results and other pieces.

Read more about this story at Slashdot and TechCrunch or read the company’s post “This is the way the world ends. Not with a bang but a whimper” where they are also listing the domain for sale.

A few key takeaways:
* Mirroring - JournalSpace had been mirroring their data, meaning two drives would have the exact same data. While often mistaken for backup because this protects from a single hard drive failure, this is open to all other causes of data loss such a virus, fire, user error, etc.
* Data Recovery - most people realize they should do backups, but they put it off, and in the back of their head think “worst case, I’ll take it to one of those drive recovery places.” Alas, as JournalSpace discovered, even the professionals at DriveSavers can only recover data in certain lucky cases.
* Cost - if you think doing backups is too expensive, try not doing backups. JournalSpace says they spent as much on their attempt to recover the data as they had made in the entire year prior, did not succeed, and paid the ultimate corporate price.

Six years of effort building a company and volumes of users’ data lost is really unfortunate; if you have not been doing backups, make this your wake up call.

1 Your backup strategy is to burn CDs and DVDs.
A diamond may be forever, but CDs and DVDs have a shelf-life. Even the Optical Storage Technology Association says an unrecorded disk will only last 5 to 10 years. And this assumes you’ve selected the right files and remember to do it, which brings me to reasons #2 and #3.

2Your backup strategy requires picking which files to backup.
If you’re like most people, you actually have no idea where your files are. Wait, you’re not like most people – you absolutely select which folders you put your files in. But some applications save files in random places. Have any idea where your iTunes playlists are? Hint: They’re not in My Documents. Windows hides folders. Vista forces older applications into a hidden sandbox. You probably know where most of your files are…but some files your care about are almost certainly somewhere else.

Continue reading…

Robyn Peterson isn’t a computer newbie - after all, he’s in charge of technology for tech publisher Ziff Davis. So how did he lose his backup? Using a popular techie solution for online backup: FTP-ing his files to a web hosting provider. A perfectly reasonable, if somewhat tedious, solution. But as he found out, not such a secure option.

“We have been cracking down on people using our services for backing up files,” Robyn was told by the provider. The Terms of Service said their “servers are not intended as a data backup or archiving service.” Don’t check the fine print? Or don’t think it’ll be enforced? Ok sometimes…but when you really need that backup, you don’t want the response Robyn got when he asked whether his deleted files were backed up: “backups go back a maximum of only two weeks, and no backups are guaranteed.”

“I’ve owned six portable USB hard drives over the past 10 years, and all six of them have failed unrecoverably. Is it just me, or is there a wider problem out there?” Mike Elgan asks. As prices have plumetted over the last few years, the USB hard drive market has exploded. More and more users are counting on these drives to add extra space or backup their current drive. But these drives don’t protect from many of the items that would kill the main computer - fire, flood, theft, etc. And judging by the 170 or so responses to Mike, he’s not the only one that has lost a hard drive or two.

According to Forrester Research there will be 1 Billion PCs in use by the end of 2008. 50 billion photos snapped by consumers in 2007. You’d think if we care enough about our travels, kids, and memories to take those photos, we’d care enough to make sure we didn’t lose them. But it seems we don’t. Almost half of us have lost data and still less than one in four of us are doing regular backups.

At Backblaze, we hope to change that and make the world just a bit better. We’re creating an online backup solution that will be so easy our office dog Gromit could use it, so secure your bank will look like a social network, and so high-performance that you’ll swear your computer runs faster when you use it. Request a beta code - it might just save your memories.