Wow. Seagate just announced the world’s first 3 TB drive. Called the Seagate FreeAgent GoFlex Desk, it’s big enough to hold about 1,000 full-length HD movies or 1 million digital photos or songs.

It is also twice the size of the Seagate drives we used in the Backblaze Storage Pod when we published the design in September. Switching to the 3 TB hard drives in our storage pods would mean one Backblaze Storage Pod would hold 135 TB of data: 45,000 full-length HD movies or 45 million digital photos or songs. That is one monster collection of media.

The Seagate 3 TB hard drives are only available for external use only at this point, so we can’t actually put them into our storage pods - but we’re keeping an eye out. If you want one one of these drives for yourself, they’re $250 at Seagate. (Note that you need to be using a 64-bit OS to take advantage of all the space.)

For my estimates, I’m assuming an HD movie is 3 GB (as Hot Tub Time Machine on iTunes is when I just looked at it a second ago) and a photo or song is 3 MB (as they are with a 5 megapixel camera and downloaded from iTunes respectively.)

CrunchGear published that the National Security Agency (NSA) is forecasting it may need yottabytes of storage to keep all of its surveillance data by 2015.

What is a yottabyte?
1000 GB = 1 Terabyte (TB)
1000 TB = 1 Petabyte (PB)
1000 PB = 1 Exabyte (EB)
1000 EB = 1 Zettabyte (ZB)
1000 ZB = 1 Yottabyte (YB)
In other words, a Yottabyte = 1,000,000,000,000,000 GB.

How much will this cost by 2015
On the one hand, what makes this even tougher is that typical storage systems cost 10x the price of the raw hard drives. Thus, the likely actual cost of storage for the NSA:
* $1,000 trillion for a complete storage system

The NSA may need to partner with NASA to see if it can spin off about 15 more planet Earth’s so their combined GDP could pay for its storage requirements.

On the other hand, this is based on prices and storage technology in 2009. But the cost per GB has dropped consistently 4% per month for the last 30 years. Assume the trend continues for the next 5 years, by when the NSA needs their yottabyte of storage. The costs in 2015 then would be:
* $8 trillion for the raw drives
*$80 trillion for a storage system

Well, that’s getting closer - a bit less than today’s global GDP.

How much space will this take by 2015?
Per historical metrics, a drive should hold 10 TB by 2015. The NSA would require:
* 100 billion hard drives
* 2 billion Backblaze storage pods

And of course, they would probably want this data backed up.
That might really test our offer of $5 for unlimited storage.

To be fair, the original analysis states that the need for yottabytes of information may not be accurate because it assumes that data is collected in a way similar to today. Instead, it purports they may only need hundreds of petabytes of data storage by 2015; significant, but completely manageable. Hard drives are also the assumed technology - which has been a good assumption for 30 years and may continue to be a good one for the next 5 years. SSD and other technologies may provide some interesting options, but in the near term, the price and density winner will likely continue to be spinning platters.

Regardless of how much data the NSA ends up needing to store, could we perhaps recommend a storage design?

Apple launched OSX 10.6 Snow Leopard on Friday and not only is it faster and smaller, but also adds a few new features and core services. But for many, the price of just $29 seemed to be the primary reason to upgrade.

So, were these really enough for people to rush out and buy the new OS?

Well, we know that Backblaze customers are early adopters, but considering nearly 1/4th of all Backblaze Mac customers have already upgraded to Snow Leopard - it seems Apple’s customers think the new Snow Leopard is a worthwhile upgrade.

Thinking about upgrading to Snow Leopard?
Read our short Snow Leopard upgrade page for Backblaze:
https://www.backblaze.com/snowleopard.html

One year ago I attended Cloud Camp, a gathering of early adopters of cloud computing. Primary questions there centered on: “How do we define cloud computing?

Last Thursday I attended GigaOm Structure 09 and the discussions were full of cloud computing success stories, explanations of deployments, insights about how to scale bigger, offerings of cloud services at various levels and more. Below I summarize my takeaways from the day’s events.

Is Cloud Computing Real or Just a New Term for Old Technologies?
The short answer: Yes.

Many of the concepts of cloud computing have existed for years: large datacenters, hosted servers, online service offerings (think Yahoo! Mail), and even multi-tenancy (in most mainframes.) In part, cloud computing is a packaging of these into a term renovated for today.

However, cloud computing is more than just marketing. It is both a business model and technological change for the providers and consumers of these computing services. While startups could have always avoid directly deploying hardware by paying a hosting provider, the ability to pay for usage rather than signing year-long contracts dramatically changes the playing field when launching a new service with unpredictable demand.

What are the Tiers of Cloud Computing?
Cloud computing is standardizing on three key tiers:
* Infrastructure-as-a-Service
Amazon Web Services‘ EC2 and S3 are primary examples in this category where raw computing power such as processing and storage can be rented. GoGrid, Nirvanix, Rackspace, Savvis, and 3Tera are other examples. Some of these offer the infrastructure as a web-service, where you simply purchase “1 GB of space.” Others offer a hardware-as-a-service model in which you actually specify how many servers you want and how they are configured. Control versus the desire for hands-off outsourcing will drive which of these two models is ideal for your organization.

* Platform-as-a-Service
Google AppEngine, Force.com, and Microsoft Azure are primary examples in this category where a hosted development environment is provided upon which applications can be built and the underlying hardware is abstracted out. These platforms are ideal for developers wishing to get products to market quickly, but lock-in to the platform can be a concern. Some vendors are attempting to offer the ability to use standardized frameworks that could be portable, but this is in its infancy. Facebook Platform is also an example of a platform-as-a-service, though one in which applications would be deployed only within the Facebook experience rather than on the public Web.

* Software-as-a-Service
Examples of this category are all around us: online banking, Yahoo! Mail, Google Apps, and, of course, Salesforce.com.

What level should you engage in? As a consumer, the answer is obvious: you are simply a user of numerous software-as-a-service offerings. As a developer, you may choose to build upon an infrastructure-as-a-service or platform-as-a-service, depending largely on your desire to tradeoff control versus speed of deployment.

Will there be a Segmentation Amongst Clouds?
Scale is a critical component of clouds since the ability to scale up and down quickly relies on having both a large deployment and multiple-tenants across which spikes are smoothed out. However, companies consuming cloud services may have differing requirements that would push vendors to offer different clouds types. A basic examples of this segmentation is the different size server instances offered through EC2. However, other possible needs include quality-of-service guarantees, HIPAA or SAS70 compliance, or specific geographic or hardware requirements.

We are already starting to see cloud differences appearing:
* Amazon focuses on a broad-based offering.
* Nirvanix focuses on clouds for enterprises.
* RackSpace focuses on customer support.

Will clouds segment? Yes, but likely into a handful of major sites that still allow for the scale required to extract the benefits of cloud computing.

Does Cloud Computing mean the end of Scalability Issues?
Startups bringing the next high-definition-video-social-network may become wildly successful and require quickly scaling to millions of users. Cloud computing purports to easily address this by enabling these companies to simply add servers and storage as quickly as they are needed. While this may seem to solve all scalability issues, developers that have dealt with scale in the past understand that the fundamental architecture of the application needs to be developed to handle scale issues and cannot purely rely on scaling hardware.

If a program requires significant computation and the code is written to allow parallel processing, adding CPUs in the cloud can enable the computation to happen much quicker. Alternatively, if the code were written single-threaded, no amount of extra hardware will help speed the process. Cloud computing may reduce the need for system administrators on staff, but it does not negate the requirement for top-tier engineers.

And Finally, is Cloud Computing Inevitable?
Cloud computing is certainly in a phase of dramatic growth, with vendors providing new offerings and customers experimenting and adopting. There is no doubt that this trend will continue due to the benefits of scale, cost, operations, and segmentation of skill-sets. However, it is not inevitable that everything will move to the cloud. Certain organizations will have needs that are too custom-tailored to create a segmented cloud and will retain their internal systems - potentially building a ‘private cloud’. Of course, companies should evaluate the pros and cons for their own environments and for most there are likely many opportunities to move applications into the cloud and derive significant benefits.

Cloud service are forecast to be a bright spot amidst dire economic times. Reaching $56.3 billion by end of 2009, cloud services are expected to generate 2x the revenue of Google (which is partially included) and a growth rate that exceeds the company’s 18% year-over-year rate. By 2013, Gartner expects cloud services to reach $150 billion in sales.

Revenues from cloud application services (such as Backblaze) “were almost twice as large as the market for systems infrastructure and will continue to show strong growth,” according to the Gartner report.

While there is a lot of discussion about terminology and taxonomy (”cloud services” vs “SaaS” etc.)…there is no doubt the market for infrastructure and applications provided over the Internet has arrived.

Kodak Gallery, formerly known as “Ofoto”, claims to be the leading online digital photo developing service. Customers upload photos to the site, use it as a way to store and share photos online, and count on it as a photo backup.

However, today Kodak updated its storage policy to say that it will delete your photos if you do not purchase products from the company each year. Specifically, every 12 months customers storing:
* Less than 2 GB of photos need to purchase at least $4.99
* More than 2 GB of photos need to purchase at least $19.99
With a $100 digital camera shooting at 8 megapixels, just 500 compressed photos or 40 uncompressed photos would take you over the 2 GB limit.

Kodak Gallery provides a good photo developing service and should be able to charge for it. However, if you are counting on this as a backup of your photos - think again. Forget to purchase your minimum requirement one year? Bought your minimum, but then added some more photos and went over 2 GB? Your photos may be deleted.

Kodak Gallery’s terms of service say,

“You should keep a copy of each image you upload in a secure place. We are not responsible for deleted images.”

Ensure your photos are safe, in a secure place, forever. Back them up with Backblaze.

Online backup service provider Carbonite was all over the news this week for suing two of its suppliers, claiming the vendors setup the systems that were responsible for losing 7,500 customers’ data. First published in The Boston Globe, the story was picked up by TechCrunch, Computerworld, Cnet, and at least 20 other media outlets.

A large number of users losing their data certainly makes for good headlines and stories about the risks of cloud computing, but I think there is a different story here.

Carbonite’s CEO complained the media response was overblown. He claimed, correctly, that the majority of these reports were misleading. While 7,500 customers’ data was lost, only a small percentage of users (54 in total) actually lost data because this data was a backup and most people still had the the files on their computers.

Numerous articles also used this as an opportunity to raise doubts about using “the cloud” in general. Again, this seems to be a red herring. As a whole, cloud computing is generally more available and reliable than systems users put in place themselves. What’s more, backup is the ideal cloud computing application because it adds to the reliability by having data offsite that you already have locally.

What was missed in the news is that a company who’s core business is to provide online backup outsourced their online backup. Carbonite hired a system integrator, and now by suing this vendor, they are effectively denying responsibility for the backup systems.

This would be the equivalent of Google outsourcing search technology.

Building robust online backup technology is difficult. There are certainly lots of complexities involved to ensure data is backed up, redundant, and secure. It is the role of the online backup service provider to have the technical expertise and laser focus to work through these items. Pushing it off on an outside company just seems a bit …risky.

Update: After writing this post, David Friend, CEO of Carbonite, emailed me to state that while they purchased the Promise Technology boxes from system integrator, Interactive Digital Systems, they now write their own software.

Ma.gnolia.com was a bookmark storing and sharing service (similar to Delicious.com) that has shut down due to data loss. The company started several years ago and built a small, but adoring user community that liked Ma.gnolia’s easy-to-use API and caching of linked pages. The service was the work of Larry Halff who nearly single-handedly built the service.

Alas, as for many companies without a good backup, data loss caused a mortal wound: Ma.gnolia completely shut down on February 17th, 2009.

Larry explained that when he started the company several years ago, no good cloud-based backup services existed, thus requiring him to develop his own backup. The backup was doing a file sync over Firewire to another computer. Unfortunately there was no integrity checking, no versioning, and the system was never tested to see if the backups worked. When it came time to restore, it turns out they did not.

People often assume simply setting up a copy or sync process is sufficient for backup, only to discover the issues with this when a restore is needed. At Backblaze, every file is compressed, encrypted, de-duplicated, and integrity-checked to ensure the backed up file exactly matches the original.

Larry intends to develop a new service and says when starting the new company, “My first priority is better backups.”

We wish Larry the best with his new service and hope others benefit from Larry’s pain.

HP is shutting down its Upline online backup service
No official announcement has been made, but HP has sent its users a notice (shown below) saying it will shut down the service on March 31st. TechCrunch has reported about previous trouble the service had and the current shutdown.

Backblaze is offering HP Upline users a special 20% one-time discount
With this discount, the Backblaze service is over 30% less than HP Upline. HP Upline users can switch to Backblaze by purchasing with coupon code “upline2backblaze” by March 31st, 2009. Sign-up now >

Start your backup now to ensure there is no interruption in your data backup.

Email sent to HP Upline customers:

Thank you for your interest in HP Upline.

HP continually evaluates product lines and has decided to discontinue the HP Upline service on March 31, 2009.

HP will no longer be backing up your files to the HP Upline servers as of Feb 26, 2009 at 8 am Pacific time. HP will keep the file restore feature of the Upline service operational through March 31, 2009 Pacific time in order for you to download any files you have backed up to Upline.

If you have a paid subscription to HP Upline, you will be refunded the full amount of the fees you paid for the service. That refund will be credited to the credit card account or PayPal account that you used to subscribe to the Upline service. If you do not receive the refund prior to March 31, 2009, please contact our customer service team at https://www.upline.com/support/email.aspx.

HP looks forward to offering you additional technology products and services in the future.

Thank you.

HP Upline team

Broadband connections in the United States are 10 times slower than those in South Korea. In fact, we are number 15 based on average connection speed, behind not only Japan, Finland, France and Sweden (which are all at least 3 times faster) but also countries including Iceland, Poland, and Portugal that are blessed with significantly faster connections.

Cloud computing, VoIP, eCommerce, Internet TV, telemedicine, telecommuting, distance learning and a myriad of other drivers of communication, business, and social benefit rely on fast, accessible, reliable broadband connectivity.

Published on Friday, The American Recovery and Reinvestment Act of 2009 provides grants to service providers to build out broadband infrastructure in unserved and underserved areas. Grant requirements include mandated speeds of:
* Advanced Broadband Services: 45 Mbps downstream, 15 Mbps upstream
* Basic Broadband Services: 5 Mbps downstream, 1 Mbps upstream

Under this definition of basic broadband service a user could download 1.5 TB and upload 300 GB of data in one month - enough to watch a movie a night, download hundreds of photos or songs a day, backup an average computer online, and still have room to work remotely and do some video-conferencing.

Part of this Bill requires “a plan for use of broadband infrastructure and services in advancing consumer welfare, civic participation, public safety and homeland security, community development, health care delivery, energy independence and efficiency, education, worker training, private sector investment, entrepreneurial activity, job creation and economic growth, and other national purposes.”

History shows that connectivity not only links us in ways that are obvious at the outset, but drives innovation that is beneficial and unforeseen.

So will we have fast nationwide broadband tomorrow? Of course, not. Within a year of the passage of this Bill, a national broadband plan is required to be created. Best case is another year or two before individuals and businesses see available connections. This plan will not put the U.S. on par with South Korea, but it is a good step forward, and within several years we may start seeing the benefits of the broader availability of faster broadband.

JournalSpace, a 6 year old blog hosting service, closed shop on Tuesday after losing all of its users’ data. Details are sketchy, but the company claims the cause was either an OS failure or a disgruntled employee that deleted the data.

Regardless of the cause, since the JournalSpace had no backups, in addition to the death of the company, all of the bloggers who hosted their sites with the company lost their data. Many are trying to reassemble their years of blogging from a combination of Google cache results and other pieces.

Read more about this story at Slashdot and TechCrunch or read the company’s post “This is the way the world ends. Not with a bang but a whimper” where they are also listing the domain for sale.

A few key takeaways:
* Mirroring - JournalSpace had been mirroring their data, meaning two drives would have the exact same data. While often mistaken for backup because this protects from a single hard drive failure, this is open to all other causes of data loss such a virus, fire, user error, etc.
* Data Recovery - most people realize they should do backups, but they put it off, and in the back of their head think “worst case, I’ll take it to one of those drive recovery places.” Alas, as JournalSpace discovered, even the professionals at DriveSavers can only recover data in certain lucky cases.
* Cost - if you think doing backups is too expensive, try not doing backups. JournalSpace says they spent as much on their attempt to recover the data as they had made in the entire year prior, did not succeed, and paid the ultimate corporate price.

Six years of effort building a company and volumes of users’ data lost is really unfortunate; if you have not been doing backups, make this your wake up call.

While the economic crisis continues to be at the forefront of our minds, business at Backblaze is booming. I would like to attribute that explicitly to our execution, but there is another factor at play: storage needs are skyrocketing.

Every day, consumers and businesses are generating more digital data both in unit volume (# of photos, # of spreadsheets) and also in size per document (photos are becoming higher resolution, spreadsheet formats are taking more space.) As these digital items become more prevalent, they are also becoming more critical.

Ziff Davis just published a survey that found that 89% of respondents would maintain or increase their storage purchasing in 2009 according to the eWeek article, “ZDE Report Validates Storage Sector as Healthiest in IT.” This reaffirms previous statements by analyst firms Gartner and IDC saying the storage sector “will continue doing well compared to other segments of the market.”

Online backup companies continue to benefit from this strong demand for storage - partially because users are creating more critical data, and partially because in a world of limited time and resources, both individuals and IT organizations make their lives easier by handing off the management for backups of this ballooning volume of data.

CloudCamp gathered a crowd of developers, entrepreneurs, business people, and investors last night to have an un-conference focused on honing the future of computing clouds. Despite spending less than a month on the event, Reuven Cohen and his team attracted several hundred people and 22 sponsors including Sun, Microsoft, Dell, Amazon and a number of smaller companies. This was a testament to the organizers, but also an indicator of the fervor surrounding cloud computing.

Run in the traditional “BarCamp” approach, following a brief introduction the audience was invited to offer discussion topics they would like to lead with a set of 30 session slots available. Many topics revolved around:

• Deploying, scaling, and automating cloud computing applications
• Best practices and common pitfalls in using cloud computing
• Vendor discussions around how to use their products with cloud computing

…but “What is Cloud Computing?” drew perhaps the largest attendance.

In this discussion, cloud computing was likened to electricity (”you should be able to plug in, use as much as you want, and have a standard interface to access it.”); people discussed whether cloud computing is a technology or a business model; some pondered whether the margins of cloud providers would eventually be driven to zero; and many struggled with what really is cloud computing and how is it different than hosting and shared servers.

Vendors like Amazon with its AWS and Nirvanix provide numerous benefits, especially for early stage companies:

• Pay-per-use computing and storage – limiting capital investments
• Ability to scale up quickly – good if you get a burst of usage
• Ability to scale down quickly – good if that burst goes away
• Allowing companies to focus on their core – which is often not infrastructure

Technology startups building social applications with viral hooks trying to fund advertising based business often use miniscule processing and storage relative to the business, marketing, and salary expenses. At least until they hit that viral hockey stick that requires scaling up at hyperspeed. If that magic happens, the question of “will we continue to see this and we should throw servers at this, or is this a flash of momentary virality and we’ll need to scale down on Wednesday” arises. Leveraging a cloud provider for these startups may be an excellent decision enabling them to scale up, scale down, and pay-per-use.

So did Backblaze use a cloud vendor to power our online backup service? No.

There is wisdom in clouds, but they aren’t for everyone. Stay tuned and I’ll explain our logic and approach in a post coming soon.

A decade ago the idea of entering your credit card information into a webpage was reserved for only a few web sites as ecommerce was in its infancy. Purchasing a book from Amazon.com was acceptable, but joesbooks.com saw no customers – even if they were a reputable offline store. Users were not ready to make that leap of faith. Over the years, much has been done to establish our trust as consumers in this medium: encryption of connections, little lock icons and green/red address bars in our browsers, Verisign/TrustE/HackerSafe/BBB Online standards and logos, fraud protection by Visa/MasterCard, and much more.

Of course, more than anything, what has garnered our trust is simply time. It’s no longer rare or surprising when someone purchases a $1000 airline ticket or applies for a $1 million mortgage online. We feel safe from experience. We feel safe from others’ experience.

This April 15^th was another milestone for trust in online services and electronic transactions: 22 million U.S. taxpayers filed their taxes from their home computers. That may not sound like a large percentage of the population, but dig a bit deeper and you see that 68% of ALL individual tax returns were filed electronically (a mix of “practitioners” and home users.) While the stats for paper forms are not broken down between practitioners and home users are not broken down, even if you assume every single paper return was submitted by an individual, that still says that of those people who did their own taxes, 38% filed them electronically.

And taxes are more than just a credit card number. Social security numbers, phone numbers, addresses, income, stocks, charities, real estate and more are fully documented. And the masses used the internet to file this data with the IRS.

Online backup is not new, but much like electronic transactions, adoption requires trust. And it is not only the encryption, little lock icons, and Verisign logos that enable us to trust these systems. It is our own experience with them. And those of our friends, and family, and mavens we trust. It doesn’t happen overnight, but usage builds usage, and the momentum is clearly building.

I don’t love paying taxes, but this April 15^th provided a pretty positive outlook.

EMC made an unsolicited bid for Iomega. While Iomega refused it, choosing to stick with the share purchase agreement it struck with ExcelStor Group Ltd. in December, it begs the question: Why was EMC interested in Iomega?

CEO Joseph Tucci has long stated exploratory interest in the consumer market: “We’re thinking about it. There are no big decisions yet, but I do think there’s a [sizeable] play in the home for a storage mini array,” but noted “We do not have expertise within EMC that understands the consumer market.”

The acquisition of Mozy and attempted acquisition of Iomega point to having made both the decision to enter and to acquire some expertise. Not surprising if Tucci believes the IDC numbers in the just-released EMC-sponsored whitepaper, “The Diverse and Exploding Digital Universe.” In this paper, IDC estimates that not only will global storage increase a factor of 10 from 2007 to 2011, but that consumers will generate 70% of that data, with organizations generating the remaining 30%.

EMC today holds a leadership position in enterprise storage, but has effectively no focus, no products, and no brand in the consumer space. Will EMC up its ante for Iomega? Try to pick up other consumer-focused offerings? And when will we start seeing animated polar bears pitching EMC during the Super Bowl?

Our PCs are more powerful than ever, so why do they feel so slow? It might be our expectations have increased. Or it might be the applications installed on them. A new PC today has 1 million times (4GB vs 4KB) more RAM than the earliest computers. Creating a program back then meant optimizing every line of code. Today, faster computers have enabled programmers to get sloppy. And sloppy code means slower computers.

Symanted launched the new “lighter” version of its Norton 360. Users had complained that after installing the previous version their “PCs experienced severe performance troughs.” It seems users are starting to push back on sloppy software, telling the software industry: I bought a faster computer so my computer would be faster - not so you can write bloated software.