What do Google providing search, Coca-Cola operating its systems to track inventory, and Backblaze backing up your data have in common? The computers that handle all of this live in data centers. And those data centers use power - lots of it.

In the U.S. alone there are over 20,000 data centers - each of which houses thousands or tens of thousands of servers. Combined, these data centers make up 3% of all U.S. energy consumption (not just electricity) - more than the entire domestic air fleet.

So when I went to an event on Wednesday called:
THE TRUTH ABOUT THE FUTURE OF THE DATA CENTER:
CLOUD, COLOCATION, & DATA CENTER REAL ESTATE
it should be no surprise that the focus was on power, power, power.

And lest you think this is people getting wrapped up in the green movement or just jumping on a marketing trend - let me dissuade you. Datacenters in the U.S. spend $23 billion a year on electricity according to KC Mares of MegaWatt Consulting. In fact, electricity can often cost over 50% of the purchase price of a server over it’s lifetime. Minor improvements can have massive implications not only on global warming but also company bottom lines.

KC provided a fascinating overview of innovations and experiments that operators of data centers and the companies building out large server deployments are pursuing. Some examples:

* VFDs - variable frequency drives to adjust the speed of blower fans that adjust to need rather than spinning at a constant rate.
* Natural cooling - using outside air and fans rather than air-conditioning to keep data centers cool; it turns out most servers are perfectly happy running at temperatures much higher than what data centers attempt to keep them at.
* Shorter cooling regions - having air flow almost directly around a server in the process of cooling it rather than through the entire building; shorter distances mean less air friction and less energy spent moving it around.
* Eliminating UPS systems - getting rid of the backup power systems and assuming servers will go down…and having backup servers or data centers instead.
* Using 480 volts - higher voltage means lower amperage and thus less heat loss and higher efficiency. More of today’s server systems are capable of handling this voltage.
* Higher efficiency power supplies - switching to 90% efficient power supplies on servers rather than using 70% or 80% ones; these are more expensive upfront but can still pay off fairly quickly.

A number of these items pay for themselves in a couple months and then generate savings ongoing from then on. KC has a variety of information on his site and blog.

Last month’s blog post about building our Backblaze storage pods generated a ton of interest and many people are building their own pods! Our post also generated a ton of questions so below we answer the common ones and provides more detail about where to get components.
Continue reading…

Three weeks ago we published how to build a Backblaze Storage Pod, the cloud storage hardware we use for our unlimited online backup service, and gave away the design to anyone who wished to build their own. We thought a few people might find it interesting. Perhaps some might even want to try to build one. We never expected would happen next.

Om Malik wrote about it at GigaOm, as did Robin Harris at StorageMojo, and Cory Doctorow on Boing Boing. Soon after, CrunchGear, VentureBeat, ZDNet, Mashable, TUAW, Electronista, MacWorld, Vator.tv, NetworkComputing, On-Storage, PSFK, Enterprise Storage Forum, eWeek and dozens of others picked it up. After digging in, SmallNetBuilder did a thorough breakdown for its DIY audience.
Continue reading…

At Backblaze, we provide unlimited storage to our customers for only $5 per month, so we had to figure out how to store hundreds of petabytes of customer data in a reliable, scalable way—and keep our costs low. After looking at several overpriced commercial solutions, we decided to build our own custom Backblaze Storage Pods: 67 terabyte 4U servers for $7,867.

In this post, we’ll share how to make one of these storage pods, and you’re welcome to use this design. Our hope is that by sharing, others can benefit and, ultimately, refine this concept and send improvements back to us. Evolving and lowering costs is critical to our continuing success at Backblaze.

Below is a video that shows a 3-D model of the Backblaze Storage Pod. Continue reading to learn the exact details of the design.

You can download the full 3-D model of the Backblaze Storage Pod here.

Continue reading…

One year ago I attended Cloud Camp, a gathering of early adopters of cloud computing. Primary questions there centered on: “How do we define cloud computing?

Last Thursday I attended GigaOm Structure 09 and the discussions were full of cloud computing success stories, explanations of deployments, insights about how to scale bigger, offerings of cloud services at various levels and more. Below I summarize my takeaways from the day’s events.

Is Cloud Computing Real or Just a New Term for Old Technologies?
The short answer: Yes.

Many of the concepts of cloud computing have existed for years: large datacenters, hosted servers, online service offerings (think Yahoo! Mail), and even multi-tenancy (in most mainframes.) In part, cloud computing is a packaging of these into a term renovated for today.

However, cloud computing is more than just marketing. It is both a business model and technological change for the providers and consumers of these computing services. While startups could have always avoid directly deploying hardware by paying a hosting provider, the ability to pay for usage rather than signing year-long contracts dramatically changes the playing field when launching a new service with unpredictable demand.

What are the Tiers of Cloud Computing?
Cloud computing is standardizing on three key tiers:
* Infrastructure-as-a-Service
Amazon Web Services‘ EC2 and S3 are primary examples in this category where raw computing power such as processing and storage can be rented. GoGrid, Nirvanix, Rackspace, Savvis, and 3Tera are other examples. Some of these offer the infrastructure as a web-service, where you simply purchase “1 GB of space.” Others offer a hardware-as-a-service model in which you actually specify how many servers you want and how they are configured. Control versus the desire for hands-off outsourcing will drive which of these two models is ideal for your organization.

* Platform-as-a-Service
Google AppEngine, Force.com, and Microsoft Azure are primary examples in this category where a hosted development environment is provided upon which applications can be built and the underlying hardware is abstracted out. These platforms are ideal for developers wishing to get products to market quickly, but lock-in to the platform can be a concern. Some vendors are attempting to offer the ability to use standardized frameworks that could be portable, but this is in its infancy. Facebook Platform is also an example of a platform-as-a-service, though one in which applications would be deployed only within the Facebook experience rather than on the public Web.

* Software-as-a-Service
Examples of this category are all around us: online banking, Yahoo! Mail, Google Apps, and, of course, Salesforce.com.

What level should you engage in? As a consumer, the answer is obvious: you are simply a user of numerous software-as-a-service offerings. As a developer, you may choose to build upon an infrastructure-as-a-service or platform-as-a-service, depending largely on your desire to tradeoff control versus speed of deployment.

Will there be a Segmentation Amongst Clouds?
Scale is a critical component of clouds since the ability to scale up and down quickly relies on having both a large deployment and multiple-tenants across which spikes are smoothed out. However, companies consuming cloud services may have differing requirements that would push vendors to offer different clouds types. A basic examples of this segmentation is the different size server instances offered through EC2. However, other possible needs include quality-of-service guarantees, HIPAA or SAS70 compliance, or specific geographic or hardware requirements.

We are already starting to see cloud differences appearing:
* Amazon focuses on a broad-based offering.
* Nirvanix focuses on clouds for enterprises.
* RackSpace focuses on customer support.

Will clouds segment? Yes, but likely into a handful of major sites that still allow for the scale required to extract the benefits of cloud computing.

Does Cloud Computing mean the end of Scalability Issues?
Startups bringing the next high-definition-video-social-network may become wildly successful and require quickly scaling to millions of users. Cloud computing purports to easily address this by enabling these companies to simply add servers and storage as quickly as they are needed. While this may seem to solve all scalability issues, developers that have dealt with scale in the past understand that the fundamental architecture of the application needs to be developed to handle scale issues and cannot purely rely on scaling hardware.

If a program requires significant computation and the code is written to allow parallel processing, adding CPUs in the cloud can enable the computation to happen much quicker. Alternatively, if the code were written single-threaded, no amount of extra hardware will help speed the process. Cloud computing may reduce the need for system administrators on staff, but it does not negate the requirement for top-tier engineers.

And Finally, is Cloud Computing Inevitable?
Cloud computing is certainly in a phase of dramatic growth, with vendors providing new offerings and customers experimenting and adopting. There is no doubt that this trend will continue due to the benefits of scale, cost, operations, and segmentation of skill-sets. However, it is not inevitable that everything will move to the cloud. Certain organizations will have needs that are too custom-tailored to create a segmented cloud and will retain their internal systems - potentially building a ‘private cloud’. Of course, companies should evaluate the pros and cons for their own environments and for most there are likely many opportunities to move applications into the cloud and derive significant benefits.

Cloud service are forecast to be a bright spot amidst dire economic times. Reaching $56.3 billion by end of 2009, cloud services are expected to generate 2x the revenue of Google (which is partially included) and a growth rate that exceeds the company’s 18% year-over-year rate. By 2013, Gartner expects cloud services to reach $150 billion in sales.

Revenues from cloud application services (such as Backblaze) “were almost twice as large as the market for systems infrastructure and will continue to show strong growth,” according to the Gartner report.

While there is a lot of discussion about terminology and taxonomy (”cloud services” vs “SaaS” etc.)…there is no doubt the market for infrastructure and applications provided over the Internet has arrived.

TechCrunch is holding its next roundtable on Feb 27th asking the question, Who’s Cloud Is It Anyway? Assembling 275 people together, TechCrunch intends to discuss whether clouds will be closed warring platforms or open, interconnected systems.

Other questions I believe should be discussed include:
* What is the appropriate level for a cloud to exist at? (providing raw computing power? API-level applications? complete services?)
* Build vs. Buy - when should organizations build their own clouds?
* Should consumers and businesses aim to consolidate into a single cloud or distribute their reliance among clouds?

I’m looking forward to a fascinating discussion. Sign up for the event, email me if you would like to meet there, and follow Backblaze on Twitter for event updates.

Broadband connections in the United States are 10 times slower than those in South Korea. In fact, we are number 15 based on average connection speed, behind not only Japan, Finland, France and Sweden (which are all at least 3 times faster) but also countries including Iceland, Poland, and Portugal that are blessed with significantly faster connections.

Cloud computing, VoIP, eCommerce, Internet TV, telemedicine, telecommuting, distance learning and a myriad of other drivers of communication, business, and social benefit rely on fast, accessible, reliable broadband connectivity.

Published on Friday, The American Recovery and Reinvestment Act of 2009 provides grants to service providers to build out broadband infrastructure in unserved and underserved areas. Grant requirements include mandated speeds of:
* Advanced Broadband Services: 45 Mbps downstream, 15 Mbps upstream
* Basic Broadband Services: 5 Mbps downstream, 1 Mbps upstream

Under this definition of basic broadband service a user could download 1.5 TB and upload 300 GB of data in one month - enough to watch a movie a night, download hundreds of photos or songs a day, backup an average computer online, and still have room to work remotely and do some video-conferencing.

Part of this Bill requires “a plan for use of broadband infrastructure and services in advancing consumer welfare, civic participation, public safety and homeland security, community development, health care delivery, energy independence and efficiency, education, worker training, private sector investment, entrepreneurial activity, job creation and economic growth, and other national purposes.”

History shows that connectivity not only links us in ways that are obvious at the outset, but drives innovation that is beneficial and unforeseen.

So will we have fast nationwide broadband tomorrow? Of course, not. Within a year of the passage of this Bill, a national broadband plan is required to be created. Best case is another year or two before individuals and businesses see available connections. This plan will not put the U.S. on par with South Korea, but it is a good step forward, and within several years we may start seeing the benefits of the broader availability of faster broadband.

While the economic crisis continues to be at the forefront of our minds, business at Backblaze is booming. I would like to attribute that explicitly to our execution, but there is another factor at play: storage needs are skyrocketing.

Every day, consumers and businesses are generating more digital data both in unit volume (# of photos, # of spreadsheets) and also in size per document (photos are becoming higher resolution, spreadsheet formats are taking more space.) As these digital items become more prevalent, they are also becoming more critical.

Ziff Davis just published a survey that found that 89% of respondents would maintain or increase their storage purchasing in 2009 according to the eWeek article, “ZDE Report Validates Storage Sector as Healthiest in IT.” This reaffirms previous statements by analyst firms Gartner and IDC saying the storage sector “will continue doing well compared to other segments of the market.”

Online backup companies continue to benefit from this strong demand for storage - partially because users are creating more critical data, and partially because in a world of limited time and resources, both individuals and IT organizations make their lives easier by handing off the management for backups of this ballooning volume of data.

CloudCamp gathered a crowd of developers, entrepreneurs, business people, and investors last night to have an un-conference focused on honing the future of computing clouds. Despite spending less than a month on the event, Reuven Cohen and his team attracted several hundred people and 22 sponsors including Sun, Microsoft, Dell, Amazon and a number of smaller companies. This was a testament to the organizers, but also an indicator of the fervor surrounding cloud computing.

Run in the traditional “BarCamp” approach, following a brief introduction the audience was invited to offer discussion topics they would like to lead with a set of 30 session slots available. Many topics revolved around:

• Deploying, scaling, and automating cloud computing applications
• Best practices and common pitfalls in using cloud computing
• Vendor discussions around how to use their products with cloud computing

…but “What is Cloud Computing?” drew perhaps the largest attendance.

In this discussion, cloud computing was likened to electricity (”you should be able to plug in, use as much as you want, and have a standard interface to access it.”); people discussed whether cloud computing is a technology or a business model; some pondered whether the margins of cloud providers would eventually be driven to zero; and many struggled with what really is cloud computing and how is it different than hosting and shared servers.

Vendors like Amazon with its AWS and Nirvanix provide numerous benefits, especially for early stage companies:

• Pay-per-use computing and storage – limiting capital investments
• Ability to scale up quickly – good if you get a burst of usage
• Ability to scale down quickly – good if that burst goes away
• Allowing companies to focus on their core – which is often not infrastructure

Technology startups building social applications with viral hooks trying to fund advertising based business often use miniscule processing and storage relative to the business, marketing, and salary expenses. At least until they hit that viral hockey stick that requires scaling up at hyperspeed. If that magic happens, the question of “will we continue to see this and we should throw servers at this, or is this a flash of momentary virality and we’ll need to scale down on Wednesday” arises. Leveraging a cloud provider for these startups may be an excellent decision enabling them to scale up, scale down, and pay-per-use.

So did Backblaze use a cloud vendor to power our online backup service? No.

There is wisdom in clouds, but they aren’t for everyone. Stay tuned and I’ll explain our logic and approach in a post coming soon.