Nearly 50,000 swimmers nationwide rely on the U.S. Masters Swimming organization for training, competitions, and various local programs. Started in 1970, the organization has grown to support 500 local clubs and its website, www.usms.org, is the hub that helps manage this large, distributed non-profit.

Yesterday, during a routine import, the database containing all the member records became corrupted. Access to the website was shutdown to prevent additional errors while the team tried to restore the systems, but to no avail.

Luckily, the USMS team understood that data gets lost and rigorously prepared for such an event. Data was backed up in three ways: incremental, transactional, and tape. Even so, it turned out their first two backups failed. Fortunately, the third backup worked, and the organization lost 3 days, not 30 years, of data.The USMS team says:

“For the next six hours, five experts attempted to recover the corrupted data and it turned out to be impossible.”

Continuing to show diligence and savviness, the USMS team has developed their plan to go forward:

What we are doing short term
We are going through all auxilliary sources of data including credit card transactions, automated emails and batch processes. We have hired extra help and we are manually copying the data from these sources as soon as we receive it.

What we are doing Long Term
We are purchasing additional hardware and leasing a separate facility to do real-time secure backups of transactional data thereby efectively adding a fourth set of backup data that is accurate up to the last transaction. In addition we have established a new set of protocals to further mitigate potential data loss.

If there is are some lessons to be learned from the USMS experience:
* Make sure your data is backed up.
* Backup to more than one place (e.g. local hard drive and online backup service.)
* Periodically check your backups.

You can never be too careful with your precious data.

Ma.gnolia.com was a bookmark storing and sharing service (similar to Delicious.com) that has shut down due to data loss. The company started several years ago and built a small, but adoring user community that liked Ma.gnolia’s easy-to-use API and caching of linked pages. The service was the work of Larry Halff who nearly single-handedly built the service.

Alas, as for many companies without a good backup, data loss caused a mortal wound: Ma.gnolia completely shut down on February 17th, 2009.

Larry explained that when he started the company several years ago, no good cloud-based backup services existed, thus requiring him to develop his own backup. The backup was doing a file sync over Firewire to another computer. Unfortunately there was no integrity checking, no versioning, and the system was never tested to see if the backups worked. When it came time to restore, it turns out they did not.

People often assume simply setting up a copy or sync process is sufficient for backup, only to discover the issues with this when a restore is needed. At Backblaze, every file is compressed, encrypted, de-duplicated, and integrity-checked to ensure the backed up file exactly matches the original.

Larry intends to develop a new service and says when starting the new company, “My first priority is better backups.”

We wish Larry the best with his new service and hope others benefit from Larry’s pain.

JournalSpace, a 6 year old blog hosting service, closed shop on Tuesday after losing all of its users’ data. Details are sketchy, but the company claims the cause was either an OS failure or a disgruntled employee that deleted the data.

Regardless of the cause, since the JournalSpace had no backups, in addition to the death of the company, all of the bloggers who hosted their sites with the company lost their data. Many are trying to reassemble their years of blogging from a combination of Google cache results and other pieces.

Read more about this story at Slashdot and TechCrunch or read the company’s post “This is the way the world ends. Not with a bang but a whimper” where they are also listing the domain for sale.

A few key takeaways:
* Mirroring – JournalSpace had been mirroring their data, meaning two drives would have the exact same data. While often mistaken for backup because this protects from a single hard drive failure, this is open to all other causes of data loss such a virus, fire, user error, etc.
* Data Recovery – most people realize they should do backups, but they put it off, and in the back of their head think “worst case, I’ll take it to one of those drive recovery places.” Alas, as JournalSpace discovered, even the professionals at DriveSavers can only recover data in certain lucky cases.
* Cost – if you think doing backups is too expensive, try not doing backups. JournalSpace says they spent as much on their attempt to recover the data as they had made in the entire year prior, did not succeed, and paid the ultimate corporate price.

Six years of effort building a company and volumes of users’ data lost is really unfortunate; if you have not been doing backups, make this your wake up call.

1 Your backup strategy is to burn CDs and DVDs.
A diamond may be forever, but CDs and DVDs have a shelf-life. Even the Optical Storage Technology Association says an unrecorded disk will only last 5 to 10 years. And this assumes you’ve selected the right files and remember to do it, which brings me to reasons #2 and #3.

2Your backup strategy requires picking which files to backup.
If you’re like most people, you actually have no idea where your files are. Wait, you’re not like most people – you absolutely select which folders you put your files in. But some applications save files in random places. Have any idea where your iTunes playlists are? Hint: They’re not in My Documents. Windows hides folders. Vista forces older applications into a hidden sandbox. You probably know where most of your files are…but some files your care about are almost certainly somewhere else.

Continue reading…

For over five years I have been tied into ancient cell phones because I had originally signed up for a no-longer-offered $99 unlimited minutes plan and converting to any current plan would double my monthly bill. When the iPhone debuted, I looked longingly at it as I continued to use my 1950′s-era RAZR. But someone at AT&T heard my cry and friends and family who knew of my woes emailed me from as far as London and Prague saying, “Did you hear AT&T/Cingular is coming out with a new $99 unlimited plan?” Hallelujah, I could finally get a new phone.

I considered the Blackberry Pearl and a few others, but decided on the iPhone. I’d like to say it was all through a logical process (big screen; best web surfing; Wi-Fi enabled; cheaper data plan) and certainly those were a consideration. And it certainly helped that my master-texter brother said the keyboard worked ok and Apple announced both the SDK and ActiveSync integration with Exchange. Of course, in the end, it was just sexy.

The 16 GB iPhone arrived last week and I immediately synchronized all my contacts, calendar appointments, photos and a chunk of my music. Then I went rock climbing. And I brought my iPhone. But then I hesitated.

Previously I would keep my phone in my pocket while climbing – they were small, solid, and cheap. Arriving at the climbing gym, though, I wondered whether the larger, pricey, and seemingly fragile iPhone would survive. Looked online a bit and it appears they are pretty solid: Mike Beauchamp claims his iPhone survived having his iPhone run over by a semi and Don’s iPhone works even after the screen cracked.

But as the crew at Will it Blend learned, they’re not indestructible. And neither is the data on them. As John C. Dvorak notes at PC Mag, they get dropped in toilets, left in taxis, and abused in numerous ways not typical for a PC. John argues the move to smaller devices is ridiculous as people use them as desktop alternatives. He says these devices are easier to steal, easier to break, and are never backed up.

But my iPhone is backed up. Every picture, every song, every contact and calendar appointment. Did I hack the iPhone with some special backup software?

No. But the iPhone is a copy of my real data – which lives on my laptop. And my laptop is backed up. Ironically, with my new iPhone, my data is more, not less, safe than it was in my older mini-bricks. Those phones may not have been as slippery and arguably may have been less likely to break…but ultimately they would…and since they did not synch, the data on them would be gone. Having my iPhone run over by a semi would be $500 of pain, but since my iPhone is backed up, at least I will never need to send the “oops, could everyone please send me your phone numbers again” email.

Robyn Peterson isn’t a computer newbie – after all, he’s in charge of technology for tech publisher Ziff Davis. So how did he lose his backup? Using a popular techie solution for online backup: FTP-ing his files to a web hosting provider. A perfectly reasonable, if somewhat tedious, solution. But as he found out, not such a secure option.

“We have been cracking down on people using our services for backing up files,” Robyn was told by the provider. The Terms of Service said their “servers are not intended as a data backup or archiving service.” Don’t check the fine print? Or don’t think it’ll be enforced? Ok sometimes…but when you really need that backup, you don’t want the response Robyn got when he asked whether his deleted files were backed up: “backups go back a maximum of only two weeks, and no backups are guaranteed.”

“I’ve owned six portable USB hard drives over the past 10 years, and all six of them have failed unrecoverably. Is it just me, or is there a wider problem out there?” Mike Elgan asks. As prices have plumetted over the last few years, the USB hard drive market has exploded. More and more users are counting on these drives to add extra space or backup their current drive. But these drives don’t protect from many of the items that would kill the main computer – fire, flood, theft, etc. And judging by the 170 or so responses to Mike, he’s not the only one that has lost a hard drive or two.