tow.com

2010-12-29: I’ve updated my backup strategy post here.

I’ve been making backups since I started using computers back in the early 1980s. Software at the time came on cassette tape and 5 1/4-inch floppy disks before transitioning to 3 1/2 inch floppies and CD-ROMs. When my love affair with the Mac began in 1989, I knew what it was like to have a cavernous storage system. I recall thinking to myself, “I’ll never run out of 40MB of hard drive space!” I still have old issues of MacUser magazine advertising 20MB hard drives for thousands of dollars. Now, you can get a hard drive with 50,000 times the storage capacity for a $100!

My current camera generates files that are 14MB in size. Two photos alone would almost fill up my poor Mac SE/30’s hard drive were it still spinning today. My current data set is topping 4TB these days, so my need to keep my data safe and sound is critical. I’ve personally experienced hard drive crashes before, and they are not fun to recover from without backups.

Over the years, I’ve gone through many types of backup strategies: multiple hard drives, NAS boxes, storage robots, online, and off-site storage. I’ve finally developed a comprehensive multi-pronged approach for keeping my data safe and available.

This article will be useful for those who have massive amounts of data to be backed up. If you just have a single hard drive to worry about, get a couple of external hard disks or a Time Capsule. Use Time Machine and be done with it. On the other hand, if you have to worry about backing up terabytes of data, read on!

1. Goals
2. Tools
3. Methodology
4. Future Plans

Goals

My goal is to be able to recover from a hard drive crash with minimal to zero data loss. In addition, I want my data stored offsite in the event — knock on wood — the house burns down or is burglarized.

The 4 terabytes of data that I need backed up is organized in the following manner:

• Boot and User Data: 400GB*
• Photos: 1.5TB
• Video: 1TB
• Archived: 1.5TB

* I know that I should separate my user data from my boot volume. One thing at a time!

Tools

Over the years, I’ve assembled a smorgasbord of hardware and software to handle backups. Here’s what I’m currently using:

• Infrant ReadyNAS 600 with 4x750GB drives configured in RAID 5
• Drobo (USB) with 2x750GB and 1x300GB drives
• WiebeTech RTX400-SV SATA enclosure with 4 drive bays
• Lots of hard drives (250GB, 750GB, 1TB, and 1.5TB varieties)
• SuperDuper

Time Machine and Infrant ReadyNAS

Three and a half years ago, I bought the ReadyNAS because I had run out of space on my 250GB Photos hard drive. Initially, the Infrant held 4x250GB drives. In a RAID 5 configuration, it offered me about 700GB of protected disk space. All of my RAW photos went onto the Infrant, and I was even using it as my primary working volume. Sadly, even though it sports Gigabit Ethernet, the Infrant’s performance was nowhere near that of using an external or internal hard drive.

Today, the Infrant is loaded with 4x750GB drives and currently functions as a Time Machine destination for my Mac Pro and two MacBook Pro computers.

Drobo

The Drobo holds completed photography projects and other infrequently accessed files. I don’t have it on all of the time because (1) it’s a little noisy and (2) it’s slow because of the USB interface. There is a faster Drobo today that offers two FireWire 800 ports. Sadly, I don’t know how much longer FireWire will be with us, seeing that Apple may be phasing out the high-speed peripheral port.

WiebeTech RTX400-SV

My previous computer, a PowerMac Quad G5, had but two internal drive bays. Since Rae and I started working on our film in early 2008, I knew we needed more fast storage. The first step was purchasing a WiebeTech RTX400-SV at the recommendation of Eric.

Unlike many other SATA hard enclosures, RTX-series does not require drive trays. You simply insert the hard drives like a floppy disk into the RTX400. The Drobo has this feature too, but the RTX400-SV uses port-multipled eSata to transfer files at internal hard drive speeds. In other words, it’s really fast!

The second step was to replace the Quad G5 with a Mac Pro. With four drive bays, the new Mac could hold my Boot, Photos, Video, and Scratch volumes. With my RAW photos now on a internal drive, instead of the ReadyNAS, working Aperture has become much faster.

SuperDuper

The final piece of my backup strategy is software. I use SuperDuper from Shirt Pocket Software to create clones of my hard drives. The application performs incremental backups, which speeds up the cloning process tremendously. You can also schedule backups to occur automatically throughout the week.

Another similar product to SuperDuper is Carbon Copy Cloner from Bombich Software.

Methodology

Here’s what happens backup-wise on a hourly, daily, weekly, and monthly basis:

1. Hourly/Daily backup to Time Machine instance on Infrant ReadyNAS RAID 5 (1.TB available): Mac Pro and 2 MacBook Pros
2. Weekly backup to hard drives: Boot, Photos, Video, Laptops
3. Monthly rotation of weekly backups with offsite backups: Boot, Photos, Volume
4. Offsite backup: Archived

Every morning, the Mac Pro wakes up and starts a SuperDuper cloning operation onto the Infrant. The initial backup for each volume was an overnight process, but each incremental backup takes between 20 minutes to an hour.

Periodically throughout the week, I fire up the WiebeTech RTX400, which holds clones of the drives in my Mac Pro, and I use SuperDuper to create an exact clone of these drives. Every month, I rotate these drives with the ones stored offsite.

Future Plans

My current strategy has one weakness. Because only the boot volume is being protected with Time Machine, the Photos and Video hard drives are at risk of losing more than a day’s worth of data in the event of a hard drive crash. I noted earlier that I have four separate drives in my Mac Pro acting as four separate volumes. Originally, I had configured the four drives in a RAID 0+1 configuration, following the directions on MacProJournal. I first attempted to partition and configure the four drives into a RAID 1+0 configuration, but despite hours of wrangling with Disk Utility and the command-line interface, I was unsuccessful at striping a set of mirrored drives.

With this setup, I was protected against a single drive failure in my Mac Pro. The downside was that four 1TB drives arranged in this configuration yielded only 2TB for my Boot, Photos, Video, and Scratch partitions. A few months ago, I began to run out of disk space, and as a result, I had to break the RAID. Since each volume was already cloned on a separate hard drive, it was easy to go back to a normal four-drive, four-volume setup in my Mac Pro.

To get both storage and redudancy, I might need to buy a MaxConnect and install 6-8 drives in the Mac Pro. With 2 TB drives now available for around $200, I could have up to 4-8TB of usable, protected space, which would be enough for me for the next one to two years?

Like death and taxes, increasing storage requirements seems to be another immutable law!