The backup you never tested

Here’s how you find out.

It’s a Tuesday. A plugin update goes sideways, or the host has an incident, or someone with admin access makes an honest mistake at exactly the wrong layer. The site is down. Not slow, not glitchy, gone. Someone says the sentence that’s supposed to make everything okay: “We have backups.”

Then comes the pause.

Where are they, exactly? Who has the login for the backup service? Is it the whole site or just the database, does it include the uploads folder or did that get excluded to save storage back when somebody was trimming costs, and is the most recent one from last night or from sometime in the spring?

Nobody knows. That’s the finding out. And it’s happening while customers are emailing, the phone is lit up, and everyone in the company is suddenly very interested in infrastructure.

An untested backup exists in two states at once: fine and useless.

You collapse it into one or the other by restoring it. The universe collapses it by breaking your site. Only one of those methods lets you pick the date.

We’ve come to think about backups the way pilots think about the emergency slide. Nobody is impressed that it’s stowed on board. The entire question is whether it inflates, there’s exactly one way to know, and you would strongly prefer to learn the answer somewhere other than the actual emergency.

The restore drill

So we run restore drills. Quarterly is plenty for most sites. Here’s the shape of one:

• Restore the latest backup to a staging server. All of it: files, database, media, configuration.
• Time it. “We can restore” is a very different fact from “we can restore in four hours.”
• Write down every step and every credential the drill required. If the person doing it got stuck, note exactly where. That document ends up worth more than the backup itself.
• Click around the restored site like a customer would. A restore that boots but is missing six months of images is a partial answer wearing a green checkmark.

The first drill is always humbling. Something’s missing. A password lives in one person’s head, and that person is at a wedding. The restore takes triple what anyone guessed. All of it is good news, because every one of those discoveries just happened on a calm afternoon with coffee instead of on the Tuesday.

The green checkmark problem

A word about the false comfort of automation. Most backup tools send a nightly email with a green checkmark, and that checkmark means one thing only: a file was created. It does not mean the file is complete, current, or capable of becoming a website again. We’ve seen setups that reported success every night for a year while quietly backing up an empty directory, because a path changed during a migration and nothing downstream ever noticed. Automation is how backups happen. It is not how you know they work.

Drills also surface the quieter questions nobody asks while things work. How far back do the backups actually go? If the bad thing happened three days ago and you only noticed today, is there a copy old enough to predate it? Does the backup live somewhere that survives the same disaster as the site, or is it sitting on the same server, keeping it company?

There’s a reason this doesn’t get done, and it isn’t laziness. Testing a backup produces nothing visible. No feature ships, no page gets faster, nothing changes that a client or a boss can see, and so the drill loses the calendar fight to literally everything else, every quarter, for years. Right up until the Tuesday.

Our take: the backup was never the plan. The restore is the plan. The backup is just an ingredient, and an ingredient you’ve never cooked with is a guess.

The day you find out whether yours works is coming either way. You can pick it, or it can pick you.