Backup and recovery reviews: evidence to gather

Published backup and recovery reviews cannot tell you whether a product will return your data, at your volume, over your connection. The only thing worth measuring is whether it returns your data, at your volume, over your connection, into your hands. Nobody can test that for you.

Judge a review instead on evidence: a drill you run, a history you can read, and a support experience you deliberately provoke while you still have the option to walk away.

What to take away

The drill scorecard

Run this on a scratch location, never over live data, and fill in every row with a time and a result. Repeat it for each candidate on the same data. A written order for what to protect and how often each copy is made belongs in the backup and recovery setup before any drill begins.

Drill	What passing looks like	Time	Result

Use these drills as scorecard rows, and record the elapsed time and result for each.

Three of those rows fail more often than the rest. The restore performed by someone else, because consoles are easy for whoever configured them. The deletion attempt, because a great many arrangements permit it. And the key retrieval, because it is usually filed in exactly the place that would be gone.

The two numbers you take away are the time for a whole share and the time for a whole machine. Everything else in the evaluation comments on those.

The National Institute of Standards and Technology's Contingency Planning Guide for Federal Information Systems discusses recovery time. A drill is the only way a small business gets a real one.

Reading the provider rather than the product

A backup service is a promise about availability, so read its record of keeping promises.

The status page and its history. Look for incidents from the last two years, and specifically for whether restores were affected as well as backups. A provider that publishes detail about its own bad days is telling you something a marketing page cannot.

How it announces changes. Retention changes, tier changes and deprecations arriving with notice, in a place you would see, is a proxy for how the relationship will feel in year three.

What happens at the end. Ask, in the trial, exactly how you would extract everything if you left, how long it would take, and what it would cost. A vendor that answers precisely is a different proposition from one that does not.

Who else can reach your data. Support staff, subcontractors, the infrastructure the service is built on. This is a question about standing access rather than about encryption, and it is the same question cloud security asks of any service holding your data.

Provoke the support line

During the trial, on a Saturday, open a case that says a restore is failing. Do not warn them.

Record when you asked, when a human answered, and whether the first answer was a link to documentation you had already read. Then ask a follow-up that needs somebody to look at your account.

That exchange is the most predictive thing available to you before signing, because the situation you are buying for is precisely a weekend when something has failed and you are not calm.

Reading what other people write

Published reviews of this category are dominated by two moments: setup, and the month somebody left. Both are worth reading and neither describes the middle.

Ignore star averages entirely. People write about backup software when they are relieved or furious, and averaging those two crowds produces a number about nothing.

The reference call worth making

Ask the vendor for a customer of your size who has restored something substantial. Ask that customer one question: how long did it take, from the decision to restore to the business working again?

If the vendor cannot produce such a customer, that is the answer. If the customer's number is much larger than the one from your own drill, ask what was different, because the difference is usually volume or connection, and both of those apply to you as you grow.

A business that has never restored anything cannot answer, and knowing that is useful too. It means the arrangement has not been tested there either, which is the condition described in security foundations as believing you have a backup.

What no review can settle

Whether anybody in your business will read the failure reports. Whether the restore order has been decided. Whether the environment you restore into has been cleaned first, which is a decision that belongs in incident response rather than in a product comparison.

The U.S. Cybersecurity and Infrastructure Security Agency's Cyber Essentials ranks organizational questions above tooling for businesses without a security team.

That ranking is right: a well-reviewed product in a business where nobody reads anything is a subscription, not a recovery plan.

Common questions

How much data should the drill use?

Enough to be honest about duration. Restoring a small folder proves the mechanism and tells you nothing about the timing, and timing is the number the whole decision turns on.

Can we run the drill on the provider's demonstration environment?

It proves the console works. It proves nothing about your volume or your connection, which are the two variables that make restores slow. Use your own data.

We have used the same arrangement for years without testing it. Where do we start?

With one file, this week. Then a whole share within the month. Most arrangements pass the first and surprise people on the second, and finding that out on a quiet Thursday is the entire point.

Is a provider with no published outage history a good sign?

It is an absent sign. Every provider has bad days; the ones that publish them are describing how they behave when things go wrong, and a business buying recovery should want exactly that information. What the platform itself retains, and where its responsibility ends, is a separate question answered in backup and recovery.