Backups that actually restore: the 3-2-1 rule and recovery testing

The backup job has shown a green tick every night for a year. Then a storage array fails — or ransomware encrypts the file server — and the restore begins. It turns out the backups were kept on a NAS that was encrypted too; the newest clean copy is three weeks old; the accounting database restores but will not start; and nobody wrote down the password for the backup encryption key.

None of these problems is exotic. They share one cause: the backups were configured, but recovery was never tested. This article covers the principles that make backups resilient and a practical way to prove they work.

A backup is only as good as its last tested restore

A successful backup job proves that data was copied somewhere. It does not prove that the copy is complete, consistent, readable, reachable when the main systems are down, or restorable in a time the business can accept. Those things are proven only by restoring — regularly, and in conditions close to a real incident.

The 3-2-1 rule — and why it is a minimum

The 3-2-1 rule is a simple, widely used baseline:

• 3 copies of the data: the production data plus at least two backups.
• 2 different types of storage, so that a single failure mode — a faulty array, a firmware bug — does not damage every copy.
• 1 copy off-site, so that a fire, flood or theft at the office does not take everything.
• Increasingly, one more: at least one copy offline or immutable, so that ransomware holding administrator credentials cannot delete or encrypt it.
• And zero unverified backups: copies are checked for integrity and restores are tested.

Start from RPO and RTO, not from software

Two numbers drive every decision. RPO (recovery point objective) is how much data the business can afford to lose, measured in time: if backups run nightly, up to a day of work can be lost. RTO (recovery time objective) is how long a system can be unavailable before the damage becomes unacceptable.

Agree them per system with the people who use it. For example, in a hypothetical office of 60 people, shared documents might tolerate losing a few hours of work and a day of downtime, while the accounting system at month-end might need hourly copies and recovery within hours. These targets — not the features of a particular product — determine backup frequency, where copies are kept and how much the solution should cost.

What is usually missing from the backup

• Cloud and SaaS data: under many providers' terms, backing up email and files held in cloud services is the customer's responsibility.
• Databases copied as open files instead of with application-consistent methods.
• Configurations of firewalls, switches, Wi-Fi controllers and the CCTV system.
• Laptops of people who keep their files locally.
• The things needed to restore: encryption keys, passwords, license files, installation media and documentation of how the systems are built.

How backups fail in practice

• Backups kept on the same network and under the same accounts as the systems they protect — and encrypted along with them.
• Jobs that silently skip locked or newly created folders.
• Retention too short to go back to before an infection that went unnoticed for weeks.
• Off-site copies that take days to download over the office internet connection.
• Restores that depend on one person — who happens to be on holiday.

Recovery testing: a practical schedule

Testing does not have to be heavy to be useful. A tiered approach works well — for example:

1. Regularly, for example monthly: restore a few random files and folders from different dates and check that they open.
2. Quarterly: restore a complete server or virtual machine into an isolated environment and check that the application actually starts and the data is current.
3. At least once a year: a full scenario exercise — the main site or main storage is lost — following the written recovery plan, with the people who would do it for real.
4. After every major change: a new system, new storage, new backup software or a network change.

Record every test: what was restored, how long it took compared with the RTO, which problems appeared and what was fixed. Over time these records become the honest answer to the question: 'How long would we actually be down?'

Make ownership explicit

Backups most often fail in the gap between people: IT assumes the software vendor handles it, the vendor assumes the customer does, and the cloud provider's terms say something else entirely. Write down who is responsible for each system's backups, who monitors them and who tests restores. If an external provider runs backups as part of managed IT support, the scope should state exactly which systems are covered and how test results are reported. Isolating backup systems on their own network segment is part of the same design.

Would you like to know whether your backups would really restore? Contact us — we can help review the current setup and plan a first recovery test.