Disaster recovery (RPO/RTO)

What is disaster recovery (RPO/RTO)?

Disaster recovery is the plan for bringing systems and data back after a major failure. The failure might be a cloud region going dark, a database that corrupts itself, a ransomware attack, or a wrong DELETE run against production at the worst possible moment.

Two numbers shape the whole plan, and they are the ones worth getting exactly right: RPO and RTO.

RPO, the recovery point objective, is how much data you can afford to lose, measured in time and counted backwards from the incident. An RPO of 15 minutes means that after a recovery you can accept losing at most the last 15 minutes of changes. That is why RPO sets how often you have to back up or replicate: to meet a 15-minute RPO, you copy data at least every 15 minutes.

RTO, the recovery time objective, is how long the service may stay down, measured forwards from the incident. An RTO of 2 hours means the service is expected to be usable again within 2 hours of going down. It says nothing about data loss; that is the other number's job.

Reading RPO and RTO on a timeline

Put the incident at a single point on a line. RPO looks backwards from it to your last good copy. RTO looks forwards from it to the moment service is usable again.

Take an orders database with an RPO of 15 minutes and an RTO of 2 hours. Your last replication checkpoint is at 14:00 and ransomware hits at 14:12. You have lost 12 minutes of orders, inside the 15-minute budget, so the data promise holds. Now the clock runs forward: the restore has to finish by 16:12 to meet the RTO. If rebuilding the environment takes three hours, you kept the data promise and broke the time one.

Different systems earn different numbers. A monthly reporting export might accept an RPO of a day and an RTO of two days. A payment ledger or a live webshop will not. Set the two numbers per system rather than once for everything, and spend the budget where downtime actually costs you.

The four common recovery strategies

The strategy you pick is a trade between what you pay to keep a second environment ready and how fast you need to be back. The four strategies that AWS documents run from cheapest and slowest to most expensive and fastest.

1. Backup and restore. You keep backups and, on failure, redeploy the environment with infrastructure as code and restore the data. It is the cheapest option, and the RTO runs into hours. Fine for the loss of a single data centre on a workload that is otherwise available.

2. Pilot light. Data is replicated continuously to a second region and the core infrastructure is always on, but the application servers sit switched off. Data loss is low, yet you still have to switch the compute on and scale it up before you can serve traffic.

3. Warm standby. A scaled-down but fully working copy of production runs in the second region. It can take traffic immediately at reduced capacity, then scale up to full size. The RTO is shorter than pilot light, and the running cost is higher.

4. Multi-site active/active. Full copies serve live traffic in more than one region at once, so recovery time is close to zero for most failures. It is the most complex and most expensive option, and even here a data corruption event falls back to backups, so the recovery point is never quite zero.

Why a read replica is not a backup

A read replica continuously applies the same writes as its primary, usually within seconds and often through change data capture. That is useful for spreading read load and for failover, and promoting a replica can give you a low RPO.

It is still not a backup. A replica copies every write, including the bad ones. A wrong DELETE, a broken migration, or a ransomware write reaches the replica almost as fast as it hit the primary. Promote it and you inherit the same damaged data.

A backup is different in kind: it is a point in time you can roll back to, a moment before the damage. That is why continuous replication protects you against a lost server but not against corruption. A serious plan keeps both, replicas for availability and point-in-time backups for recovery.

Test the recovery, not just the backup

The most common way disaster recovery fails is trusting a plan nobody has run. A backup you have never restored is an assumption, not a recovery, and a failover you have never rehearsed will surprise you on the day you can least afford it.

So rehearse it. Restore from backup on a schedule and time how long it takes, because that number is the only honest test of your RTO. Run failover drills that include the people and the communication steps, not only the technical ones. A recovery that quietly fails just extends the data downtime you were trying to end.

Write the steps into a runbook, and tie each system's RPO and RTO to a written service level objective so the promise is explicit rather than assumed. For data teams there is more to restore than a database: you need to know, through data lineage, which pipelines must rerun, which dashboards showed wrong numbers, and which downstream systems were fed bad data. That is where idempotence earns its place, because an idempotent rerun can repeat without double-counting the orders you just recovered.