A client asked me to help with a database migration. The standard process is: snapshot, spin up a new RDS instance, restore, verify, cut over. They had 14 months of automated daily snapshots, and the status was green everywhere.
The restore took 4 hours instead of 45 minutes.
What went wrong
The snapshot was complete. But the database engine had been upgraded 8 months earlier, and the snapshot pre-dated the upgrade. The restore hit compatibility issues nobody had seen, because nobody had ever actually run a restore.
They had been backing up a database they could not restore in a crisis.
This is not rare
Backup configuration gets treated as a one-time setup task. Restore testing gets treated as something you do when you have time. You do not find out your backup process is broken until the moment you need it.
What to do
Test a restore to a separate instance once a quarter. It does not need to run production traffic. It just needs to confirm the process works and tell you how long it actually takes.
If you do not know how long your database takes to restore from a snapshot, that is the answer you need right now.