A collaborative workspace product must define RTO and RPO. The product team says "we can never lose a user's work". What does that requirement actually mean, and what does it cost?
Show the full answer Hide the answer
Unpacking the requirement
"Never lose work" conflates several distinct guarantees that have very different costs:
1. Durability of committed writes. Once the product has told the user their change is saved, it must survive. This is achievable and non-negotiable — RPO of zero for acknowledged writes, via synchronous replication before acknowledgement.
2. Durability of unacknowledged writes. A user typing when the network drops. The server never received it, so no server-side mechanism can protect it. The answer is entirely client-side: local persistence and replay on reconnect. This is where most perceived data loss actually originates, and no amount of server-side replication addresses it.
3. Recoverability from logical corruption. A bug, a bad migration, or a user deleting a page. Replication faithfully replicates the mistake, so this needs point-in-time recovery and version history — a different mechanism from durability entirely.
4. Availability of the work. Not losing it but being unable to reach it for four hours is, experientially, close to losing it.
What each costs
| Guarantee | Mechanism | Cost |
|---|---|---|
| Committed writes durable | Synchronous replication before ack | Write latency, especially cross-region |
| Unacknowledged writes | Client-side local persistence and replay | Client complexity, merge handling |
| Logical corruption | PITR, version history, tombstones | Storage, retention, restore tooling |
| Availability | Warm standby or active-active | Continuous infrastructure cost |
The important observation is that the first two are the ones users mean, and only one of them is a server-side problem.
Where the RPO conversation usually goes wrong
Teams set RPO from the database's replication configuration and consider it answered. But the system's real RPO is the worst of its components: the database may replicate synchronously while object storage holding attachments replicates asynchronously, the search index rebuilds from scratch, and the configuration store is backed up nightly.
A workspace product's "work" spans all of those. A page whose text survives but whose attached files are lost has lost the user's work by any definition the user cares about.
The practical recommendation
- RPO of zero for the document content path, achieved by synchronous replication plus client-side buffering.
- Version history as a product feature, which doubles as the recovery mechanism for logical corruption and is far more usable than a database restore.
- A stated, non-zero RPO for derived data — search indexes, previews, analytics — because these are regenerable and paying for zero RPO on regenerable data is waste.
- An explicit inventory of every store, with its own RPO, so the real number is known rather than assumed.
The final point is the one worth insisting on: an unstated RPO is not zero, it is unknown — and discovering it during a recovery is the worst possible moment.