beginner 2 min answer

A 6 TB order history must be backfilled into a new store before cutover. The link sustains 1 Gbps and the target ingests about 40 MB/s per writer. Roughly how long is the backfill, and which assumption decides whether it fits one weekend?

backfillcapacity-planningmigrationiopsverification
Show the full answer Hide the answer

The assumptions worth stating

  • 6 TB means roughly 6,000,000 MB of payload. Index and format overhead on the target will add to that, so treat 6 TB as a floor.
  • 1 Gbps is about 125 MB/s of theoretical throughput, and perhaps 100 MB/s sustained after protocol overhead.
  • The target ingests about 40 MB/s per writer, and the question does not say how many writers can run.
  • The source is a production database still serving customers.

The arithmetic

With a single writer at 40 MB/s:

6,000,000 MB ÷ 40 MB/s = 150,000 s ≈ 42 hours

A weekend maintenance window from Friday 20:00 to Monday 06:00 is about 58 hours. So 42 hours fits, with roughly 16 hours of slack, and the network is not the constraint because 40 MB/s is well under the link's 100 MB/s.

Add a verification pass. A full row-by-row comparison reads both sides again, so plan on something between a third and a full repeat of the copy time unless verification runs concurrently with the copy. That is the difference between 42 hours and 60 to 80, and it is what turns a plan that fits into one that does not.

The assumption that dominates the error

Not the target and not the link: the read budget on the source. Sustaining 40 MB/s of extra sequential reads for two days means roughly 40 MB/s of additional I/O on a system that is also serving customers, and a backfill that competes with production reads will either be throttled or will raise customer-facing latency until someone throttles it.

The honest planning number is therefore the source's spare I/O during the window, not the target's ingest rate. Measure it before promising a date: run the extract at full rate for fifteen minutes in business hours and watch the production p99. If the answer is "we can spare 15 MB/s", the arithmetic becomes 6,000,000 ÷ 15 ≈ 111 hours, and the weekend plan is dead.

What the number rules in or out

  • 42 hours rules in a single-window backfill with a freeze, if and only if the source can sustain the read rate.
  • 111 hours rules out the freeze entirely and forces the standard shape: bulk copy over days at a throttled rate while the system runs, then a change-capture stream to catch everything written since the snapshot, then a short cutover that only has to drain the tail.
  • Either way, parallel writers help only until the source is the bottleneck, which is usually at two or three. Adding writers past that point converts a migration problem into an incident.

When this is the wrong answer

Below roughly a few hundred gigabytes, none of this matters: the copy finishes inside an hour and the effort belongs in verification and rollback instead. Estimating a backfill is worth doing when the number is large enough to change the shape of the plan, and the test for that is whether it exceeds the largest window the business will grant.