A data platform wants to use interruptible capacity to reduce cost. Which workloads are suitable, and what must be true of them?

What must be true

• The work must be interruptible without losing significant progress. That means checkpointing, and the checkpoint interval becomes an economic parameter: too frequent and it costs throughput, too rare and an interruption destroys hours.
• The work must be restartable elsewhere, so no instance-local state that matters.
• The deadline must have slack, because reclamation can mean waiting for capacity.
• The failure must be handled automatically, or the operational cost of manual restarts exceeds the saving.

Suitable workloads

Batch data processing with checkpointing · model training with regular checkpoints · CI and build workers, where a lost build is retried · stateless request serving at partial capacity, mixed with on-demand instances so that reclamation reduces headroom rather than causing an outage · queue consumers, where an interrupted message is redelivered.

Unsuitable workloads

Stateful databases · anything with a hard latency SLA and no on-demand baseline · long single-shot jobs with no checkpointing, where an interruption at 90% destroys the whole run · anything where the restart is non-idempotent, since a preempted worker may have completed its side effect before dying.

The design that makes it work at scale

A mixed fleet with a guaranteed on-demand or reserved baseline plus interruptible capacity above it. The baseline guarantees the SLA; the interruptible portion provides the elasticity cheaply. Reclamation then degrades headroom rather than availability, which is a completely different risk profile from an all-spot fleet.

Diversify across instance types and zones, because reclamation is correlated within a type — a fleet entirely on one instance type can lose most of its capacity simultaneously.

The customer-facing version

For a platform reselling compute, the mature form is to expose the trade as priced classes — on-demand and preemptible — so interruption is a contract the customer accepted at a lower price rather than a scheduler decision about work they believed was safe. That converts a technical risk into a product feature and moves the checkpointing obligation to where the knowledge is.

The trap

Assuming the saving is the discount. The real saving is the discount minus the wasted work from interruptions, minus the engineering cost of making the workload interruptible, minus the operational cost of handling reclamation. For a workload that is genuinely batch and already checkpointed, that arithmetic is excellent; for one that needs to be rewritten, it frequently is not.