A multi-tenant SaaS platform uses a service registry for discovery. The registry becomes unavailable during an incident. What should happen to running traffic, and what does that requirement imply about the design?

The requirement

A discovery outage must not be a traffic outage. Service discovery is control-plane machinery, and the data plane should keep working when the control plane is down — the property called static stability. A system where every request consults the registry has made an administrative component a hard dependency of every customer interaction.

What that implies about the design

• Clients cache resolved endpoints locally and keep serving from the cache when the registry is unreachable. The cache should be treated as authoritative-until-refreshed rather than as an optimisation with a short expiry that fails closed.
• Registry data has a long soft expiry and no hard expiry during an outage. Stale endpoints are far better than none: most instances are still where they were.
• Health checking is separate from discovery. The client should remove an endpoint that is failing its own requests without waiting for a registry to tell it. This is what makes stale data safe — the client self-corrects against reality.
• Registration is not on the request path. An instance failing to register should not fail to serve; it should serve and keep trying to register.

The residual risk

Serving from stale data means traffic may be sent to instances that have gone away, which is why client-side health checking, connection-level failure detection and retry-to-another-endpoint are not optional in this design. The combination — stale list plus live health checking — is what keeps availability high, and either half alone is insufficient.

The remaining exposure is a scaling event during the outage: new instances cannot be discovered, so capacity cannot grow. That is an acceptable degradation and should be stated as one: during a registry outage the system is statically stable at its current capacity, which is a defensible position as long as somebody has decided it deliberately rather than discovered it during an incident.