Negative Caching

A failover creates a DNS record and the record's TTL is 60 seconds. Roughly 8% of clients keep failing for 30 minutes. Every manual check returns the new record immediately, from the authoritative servers and from public resolvers. Nothing is broken and nothing is misconfigured for the record in question - what those clients hold is a cached answer saying the name does not exist.

Negative caching is specified behaviour, defined in RFC 2308 (1998). A resolver that receives NXDOMAIN caches the absence, and the lifetime of that cached absence comes from the zone's SOA minimum field (bounded by the resolver's own cap), not from the TTL of the record that was eventually created.

Why it matters

It exists for a good reason: without it, every typo and every lookup of a non-existent name would reach the authoritative servers, and junk queries are a large share of DNS traffic. The cost is that absence is sticky, and absence is what a failover-by-record-creation produces.

The symptom is distinctive and widely misdiagnosed: a minority of clients failing, for a fixed duration unrelated to any TTL anyone configured, while every diagnostic the operator runs succeeds. The reason the diagnostics succeed is that they query a resolver that never saw the negative answer, or the authority directly - so the one test that matters is to query the failing client's own resolver.

Implementation patterns

• Set the SOA minimum deliberately. A value of 60 to 300 seconds suits a zone involved in failover; the default in many zone templates is 1800 or more, inherited and never examined.
• Never fail over by creating a name. Pre-create every name you might need - regional names, failover names, maintenance names - pointing somewhere harmless, and fail over by changing a value. A positive answer has a bounded lifetime you control; a negative one does not.
• Prefer health-checked record values or anycast withdrawal where the recovery target is tight, which removes resolver caching from the recovery path entirely.
• Watch for the wildcard interaction. A wildcard record prevents NXDOMAIN entirely, which solves negative caching and also means no name under the zone ever fails to resolve, with its own consequences for typos and for certificate issuance.
• Check the resolver floors. Public resolvers enforce their own minimum and maximum caching times, so a 5-second SOA minimum is not honoured as 5 seconds everywhere.

Industry example

The mechanism is routine in post-incident write-ups of DNS-based failovers, and the standard remediation in them is the same: stop using record existence as a signal. Operator guidance for managed DNS services recommends health-checked failover on an existing record, or routing-level failover, precisely because TTL honouring is inconsistent across resolvers and client stacks, and negative caching is the least predictable part of it. Long-lived application processes compound it: many runtimes and connection pools cache a resolution for the life of the process, so even a correctly short TTL does not reach them.

Failure scenarios

• Failover by record creation, holding a fraction of clients on the failed path for the SOA minimum.
• A new regional endpoint rolled out progressively, where early clients cached the absence and cannot reach it after it exists.
• A deleted-then-recreated record during a migration, producing the same effect without anyone connecting the two events.
• Certificate issuance failing because a validation record was queried before it propagated and the absence was cached past the issuer's retry window.
• An SOA minimum of several hours inherited from a template, turning a minor DNS mistake into an afternoon.

Trade-offs

Lowering the SOA minimum buys faster recovery from any situation where a name was briefly absent, and pays in query load on the authoritative servers for names that do not exist - modest for most zones and not nothing for one attracting junk traffic. The deeper trade-off is architectural: treating DNS as a control plane is cheap and slow; treating routing as the control plane is fast and operationally heavier. Design for a recovery objective of minutes if DNS is in the path, and do not promise seconds.

When not to use it

This is not a lever to pull routinely. For a planned migration with hours of notice, negative caching is irrelevant - create the record, let it propagate, then switch - and tuning the SOA minimum for a zone with no failover role is churn with a small cost attached. Where the recovery target genuinely is seconds, do not tune TTLs at all: the mechanism has to be anycast withdrawal or a stable endpoint in front of the change, because no TTL value makes resolver behaviour reliable enough to depend on.

Interview question

Q: You fail over by adding an A record for a new endpoint. Most clients recover immediately; a few per cent fail for exactly half an hour. Diagnose it, then tell me how you would redesign the failover.

What a strong answer covers: identifying cached NXDOMAIN and naming the SOA minimum as its governing value rather than the record TTL; explaining why authoritative and third-party checks all look clean and querying the client's own resolver instead; the redesign - pre-created names with health-checked values, so failover changes a value and never creates one; lowering the SOA minimum to a sensible window with the query-load trade acknowledged; and recognising that long-lived processes cache resolutions in-process, so a seconds-level recovery target needs routing rather than DNS.

Quick check

Quiz: Which field governs how long a resolver remembers that a name does not exist? The zone's SOA minimum, subject to the resolver's own limits - not the TTL of the record created later.

Flashcard: Why do all your DNS checks look clean while some clients still fail? Your checks hit the authority or a resolver with no cached negative answer; the failing clients' resolver cached NXDOMAIN before the record existed.