intermediate 3 min answer

A team adopts feature flags enthusiastically and two years later the codebase is full of them. What went wrong, and how should flags be managed as a lifecycle?

feature-flagstechnical-debttestinglifecycleincidents
Show the full answer Hide the answer

The four distinct things called "feature flags"

Conflating them is the root of the problem, because they have completely different lifetimes:

  • Release toggles — decouple deployment from release. Lifetime: days to weeks. Removed once the feature is fully rolled out.
  • Experiment flags — A/B tests. Lifetime: the experiment, then removed with the losing variant.
  • Operational toggles / kill switches — disable a feature under load or during an incident. Lifetime: permanent, deliberately, and part of the degradation ladder.
  • Permission flags — entitlements by plan or customer. Not feature flags at all; they are product configuration and belong in the domain model, not in a flag system.

Most accumulated flag debt is release toggles that were never removed, and treating them as permanent configuration is how a codebase acquires hundreds of them.

What goes wrong

  • No removal step. The flag is added as part of the feature work and its removal is nobody's task.
  • Combinatorial state space. N boolean flags mean 2^N configurations, of which the team tests a handful — so production runs a configuration that has never been exercised, and the bug appears only for users in that combination.
  • Flags in the data model rather than in behaviour, so removing one requires a data migration.
  • Nested and interacting flags, where the effect of one depends on another and nobody can reason about it.
  • Flags becoming permanent branches, with two implementations maintained indefinitely.
  • A flag service that becomes a hard dependency, so its outage is an outage — and flag evaluation on the request path with no cached default is a genuinely common cause of incidents.

Managing the lifecycle

  • Declare the type and an expiry date at creation. A release toggle without an expiry is permanent by default.
  • Automated reporting of stale flags — past expiry, or fully rolled out for a period, or not evaluated at all — with a task raised to the owning team.
  • Removal is part of the feature's definition of done, not a follow-up.
  • A hard cap on concurrent release toggles per team, which forces cleanup by making it a constraint rather than an aspiration.
  • Test the flag states that will actually occur: the current production configuration and the one being rolled out to. Testing all combinations is impossible; testing the two that matter is straightforward and is what most teams omit.
  • Flags evaluated from a cached local snapshot with a safe default, so the flag service being unavailable degrades to last-known-good rather than failing — the same reasoning as any configuration dependency.
  • Kill switches exercised on a schedule, since a switch not thrown in six months does not work.

The reframing

A release toggle is a temporary construction and should feel like one. Its presence in the codebase is a small ongoing cost — in reading, testing, and reasoning — that is worth paying for a few weeks and is not worth paying for three years.

Operational toggles are the opposite: deliberately permanent, actively maintained, tested, and among the most valuable resilience mechanisms available. The mistake is applying one policy to both.