How should an architect make defensible decisions in a domain where the technology changes faster than the decision cycle?
Show the full answer Hide the answer
The shift in what "defensible" means
From "this is the right choice" to "this is a reasonable choice and we can change it cheaply". In a fast-moving domain, being right about the current best option is worth much less than being able to re-evaluate in three months.
What that implies
1. Make the volatile choices configuration, not code. A routing layer that selects a model, a provider or an implementation means the decision changes without a deployment, and comparison on real traffic becomes possible. Choices scattered through a codebase cannot follow a moving landscape.
2. Build the evaluation capability, not the answer. A held-out set of real tasks with known-good outputs, an automated comparison harness, and traffic-based comparison for the final check. This is the durable asset; the current choice is not.
Without it, an organisation runs whatever it chose originally, indefinitely, because it has no way to know whether anything better exists for its workload.
3. Keep the abstraction thin. An abstraction over providers that becomes a lowest common denominator forfeits the capability differences that made re-evaluation worthwhile. Thin enough to swap, not so thick that everything is reduced to the intersection.
4. Separate the stable from the volatile. Retrieval quality, permission enforcement, cost attribution, evaluation infrastructure and observability are stable concerns that will matter regardless of which model or provider is current. Invest there.
5. Record decisions with revisit triggers, so a change in the landscape produces a review rather than an argument about whether to have one.
What not to do
Wait for the landscape to settle, which forfeits the period in which the capability is being built and the requirements learned. Or commit deeply to the current option, which is the same mistake with more confidence.
The framing
In a volatile domain, optionality is the architecture. The decisions worth agonising over are the ones that determine how cheaply you can change your mind — and those are usually about interfaces, configuration and measurement rather than about the technology being selected.