A search company must decide where the rules deciding which documents are eligible to appear for a query should live - the crawler, the indexer, the query serving layer, or a dedicated component. Which choice best reflects cohesion?
Show the full answer Hide the answer
Why the obvious answers are wrong
Crawler. It changes when fetching changes — politeness, robots handling, discovery, scheduling. Eligibility changes when legal, safety and quality policy changes. Different reasons, cadences and owners: coincidental cohesion.
Serving layer. Tempting because it is where results appear, but serving changes for latency reasons. Every policy change would then touch the most latency-critical, highest-blast-radius component in the system.
Duplicated in both. This is what teams actually ship, and it produces the failure that matters: index and serving disagree about eligibility. A document is indexed but suppressed, or suppressed at index time but served from a cache built before the rule existed. Not a performance bug — a correctness bug that appears only under a rule change, which is exactly when nobody is looking.
The cohesion test
Cohesion is not "related things nearby". It is things that change together, for the same reason, at the same time, on the same person's say-so. Eligibility policy has one owner, one cadence and one reason. That is a cohesive unit.
What the design looks like
A policy component both pipelines consume — evaluated at index time to decide inclusion, and again at serve time as a filter, from the same versioned rule set.
Why evaluate twice? Indexing is asynchronous and the corpus is huge. A rule effective now cannot wait for a full reindex; the serving-time filter gives immediate effect and the index-time evaluation gives efficiency. Both read one source of truth, and the rule set carries a version so you can answer "which rules produced this result set" during an investigation.
The trade-off you accept
Two evaluation points cost latency on the serving path and add a dependency to the most critical component. That price buys immediate policy effect and a single definition. For a system where policy errors are legal exposure it is worth it; where they are cosmetic it is not. That judgement is the actual question.