intermediate 1 min answer

A company deploys a data catalogue. Six months later almost nobody uses it and most entries are stale. What went wrong, and what would make it work?

catalogmetadataadoptionautomationatlassian
Show the full answer Hide the answer

Why catalogues fail

They depend on humans to populate and maintain metadata that nobody is measured on. Documentation written once decays immediately, and a catalogue that is 40% accurate is worse than none, because a user who is misled twice stops trusting it entirely.

The second failure is that the catalogue is not on the path of any workflow. People discover data by asking a colleague in chat, which is faster and gives an answer they trust. A tool that is slower than the informal alternative loses.

What makes it work

  • Automated harvesting of schema, lineage, freshness, ownership and usage — everything derivable should be derived, so the human contribution is limited to what only a human knows: what the data means and what it should not be used for.
  • Usage-ranked results. The most-queried dataset is usually the right answer, and popularity is a better relevance signal than completeness of documentation.
  • Ownership from the system of record, not from a free-text field that names someone who left.
  • Embedded in the workflow — in the query tool, the notebook, the BI tool — rather than as a separate destination.
  • Staleness visible, so users can weigh an entry's age rather than assuming currency.

The honest measure

Not entries catalogued, but questions answered without asking a person. Coverage metrics reward populating the catalogue; the outcome metric rewards making it useful, and those lead to very different investments.