Building a Graph from Unstructured Text

Most of what an organisation knows is in documents. Turning that into a graph means extracting entities and relations at scale, and the difference between a graph that answers questions and a pile of triples nobody queries is decided by a few choices early in the pipeline.

The pipeline

Entity recognition and typing finds spans and assigns them to classes. A model can do this well for common types and needs domain adaptation for specialised ones, where the entities are internal identifiers, product codes or technical terms no general model has seen.

Entity linking maps each mention to a canonical identifier, which is the entity resolution problem arriving at extraction time. A mention linked to the wrong entity attaches a fact to the wrong node, which is worse than not extracting it.

Relation extraction identifies which relation holds between a pair of mentions. Sentence-level extraction is tractable; the relations that matter are frequently expressed across sentences or across a document, and document-level extraction is substantially harder.

Normalisation turns extracted values into comparable forms: dates into a standard representation, quantities into consistent units, names into canonical spellings. Skipping it produces a graph where "3 kg" and "3000 g" are different objects.

Validation checks the result against the schema and against existing assertions, rejecting or flagging triples that violate a constraint or contradict something already known.

Open versus closed extraction

Open extraction pulls whatever relation the text expresses, with the predicate as free text. It has no schema, requires no preparation, and produces a graph where the same relationship appears as acquired, bought, purchased, took over and acquisition of. Nothing can be queried reliably because the query must guess the surface form. It is useful for discovery and for building a schema, and it is not a substitute for one.

Closed extraction maps to a fixed relation set, which is what makes the graph queryable and requires the relation set to be defined in advance and to actually cover the domain.

The workable sequence is to run open extraction first, cluster the resulting predicates to see what the corpus actually expresses, define a closed schema from that evidence, and then extract against it. Designing the schema before looking at the data produces a schema the data does not fit.

When it breaks

Extraction confidence must be retained. Some triples are certain and some are guesses, and a graph that stores both identically cannot distinguish them later. Confidence scores and source provenance per triple are what allow a consumer to filter and what make correction possible.

Contradictions are information. Two sources asserting incompatible facts is a signal about the sources, not an error to resolve silently by taking the most recent. Recording both with provenance, and surfacing the conflict, is more useful than a graph that quietly picked one.

Temporal validity is usually ignored. A fact true in 2019 and false now is stored as true, because the triple model has nowhere to put the interval. This is the reification problem arriving concretely, and a graph built from historical documents without it accumulates a growing body of stale assertions presented as current.

Recall on rare relations is poor and invisible. Extraction models are trained on what is common, so relations appearing rarely are extracted rarely, and the resulting graph is systematically incomplete in a way that a query cannot detect. Per-relation extraction evaluation is what surfaces it.