Term Kind Topic What it is
Bloom Filter tool Data Architecture A compact probabilistic structure that answers "is this key definitely absent, or possibly present?" — no false negatives, tunable false positives.
Bloom Filter Cache Guard pattern Caching Strategies Placing a Bloom filter in front of an expensive lookup so that keys which certainly do not exist never reach it.
Cache Invalidation concept Data Architecture The problem of removing or refreshing cached data when the underlying source changes, and the reason caching is harder than it looks.
Cache Penetration concept Cache Invalidation Repeated lookups for keys that do not exist, which miss the cache every time by definition and pass straight through to the store.
Cache-Loss Survivability Cache Failure Blast Radius, Origin Protection practice Cache Invalidation Designing so that losing the cache degrades the system rather than destroying it - because a cache absorbing most of the read traffic is a capacity dependency, not an optimisation.
Caching Strategies pattern Caching Strategies Which cache pattern to use, how staleness is bounded, and the failure modes that only appear under load.
Caching Strategies in Practice Cache-Aside, Write-Through, Read-Through pattern Caching Strategies The write and read strategies, the stampede that takes down origins, and why cache invalidation deserves its reputation.
Cardinality Estimation concept Query Optimisation The planner's prediction of how many rows each step of a query will produce — the input that determines every other choice it makes.
CDC Initial Snapshot concept Change Data Capture The consistent full copy taken when a CDC pipeline starts, before streaming begins — and the step that determines whether the target is correct.
Change Data Capture CDC pattern Data Architecture Publishing a stream of a database's row-level changes by reading its replication log, without modifying the application that owns it.
Change Data Capture in Practice CDC pattern Change Data Capture Turning a database's replication log into an event stream, so downstream systems learn about changes without the application publishing them.
Claim Provenance As-Of Metadata, Source-Tagged Data practice Data Architecture Storing every externally sourced value with its timestamp, its source and a confidence derived from that source's history - so that downstream decisions about staleness are explicit rather than accidental.
Consistent Hashing Ring Hashing concept Partitioning & Sharding A partitioning scheme where adding or removing a node moves only a small fraction of keys, instead of remapping everything.
Covering Index Index-Only Scan concept Indexing An index that contains every column a query needs, so the query is answered from the index without reading the table at all.
CQRS Command Query Responsibility Segregation pattern Data Architecture Separating the model used to change state from the model used to read it, so each can be optimised independently.
CQRS as a Data Architecture pattern CQRS Treating the serving layer as a derived, rebuildable projection of an authoritative write store.
Crypto-Shredding Cryptographic Erasure, Key Destruction Deletion, Per-Subject Encryption pattern Data Lifecycle & Retention Encrypting each data subject's personal data under a key unique to them, so that a deletion request is satisfied by destroying the key rather than by rewriting immutable or widely-replicated data.
Crypto-Shredding Cryptographic Erasure practice Data Lifecycle & Retention Making data permanently unreadable by destroying its encryption key, so immutable copies in backups and archives are erased without being modified.
Data Catalog tool Data Governance A searchable inventory of datasets with their schema, owner, meaning, freshness, quality and classification.
Data Governance practice Data Governance Making data ownership, meaning, quality and access explicit — the mechanism that keeps a data platform trustworthy as it grows.
Data Lakehouse concept Data Architecture A pattern that puts warehouse-style transactions, schema and governance on top of cheap open-format object storage.
Data Lifecycle practice Data Lifecycle & Retention Managing data from creation through tiering to deletion — the discipline that keeps storage cost and legal exposure bounded.
Data Lineage practice Data Governance A record of where each dataset came from, what transformed it, and what depends on it — traced at table and ideally column level.
Data Mesh concept Data Architecture An organisational approach that gives domain teams ownership of their analytical data as a product, with a self-serve platform and federated governance.
Data Retention Policy practice Data Architecture A defined rule for how long each class of data is kept, where, and what happens at the end of it.
Data Warehousing concept Data Warehousing A separate analytical store modelled for questions rather than transactions, physically isolated from the systems that generate the data.
Database Index concept Data Architecture A secondary structure that lets the engine find rows without scanning, trading write cost and storage for read speed.
Database per Service pattern Polyglot Persistence Each service owning its own datastore, with no other service reading or writing it directly.
Denormalisation practice Data Architecture Deliberately duplicating data across records to make reads cheap, accepting the write-time cost of keeping copies in step.
Derived Store Discipline One Authoritative Store Per Fact, No Dual Authority concept Polyglot Persistence The rule that every fact has exactly one authoritative store and every other copy is explicitly derived, rebuildable and populated by a documented pipeline - the discipline that makes multiple datastores defensible.
Discord's Message Store Migrations case-study Data Architecture Discord moved from MongoDB to Cassandra to ScyllaDB as message volume grew from millions to trillions, each time for a specific and different reason.
Discord: Hot Partitions at Trillions of Messages Discord Message Storage case-study Partitioning & Sharding Discord partitions messages by channel and time bucket, because a single very busy channel would otherwise concentrate load on one partition.
Document Store tool NoSQL Stores A store that keeps semi-structured documents — typically JSON — retrievable by key and queryable by their contents.
ETL and ELT practice ETL & ELT Whether to transform before loading or after — a choice that follows from where compute is cheap and who owns the transformation.
ETL vs ELT concept Data Architecture Whether data is transformed before loading into the target or after it, which decides where the compute happens and how much raw history you keep.
Event Sourcing pattern Data Architecture Storing the full sequence of state-changing events as the system of record, and deriving current state by replaying them.
Event Sourcing in Practice pattern Event Sourcing Storing state as an immutable sequence of events and deriving current state by replay — powerful where history is the domain, expensive everywhere else.
Event Store tool Event Sourcing An append-only store of domain events organised into per-entity streams, serving as the system of record rather than as a log beside it.
Event Upcasting pattern Event Sourcing Transforming an old event's stored form into the current shape as it is read, so historical events remain replayable after the schema changes.
Expand and Contract Migration Parallel Change, Expand-Migrate-Contract practice Data Governance Changing a schema through a sequence of individually backward-compatible steps, so old and new application versions can run simultaneously.
Figma's Postgres Sharding case-study Data Architecture Figma delayed sharding for years using replicas and vertical partitioning, then sharded Postgres horizontally without downtime using logical shards and a proxy layer.
Foreign Key Constraint concept Relational Modelling A database-enforced rule that a referencing value must exist in the referenced table — referential integrity that no application bug can violate.
Freshness Tiering Hot Index, Recent-Document Tier pattern Indexing Splitting a search corpus into a small aggressively-refreshed index of recent documents and a large lazily-refreshed main index, so sub-second freshness is paid for only on the documents that need it.
Graph Database tool NoSQL Stores A store whose first-class citizens are nodes and the relationships between them, making multi-hop traversal cheap.
Hot Key Hot Partition, Hot Row, Key Skew, Celebrity Problem concept Partitioning & Sharding A single key or narrow key range receiving a disproportionate share of traffic, so one partition saturates while the rest of the cluster is idle - the failure that no amount of horizontal scaling fixes.
Hot Key Hot Partition, Hot Shard concept Partitioning & Sharding A single key or partition receiving a disproportionate share of traffic, so that a well-balanced key space still produces one overloaded node.
Hot Partition Hot Shard, Hot Key concept Partitioning & Sharding One partition receiving disproportionate traffic, so the system saturates at a fraction of its aggregate capacity.
Hot, Warm and Cold Data concept Data Lifecycle & Retention Classifying data by how frequently and how urgently it is accessed, so each tier can be stored on media priced for that access pattern.
Idempotent Pipeline practice ETL & ELT A pipeline whose task can be re-run for the same input window any number of times and produce the same result.
Identity-Record Separation Pseudonymous Subject Reference, Retention-Erasure Split pattern Data Lifecycle & Retention Holding regulated business records under a pseudonymous reference and the reference-to-person mapping in a separate governed store, so a retention obligation and an erasure right can both be satisfied.
Index Selectivity metric Indexing The fraction of rows a predicate eliminates — the property that determines whether an index is worth using at all.
Indexing concept Indexing Auxiliary structures that turn a scan into a lookup — the highest-leverage database intervention and the one most often applied blindly.
Indexing Strategy practice Data Architecture Choosing the set of indexes a table carries by working backwards from its actual queries, and accepting the write cost that each one adds.
Integrated Cache Tier Cache-Behind-the-Client, Storage-Managed Cache, Transparent Cache pattern Caching Strategies Placing the cache inside the storage layer's own client rather than in each calling application, so invalidation is driven by the change stream and consistency semantics are defined once instead of per team.
Join Strategies concept Query Optimisation The three ways a database combines two row sets — nested loop, hash join and merge join — and the conditions under which each is correct.
Lakehouse pattern Data Lakes & Lakehouses Open table formats over object storage that add transactions, schema enforcement and incremental updates to a data lake.
LinkedIn Databus: Change Capture as a Product Databus case-study Change Data Capture LinkedIn built a change capture system so that derived stores — search, graph, caches — could stay current without every application dual-writing to them.
Log-Based CDC pattern Change Data Capture Capturing changes by reading the database's own write-ahead log, which sees every change with no load on the source and no application involvement.
Materialised Read Model Projection, Read Model practice CQRS A precomputed, query-shaped copy of data maintained asynchronously from the system of record - and the operational obligations that come with it.
Materialized View pattern Data Architecture A precomputed, stored result of a query, refreshed on a schedule or from a change stream, read instead of recomputing.