Portfolio ↗

Terminology

2225 terms, tools, patterns and metrics an architect is expected to use precisely. Each one gets a short explanation of what it is, and — where it matters — what it is commonly confused with. Search filters as you type; the column headers sort.

All areas2225 Architecture Fundamentals77 Distributed Systems107 Data Architecture110 Cloud Architecture94 Networking88 API & Integration Architecture82 Reliability & Resilience75 Observability70 Performance & Capacity Engineering72 Security Architecture86 Cost Architecture & FinOps71 Business Architecture67 Architecture Communication67 Enterprise Architecture66 Legacy Modernization71 AI-Era Architecture74 Software Architecture & Engineering71 Architecture Patterns71 Architecture Decision-Making63 The Architect's Meta-Skills61 Delivery & Release Engineering66 Platform Engineering & Developer Experience70 Testing & Quality Architecture66 Data Platform Architecture69 Streaming & Real-Time Data69 Data Governance & Semantics74 Frontend & Experience Architecture66 Edge, Mobile & IoT68 Regulatory & Data Protection Architecture65 Assurance, Audit & Model Risk69
Kind Any concept801 practice736 pattern377 metric127 case-study77 tool62 protocol45

69 terms shown.

Term Kind Topic What it is
Backpressure in Streaming concept Streaming & Real-Time Data A slow downstream stage signalling upstream to slow down, so that queues stay bounded instead of consuming memory until the job dies.
Catch-Up Throttling Recovery Rate Limiting, Drain Control practice Streaming & Real-Time Data Deliberately limiting how fast a recovered consumer drains its backlog, so that recovery does not become a second and larger incident.
Changelog Stream concept Stream-Table Duality A stream whose records are keyed updates, so replaying it from the beginning reconstructs a table — the same information in the other of its two forms.
Checkpoint Interval metric Stream Processing Frameworks How often a stateful processor persists its state and offsets, which trades steady-state overhead against how much work is redone after a failure.
Compaction Lag Cleaner Lag, Dirty Segment Lag concept Stream-Table Duality The delay between a key being overwritten and the superseded record actually being removed, which decides how much larger a compacted topic is than its key set and how long a rebuild from it takes.
Completeness Indicator As-Of Metadata, Result Confidence Marker practice Streaming SLOs Metadata published alongside a streaming result stating how current it is and what proportion of expected input it reflects - so consumers can reason about staleness instead of assuming currency.
Completeness SLO Did Everything Arrive, Delivery Completeness metric Streaming SLOs The proportion of produced events that actually reached the destination - the streaming SLO that is measured least often and whose absence conceals the most damaging failure.
Consumer Lag metric Streaming SLOs The gap between the latest offset written to a partition and the offset a consumer group has processed, expressed in records or in time.
Dead Letter Queue DLQ, Poison Message Queue pattern Dead Letter Handling A separate destination for messages that cannot be processed successfully, so one bad message does not halt the stream behind it.
Dead-Letter Budget Diversion Budget, Poison Rate Budget metric Dead Letter Handling A declared ceiling on the share and the age of records a pipeline may divert before it counts as failing rather than coping - with an owner and a defined action attached to each breach.
Denormalised Log Resolved-Asset Topic, Fat Event Stream pattern Kappa vs Lambda A second topic that republishes each record with its referenced data already resolved, so consumers read complete payloads instead of each performing the same joins against the normalised source of truth.
DoorDash Dispatch: Matching as an Optimisation Problem Deep Red, Assignment Optimisation case-study Streaming & Real-Time Data Assigning deliveries greedily to the nearest driver is locally sensible and globally poor, so the assignment is batched and solved as an optimisation.
Event Time Versus Processing Time concept Windowing The distinction between when something happened and when the system saw it, which determines whether results are reproducible.
Event Time vs Processing Time concept Windowing The distinction between when something happened in the world and when the system got round to handling it.
Exactly-Once Semantics EOS, Effectively-Once concept Exactly-Once Semantics The guarantee that each input record affects the output state precisely once, achieved through atomic state-and-offset commits rather than through delivering messages once.
Feature Freshness metric Feature Freshness How recent the data behind a model's input features is at the moment of inference, and the training-serving consistency problem it creates.
Feature Staleness metric Feature Freshness The age of the feature values a model scores against, and the divergence between how they are computed at training time and at inference time.
Follower Fetching Closest-Replica Fetch, Rack-Aware Consumption pattern Streaming Cost Letting a consumer read from the in-sync replica in its own availability zone instead of from the partition leader, which removes consumer-side cross-zone transfer charges and adds replication delay to every read.
Freshness Requirement concept Streaming vs Batch How stale data may be before the decision it supports degrades — the only question that justifies streaming over batch.
Idempotent Consumer pattern Exactly-Once Semantics A message consumer whose effect is the same whether a message is processed once or many times, which is what makes at-least-once delivery safe.
Idempotent Sink pattern Exactly-Once Semantics A destination where writing the same record twice has the same effect as writing it once, which is what makes end-to-end exactly-once achievable at all.
Idle Source Stall Watermark Stall, Silent Pipeline Halt, Quiet Partition Problem concept Streaming SLOs A stream processor whose event-time progress halts because one partition or source has stopped producing - so output ceases entirely while every component reports healthy.
Join Replay Misalignment One-Sided Replay, Split-Watermark Join concept Streaming Joins The failure that follows from replaying one input of a stateful join while the other stays live - event times no longer overlap, window state is never cleaned, and the output is wrong in both directions with n…
Kappa Architecture pattern Kappa vs Lambda Treating the stream as the single processing path, and handling historical reprocessing by replaying the log through the same code.
Key Skew concept Partition Keys & Ordering An uneven distribution of records across partitions, which caps throughput at the busiest partition regardless of how many exist.
Keyed State Size metric Stateful Stream Processing The total state a job holds per key across all keys, which governs memory, checkpoint duration and recovery time.
Lambda Architecture Batch and Speed Layer, Dual-Path Analytics pattern Kappa vs Lambda Running the same computation as both a scheduled recomputation over all retained history and a streaming job over recent events, so that any defect in the fast path is erased by the next full recomputation.
Late-Data Policy Lateness Handling, Allowed Lateness practice Watermarks & Late Data The explicit decision about what happens to an event that arrives after its window has closed - drop it, route it aside, or update the emitted result.
Log Compaction concept CDC to Stream Retaining only the most recent value for each key in a topic, so the log becomes a durable snapshot of current state rather than a bounded window of history.
Micro-Batching Mini-Batch Streaming, Discretised Stream Processing pattern Streaming vs Batch Executing a stream as a sequence of small bounded jobs on a fixed trigger interval, which buys batch failure semantics and atomic sink commits at the price of a latency floor equal to the interval.
Online Training Loop Continuous Training, Streaming Model Updates pattern Feature Freshness Training a model continuously on interaction events as they arrive rather than on a scheduled batch, trading the ability to gate a model artefact before serving for the ability to react to feedback within minutes.
Partition Count Immutability Fixed Partition Count, Key-to-Partition Remap Hazard concept Partition Keys & Ordering The partition count of a keyed log is effectively permanent, because raising it changes which partition a key hashes to and therefore breaks per-key ordering and every table rebuilt by replaying that log.
Partition Key Skew Hot Partition, Key Skew concept Partition Keys & Ordering One partition receiving disproportionate traffic because a small number of key values dominate, capping throughput at what a single consumer can handle.
Poison Message concept Dead Letter Handling A record that a consumer cannot process and cannot skip, which halts its partition entirely until someone intervenes.
Primary-Key-Partitioned Upsert Upsert Partitioning Contract, Key-Routed Upsert pattern Real-Time Analytical Stores Routing every version of a record to one partition so a real-time store can resolve the latest value locally - which buys sub-second latest-value queries and fixes the partition count and key budget for the ta…
Provisional Result Restatement, Unsettled Aggregate concept Watermarks & Late Data An aggregate emitted before its window is definitively complete, which may be corrected when late events arrive - correct behaviour that destroys trust when it is not made visible.
Real-Time Analytics Store OLAP Serving Layer tool Real-Time Analytical Stores A database built for sub-second aggregate queries over freshly ingested data at high concurrency, serving user-facing analytics rather than internal reporting.
Real-Time Serving Layer pattern Real-Time Serving Layer The read path that answers queries in milliseconds from precomputed or continuously updated state — where the design question is what to compute when.
Rebalance Cost Partition Reassignment Duration, State Restore Time concept Stateful Stream Processing The time a stateful consumer group is unavailable while partitions are reassigned and state restored - proportional to state size, incurred on every deployment, and invisible in throughput and lag metrics.
Replay Effect Suppression Separating Computation From Effects, Safe Reprocessing practice Backfill & Reprocessing Separating a stream processor's computation from the components that act on its output, so that reprocessing a month of data does not re-send a month of notifications.
Replay Pipeline pattern Kappa vs Lambda A single processing path that produces both live and historical results by re-running the same code over retained input, replacing the two-path Lambda arrangement.
Replication Slot Retention Slot Lag, WAL Retention Pressure concept CDC to Stream The write-ahead log a database must keep because a change-data-capture consumer has not yet confirmed it, which turns a slow or stopped CDC pipeline into a disk-space failure on the primary database.
Reprocessing Window concept Backfill & Reprocessing How far back a pipeline can be replayed, set by the shortest retention anywhere along the path rather than by intent.
Retention Cost metric Streaming Cost The storage bill for keeping a log replayable, which is set by retention multiplied by throughput multiplied by the replication factor.
Rollup Policy Pre-Aggregation Policy, Retention Tiering practice Real-Time Analytical Stores The decision - made before ingestion, not after - about what granularity of data is retained for how long, and at what point raw events are collapsed into aggregates.
Schema Compatibility Mode concept Streaming Schema Evolution The rule stating which schema changes a registry will accept, which encodes whether producers or consumers are expected to upgrade first.
Semantic Schema Break Wire-Compatible Meaning Change concept Streaming Schema Evolution A change that keeps a field's wire format valid while changing what its values mean, so every compatibility check passes and every consumer computes a wrong answer without raising an error.
Serving Latency Budget concept Real-Time Serving Layer The end-to-end time from an event occurring to its effect being queryable, allocated across ingest, processing and serving.
State Time-to-Live State TTL, State Expiry practice Stateful Stream Processing An expiry policy on stream-processor state so that state size tracks the active key set rather than the accumulated history of everything the job has ever seen.
Stateful Stream Processing concept Stateful Stream Processing Stream computations that must remember information across events — aggregations, joins, deduplication, sessionisation — and the durability machinery that makes them recoverable.
Stream Enrichment pattern Streaming Joins Attaching reference data to a stream by lookup against a materialised table rather than by joining two unbounded streams.
Stream Processing Frameworks tool Stream Processing Frameworks Engines that process unbounded data with managed state, windowing and fault tolerance — chosen by state and time semantics rather than by throughput.
Stream-Table Duality concept Stream-Table Duality The equivalence between a stream of changes and a table of current state, where each can be derived from the other.
Streaming Cost Model concept Streaming Cost The cost structure of an always-on streaming system, where compute runs continuously regardless of volume and cross-zone data movement is a major line item.
Streaming Join concept Streaming Joins Combining two streams, or a stream and a table, where the records to be joined may not arrive at the same time or in the same order.
Streaming Schema Evolution practice Streaming Schema Evolution Changing the shape of events in a topic that has multiple independently deployed producers and consumers, plus retained history that still has to be readable.
Streaming vs Batch Decision practice Streaming vs Batch Choosing between continuous and periodic processing based on the decision latency the business actually requires, not on the appeal of real-time.
Sub-Second Aggregation Store tool Real-Time Analytical Stores A database built to ingest continuously and answer aggregate queries over recent data in milliseconds, occupying the gap between OLTP and the warehouse.
Table Bootstrap Lag Join Side Warm-Up, Lookup State Load Time concept Streaming Joins The interval after a stream job starts during which its lookup table is only partly loaded, so enrichment lookups miss for keys that appear late in the source topic and the job emits wrong output while looking…
Temporal Join As-Of Join, Point-in-Time Join pattern Streaming Joins A join that enriches an event with the reference value as it was when the event occurred, rather than as it is now - which is the difference between a reproducible result and a drifting one.