Search the practice set
275 questions, 991 terms and 600 topics in 30 areas.
60 results for “Data Platform Architecture”
Data Mesh
An organisational approach that gives domain teams ownership of their analytical data as a product, with a self-serve platform and federated governance.
Analytical Estate Topology
The full picture of where analytical data lands, is transformed and is served, including the paths that bypass the intended one.
Booking.com's Experimentation Platform
Booking.com runs over a thousand concurrent experiments and treats the ability to test any change safely as a platform capability rather than a product feature.
Capital One's Data Centre Exit
A major US bank closed all eight of its data centres and moved fully to public cloud, treating governance automation as the enabling technology rather than a constraint.
Change Data Capture
Publishing a stream of a database's row-level changes by reading its replication log, without modifying the application that owns it.
Control Plane and Data Plane
The separation between the machinery that makes changes to a system and the machinery that serves its traffic.
Cross-Zone Data Transfer
Charges incurred when data moves between availability zones within a region — invisible on architecture diagrams and a recurring surprise on cloud bills.
Data Lakehouse
A pattern that puts warehouse-style transactions, schema and governance on top of cheap open-format object storage.
Data Residency
A requirement that specific data be stored and sometimes processed only within a defined geography.
Data Retention Policy
A defined rule for how long each class of data is kept, where, and what happens at the end of it.
A business sponsor asks for a real-time data platform because "the competition has one". Reporting is currently a nightly batch that lands at 06:00 and nobody has complained. How do you handle this?
Do not answer the technology question "Real time platform" is a solution, and it has arrived without a problem attached. Answering it directly leads either to a
A new platform must serve a public partner API, three internal front-ends with different data needs, and high-volume service-to-service traffic. Choose the API styles and defend the choice.
Resist "pick one" These are three different problems with three different consumers. Standardising on one style optimises for architectural tidiness at the expe
A regulated client requires that no traffic between their data centre and your SaaS platform traverses the public internet. Design the connectivity and justify the cost.
Two distinct requirements hiding in one sentence Traffic must not traverse the public internet — a routing requirement. The client must be able to demonstrate i
Product wants to add a recommendation feature using browsing history. Legal asks for a data protection impact assessment. What does architecture need to supply?
What the assessment actually needs from architecture Legal cannot assess a feature description. They need the data facts, which only the design supplies: A data
Six weeks before launch, legal confirms that customer data for one market must be processed and stored in-country. The architecture is single-region in another jurisdiction. What do you do?
Establish exactly what the requirement covers Before designing anything, get three things in writing from legal: which data is in scope — usually a defined cate
You are asked to choose between a cloud data warehouse and a lakehouse for a new analytics platform. How do you decide?
What actually differs The gap has narrowed considerably, so the decision turns on fewer things than the marketing suggests. Format ownership. In a lakehouse, da
You are starting a greenfield platform. How do you establish what the architecture must satisfy before designing anything?
Separate three different things first Requirements can be traded against one another. Constraints cannot — a design violating one is not a trade off but a failu
A CDO proposes moving to a data mesh because the central data team is a bottleneck with a nine-month backlog. How do you assess the proposal?
Agree with the diagnosis, examine the prescription The bottleneck is real and it is structural rather than a matter of capacity. A central team receives data fr
An AI feature launched two months ago now costs more per month than the rest of the platform. What do you investigate?
Get cost per request, decomposed Token cost splits into input and output, and they price differently. Break the bill down by feature, by user, and by input vers
Data Platform Architecture
General material on designing the analytical data estate end to end.
Data Platform Tenancy
Multiple domains on shared storage and compute, with separable access and cost.
Data Vault Modelling
Hubs, links and satellites, and the auditability and load parallelism they buy.
Data Virtualisation
Querying across sources without moving data, and the performance ceiling that imposes.
Medallion Architecture
Bronze, silver and gold layers, and what each layer is allowed to guarantee.
Data Architecture
General material on structuring, storing and governing data.
Regulatory & Data Protection Architecture
General material on designing under legal and regulatory obligation.
Analytics Cost Control
Scanned bytes, idle warehouses, and the query nobody knew was running hourly.
Batch Orchestration
DAGs, dependencies, retries, and the difference between a schedule and an orchestration.
CDC Pipeline Design
Building on a change stream: snapshot plus delta, tombstones, and merge into the target.
Dimensional Modelling
Facts, dimensions, grain, and the star schema's continued relevance.
File Formats & Compaction
Columnar formats, the small-file problem, and the maintenance nobody schedules.
Ingestion Patterns
Full load, incremental, append-only and merge, and the source system each one suits.
Open Table Formats
Iceberg, Delta and Hudi — transactions, snapshots and time travel over object storage.
Reverse ETL
Pushing modelled analytical data back into operational systems, and who owns it then.
Slowly Changing Dimensions
Overwriting, versioning or timestamping attribute history, and the reporting each enables.
Storage Layout & Partitioning
Partition keys, clustering, and the scan the query planner is left able to skip.
Transformation Frameworks
Declarative SQL transformation with tests, lineage and versioned models.
Warehouse Migration
Moving off a legacy warehouse with thousands of reports pointed at it.
Warehouse, Lake & Lakehouse
Three answers to where analytical data lives, and the workloads that separate them.
Workflow Schedulers
Airflow, Dagster and their kin — where the control plane sits and what it can recover.
Workload Isolation
Keeping an analyst's query off the pipeline's compute, and both off the dashboard's.
Architecture Roles
Solution, enterprise, domain and platform architecture, and where each is accountable.
Change Data Capture
Turning a database's replication log into a stream, and its coupling risk.
Client Caching & Data Layer
Stale-while-revalidate, invalidation and optimistic updates on the client.
Cluster Architecture
How many clusters, split by what, and the blast radius each split buys.
Consent Architecture
Capturing, versioning and propagating consent to every system that acts on the data.
Data Classification
Knowing which fields are regulated, because every control depends on it.
Data Governance
Ownership, lineage, quality, catalogues and who may see what.
Data Lakes & Lakehouses
Open formats on object storage with transactional metadata on top.
Data Lifecycle & Retention
How long data is kept, where it ages to, and how it is actually deleted.
Data Mesh
Domain ownership, data as a product, self-serve platform, and federated governance.
Data Residency
Keeping data within a jurisdiction, including backups, logs and support access.
Data Subject Rights
Access, correction, portability and erasure across systems that never planned for them.
Data Platform Architecture
The analytical estate: storage layout, ingestion, transformation and the compute that reads it.
Data Architecture
Where state lives, how it is modelled, replicated, partitioned and governed.
Regulatory & Data Protection Architecture
The obligations that constrain a design before a single quality attribute is discussed.
AI-Era Architecture
AI workloads meeting security, data, networking, reliability and cost.