Portfolio ↗

Data Lakes & Lakehouses

Open formats on object storage with transactional metadata on top.

4Questions
11Flashcards
4Terms
Drill this topic All of Data Architecture
Questions

4 to work through

  1. intermediate Multiple choice

    A company is building a new analytics platform. Moderate data volume, heavy BI usage, strong governance requirements, a small data team. Warehouse or lakehouse?

    2 min answer
  2. intermediate Multiple choice

    A data platform runs interactive analyst queries alongside multi-hour batch pipelines on shared compute. Interactive latency becomes unpredictable. What is the architectural fix?

    2 min answer
  3. advanced Multiple choice

    A recommendation platform stores petabytes of interaction events used for both ad-hoc analysis and model training, with occasional need to delete individual users' data. Which storage architecture fits, and why does the deletion requirement drive the choice?

    2 min answer
  4. advanced

    You are asked to choose between a cloud data warehouse and a lakehouse for a new analytics platform. How do you decide?

    2 min answer
Terminology

4 terms in this topic

pattern

Lakehouse

Open table formats over object storage that add transactions, schema enforcement and incremental updates to a data lake.

 concept

Schema Evolution

Changing a table's structure over time while keeping existing data readable and existing consumers working.

 concept

Time Travel

Querying a table as it existed at a previous version or timestamp, made possible by keeping the metadata and files of prior commits.

 pattern

Workload Class Isolation

Running interactive, scheduled and batch analytical workloads on separate compute over shared storage, so that a long-running job cannot make an anal…
Data Architecture

Neighbouring topics

Data Architecture

General material on structuring, storing and governing data.

10 quiz 31 cards 25 terms

Relational Modelling

Normalisation, keys, constraints and the invariants a schema enforces.

3 quiz 10 cards 4 terms

NoSQL Stores

Key-value, document, wide-column and graph — what each buys and forbids.

5 quiz 13 cards 4 terms

Indexing

Designing indexes per query shape, and paying for them on every write.

5 quiz 12 cards 6 terms

Query Optimisation

Reading a plan, fixing statistics, and finding the real bottleneck.

3 quiz 11 cards 4 terms

Transactions & Isolation

ACID, isolation levels, and the anomalies each level permits.

3 quiz 10 cards 4 terms

Replication

Primaries, replicas, lag, and synchronous versus asynchronous durability.

4 quiz 14 cards 6 terms

Partitioning & Sharding

Splitting data across machines, and the one-way door of a partition key.

6 quiz 12 cards 10 terms

Caching Strategies

Cache-aside, read-through, write-through and where each belongs.

7 quiz 15 cards 6 terms

Cache Invalidation

Stampedes, penetration, staleness windows and versioned keys.

3 quiz 7 cards 2 terms

CQRS

Separating the write model from the read models that serve queries.

4 quiz 10 cards 5 terms

Event Sourcing

Storing the change log as the system of record, and what that costs forever.

5 quiz 13 cards 4 terms

Change Data Capture

Turning a database's replication log into a stream, and its coupling risk.

5 quiz 14 cards 5 terms

Data Warehousing

Dimensional modelling, star schemas and analytical workloads.

4 quiz 13 cards 3 terms

ETL & ELT

Where transformation happens, and how much raw history you keep.

3 quiz 9 cards 3 terms

Streaming Data

Windowing, watermarks, late arrivals and exactly-once semantics.

3 quiz 10 cards 2 terms

Data Governance

Ownership, lineage, quality, catalogues and who may see what.

3 quiz 9 cards 4 terms

Data Lifecycle & Retention

How long data is kept, where it ages to, and how it is actually deleted.

4 quiz 11 cards 5 terms

Polyglot Persistence

Choosing a store per workload, and the operational cost of variety.

6 quiz 12 cards 4 terms