advanced 2 min answer

A fintech in the mould of Razorpay has 6 analysts, 2 TB of transactional and event data, one data engineer and a target of dashboards plus occasional ML features. The CTO asks you to plan a "lakehouse with Iceberg, Spark and a catalogue". How do you answer in the room?

lakehousewarehousesmall teambuild vs buyoperational burdeniceberg
Show the full answer Hide the answer

What the interviewer is testing

Whether you can say no to sophistication with evidence, and propose what would justify yes. The question is a trap for candidates who match "data platform" to the most modern stack.

Clarifying questions that change the answer

  • Growth and shape: does 2 TB become 20 TB in a year? Is most of it structured, or logs and documents?
  • Consumers: are the ML workloads training on raw files, or are features computed in SQL?
  • Engines: does any consumer need a non-SQL engine, or multiple engines against the same data?
  • Compliance and cost: any residency rule, and what is the current warehouse bill?
  • Team: who is on call for a Spark cluster at 2 a.m., given one engineer?

The arc of a strong answer

At 2 TB, SQL users and one engineer, a managed warehouse is almost certainly the right default. Storage and compute are billed together, there is no cluster to size, the catalogue is built in, and a single engineer can run the whole stack. A rough check: 2 TB scanned a few hundred times a month costs low hundreds of dollars on on-demand pricing (2024-era list prices were around $5 to $6.25 per TB scanned), a rounding error next to an engineer's salary.

The lakehouse earns its place when specific conditions hold: multiple engines need the same tables, data volume reaches the point where warehouse storage dominates cost, unstructured or ML data must sit beside tables, or vendor exit is a stated requirement. Open table formats on object storage also give a migration path that warehouse-native tables do not.

I would propose: managed warehouse now, land raw data in object storage in an open format (cheap and replayable), and revisit when a named trigger fires.

Common weak answers

  • "Lakehouse is the modern standard." It is a trade of operating effort for flexibility.
  • "Warehouse lock-in is dangerous." Real, but a landing zone in open formats removes most of it for little effort.
  • "Spark can do everything." It can, and one engineer cannot run it and everything else.

What a strong answer adds

The triggers written down: a monthly warehouse bill above a stated threshold, a second engine requirement, a table above ~50 TB. Also the cost of waiting: low, because the raw landing zone preserves the option. And an honest estimate of operational burden: a lakehouse adds a catalogue, compaction, snapshot expiry and access control to someone's job.

When the lakehouse answer is right

A platform team of five or more, several engines (SQL, Spark, a vector or ML job) sharing governed tables, tens of terabytes, and a cost curve where warehouse storage and egress dominate. There, owning the table format is how you avoid being priced by one engine.