Flipkart Marketplace Platform  ·  View 12 of 25  ·  4 · Data

Event Backbone

Who publishes, what the topic guarantees, who reacts, and where the result is finally readable.

Editable source SVG draw.io All views
Producer Topic Stream processing Consumers Materialised in Commerce Order service orders.v2 256 partitions Order state fan-out Flink Fulfilment, notify, care Order timeline Pinot Inventory Inventory service inventory.delta compacted Availability rollup Flink Search, PDP, Buy Box Availability cache Redis Payments Payment orchestrator payments.v1 PII redacted Settlement accrual Flink Order, settlement, risk Ledger entries Citus Catalog Catalog and pricing catalog.changes log compacted Enrich and embed Flink + AOAI Search indexer Search index OpenSearch Behaviour Apps and web clickstream.raw 4B/day Sessionise Flink Reco, ads, growth Feature store Feast Trust Every domain risk.signals Rules + model score Flink + KServe Case queue, blocks Risk decisions Citus Event Backbone — Who Publishes, Who Reacts Application we own Queue / topic Data store Security / platform Every topic carries a versioned Avro schema and an idempotency key. Compacted topics hold current state; append-only topics hold history and are the replay source for every derived store. v 1.0 · owner Platform Engineering · date 2026-09

Decisions

  • Self-managed Kafka on Strimzi rather than Event Hubs: partition counts, compaction, tiered storage and quota policy are all things this design needs to control
  • Append-only topics carry history and are the replay source; compacted topics carry current state
  • Every event has a versioned Avro schema and an idempotency key, because at-least-once delivery is the only realistic guarantee

Sizing

  • About 300 brokers, orders.v2 at 256 partitions sized for peak-hour fan-out
  • Consumer autoscaling on lag via KEDA, not on CPU
  • Dead-letter topics per consumer group with an operator-facing replay tool

Deliberately omitted

  • The other 32 topics; six domain families are shown as the pattern
  • Exactly-once semantics detail — Flink checkpointing with a transactional sink, described in the ADR
  • Cross-region mirroring, which is on the deployment view
◀  Data Flow All views Core Data Model  ▶
Solution Architecture v1.0 · Open source on Microsoft Azure · Enterprise Architecture · 2026-0912-event-backbone