No-Code SaaS Automation Platform  ·  View 09 of 21  ·  Data

Data Flow

A provider payload becomes an authoritative record twice, and everything after that is derived.

Editable source SVG draw.io All views
Source Provider payload untrusted Accept Authenticate, validate size and schema Deduplicate (conn, event id) Authority Trigger event log 30 d, replayable Execute Bind and call by reference Authority Step ledger 90 d, append only Projections Run history rebuildable Cost and analytics replay reject Data Flow — Payload to Projection External / third party Interface / broker Application we own Queue / topic Data store synchronous event / async failure / alternate Only two stores are authoritative. Everything right of the ledger can be dropped and rebuilt. v 1.0 · owner Integration Platform Architecture · date 2026-10

Decisions

  • Exactly two stores are authoritative. Everything right of the step ledger can be deleted and rebuilt, which is what makes a schema change to run history a routine operation.
  • Validation happens before the durable commit, so a payload that cannot be mapped is quarantined rather than poisoning the partition behind it.
  • Replay reads the trigger event log, never the provider — which is what makes the 30-day retention a recovery commitment rather than a storage preference.

Assumptions

  • Trigger payloads retained 30 days; step ledger 90 days hot and 400 days archived; run history 30 days free, 12 months paid.
  • RTO 2 h to rebuild the full run-history projection from the ledger, with author queries degraded but available during the rebuild.

Risks

  • Retaining 30 days of payloads makes the platform the custodian of a month of its customers' data from every connected product — the privacy cost of exact replay, and the subject of Core Architecture Question 5.
◀  Interface Catalogue All views Data Architecture  ▶
Solution Architecture v1.0 · Amazon Web Services · Integration Platform Architecture · 2026-10 · 21 views09-data-flow