A modern service must read from a legacy system that offers only nightly batch files and a synchronous interface that cannot handle concurrent load. What integration approach fits?
Show the full answer Hide the answer
Why the intermediary is necessary
The legacy system imposes two hard constraints: it cannot serve concurrent load, and its only bulk path is nightly. New services need reasonable freshness and unbounded read concurrency. Those requirements are irreconcilable through direct calls.
An integration service owns the reconciliation. It reads from the legacy system at a rate the legacy system can tolerate, materialises the data into a store built for the read pattern, and exposes a modern interface. New services scale their reads against that store, not against the legacy system.
Why the alternatives fail
Direct synchronous calls. Every new service becomes coupled to the legacy system's availability, concurrency limits and data model. As new services multiply, load on the legacy system grows without bound. The first serious traffic event takes down a system that cannot be scaled.
Nightly batch only. Sometimes acceptable, often not — and it makes the worst freshness the only freshness, which forecloses use cases that would tolerate minutes.
Replace first. The most common way modernisation programmes stall. Nothing new ships until the legacy system is gone, so the business case never materialises and the programme is cancelled.
The design
1. Choose the extraction mechanism by what the legacy system permits, in order of preference: - Change data capture from the database if accessible — near-real-time, no load on the application, captures deletes. - Incremental extraction on a modified-timestamp if a suitable column exists, accepting that it misses deletes and has a boundary race. - The nightly batch as the baseline, supplemented by targeted synchronous reads for the small number of cases needing freshness.
2. An anti-corruption layer, non-negotiable. The new services must not inherit legacy naming, codes and structure. Translation happens here, or the "new" architecture becomes a distributed copy of the old one with extra network calls — the most common way strangler migrations fail.
3. A read store shaped for the new access pattern, with explicit and monitored freshness. Publish the staleness so consumers can reason about it rather than assume.
4. Writes handled separately and deliberately. If new services must write back, that path is synchronous, rate-limited, queued, and idempotent — because the legacy system will reject or fail under concurrency and retries must be safe.
5. Reconciliation. Periodically compare the materialised store against the source and alert on divergence, because in any asynchronous copy, drift is a certainty.
The strategic benefit
The integration service is also the seam for eventual replacement. When the legacy system is finally retired, only that service changes; the new services see the same interface throughout. That is the strangler pattern's essential move — establish the boundary first, so that replacement becomes an internal change rather than a coordinated migration of everything.