Multi-system data flow design = which system owns what, how data syncs, what's the latency tolerance.
Step 1: Map data domains.
For each major entity (Account, Customer, Product, Order, Invoice):
- Source of truth — which system owns the canonical version?
- Consumers — which systems read it?
- Update paths — which systems can change it?
Step 2: Define sync patterns per flow.
- Real-time sync — sub-second latency. Use Pub/Sub API, CDC, Platform Events.
- Near-real-time — minutes. Mulesoft / iPaaS event-driven.
- Batch sync — hourly / daily. Bulk API, ETL.
- On-demand — pull when needed. External objects via Salesforce Connect.
- Event-driven — fire on state change. Platform Events + subscribers.
Pick based on: latency requirement, volume, reliability needs.
Step 3: Conflict resolution.
When multiple systems can update the same field:
- Source-of-truth wins — non-source updates are rejected or queued for SOR.
- Last write wins — simple but loses data.
- Merge logic — application-specific resolution.
- Avoidance — design so only one system writes per field.
Step 4: Idempotency.
Every receiving system must handle duplicate events. Use idempotency keys (External Id) to prevent double-processing.
Step 5: Failure handling.
For each integration:
- Retry policy (exponential backoff).
- Dead-letter queue for permanent failures.
- Alerting on threshold crossed.
- Monitoring dashboard.
Step 6: Testing.
- Unit tests per integration.
- End-to-end tests covering happy paths.
- Failure injection to test recovery.
Architectural patterns:
- Hub-and-spoke with iPaaS as hub.
- Mesh — each system integrates directly with others (gets messy at scale).
- Event-driven via message broker (Kafka, AWS SQS).
- CDC + warehouse for analytics-only consumers.
Performance:
- Caching at consumer side reduces source load.
- Eventual consistency acceptable for many flows.
- Strong consistency required for financial / audit flows.
Documentation:
Maintain an integration architecture diagram. For each flow:
- Source / target.
- Direction.
- Frequency.
- Trigger.
- Transformation.
- SLA.
- Owner.
Senior architect insight: data flow architecture is one of the most under-documented areas. Without an integration map, troubleshooting incidents becomes archaeology. Maintain the map proactively.
The most senior framing: don't ask "how do we sync these?" — ask "should we sync these at all?" Sometimes the right answer is "different systems own different concerns; reconcile only at the analytics layer".