Arc 3 Recap — Observable Replication, Partition, and Repair (Chapters 10–14)
Arc 3 expands "consistent after network recovery" into an observable process: where loss occurs, who detects it, how it is repaired, and why returned business responses still need external protection.
The path from membership to convergence
membership observation stabilizes
→ local mutation creates delta
→ online propagation attempts remote application
→ digest/anti-entropy detects a gap
→ replay or snapshot repairs it
→ another comparison confirms convergence
ChangeStream offers one observation surface, while its own bounded buffer can lose events. Final judgement returns to collection state, digest, and the business system of record.
Five boundaries
| Boundary | Common misconception | Working contract |
|---|---|---|
| membership | one join/leave event is a global fact | wait for consecutive stable views before high-risk quorum decisions |
| propagation | local success means remote write complete | record local commit and remote application separately |
| repair | more replay/snapshot is always safer | avoid regression; choose from retention and cost |
| subscription | ChangeStream is a durable event log | bounded buffer, explicit overflow, reread state when needed |
| response | one eventual winner revokes the loser's OK | transaction, idempotency, fencing, or compensation protects effects |
How this arc realizes DSM's value
Runtime reuses membership handling, online delta, difference detection, and replica repair across services. Business teams avoid rebuilding those pipelines for each service. DSM restores current coordination state; callers still interpret local success, protect external effects, and choose acceptable ChangeStream loss/failure behavior.
Fault-to-evidence map
| Fault | Minimal evidence | Still unproven |
|---|---|---|
| membership churn | stable-round reset; Lease returns
membership-unstable |
real-network threshold suitability |
| missed online delta | local success first; repair catches up after recovery | exactly-once history delivery |
| lagging requester | planner chooses replay/snapshot without regression | optimal large-scale cost |
| slow subscriber | four overflow outcomes and metrics | lossless business-event flow |
| network partition | autonomous/quorum Lease contrast | protection of external effects |
Six review questions
- Does the decision use visible membership, original cluster size, or stable size?
- When local success returns, which remote stage has completed?
- Who detects a missed online delta?
- How does snapshot take over safely when replay is unavailable or costlier?
- When a subscription buffer fills, does it drop, wait, or fail, and which metric records it?
- Did local success during a partition trigger an irreversible effect?
Migration exercise
Replace the route hint with a feature-rollout flag. Record its authority, locator, impact of two local OK responses during a partition, repair mode, ChangeStream overflow policy, and categories of flags excluded from DSM.
Next
Arc 4 wraps collections and failure boundaries in business ports, integrates Spring Boot, and protects engineering boundaries with two-layer identity isolation and layered test evidence.