Chapter 24 — Collection Semantics across Clusters
Chapter objective: After this chapter, a developer can describe federation propagation, recovery, and ownership boundaries separately for Register, Lease, and CRDT, and reject claims of automatic global strong consistency.
Learning objectives
- Identify remote cluster, service, and locator with
FederationTarget. - Distinguish federation semantics for the three collection types.
- Interpret relay health and binding status.
- Identify the gap between an in-process baseline and production cross-region transport.
Prerequisites
- Collection choice, repair, and security gates from Chapters 9, 12, and 20 are complete.
- Intra-cluster replication and federation are separate failure domains.
- A cross-cluster link needs independent security, capacity, and recovery validation.
Case progress
The fulfillment team runs independent clusters in two regions. Route hints need cross-region visibility, shard owners are observed remotely, and request counters can merge. One vague "synchronization policy" cannot cover all three invariants.
Three collections cross one boundary with different semantics
| Type | Initial state | Live changes | Catch-up path | Cross-region boundary |
|---|---|---|---|---|
| Register | bootstrap current visible snapshot | forward put/remove | periodic digest repair; rebootstrap if needed | eventual convergence, no cross-region transaction |
| Lease | no bootstrap | forward LeaseSnapshot |
no anti-entropy sweep | remote read-only observation; no acquire/renew |
| CRDT | no bootstrap | forward committed delta | no federation sweep | depends on mergeable/idempotent delta and eventual delivery |
A remote Lease snapshot does not create cross-region ownership. Owner and fencing authority remain in the source cluster; the snapshot can support routing or operations observation, not permission for an external effect.
Target is complete identity, not a URL
FederationTarget contains remoteClusterId,
remoteServiceId, and remoteLocator. The relay
finds Runtime by cluster/service and collection by locator. A mismatch
at any layer fails explicitly; equal collection names do not imply equal
state.
Health qualifies confidence in propagation
A binding is SYNCING, HEALTHY,
DEGRADED, or DISCONNECTED. Forwarding stops
after relay failure; Register performs catch-up bootstrap after
recovery. Binding health says nothing about global confirmation of
business facts.
The current implementation boundary
InMemoryFederationRelay connects independent Runtimes in
one process. It executes real bridge semantics for three collections but
has no gRPC/HTTP, cross-host link, authentication, retry budget, or
production latency. A production relay implements bootstrap, put/remove,
repair, Lease snapshot, and CRDT delta, then separately satisfies the
security/capacity contracts from Chapters 20–22.
Counterexample and fault injection
- Set a wrong target service ID and observe failure to find the remote Runtime/collection.
- Write Register during relay failure and confirm queued/bootstrap catch-up on recovery.
- Try to infer Lease-acquire permission from a remote snapshot and identify the read-only violation.
- Drop a CRDT delta and note that current federation has no CRDT anti-entropy sweep.
Experiment
cd submodule/dsm
mvn -q -pl dsm-federation,dsm-integration-test -am \
-Dtest=InMemoryFederationBridgeTest,FederationIntegrationTest \
-Dsurefire.failIfNoSpecifiedTests=false testUse federation-contract.json to check bootstrap, live
delivery, repair, and ownership for each collection.
Experiment acceptance card
| Field | Content |
|---|---|
| Command | Federation unit/integration tests plus book assets |
| Input or fault | bootstrap, live mutation, relay failure/recovery, Lease snapshot, CRDT delta |
| Observable result | Register catches up; Lease remains observational; CRDT merges; status follows relay health |
| Evidence level | E2/E3: two independent Runtimes plus in-process relay |
| This experiment does not prove | Real cross-region transport, security, latency, bandwidth, complete history, or global transaction |
Review
- Federation is a cross-failure-domain replication contract, not an extended cluster.
- Register, Lease, and CRDT retain distinct semantics.
- A remote target includes cluster, service, and locator.
- The current relay is an executable baseline, not production networking.
Next
Chapter 25 asks which state should stay outside DSM and when a database, cache, event log, or consensus service fits better.