carlos / platform Public

Found by Astra in joint round 7 of the moves and host-lifecycle claim review (/home/paulca/.cache/nearest-region/CODEX-FINDINGS-JOINT-R7.md, R7-1). The moves session's model first raised it. It is not specific to moves: it applies whenever a second box restores a hibernating instance's replica while the first box's upload may still be in flight.

Schedule:

1. The replica is restorable through TXID N.
2. Process A starts a PUT for L0 [N+1,N+1]. The request is outstanding when A dies or loses its lease (box death, network partition, freeze).
3. B takes the lease, cold-restores through N, accepts writes, and uploads its own [N+1,N+1].
4. A's PUT completes afterwards. litestream v0.5.14's S3 client PUTs unconditionally (no If-Match, If-None-Match or token), to a key made only of prefix, level and TXID range, so A's bytes replace B's.
5. The next cold restore reads A's file and loses B's acknowledged writes. Both files pass their CRC.

A second schedule: A's file lands after B initializes its local position but before Replica.Sync computes the remote position. B then treats N+1 as already uploaded and skips its own.

The lease (lock.json) cannot revoke a data PUT that was already accepted, so waiting out the lease TTL does not prove old requests have finished.

Fix direction (the moves spec rev 7.8 and claim will specify it): any recovery that loses replication state starts a fresh, durably selected generation or writer epoch, published under the claim before writes are admitted, with every L0, compaction, snapshot and retention operation confined to that epoch's prefix. Conditional PUTs alone are not enough: delete markers re-enable creation, multipart needs the same condition, and the remote-position shortcut must check identity.

Exposure today: shared-pool lease failover between boxes in one region. How often it happens depends on how often a box dies with an upload in flight and another box wakes the instance before that request finishes. Worth checking the S3 SDK's retry and timeout bounds in internal/lease, since they cap how late a PUT can land.