amadan
design: self-serve offline moves, rev 6 (Astra round 5)
Reconciliation writes the record's move_seq with any ready row it mints after a finished or aborted move, so a rebuilt row outranks a stale holding entry; a per-record move_lineage separates a recreated host's publications from a previous life's; the table orders reservation, queued record, reports, staging; the unconditional ready-route precedence and the 'data is live' wording are gone.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
1 file changed,
+8
−7
docs/superpowers/specs/2026-10-21-self-serve-offline-moves-design.md+8 −7
CI passed — run details
CI log
=== gate ===
.amadan/test-gate.sh
=== launchctx (0 s)
ok amadan.net/carlos/platform.git/launchctx 0.015s
? amadan.net/carlos/platform.git/launchctx/controlname [no test files]
? amadan.net/carlos/platform.git/launchctx/wireid [no test files]
=== vet (1 s)
=== gofmt (0 s)
gofmt -l .: clean
=== rest (139 s)
? github.com/carlosframework/platform/cmd/changelog [no test files]
ok github.com/carlosframework/platform/cmd/console 3.588s
ok github.com/carlosframework/platform/cmd/moveprobe 11.915s
ok github.com/carlosframework/platform/cmd/zddprobe 0.734s
ok github.com/carlosframework/platform/internal/activator 59.444s
ok github.com/carlosframework/platform/internal/agentsock 0.026s
ok github.com/carlosframework/platform/internal/alerting 0.004s
ok github.com/carlosframework/platform/internal/analytics 0.023s
ok github.com/carlosframework/platform/internal/appconfig 2.891s
ok github.com/carlosframework/platform/internal/apperr 0.179s
ok github.com/carlosframework/platform/internal/applog 0.239s
ok github.com/carlosframework/platform/internal/awsinv 0.006s
ok github.com/carlosframework/platform/internal/boxchoice 0.006s
ok github.com/carlosframework/platform/internal/boxid 0.993s
? github.com/carlosframework/platform/internal/boxid/boxidtest [no test files]
? github.com/carlosframework/platform/internal/boxid/internal/signtest [no test files]
ok github.com/carlosframework/platform/internal/boxvolumes 0.008s
ok github.com/carlosframework/platform/internal/caps 0.024s
ok github.com/carlosframework/platform/internal/certwarm 0.004s
ok github.com/carlosframework/platform/internal/cgroup 0.746s
ok github.com/carlosframework/platform/internal/changelog 0.509s
ok github.com/carlosframework/platform/internal/claimactivation 1.174s
ok github.com/carlosframework/platform/internal/clamd 0.056s
? github.com/carlosframework/platform/internal/cliflag [no test files]
? github.com/carlosframework/platform/internal/controlname [no test files]
? github.com/carlosframework/platform/internal/controlsock [no test files]
ok github.com/carlosframework/platform/internal/dbdoor 6.034s
ok github.com/carlosframework/platform/internal/dbdoor/dbdoortest 0.012s
ok github.com/carlosframework/platform/internal/dbverify 0.005s
ok github.com/carlosframework/platform/internal/dnsrec 0.002s
ok github.com/carlosframework/platform/internal/docsite 0.010s
ok github.com/carlosframework/platform/internal/domains 2.928s
ok github.com/carlosframework/platform/internal/doorkey 0.007s
ok github.com/carlosframework/platform/internal/economics 0.009s
ok github.com/carlosframework/platform/internal/edge 45.140s
? github.com/carlosframework/platform/internal/edgeproof [no test files]
ok github.com/carlosframework/platform/internal/edgesfile 0.012s
ok github.com/carlosframework/platform/internal/fleetapi 0.230s
ok github.com/carlosframework/platform/internal/gate 1.789s
ok github.com/carlosframework/platform/internal/genselect 0.019s
ok github.com/carlosframework/platform/internal/geo 0.016s
ok github.com/carlosframework/platform/internal/geoip 0.038s
ok github.com/carlosframework/platform/internal/historyreadback 0.267s
ok github.com/carlosframework/platform/internal/host 136.317s
ok github.com/carlosframework/platform/internal/hostactors 6.454s
ok github.com/carlosframework/platform/internal/hostactors/awsmembership 0.004s
ok github.com/carlosframework/platform/internal/hostbox 3.183s
ok github.com/carlosframework/platform/internal/hostclaim 0.826s
ok github.com/carlosframework/platform/internal/hostenv 0.007s
ok github.com/carlosframework/platform/internal/hostevidence 0.010s
ok github.com/carlosframework/platform/internal/hostexec 5.545s
ok github.com/carlosframework/platform/internal/hostfence 0.032s
ok github.com/carlosframework/platform/internal/hostfiles 10.213s
ok github.com/carlosframework/platform/internal/hostfloor 0.018s
ok github.com/carlosframework/platform/internal/hostguard 1.081s
ok github.com/carlosframework/platform/internal/hostops 2.287s
? github.com/carlosframework/platform/internal/hostops/fencerelease [no test files]
ok github.com/carlosframework/platform/internal/hostops/internal/fencerelease 0.004s
ok github.com/carlosframework/platform/internal/hostquarantine 0.004s
ok github.com/carlosframework/platform/internal/hostrelocate 0.665s
ok github.com/carlosframework/platform/internal/hostresolve 0.222s
ok github.com/carlosframework/platform/internal/hostwrite 0.002s
ok github.com/carlosframework/platform/internal/infraview 0.004s
ok github.com/carlosframework/platform/internal/instancectl 5.246s
ok github.com/carlosframework/platform/internal/instanceexec 2.804s
ok github.com/carlosframework/platform/internal/instancefs 0.009s
ok github.com/carlosframework/platform/internal/instancekey 0.004s
ok github.com/carlosframework/platform/internal/instanceupdates 0.009s
ok github.com/carlosframework/platform/internal/layout 1.946s
ok github.com/carlosframework/platform/internal/lbhealth 0.007s
ok github.com/carlosframework/platform/internal/lease 10.218s
ok github.com/carlosframework/platform/internal/lease/leasetest 0.550s
ok github.com/carlosframework/platform/internal/ledger 0.018s
ok github.com/carlosframework/platform/internal/mail 0.216s
ok github.com/carlosframework/platform/internal/maintenance 0.080s
ok github.com/carlosframework/platform/internal/maintenanceexec 5.610s
ok github.com/carlosframework/platform/internal/maintenanceguard 0.093s
ok github.com/carlosframework/platform/internal/metrics 0.005s
ok github.com/carlosframework/platform/internal/moves 21.533s
ok github.com/carlosframework/platform/internal/moves/archive 1.150s
ok github.com/carlosframework/platform/internal/objstore 0.284s
ok github.com/carlosframework/platform/internal/ops 0.202s
ok github.com/carlosframework/platform/internal/park 0.275s
ok github.com/carlosframework/platform/internal/plans 0.005s
ok github.com/carlosframework/platform/internal/privatefile 0.003s
ok github.com/carlosframework/platform/internal/provision 13.399s
ok github.com/carlosframework/platform/internal/publicrelease 0.011s
? github.com/carlosframework/platform/internal/regionname [no test files]
ok github.com/carlosframework/platform/internal/registry 5.637s
ok github.com/carlosframework/platform/internal/relay 7.249s
ok github.com/carlosframework/platform/internal/release 24.614s
ok github.com/carlosframework/platform/internal/rollout 3.810s
ok github.com/carlosframework/platform/internal/rolloutwords 0.007s
ok github.com/carlosframework/platform/internal/rootaws 0.008s
ok github.com/carlosframework/platform/internal/rootparticipation 0.068s
ok github.com/carlosframework/platform/internal/rootstate 0.030s
ok github.com/carlosframework/platform/internal/runs 0.643s
ok github.com/carlosframework/platform/internal/s3compat 0.005s
ok github.com/carlosframework/platform/internal/safefile 0.007s
ok github.com/carlosframework/platform/internal/schedule 2.828s
ok github.com/carlosframework/platform/internal/seeds 0.011s
ok github.com/carlosframework/platform/internal/sim/hostlife 10.901s
ok github.com/carlosframework/platform/internal/sim/litestreamfacts 3.536s
ok github.com/carlosframework/platform/internal/status 0.086s
ok github.com/carlosframework/platform/internal/steering 0.009s
ok github.com/carlosframework/platform/internal/stopevidence 0.229s
ok github.com/carlosframework/platform/internal/svcrole 0.002s
ok github.com/carlosframework/platform/internal/telemetry 0.008s
ok github.com/carlosframework/platform/internal/termfence 0.005s
ok github.com/carlosframework/platform/internal/tunnelclient 0.006s
ok github.com/carlosframework/platform/internal/unixdial 0.009s
ok github.com/carlosframework/platform/internal/updatewords 0.010s
ok github.com/carlosframework/platform/internal/usage 0.007s
ok github.com/carlosframework/platform/internal/version 0.002s
ok github.com/carlosframework/platform/internal/vet 1.659s
=== internal_console@0 (77 s)
ok github.com/carlosframework/platform/internal/console 75.778s
=== internal_console@1 (37 s)
ok github.com/carlosframework/platform/internal/console 34.902s
=== internal_console@2 (42 s)
ok github.com/carlosframework/platform/internal/console 39.939s
=== internal_console@3 (37 s)
ok github.com/carlosframework/platform/internal/console 35.617s
=== internal_console@4 (37 s)
ok github.com/carlosframework/platform/internal/console 34.948s
=== internal_console@5 (35 s)
ok github.com/carlosframework/platform/internal/console 33.117s
=== internal_console@6 (36 s)
ok github.com/carlosframework/platform/internal/console 34.091s
=== internal_console@7 (43 s)
ok github.com/carlosframework/platform/internal/console 40.809s
=== root@0 (32 s)
ok github.com/carlosframework/platform 30.359s
=== root@1 (44 s)
ok github.com/carlosframework/platform 42.221s
=== root@2 (41 s)
ok github.com/carlosframework/platform 39.333s
=== cross ===
--- shard 0 of 5 ---
cross-compiling darwin/arm64
--- shard 1 of 5 ---
cross-compiling darwin/amd64
--- shard 2 of 5 ---
cross-compiling linux/arm64
linux/arm64 carlos binary: 50069666 bytes (ceiling 52000000)
--- shard 3 of 5 ---
cross-compiling windows/amd64
--- shard 4 of 5 ---
cross-compiling windows/arm64
=== minio-contract ===
minio contract: arm64, minio RELEASE.2025-09-07T16-13-09Z, mc RELEASE.2025-08-13T08-35-41Z
minio: cached, digest verified
mc: cached, digest verified
minio healthy on 127.0.0.1:20999
Added `local` successfully.
Bucket created successfully `local/contract-bucket`.
Bucket created successfully `local/denied-bucket`.
Created policy `contract-only` successfully.
Added user `scopeduser` successfully.
Attached Policies: [contract-only]
To User: scopeduser
buckets and scoped user ready
=== RUN TestDeliveredContractMinio
=== RUN TestDeliveredContractMinio/put
=== RUN TestDeliveredContractMinio/get_round-trips_the_body
=== RUN TestDeliveredContractMinio/list_shows_the_key
=== RUN TestDeliveredContractMinio/head_and_list_agree_on_the_version
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_its_own_object
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed
=== RUN TestDeliveredContractMinio/delete_removes_it
=== RUN TestDeliveredContractMinio/cross-bucket_denial
--- PASS: TestDeliveredContractMinio (0.02s)
--- PASS: TestDeliveredContractMinio/put (0.00s)
--- PASS: TestDeliveredContractMinio/get_round-trips_the_body (0.00s)
--- PASS: TestDeliveredContractMinio/list_shows_the_key (0.00s)
--- PASS: TestDeliveredContractMinio/head_and_list_agree_on_the_version (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route (0.01s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed (0.00s)
--- PASS: TestDeliveredContractMinio/delete_removes_it (0.00s)
--- PASS: TestDeliveredContractMinio/cross-bucket_denial (0.00s)
PASS
ok github.com/carlosframework/platform/internal/objstore 0.023s
=== RUN TestT4StorageProbeMinio
2026/10/21 22:44:16 moves: T4: STORE: list: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: 18E0ACCED9F7D52F, HostID: 7c420053e0fbcd7f41653fdd92ffa2802c903d05303b1aa15adc2e8bb65d183f, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
--- PASS: TestT4StorageProbeMinio (0.01s)
PASS
ok github.com/carlosframework/platform/internal/host 0.021s
minio contract: PASS
diff --git a/docs/superpowers/specs/2026-10-21-self-serve-offline-moves-design.md b/docs/superpowers/specs/2026-10-21-self-serve-offline-moves-design.md| index 5db176be2..498335db0 100644 |
| --- a/docs/superpowers/specs/2026-10-21-self-serve-offline-moves-design.md |
| +++ b/docs/superpowers/specs/2026-10-21-self-serve-offline-moves-design.md |
| @@ -1,6 +1,6 @@ |
| # Self-serve offline moves: a tenant's customer moves their own instance |
| -Rev 5, 2026-10-21. Status: design for review (Paul). Branch `design-self-serve-offline-moves`. Rev 1 (062387a1) was reviewed by Astra (`after236/selfserve-spec-review.md`, NOT READY, 14 findings) and by a cold builder check (7 questions); rev 2 (0e58e51d) answered them and was reviewed again (`selfserve-spec-review2.md`, NOT READY, 12 findings). Rev 2's automatic return after `open` was the source of most of round 2: whether the new region ever accepted a write cannot be proven from the bucket (litestream releases the lease even when its final sync failed; file-only changes leave no LTX; an absent lock does not tell "never taken" from "released"), and a safe post-open revert would need a new admission barrier and stop protocol for the target, which is Phase 1's work. **Rev 3 narrows the promise to what is provable**: every failure before `open` returns the account to the old region automatically and losslessly, everything checkable is checked before `open`, and a failure after `open` is paged with the holding page kept until the app answers, the return being an operator's run of the same offline move the other way. This changes one of Paul's rulings and is §10 Q3. Rev 3 (b77c01a9) was reviewed again (`selfserve-spec-review3.md`, NOT READY, 8 findings, the abort boundary accepted); rev 4 answers them: which failures stay after `open` (§7.5), the job's durable admission order (§7), cleanup's own reservation and the root handoff (§7.6), the holding entries' causal ordering (§6.4), the probe mode (§7.4), deadlines as initiate-and-retry with blocked states (§7.2), and the stale contradictions. Rev 4 (20fdcf18) was reviewed again (`selfserve-spec-review4.md`, NOT READY, 2 P1 and 2 P2: holding entries ordered by move identity cannot order consecutive moves or show an abort completed; a timed-out root cleanup request could run after its reservation was released; the queued handshake's producers; stale text). Rev 5 answers them with a per-host `move_seq` published with every route and holding entry (§6.4), a root request that ends only with root's `done` or acknowledged `cancelled` (§7.6), queued defined as no copying with the participants' reports allowed (§7), and the stale passages aligned. |
| +Rev 6, 2026-10-21. Status: design for review (Paul). Branch `design-self-serve-offline-moves`. Rev 1 (062387a1) was reviewed by Astra (`after236/selfserve-spec-review.md`, NOT READY, 14 findings) and by a cold builder check (7 questions); rev 2 (0e58e51d) answered them and was reviewed again (`selfserve-spec-review2.md`, NOT READY, 12 findings). Rev 2's automatic return after `open` was the source of most of round 2: whether the new region ever accepted a write cannot be proven from the bucket (litestream releases the lease even when its final sync failed; file-only changes leave no LTX; an absent lock does not tell "never taken" from "released"), and a safe post-open revert would need a new admission barrier and stop protocol for the target, which is Phase 1's work. **Rev 3 narrows the promise to what is provable**: every failure before `open` returns the account to the old region automatically and losslessly, everything checkable is checked before `open`, and a failure after `open` is paged with the holding page kept until the app answers, the return being an operator's run of the same offline move the other way. This changes one of Paul's rulings and is §10 Q3. Rev 3 (b77c01a9) was reviewed again (`selfserve-spec-review3.md`, NOT READY, 8 findings, the abort boundary accepted); rev 4 answers them: which failures stay after `open` (§7.5), the job's durable admission order (§7), cleanup's own reservation and the root handoff (§7.6), the holding entries' causal ordering (§6.4), the probe mode (§7.4), deadlines as initiate-and-retry with blocked states (§7.2), and the stale contradictions. Rev 4 (20fdcf18) was reviewed again (`selfserve-spec-review4.md`, NOT READY, 2 P1 and 2 P2: holding entries ordered by move identity cannot order consecutive moves or show an abort completed; a timed-out root cleanup request could run after its reservation was released; the queued handshake's producers; stale text). Rev 5 answers them with a per-host `move_seq` published with every route and holding entry (§6.4), a root request that ends only with root's `done` or acknowledged `cancelled` (§7.6), queued defined as no copying with the participants' reports allowed (§7), and the stale passages aligned. Rev 5 (89e0f37a) was reviewed again (`selfserve-spec-review5.md`, NOT READY, 1 P1 and 2 P2: a serving row rebuilt by ordinary reconciliation after a finished move would carry sequence 0 and lose to a stale holding entry, and a recreated host's publications need separating from a previous life's; the queued table order; two stale sentences). Rev 6 answers them: reconciliation persists the record's sequence with a row it mints after a move (§6.4), a per-record `move_lineage` separates lives (§6.4), the table order and the two sentences are fixed. |
| Builds on the offline operator move (`docs/superpowers/plans/2026-10-20-offline-operator-move.md`, shipped in v0.28.5 to v0.28.7) and its runbook (`docs/superpowers/plans/2026-10-21-offline-move-runbook.md`). Supersedes nothing: Phase 1 (`docs/superpowers/specs/2026-10-02-move-any-app-design.md`) stays the live-move path for claims-on deployments, and the M1 to M18 rulings stand. M18 ("Tito waits; move mutations are operator-only") is answered here for one door, not lifted: §5 says exactly what the customer's request may do. |
| @@ -116,10 +116,11 @@ The legacy move doors keep their paths and roles. With `CARLOS_SELF_SERVE_MOVES= |
| From `start` until the target's row is ready, the host is held on every source box and has no ready row anywhere. Today every edge answers a platform-domain host with the "not here yet" 503 retry page (`internal/edge/pending.go`) and a customer domain with 404. This design adds a maintenance answer, carried on the existing box to edges-file to edge channel: |
| - **Publication.** Every box that holds the host (every source-region box; and the target box from `flip` until its row is ready) lists it in a new edges-file field, `offline_moves: [{host, aliases, move_id, serving_seq, offline_at, estimate_seconds, display_name}]`, and every route in `routes` gains `serving_seq`; separate from `routes` so no decoder counts it as served (the seal door's "still listed" check reads `routes`; the builder audits every edges-file reader). `aliases` are exactly the alias rows the edges writer would publish for the host as a served route, under the same eligibility rules (DNS pointing, ownership, wildcard handling), so every custom domain that would reach the account gets the page and no other. A box stops publishing the entry when the record no longer carries that move id or when its own row for the host is ready. |
| -- **Rendering.** An edge answers a request for a host or alias whose highest-sequence publication is a holding entry (the ordering rule below) with 503, `Retry-After: 30`, `noindex`, and the holding page: "`<display_name>` is temporarily offline. Please check back in approximately N minutes." with N from `until` and the estimate, counting down; without a display name, "This account is temporarily offline. ...". A ready route anywhere wins over a holding entry (precedence), so the window between `open` and the first edges-file refresh is the pending page's known 503-then-retry window and nothing worse. The source box's own edge answers its held rows the same way from its registry (the home map excludes a box's own publication). The page reuses the pending page's safe reload mechanics (location.replace, sessionStorage) with its give-up replaced by `until`. |
| +- **Rendering.** An edge answers a request for a host or alias whose highest-sequence publication is a holding entry (the ordering rule below) with 503, `Retry-After: 30`, `noindex`, and the holding page: "`<display_name>` is temporarily offline. Please check back in approximately N minutes." with N from `until` and the estimate, counting down; without a display name, "This account is temporarily offline. ...". The highest-sequence rule below decides between a ready route and a holding entry; the window between `open` and the first edges-file refresh of the target's publication is the pending page's known 503-then-retry window and nothing worse. The source box's own edge answers its held rows the same way from its registry (the home map excludes a box's own publication). The page reuses the pending page's safe reload mechanics (location.replace, sessionStorage) with its give-up replaced by `until`. |
| - **TLS.** A host or alias in a holding entry is allowed for TLS exactly as it would be as a served route (the same `Allowed` decision, wildcard and mirrored certificates included), so the page is reachable on HTTPS; the entry never widens what the host could already present. |
| - **The display name.** Tito supplies it per request; the console validates it (at most 80 characters, printable, no control characters, no newlines) and stores it in the move; the edge renders it through contextual template escaping. It is the one piece of request-supplied text the page shows, by design (the customer named their own account), and it is shown to unauthenticated visitors, as the account's own pages are. Nothing else from the request, no header, path, diagnostic or `requested_by`, reaches the page. |
| -- **Ordering by a per-host sequence.** The record gains `move_seq`, a per-host counter the console initialises to 1 on a host's first offline move and increments on every offline transition that changes who serves the host: `start` (the hold), `open` (the target serves), the abort's reopening of the source, and `finish`. Every box persists on its registry row the `move_seq` of the transition that produced the row's current state (`serving_seq`: the hold's for a held row, `open`'s for a landed ready row, the abort's for a reopened source row; a row never touched by a move carries 0), and publishes it with the route and with any holding entry. An edge orders a host's publications by `serving_seq` and serves the highest: a holding entry (the hold's sequence) beats the source's stale ready publication (lower); the target's ready route after `open` beats the holding entries (higher); a source reopened by an abort publishes the abort's sequence, higher than the hold's, so it beats every holding entry of the aborted move still retained as last-good elsewhere; the next move's hold is higher again. Ties cannot occur between a ready route and a holding entry (one transition produces one state per box); a tie between two ready routes falls to the existing home-map rule. Nothing is inferred from move ids. The abort's reopening is therefore its own completion evidence at the edges, independent of the target's recovery report, which the finish gate does not require. |
| +- **Ordering by a per-host sequence.** The record gains `move_seq`, a per-host counter the console initialises to 1 on a host's first offline move and increments on every offline transition that changes who serves the host: `start` (the hold), `open` (the target serves), the abort's reopening of the source, and `finish`. Every box persists on its registry row the `move_seq` of the transition that produced the row's current state (`serving_seq`: the hold's for a held row, `open`'s for a landed ready row, the abort's for a reopened source row), written in the same row write as the status, and publishes it with the route and with any holding entry. **Rows minted outside a move:** when ordinary reconciliation mints or re-mints a ready row for a host from a record that carries no move (a rebuilt registry, a box returning from an outage, a row removed and restored), it writes `serving_seq` = the record's current `move_seq` (the last finished or aborted move's final value) in the same write; a record with no `move_seq` (a host with no move history) gives 0. Zero therefore means "no move history", never "a new row". Observing a newer record does not by itself move a row's sequence; only applying a transition, or minting the row from the record, does. |
| +- **Lineage.** The record also carries `move_lineage`, a random id the console sets together with `move_seq` at the host's first offline move; a host deleted and recreated gets a new record and, at its first move, a new lineage. Routes and holding entries publish the lineage with the sequence. An edge orders a host's publications only within one lineage, and ignores a holding entry whose lineage differs from the lineage of any fresh ready route for the host (a previous life's entry, retained as last-good by a publisher that has not refreshed). Publications without a lineage (no move history) and a holding entry cannot coexist for one host, because the first move sets the lineage on the record before any box holds. An edge orders a host's publications by `serving_seq` and serves the highest: a holding entry (the hold's sequence) beats the source's stale ready publication (lower); the target's ready route after `open` beats the holding entries (higher); a source reopened by an abort publishes the abort's sequence, higher than the hold's, so it beats every holding entry of the aborted move still retained as last-good elsewhere; the next move's hold is higher again. Ties cannot occur between a ready route and a holding entry (one transition produces one state per box); a tie between two ready routes falls to the existing home-map rule. Nothing is inferred from move ids. The abort's reopening is therefore its own completion evidence at the edges, independent of the target's recovery report, which the finish gate does not require. |
| - **Freshness.** The home map's existing behaviour stands: publications are refreshed on the edges' own cadence, unchanged publications are not rewritten, and a last-good publication is kept through a read failure. A holding entry therefore lives exactly as long as its publisher lists it, which is while the record carries the move and the publisher's row is not ready; there is no age rule and no fixed expiry. The displayed estimate is separate from validity: past `offline_at + estimate` the page says "...is taking longer than expected. Please check back in a few minutes." and keeps retrying. |
| - **Reopening.** The public reopens when the target's row is ready and its publication, at `open`'s sequence, reaches the edges: visitors' first requests wake the app as any cold wake does. The console's health check (§7.4) is detection, not a gate on reopening. A failure after that is visible to visitors as the app's own answers, as any instance failure is. |
| - **The health probe** (§7.4) goes to the target box directly, by its address from its edges file with the host as SNI, so it reaches the app whatever the edges show. |
| @@ -156,8 +157,8 @@ Every wait has a deadline derived from the estimate's terms with a floor; a stag |
| | phase (view) | waits for | deadline | on expiry | |
| |---|---|---|---| |
| | reserving (`preparing`) | the participants' reservations | none; `reason` says waiting | 24 h, then abort "not started in time" | |
| -| lane (`preparing`) | the lane probe report (§7.3) | 5 min | abort, reason names the lane | |
| | queued (`preparing`) | the record write of `offline-queued` landing | one tick | reread by id | |
| +| reports (`preparing`) | the source's lane report and the target's initial-cleanliness and target-ready report, both produced against the queued record (§7.3) | 5 min | abort, reason names the lane or the fact | |
| | staging (`preparing`) | a matching, successful round under the ready rule | 30 min or 10 rounds (§6.2) | ready only from a completed successful round (the last one); a failed or still-running round is not ready: wait for it, then abort if it failed | |
| | pre-checks (`preparing`) | target-ready and source reports | 5 min | abort, reason names the check | |
| | ready | the cut | 24 h | abort "not confirmed in time" | |
| @@ -167,7 +168,7 @@ Every wait has a deadline derived from the estimate's terms with a floor; a stag |
| | landing (`cutting`) | the landing report with every item OK | max(10 min, 3 × landing estimate) | abort | |
| | opening (`cutting`) | the target's opened report (the existing finish gate needs it; a ready row alone is not enough) | max(3 min, 2 × open estimate) | blocked: page; stay in opening; the record names the target; retry each tick; an operator fixes the missing prerequisite (§7.5) | |
| | health (`cutting`) | two passes of §7.4 | max(3 min, 2 × promotion estimate) + 30 s | go to finishing with `last.warning`; page (§7.5) | |
| -| finishing (`cutting`) | the existing two finish gates in turn: from `offline-open`, the target's opened report and every source box's parked report; then from `offline-finishing`, the sealer's lifted report and every source box's gone report | 10 min each | blocked: page; retry each tick; the data is live on the target; an operator fixes the missing report, never bypasses a gate | |
| +| finishing (`cutting`) | the existing two finish gates in turn: from `offline-open`, the target's opened report and every source box's parked report; then from `offline-finishing`, the sealer's lifted report and every source box's gone report | 10 min each | blocked: page; retry each tick; the record names the target, and whether the app serves there is unverified or failed (§7.5); an operator fixes the missing report, never bypasses a gate | |
| | reverting (`cutting`) | abort's reports: the stager's prefix removed, every source box reopened | 10 min | blocked: page; retry each tick; the source's copy was never removed; `source reopened` is shown separately from `abort finished` | |
| `abort` before `open` is the existing abort (`offline-reverting`), lossless: the source's database was never removed, the target never took a lease, and the tombstone lifts. `open` is the point of no automatic return (§7.5). A lost response to any door write is an uncertain commit: the next tick rereads the record before deciding, never assumes. If `CARLOS_SELF_SERVE_MOVES` is turned off with moves in flight, the job keeps running them to completion and admits no new one. |
| @@ -256,10 +257,10 @@ Infra: both staging lanes for every region pair the console arms (today IE to UK |
| - The estimate's storage and the rolling median; the phase timings' recording. |
| - The `moves` column migration, the mint flag and the instance-bound refusal. |
| - The return's leftover acceptance in the operator start door's target-ready contract, keyed by the forward move id. |
| -- `move_seq` on the record and `serving_seq` on rows, routes and holding entries; the edges' highest-sequence rule; the probe mode (`X-Carlos-Probe`, `X-Carlos-Served-By`); the "taking longer" sentence's trigger. |
| +- `move_seq` and `move_lineage` on the record, `serving_seq` on rows (written with the status, by moves and by reconciliation), both published with routes and holding entries; the edges' highest-sequence-within-lineage rule; the probe mode (`X-Carlos-Probe`, `X-Carlos-Served-By`); the "taking longer" sentence's trigger. |
| - The root cleanup request, done and cancel files and root's check before each deletion. |
| - The `offline-queued` state and the `self_serve_moves` row's phases; the cleanup reservation kind in `active.json` and the create door's check of it. |
| -- Tests: the two-box end-to-end witness of the offline plan (Task 8) extended to the job: forward; cancel in ready; a cut naming a stale id; each pre-check refusal (a stale live link, a non-empty extra directory, an ineligible archive, a failed lane probe); each deadline before open ending in a lossless abort; a post-open failure to answer ending in `done` with a warning and a page; the holding page on a third edge and a custom domain; a ready route winning over a stale holding entry and a holding entry winning over a stale route; two moves at once on disjoint boxes and two queued on one; a reservation reclaimed only on evidence; cleanup refusing when the record moved again, when a current version differs from the inventory, and when the reservation is held by a move. |
| +- Tests: the two-box end-to-end witness of the offline plan (Task 8) extended to the job: forward; cancel in ready; an abort after landing followed by the source row's reconstruction by reconciliation while the target's holding publication is retained, with the reconstructed route winning; a host recreated after a move with a stale holding entry from the old lineage ignored; a cut naming a stale id; each pre-check refusal (a stale live link, a non-empty extra directory, an ineligible archive, a failed lane probe); each deadline before open ending in a lossless abort; a post-open failure to answer ending in `done` with a warning and a page; the holding page on a third edge and a custom domain; a ready route winning over a stale holding entry and a holding entry winning over a stale route; two moves at once on disjoint boxes and two queued on one; a reservation reclaimed only on evidence; cleanup refusing when the record moved again, when a current version differs from the inventory, and when the reservation is held by a move. |
| ## 12. Risks |