amadan
plan: self-serve moves 2, rev 11 (Astra review 10: monotonic proposed stamps for every writer; every request carries its own deadline so an expired open is never replayed from open-uncertain)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
1 file changed,
+7
−7
docs/superpowers/plans/2026-10-22-self-serve-moves-2-door-token-job.md+7 −7
CI passed — run details
CI log
=== gate ===
.amadan/test-gate.sh
=== launchctx (1 s)
ok amadan.net/carlos/platform.git/launchctx 0.015s
? amadan.net/carlos/platform.git/launchctx/controlname [no test files]
? amadan.net/carlos/platform.git/launchctx/wireid [no test files]
=== vet (2 s)
=== gofmt (1 s)
gofmt -l .: clean
=== rest (152 s)
? github.com/carlosframework/platform/cmd/changelog [no test files]
ok github.com/carlosframework/platform/cmd/console 7.113s
ok github.com/carlosframework/platform/cmd/moveprobe 14.157s
ok github.com/carlosframework/platform/cmd/zddprobe 0.896s
ok github.com/carlosframework/platform/internal/activator 63.218s
ok github.com/carlosframework/platform/internal/agentsock 0.020s
ok github.com/carlosframework/platform/internal/alerting 0.012s
ok github.com/carlosframework/platform/internal/analytics 0.031s
ok github.com/carlosframework/platform/internal/appconfig 6.981s
ok github.com/carlosframework/platform/internal/apperr 0.349s
ok github.com/carlosframework/platform/internal/applog 0.420s
ok github.com/carlosframework/platform/internal/awsinv 0.040s
ok github.com/carlosframework/platform/internal/boxchoice 0.005s
ok github.com/carlosframework/platform/internal/boxid 2.034s
? github.com/carlosframework/platform/internal/boxid/boxidtest [no test files]
? github.com/carlosframework/platform/internal/boxid/internal/signtest [no test files]
ok github.com/carlosframework/platform/internal/boxvolumes 0.012s
ok github.com/carlosframework/platform/internal/caps 0.016s
ok github.com/carlosframework/platform/internal/certwarm 0.029s
ok github.com/carlosframework/platform/internal/cgroup 0.918s
ok github.com/carlosframework/platform/internal/changelog 1.534s
ok github.com/carlosframework/platform/internal/claimactivation 2.779s
ok github.com/carlosframework/platform/internal/clamd 0.085s
? github.com/carlosframework/platform/internal/cliflag [no test files]
? github.com/carlosframework/platform/internal/controlname [no test files]
? github.com/carlosframework/platform/internal/controlsock [no test files]
ok github.com/carlosframework/platform/internal/dbdoor 6.748s
ok github.com/carlosframework/platform/internal/dbdoor/dbdoortest 0.011s
ok github.com/carlosframework/platform/internal/dbverify 0.008s
ok github.com/carlosframework/platform/internal/dnsrec 0.003s
ok github.com/carlosframework/platform/internal/docsite 0.012s
ok github.com/carlosframework/platform/internal/domains 6.471s
ok github.com/carlosframework/platform/internal/doorkey 0.015s
ok github.com/carlosframework/platform/internal/economics 0.004s
ok github.com/carlosframework/platform/internal/edge 49.275s
? github.com/carlosframework/platform/internal/edgeproof [no test files]
ok github.com/carlosframework/platform/internal/edgesfile 0.015s
ok github.com/carlosframework/platform/internal/fleetapi 0.332s
ok github.com/carlosframework/platform/internal/gate 2.762s
ok github.com/carlosframework/platform/internal/genselect 0.038s
ok github.com/carlosframework/platform/internal/geo 0.061s
ok github.com/carlosframework/platform/internal/geoip 0.186s
ok github.com/carlosframework/platform/internal/historyreadback 0.406s
ok github.com/carlosframework/platform/internal/host 147.788s
ok github.com/carlosframework/platform/internal/hostactors 10.132s
ok github.com/carlosframework/platform/internal/hostactors/awsmembership 0.013s
ok github.com/carlosframework/platform/internal/hostbox 6.069s
ok github.com/carlosframework/platform/internal/hostclaim 2.576s
ok github.com/carlosframework/platform/internal/hostenv 0.041s
ok github.com/carlosframework/platform/internal/hostevidence 0.014s
ok github.com/carlosframework/platform/internal/hostexec 11.193s
ok github.com/carlosframework/platform/internal/hostfence 0.068s
ok github.com/carlosframework/platform/internal/hostfiles 13.337s
ok github.com/carlosframework/platform/internal/hostfloor 0.126s
ok github.com/carlosframework/platform/internal/hostguard 1.252s
ok github.com/carlosframework/platform/internal/hostops 5.444s
? github.com/carlosframework/platform/internal/hostops/fencerelease [no test files]
ok github.com/carlosframework/platform/internal/hostops/internal/fencerelease 0.042s
ok github.com/carlosframework/platform/internal/hostquarantine 0.006s
ok github.com/carlosframework/platform/internal/hostrelocate 1.922s
ok github.com/carlosframework/platform/internal/hostresolve 0.485s
ok github.com/carlosframework/platform/internal/hostwrite 0.003s
ok github.com/carlosframework/platform/internal/infraview 0.004s
ok github.com/carlosframework/platform/internal/instancectl 5.363s
ok github.com/carlosframework/platform/internal/instanceexec 3.346s
ok github.com/carlosframework/platform/internal/instancefs 0.019s
ok github.com/carlosframework/platform/internal/instancekey 0.004s
ok github.com/carlosframework/platform/internal/instanceupdates 0.007s
ok github.com/carlosframework/platform/internal/layout 3.930s
ok github.com/carlosframework/platform/internal/lbhealth 0.003s
ok github.com/carlosframework/platform/internal/lease 10.770s
ok github.com/carlosframework/platform/internal/lease/leasetest 1.171s
ok github.com/carlosframework/platform/internal/ledger 0.019s
ok github.com/carlosframework/platform/internal/mail 0.414s
ok github.com/carlosframework/platform/internal/maintenance 0.255s
ok github.com/carlosframework/platform/internal/maintenanceexec 6.468s
ok github.com/carlosframework/platform/internal/maintenanceguard 0.085s
ok github.com/carlosframework/platform/internal/metrics 0.017s
ok github.com/carlosframework/platform/internal/moves 28.405s
ok github.com/carlosframework/platform/internal/moves/archive 1.703s
ok github.com/carlosframework/platform/internal/objstore 0.535s
ok github.com/carlosframework/platform/internal/ops 0.628s
ok github.com/carlosframework/platform/internal/park 0.818s
ok github.com/carlosframework/platform/internal/plans 0.018s
ok github.com/carlosframework/platform/internal/privatefile 0.007s
ok github.com/carlosframework/platform/internal/provision 23.324s
ok github.com/carlosframework/platform/internal/publicrelease 0.011s
? github.com/carlosframework/platform/internal/regionname [no test files]
ok github.com/carlosframework/platform/internal/registry 8.479s
ok github.com/carlosframework/platform/internal/relay 7.261s
ok github.com/carlosframework/platform/internal/release 34.112s
ok github.com/carlosframework/platform/internal/rollout 5.779s
ok github.com/carlosframework/platform/internal/rolloutwords 0.009s
ok github.com/carlosframework/platform/internal/rootaws 0.023s
ok github.com/carlosframework/platform/internal/rootparticipation 0.071s
ok github.com/carlosframework/platform/internal/rootstate 0.028s
ok github.com/carlosframework/platform/internal/runs 0.582s
ok github.com/carlosframework/platform/internal/s3compat 0.009s
ok github.com/carlosframework/platform/internal/safefile 0.010s
ok github.com/carlosframework/platform/internal/schedule 3.745s
ok github.com/carlosframework/platform/internal/seeds 0.025s
ok github.com/carlosframework/platform/internal/sim/hostlife 25.914s
ok github.com/carlosframework/platform/internal/sim/litestreamfacts 4.134s
ok github.com/carlosframework/platform/internal/status 0.109s
ok github.com/carlosframework/platform/internal/steering 0.019s
ok github.com/carlosframework/platform/internal/stopevidence 0.273s
ok github.com/carlosframework/platform/internal/svcrole 0.005s
ok github.com/carlosframework/platform/internal/telemetry 0.005s
ok github.com/carlosframework/platform/internal/termfence 0.008s
ok github.com/carlosframework/platform/internal/tunnelclient 0.007s
ok github.com/carlosframework/platform/internal/unixdial 0.003s
ok github.com/carlosframework/platform/internal/updatewords 0.011s
ok github.com/carlosframework/platform/internal/usage 0.007s
ok github.com/carlosframework/platform/internal/version 0.004s
ok github.com/carlosframework/platform/internal/vet 2.227s
=== internal_console@0 (88 s)
ok github.com/carlosframework/platform/internal/console 82.915s
=== internal_console@1 (47 s)
ok github.com/carlosframework/platform/internal/console 42.217s
=== internal_console@2 (50 s)
ok github.com/carlosframework/platform/internal/console 46.041s
=== internal_console@3 (46 s)
ok github.com/carlosframework/platform/internal/console 41.869s
=== internal_console@4 (46 s)
ok github.com/carlosframework/platform/internal/console 42.125s
=== internal_console@5 (43 s)
ok github.com/carlosframework/platform/internal/console 40.520s
=== internal_console@6 (45 s)
ok github.com/carlosframework/platform/internal/console 41.434s
=== internal_console@7 (51 s)
ok github.com/carlosframework/platform/internal/console 48.059s
=== root@0 (46 s)
ok github.com/carlosframework/platform 41.943s
=== root@1 (51 s)
ok github.com/carlosframework/platform 47.359s
=== root@2 (49 s)
ok github.com/carlosframework/platform 46.147s
=== cross ===
--- shard 0 of 5 ---
cross-compiling darwin/arm64
--- shard 1 of 5 ---
cross-compiling darwin/amd64
--- shard 2 of 5 ---
cross-compiling linux/arm64
linux/arm64 carlos binary: 50069666 bytes (ceiling 52000000)
--- shard 3 of 5 ---
cross-compiling windows/amd64
--- shard 4 of 5 ---
cross-compiling windows/arm64
=== minio-contract ===
minio contract: arm64, minio RELEASE.2025-09-07T16-13-09Z, mc RELEASE.2025-08-13T08-35-41Z
minio: cached, digest verified
mc: cached, digest verified
minio healthy on 127.0.0.1:20999
Added `local` successfully.
Bucket created successfully `local/contract-bucket`.
Bucket created successfully `local/denied-bucket`.
Created policy `contract-only` successfully.
Added user `scopeduser` successfully.
Attached Policies: [contract-only]
To User: scopeduser
buckets and scoped user ready
=== RUN TestDeliveredContractMinio
=== RUN TestDeliveredContractMinio/put
=== RUN TestDeliveredContractMinio/get_round-trips_the_body
=== RUN TestDeliveredContractMinio/list_shows_the_key
=== RUN TestDeliveredContractMinio/head_and_list_agree_on_the_version
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_its_own_object
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed
=== RUN TestDeliveredContractMinio/delete_removes_it
=== RUN TestDeliveredContractMinio/cross-bucket_denial
--- PASS: TestDeliveredContractMinio (0.02s)
--- PASS: TestDeliveredContractMinio/put (0.00s)
--- PASS: TestDeliveredContractMinio/get_round-trips_the_body (0.00s)
--- PASS: TestDeliveredContractMinio/list_shows_the_key (0.00s)
--- PASS: TestDeliveredContractMinio/head_and_list_agree_on_the_version (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route (0.01s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed (0.00s)
--- PASS: TestDeliveredContractMinio/delete_removes_it (0.00s)
--- PASS: TestDeliveredContractMinio/cross-bucket_denial (0.00s)
PASS
ok github.com/carlosframework/platform/internal/objstore 0.025s
=== RUN TestT4StorageProbeMinio
2026/10/22 10:47:53 moves: T4: STORE: list: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: 18E0D44BBDDE10A4, HostID: 7c420053e0fbcd7f41653fdd92ffa2802c903d05303b1aa15adc2e8bb65d183f, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
--- PASS: TestT4StorageProbeMinio (0.01s)
PASS
ok github.com/carlosframework/platform/internal/host 0.023s
minio contract: PASS
diff --git a/docs/superpowers/plans/2026-10-22-self-serve-moves-2-door-token-job.md b/docs/superpowers/plans/2026-10-22-self-serve-moves-2-door-token-job.md| index 9c436c756..977284467 100644 |
| --- a/docs/superpowers/plans/2026-10-22-self-serve-moves-2-door-token-job.md |
| +++ b/docs/superpowers/plans/2026-10-22-self-serve-moves-2-door-token-job.md |
| @@ -2,7 +2,7 @@ |
| > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
| -Rev 10, 2026-10-22. Rev 9's review (`selfserve-plan2-review9.md`: 8, 10, 13, 20, N7 closed) is answered: **a committed confirmation expiry wins over a later cut** (`MarkSelfServeCut` also requires `expiry_pending = 0`; the cut door answers 409 "the move expired. prepare again"), and the persisted expiry carries its kind and deadline so execution revalidates its identity (N6); **an operator's newer staging stamp supersedes the job's outstanding wait**: when the record's stamp for the awaited field has moved past `request_proposed` without the receipt changing, the job abandons its request and issues a fresh bound round, with the crash between the two stated (N8). Rev 8's review (`selfserve-plan2-review8.md`: 21, N4 closed) is answered: reconciliation maps the record to a **phase and substep** (the first substep whose transition the record has not made), so an operator's progress never makes the job repeat a refused transition (item 10); abandonment is only for **leaving** a wait (abort, cancel, operator supersession), never for a page-and-retain expiry, with the phase table stated (item 13); the confirmation-deadline expiry commits only against `cut_requested_at = 0` and the deadline it saw, and the cut's copy expiry is a different decision (N6); the cut handoff persists which continuation it chose as a `restage` substep (`adopted` or `issue`) in the advancing transaction (N7); acceptance and discovery share one participant predicate (item 8); the superseded tag arrangement is deleted (item 20). Rev 7's review (`selfserve-plan2-review7.md`: N1, N2, N3, N5 closed) is answered: a phase with several writes has a durable **substep** (`phase_step`), advanced atomically with the consumption that ends the previous substep, and the phase function dispatches on it (item 10); confirmation is the **receipt alone** (no state predicate), phase progress is reconciliation's business (item 10); issuance refused with no outstanding request is its own result (`errRequestRefused`) (item 10); a confirmed request whose wait will never be satisfied is **abandoned** (consumed without its wait) by abort, expiry and operator supersession, and the finaliser consumes a confirmed clearing request in the transaction that records the outcome (item 13); `maintain` updates the stage identity like `stage` (N4); the cut's `MarkSelfServeCut` retires the ready deadline in the same UPDATE and ready expiry is conditional on no accepted cut (N6); return reports are **immutable per request key** (item 21); an id-less forward keeps `BoxID` membership and drops only the instance comparison (item 8); one untagged bridge file with the complete helper closure (item 20). Rev 6's review (`selfserve-plan2-review6.md`: items 1, 6, 17 closed) is answered: the request lifecycle is a complete state machine (`issued → confirmed → consumed`, or `voided`) with a **resume** path for a confirmed-unconsumed request, a standalone consume for same-phase sequences (start then seal, finish gate 1 then finish, round after round, estimate), issuance that returns the existing request so a caller of another kind or phase stops instead of interpreting it, and no-op-free receipts (a request-bearing mutation always writes its receipt) (item 10); the bound write checks **incarnation and sequence** (`provision.Expect{Incarnation, Seq}`) for the job and the operator doors (N5, item 13); every finalisation requires no outstanding request after a fence (item 13); the expiry branch classifies the fenced observation (N3); the cut is a **handoff**: the door marks `cut_requested_at` and the job turns the outstanding maintenance round into the first cut round (N4); event delivery marks `emitted` inside the transaction before delivering and the terminal event is inserted with the outcome (N1); per-box FIFO is stated (N2); the open intent is prepared before the request is issued so adoption and the persisted proposal agree, return requests carry a full identity ordered deterministically, reports name the request key, and acceptance binds the report to the reverse move's selected target (item 21); `Expect`'s real fields, `Load`'s absence distinction and one id-less rule (item 8); the fixtures move to an untagged `linux` file so today's tests keep compiling, `CopyEvent` carries its completion fields, the saved book is validated before seeding, `DiskFreeBytes` is numeric (item 20). Rev 5's review (`selfserve-plan2-review5.md`: items 2, 3, 4, 5, 7, 9, 11, 12, 14, 15, 16, 18, 19 closed) is answered: every job and operator record write is **predecessor-bound** (`provision.SetRecordMoveAt(ctx, s, acct, app, host, expectSeq, mut)`: one CAS write against the record whose `record_seq` equals the persisted expectation, no fresh-read retry), so a voided request can never land and an operator door never rebuilds a write after a reclaim fence (items 10, 13); every self-serve mutation stamps a **nonce receipt** on the record (`SelfServe.LastRequest`, or `LastMove.Request` for a clearing write), so confirmation is exact for every transition and distinct from an operator's reconciliation (item 10); confirm and advance require an unconsumed, unvoided request, so a stale worker cannot confirm a voided one (item 10); "pending" means **no confirmed request for the phase's write**, so the interval before issuance is compatible with the predecessor state (item 10); an expired uncommitted open is fenced and aborted, never replayed (N1); the stage book is reconciled against each complete successful round so deleted keys leave it (item 6); identity records are loaded by `boxid.Key(BoxID)` and `Verified` is narrowed to what `Enroll` proves (item 8); the gate's exception for an authorised malformed hostname and plan 1's outer 503 are declared (item 1); the view joins row id, move id, self-serve id and incarnation (item 17); the open record adopts an existing complete intent, return discovery and acceptance compare `InstanceID`, incarnation is re-checked at the door and in the guarded write, and `Layout` comes from the source's `stopped` report so every operator move has it (item 21); a maintenance restage in `ready` has its own kind and keeps the confirmation deadline (N2); the gate types, `StagedRequest`, failed diagnostics, the request argument to `confirmed`, and the bridge's compile rule are stated (item 20). Reviews: rev 1 (`after236/selfserve-plan2-review.md`, 20 findings), rev 2 (`selfserve-plan2-review2.md`, 21), rev 3 (`selfserve-plan2-review3.md`, 21 items), rev 4 (`selfserve-plan2-review4.md`, 21 items, 9 closed). Rev 5 is **self-contained** (nothing is "as rev N"; every schema, phase, predicate, sentence and test list is in this document) and answers rev 4's open items: the gate orders authentication, host shape and the capability check before the limiter and the no-bucket rail (item 1); operator moves keep today's admission (nil or stale metrics, no identity record) and only the self-serve path requires fresh identity; `BootID` is dropped (items 4, 8); the evidence reader takes the move and a freshness table is stated (item 5); the staged inventory is a **move-bound book** built from every copy event the mirror reports, including same-ETag and deferred copies, through a new observer hook that leaves ordinary mirroring unchanged (item 6); a participant is identified by `BoxID` **and** `InstanceID` in the resolver, in report validation and at every gate, and abort never requires the target (item 8); every request persists its **nonce, expected sequence and proposed stamp before the write**, confirmation is conditional on the record carrying that exact stamp, replay is idempotent, and consumption and the phase advance are one UPDATE predicated on the nonce (item 10); compatible sets include the predecessor state while a phase's write is pending (item 10); expiry is bound to its phase and cleared on reconciliation into another phase, with the health rule taking precedence (item 12); abandonment is conditional on the fenced observation and persisted before release, and every reclaim fences first, operator reservations included (item 13); own retries are accepted only for a complete expected selection after the whole table validates (item 14); the status view joins rows regardless of the mode (item 17); events carry a per-row sequence and are delivered inside the caller's transaction, the harness is an exported bridge **inside package host** behind the `e2e` tag with the CI step named, the Go declarations are complete, and queue ordering is stated (item 20); the open record is written **before** the open transition with the proposed open time, return requests have unique durable identities, the report path is named, acceptance is bound to the lifetime and the forward's participants, and leftovers are judged by the forward's layout (item 21). |
| +Rev 11, 2026-10-22. Rev 10's review (`selfserve-plan2-review10.md`: N6 closed; N8 narrowed; N9 new) is answered: **every** writer, the operator doors included, proposes its stamp as `max(now, the observed record's previous stamp + 1 ns)` and the mutation stores it verbatim, so stamps are monotonic under a backward clock and supersession by stamp is sound (N8); every request persists **its own deadline** (`request_deadline`, the issuing phase's step deadline), and an unconfirmed request whose deadline has passed while the record is at or before `offline-landed` is never replayed from **any** row phase, `open-uncertain` included: fence, classify, confirm if the write won, else void and abort (N9). Rev 9's review (`selfserve-plan2-review9.md`: 8, 10, 13, 20, N7 closed) is answered: **a committed confirmation expiry wins over a later cut** (`MarkSelfServeCut` also requires `expiry_pending = 0`; the cut door answers 409 "the move expired. prepare again"), and the persisted expiry carries its kind and deadline so execution revalidates its identity (N6); **an operator's newer staging stamp supersedes the job's outstanding wait**: when the record's stamp for the awaited field has moved past `request_proposed` without the receipt changing, the job abandons its request and issues a fresh bound round, with the crash between the two stated (N8). Rev 8's review (`selfserve-plan2-review8.md`: 21, N4 closed) is answered: reconciliation maps the record to a **phase and substep** (the first substep whose transition the record has not made), so an operator's progress never makes the job repeat a refused transition (item 10); abandonment is only for **leaving** a wait (abort, cancel, operator supersession), never for a page-and-retain expiry, with the phase table stated (item 13); the confirmation-deadline expiry commits only against `cut_requested_at = 0` and the deadline it saw, and the cut's copy expiry is a different decision (N6); the cut handoff persists which continuation it chose as a `restage` substep (`adopted` or `issue`) in the advancing transaction (N7); acceptance and discovery share one participant predicate (item 8); the superseded tag arrangement is deleted (item 20). Rev 7's review (`selfserve-plan2-review7.md`: N1, N2, N3, N5 closed) is answered: a phase with several writes has a durable **substep** (`phase_step`), advanced atomically with the consumption that ends the previous substep, and the phase function dispatches on it (item 10); confirmation is the **receipt alone** (no state predicate), phase progress is reconciliation's business (item 10); issuance refused with no outstanding request is its own result (`errRequestRefused`) (item 10); a confirmed request whose wait will never be satisfied is **abandoned** (consumed without its wait) by abort, expiry and operator supersession, and the finaliser consumes a confirmed clearing request in the transaction that records the outcome (item 13); `maintain` updates the stage identity like `stage` (N4); the cut's `MarkSelfServeCut` retires the ready deadline in the same UPDATE and ready expiry is conditional on no accepted cut (N6); return reports are **immutable per request key** (item 21); an id-less forward keeps `BoxID` membership and drops only the instance comparison (item 8); one untagged bridge file with the complete helper closure (item 20). Rev 6's review (`selfserve-plan2-review6.md`: items 1, 6, 17 closed) is answered: the request lifecycle is a complete state machine (`issued → confirmed → consumed`, or `voided`) with a **resume** path for a confirmed-unconsumed request, a standalone consume for same-phase sequences (start then seal, finish gate 1 then finish, round after round, estimate), issuance that returns the existing request so a caller of another kind or phase stops instead of interpreting it, and no-op-free receipts (a request-bearing mutation always writes its receipt) (item 10); the bound write checks **incarnation and sequence** (`provision.Expect{Incarnation, Seq}`) for the job and the operator doors (N5, item 13); every finalisation requires no outstanding request after a fence (item 13); the expiry branch classifies the fenced observation (N3); the cut is a **handoff**: the door marks `cut_requested_at` and the job turns the outstanding maintenance round into the first cut round (N4); event delivery marks `emitted` inside the transaction before delivering and the terminal event is inserted with the outcome (N1); per-box FIFO is stated (N2); the open intent is prepared before the request is issued so adoption and the persisted proposal agree, return requests carry a full identity ordered deterministically, reports name the request key, and acceptance binds the report to the reverse move's selected target (item 21); `Expect`'s real fields, `Load`'s absence distinction and one id-less rule (item 8); the fixtures move to an untagged `linux` file so today's tests keep compiling, `CopyEvent` carries its completion fields, the saved book is validated before seeding, `DiskFreeBytes` is numeric (item 20). Rev 5's review (`selfserve-plan2-review5.md`: items 2, 3, 4, 5, 7, 9, 11, 12, 14, 15, 16, 18, 19 closed) is answered: every job and operator record write is **predecessor-bound** (`provision.SetRecordMoveAt(ctx, s, acct, app, host, expectSeq, mut)`: one CAS write against the record whose `record_seq` equals the persisted expectation, no fresh-read retry), so a voided request can never land and an operator door never rebuilds a write after a reclaim fence (items 10, 13); every self-serve mutation stamps a **nonce receipt** on the record (`SelfServe.LastRequest`, or `LastMove.Request` for a clearing write), so confirmation is exact for every transition and distinct from an operator's reconciliation (item 10); confirm and advance require an unconsumed, unvoided request, so a stale worker cannot confirm a voided one (item 10); "pending" means **no confirmed request for the phase's write**, so the interval before issuance is compatible with the predecessor state (item 10); an expired uncommitted open is fenced and aborted, never replayed (N1); the stage book is reconciled against each complete successful round so deleted keys leave it (item 6); identity records are loaded by `boxid.Key(BoxID)` and `Verified` is narrowed to what `Enroll` proves (item 8); the gate's exception for an authorised malformed hostname and plan 1's outer 503 are declared (item 1); the view joins row id, move id, self-serve id and incarnation (item 17); the open record adopts an existing complete intent, return discovery and acceptance compare `InstanceID`, incarnation is re-checked at the door and in the guarded write, and `Layout` comes from the source's `stopped` report so every operator move has it (item 21); a maintenance restage in `ready` has its own kind and keeps the confirmation deadline (N2); the gate types, `StagedRequest`, failed diagnostics, the request argument to `confirmed`, and the bridge's compile rule are stated (item 20). Reviews: rev 1 (`after236/selfserve-plan2-review.md`, 20 findings), rev 2 (`selfserve-plan2-review2.md`, 21), rev 3 (`selfserve-plan2-review3.md`, 21 items), rev 4 (`selfserve-plan2-review4.md`, 21 items, 9 closed). Rev 5 is **self-contained** (nothing is "as rev N"; every schema, phase, predicate, sentence and test list is in this document) and answers rev 4's open items: the gate orders authentication, host shape and the capability check before the limiter and the no-bucket rail (item 1); operator moves keep today's admission (nil or stale metrics, no identity record) and only the self-serve path requires fresh identity; `BootID` is dropped (items 4, 8); the evidence reader takes the move and a freshness table is stated (item 5); the staged inventory is a **move-bound book** built from every copy event the mirror reports, including same-ETag and deferred copies, through a new observer hook that leaves ordinary mirroring unchanged (item 6); a participant is identified by `BoxID` **and** `InstanceID` in the resolver, in report validation and at every gate, and abort never requires the target (item 8); every request persists its **nonce, expected sequence and proposed stamp before the write**, confirmation is conditional on the record carrying that exact stamp, replay is idempotent, and consumption and the phase advance are one UPDATE predicated on the nonce (item 10); compatible sets include the predecessor state while a phase's write is pending (item 10); expiry is bound to its phase and cleared on reconciliation into another phase, with the health rule taking precedence (item 12); abandonment is conditional on the fenced observation and persisted before release, and every reclaim fences first, operator reservations included (item 13); own retries are accepted only for a complete expected selection after the whole table validates (item 14); the status view joins rows regardless of the mode (item 17); events carry a per-row sequence and are delivered inside the caller's transaction, the harness is an exported bridge **inside package host** behind the `e2e` tag with the CI step named, the Go declarations are complete, and queue ordering is stated (item 20); the open record is written **before** the open transition with the proposed open time, return requests have unique durable identities, the report path is named, acceptance is bound to the lifetime and the forward's participants, and leftovers are judged by the forward's layout (item 21). |
| Builds on plan 1 (`2026-10-22-self-serve-moves-1-lifetimes-and-order.md`, rev 13): records carry a lifetime, `record_seq` and `move_seq`, rows carry the pair, readers and writers are judged, `fenceRecord(ctx, acct, app, host, op fenceOwner) (fenceResult, Observation, error)` exists, the console runs behind the lifetimes cutover. This plan adds everything a Tito customer's request needs up to a finished move except the holding page, the edge probe mode and cleanup (plans 3 and 4). |
| @@ -67,7 +67,7 @@ Builds on plan 1 (`2026-10-22-self-serve-moves-1-lifetimes-and-order.md`, rev 13 |
| - `const OfflineQueued = "offline-queued"`. |
| - `type SelfServeMove struct { ID string \`json:"id"\`; Incarnation uint64 \`json:"incarnation"\`; DisplayName string \`json:"display_name,omitempty"\`; RequestedBy string \`json:"requested_by,omitempty"\`; EstimateSeconds int \`json:"estimate_seconds,omitempty"\`; OfflineAt *time.Time \`json:"offline_at,omitempty"\`; LastRequest string \`json:"last_request,omitempty"\` }` (**the receipt**: every self-serve mutation sets it to the request's nonce); `LastMove` gains `Request string \`json:"request,omitempty"\`` (the receipt of the clearing write: `OfflineFinish`, `OfflineAbortAt`, the abort path's finish). **Every transition the job writes takes `req provision.Receipt{Nonce string}`** as an additive final parameter (`OfflineQueue`, `OfflineQueuedToStaging`, `OfflinePrecheck`, `OfflineRestage`, `OfflineSetEstimate`, `OfflineStartStaged`, `OfflineSeal`, `OfflineFlip`, `OfflineOpenTransition`, `OfflineFinishingTransition`, `OfflineFinish`, `OfflineAbort`, `OfflineAbortAt`): a non-empty nonce is written to `SelfServe.LastRequest` (or `LastMove.Request`); the operator doors pass an empty receipt, which leaves the field unchanged, so an operator transition never confirms a job request. **With a non-empty receipt no transition is a no-op**: `OfflineSetEstimate` with an equal estimate and `OfflineAbort` on a reverting record still write the receipt (today's `errRecordUnchanged` short-cut applies only to an empty receipt), so every request-bearing write can be confirmed. |
| - `type Participant struct { BoxID string \`json:"box_id"\`; Label string \`json:"label"\`; InstanceID string \`json:"instance_id,omitempty"\`; Verified bool \`json:"verified,omitempty"\` }`; `type Participants struct { Source []Participant \`json:"source"\`; Target Participant \`json:"target"\` }`. |
| -- `func OfflineQueue(id, from, to, by string, st StageTarget, p Participants, ss SelfServeMove, at time.Time) func(*Record) error`: `OfflineStage`'s refusals (a move in flight, a seed block, an open run, the same region) plus `rec.Incarnation != ss.Incarnation` → `fmt.Errorf("%w: the instance was recreated since the move was requested", ErrMoveConflict)`; creates the move in `offline-queued` with `Kind: offline`, `From`, `To`, `Stager: p.Source[0].Label`, `StageBucket/StageRegion`, `Participants`, `SelfServe`, `RequestedAt: at`, `QueuedAt: &at`, `StagedAt: &at` (the pre-copy intent exists, so `StartedOffline()` is false until the start), `PrecheckAt: &at`, `Seq: rec.MoveSeq`; `move_seq` unchanged. **`at` is the caller's proposed stamp** and is stored verbatim (no clock read inside the mutation); the same holds for every transition below that takes `at`. |
| +- `func OfflineQueue(id, from, to, by string, st StageTarget, p Participants, ss SelfServeMove, at time.Time) func(*Record) error`: `OfflineStage`'s refusals (a move in flight, a seed block, an open run, the same region) plus `rec.Incarnation != ss.Incarnation` → `fmt.Errorf("%w: the instance was recreated since the move was requested", ErrMoveConflict)`; creates the move in `offline-queued` with `Kind: offline`, `From`, `To`, `Stager: p.Source[0].Label`, `StageBucket/StageRegion`, `Participants`, `SelfServe`, `RequestedAt: at`, `QueuedAt: &at`, `StagedAt: &at` (the pre-copy intent exists, so `StartedOffline()` is false until the start), `PrecheckAt: &at`, `Seq: rec.MoveSeq`; `move_seq` unchanged. **`at` is the caller's proposed stamp** and is stored verbatim (no clock read and no clamp inside the mutation); the same holds for every transition below that takes `at`. **Every caller proposes monotonically**: `at = max(now, the latest stamp on the record it observed + 1 ns)` (`provision.ProposeStamp(rec, now)`), the job from its persisted request and the operator doors from the record they observed before reserving, so a backward clock never produces an earlier stamp and today's in-mutation clamp is retired in favour of the caller's rule (`TestStampsAreMonotonicUnderABackwardClock`: an operator stage after a job restage with the clock moved back still writes a later stamp). |
| - `func OfflineQueuedToStaging(id string, at time.Time) func(*Record) error` (`StagedAt = &at`); `func OfflinePrecheck(id string, at time.Time) func(*Record) error` (queued or staging; `PrecheckAt = &at`); `OfflineRestage(id, at)` stores `at` verbatim (today's signature; the console passes its proposed stamp); `func OfflineSetEstimate(id string, seconds int) func(*Record) error` (queued or staging; `SelfServe == nil` → `ErrMoveConflict` "not a self-serve move"; equal → `errRecordUnchanged`); `func OfflineAbortAt(id string, at time.Time, req Receipt) func(*Record) error` (from `offline-queued` only: clears the move with `LastMove = {ID, aborted, At: at, OfflineS: 0}`, `move_seq` unchanged; every other state → `OfflineAbort`'s behaviour; a repeated abort on a reverting record stays a no-write success); `OfflineStage` and `OfflineStart` gain `returnOf string` (additive; "" for every existing caller); `OfflineStartStaged(id, at)` sets `SelfServe.OfflineAt = &at` when `SelfServe != nil`; `OfflineOpenTransition(id, at)` sets `OpenedAt = at`. Every other transition refuses `offline-queued` with `ErrMoveConflict` and the existing `offlineStateRefusal` words. |
| - **The open record** `type moves.OpenRecord struct { MoveID string \`json:"move_id"\`; Host string \`json:"host"\`; Account string \`json:"account"\`; App string \`json:"app"\`; Incarnation uint64 \`json:"incarnation"\`; From string \`json:"from"\`; To string \`json:"to"\`; StageBucket string \`json:"stage_bucket"\`; StageRegion string \`json:"stage_region"\`; Participants *provision.Participants \`json:"participants,omitempty"\`; Layout string \`json:"layout"\` (`flat` or `nested`, from the source precheck's `Layout`); OpenedAt time.Time \`json:"opened_at"\` }` at `OpenRecordKey(host, id) = "control/offline-moves/<host>/<id>/open.json"`, written **create-only before** the open transition, by the operator open door and by the job alike: the writer first reads the key; an existing **complete** record for this `MoveID` is **adopted** (its `OpenedAt` becomes the proposed open stamp of this attempt, so a retry after a crash between the record and the transition opens at the recorded time); absent → written with a fresh proposed stamp; **this preparation runs before the open request is issued** (`selfServeApply` takes `proposed *time.Time`: the job passes the open record's `OpenedAt`, so the persisted proposal and the adopted intent are one value; a create-only conflict on the write re-reads and adopts); present for another move id → 409 "another move's open record is in the way" (operator repair). `Layout` comes from the source's `stopped` report (`OfflineReport.Layout`, filled by every source at stop, so a direct operator move has it too). The open transition then carries the same `at`. Proof that the open **committed** is `rec.Move.OpenedAt == open.OpenedAt` while the move is on the record, or `LastMove.ID == open.MoveID && LastMove.Outcome == done` after it cleared; the return door requires the latter (a region comparison proves nothing: `OfflineFlip` changes the region before open). Because the record exists before the open can commit, no clearing transition can outrun it. |
| @@ -87,7 +87,7 @@ Builds on plan 1 (`2026-10-22-self-serve-moves-1-lifetimes-and-order.md`, rev 13 |
| - `func (s *Server) reserveBoxes(ctx context.Context, sel offlineSelection, r moves.OfflineReservation) *offlineRefusal`, in order: (1) read `active.json` (absent = empty); a body with no `Reservations` and `Host != ""` is a **legacy lock** reserving every box for `{move, Op, Host, Account, App, At}`, normalised into the table on the first new write; (2) **validate the whole object**: any malformed entry → `unreadable`; (3) **own retry**: entries with this `(Kind, ID)` exist → accepted only when they cover **exactly** the expected keys (every participant's `BoxID`, or `"*"` for a sentinel selection) with no extra; a partial own reservation → the missing keys are acquired in the same write if free, else `busy`; an own reservation whose keys differ from the expected selection → refuse "the reservation does not match this move's participants" (the job aborts); an accepted retry keeps `ReservedAt`; (4) `cfg.SelfServeMoves != "offline"` → refuse `busy` when any entry of another operation exists (today's deployment-wide semantics); (5) any expected key held by another `(Kind, ID)` → `busy` ("<host> is moving offline. move one instance at a time") unless reclaimable (below); (6) one conditional `Put` with every expected key and the projection (`Host, Op, Account, App, At` of the oldest `move` entry; of the oldest entry of any kind when no move holds); a CAS conflict re-reads once and retries from (2), then `busy`. |
| - `func (s *Server) releaseBoxes(ctx context.Context, kind, id string) error`: removes every entry with that `(Kind, ID)`, rewrites the projection, deletes the object when empty (`DeleteIfMatch`); absence is success. |
| - **Reclaim** (`time.Since(ReservedAt) >= offlineLockGrace`), for `move`: (a) `fenceRecord(ctx, acct, app, host, fenceOwner{})` (plan 1) must return `fenced` or `absent` (an error → not reclaimable); (b) the fenced observation carries no move with `ID`; (c) `SelfServeMove(id)` is absent or terminal with `release_pending = 0`; (d) no row for the host has `request_issued` set. An **operator** reservation (no row) is reclaimed on (a) and (b) alone: the fence makes a delayed operator start or stage PUT fail its CAS, so a negative observation after the fence is conclusive. For `cleanup` (plan 4): the owning box's status report (`status/boxes/<label>.json`, `GeneratedAt` within 2 status intervals, `Metrics.BoxID` equal to the key) lists the id in a terminal state with `RootRequest == ""` and `ReleasePending == false`, or its `Retirements` block is complete (`RetirementsComplete`) and lists no such id. Any read failure, a malformed body, or an absent or incomplete status → not reclaimable. |
| -- **Operator admission is predecessor-bound too:** the start and stage doors observe the record (judged) **before** reserving, reserve with `OfflineReservation.ExpectSeq = rec.RecordSeq` and `ExpectIncarnation = rec.Incarnation` (new tagged fields), then write the record with `provision.SetRecordMoveAt(ctx, ..., provision.Expect{Incarnation, Seq}, mut)` (a recreated host at a new incarnation refuses even when its fresh sequence happens to match); a CAS conflict → 409 "the instance changed while the move was being started. try again" and the door releases its reservation; the door never re-reads and retries. A reclaim's fence advances `record_seq`, so a paused door that resumes after the reclaim cannot write (its bound CAS fails) and cannot rebuild a write. |
| +- **Operator admission is predecessor-bound too:** the start and stage doors observe the record (judged) **before** reserving, reserve with `OfflineReservation.ExpectSeq = rec.RecordSeq` and `ExpectIncarnation = rec.Incarnation` (new tagged fields), then write the record with `provision.SetRecordMoveAt(ctx, ..., provision.Expect{Incarnation, Seq}, mut)` with `at = provision.ProposeStamp(observed, now)` (a recreated host at a new incarnation refuses even when its fresh sequence happens to match); a CAS conflict → 409 "the instance changed while the move was being started. try again" and the door releases its reservation; the door never re-reads and retries. A reclaim's fence advances `record_seq`, so a paused door that resumes after the reclaim cannot write (its bound CAS fails) and cannot rebuild a write. |
| - **Rollout and rollback:** old consoles cannot be reintroduced while any reservation exists that the old console did not create (its release deletes the whole object when the projection names its own move; `TestOldConsoleReleaseErasesATable` documents the hazard); the supported sequence is roll consoles → turn the switch on; rollback is switch off → wait until **every incompatible operation** ends (every self-serve row terminal and released, every cleanup reservation gone, every operator reservation created under the table form released) → the old console. |
| - [ ] **Step 1: Write the failing tests**: `TestReserveWritesEveryParticipantAndTheProjection`; `TestReserveRefusesAHeldParticipant`; `TestOwnRetryNeedsTheCompleteSelection` (exact → accepted keeping `ReservedAt`; partial with free keys → completed in one write; partial with a held key → busy; a changed selection → the mismatch refusal; a malformed unrelated entry is detected before the retry is accepted); `TestSentinelReservationConflictsBothWays`; `TestDisjointBoxesReserveAtOnceWithTheSwitch`; `TestWithoutTheSwitchOneReservationAtATime`; `TestLegacyLockBodyIsNormalised`; `TestMalformedEntryIsUnreadable`; `TestReclaimOnlyOnEvidence` (each positive case and each refusing case above; a delayed queue PUT that commits after a negative re-read and before the fence → the fence observes it and the reservation is kept; a delayed operator start PUT → the fence fails it and the reservation is reclaimed; **an operator door paused between reserving and its record write** while the reclaim fences and another operation is admitted → the door's bound write fails with 409, nothing starts; the same door paused across a delete and recreate of the host whose new record is at the same sequence → refused on incarnation; a fence error → kept; a SQL failure → kept; a cleanup-only obligation at switch-off → kept and recovery proceeds); `TestReleaseRemovesOnlyItsEntriesAndDeletesWhenEmpty`; `TestOldConsoleAlgorithmsAgainstTheNewBody`; `TestSwitchOffRecoveryReacquiresItsOwnReservation` (two disjoint reservations, each retries its own → accepted); `TestOperatorDoorsUseReservations`. |
| @@ -102,8 +102,8 @@ Builds on plan 1 (`2026-10-22-self-serve-moves-1-lifetimes-and-order.md`, rev 13 |
| - Test: `internal/console/selfserve_store_test.go`, `internal/console/selfserve_pass_test.go` |
| **Interfaces:** |
| -- Table `self_serve_moves (id TEXT PRIMARY KEY, account TEXT, app TEXT, host TEXT, incarnation INTEGER, from_region TEXT, to_region TEXT, participants TEXT, display_name TEXT, requested_by TEXT, phase TEXT, phase_started_at INTEGER, step_deadline INTEGER, window_deadline INTEGER, expiry_pending INTEGER, expiry_phase TEXT, expiry_kind TEXT, expiry_deadline INTEGER, staging_started_at INTEGER, request_issued INTEGER, request_kind TEXT, request_phase TEXT, request_nonce TEXT, request_expect_seq INTEGER, request_proposed INTEGER, request_confirmed INTEGER, request_consumed INTEGER, request_voided INTEGER, request_abandoned INTEGER, request_expect_inc INTEGER, cut_requested_at INTEGER, phase_step TEXT, stage_request INTEGER, precheck_request INTEGER, rounds INTEGER, estimate_seconds INTEGER, estimate_terms TEXT, reason TEXT, health_attempts INTEGER, health_consecutive_ok INTEGER, health_first_ok_at INTEGER, health_last_probe_at INTEGER, health_last_error TEXT, finish_gate TEXT, notified_seq INTEGER, event_seq INTEGER, outcome TEXT, outcome_reason TEXT, warning TEXT, abort_requested_at INTEGER, source_reopened_at INTEGER, abort_finished_at INTEGER, effects_done INTEGER, release_pending INTEGER, open_record_pending INTEGER, created_at INTEGER, updated_at INTEGER)`; `CREATE UNIQUE INDEX IF NOT EXISTS self_serve_moves_open ON self_serve_moves(account, app, host) WHERE outcome = ''`. |
| -- **The request lifecycle** (one request at a time per row): `IssueSelfServeRequest(id, kind, phase string, pr persistedRequest, now time.Time) (issued bool, existing *persistedRequest, err error)` (`WHERE phase = ? AND outcome = '' AND (request_consumed <> 0 OR request_issued = 0)`: sets `request_issued = now, request_kind, request_phase, request_nonce, request_expect_inc, request_expect_seq, request_proposed, request_confirmed = 0, request_consumed = 0, request_voided = 0`; not issued → `existing` is the outstanding request, whatever its kind and phase); `ConsumeSelfServeRequest(id, nonce, nextStep string) (bool, error)` (`WHERE request_nonce = ? AND request_confirmed <> 0 AND request_consumed = 0`: `request_consumed = 1, phase_step = nextStep` in one statement; the standalone consume for a same-phase sequence: the substep advances exactly when the previous request is consumed, so a crash between them cannot lose the position); `AbandonSelfServeRequest(id, nonce string) (bool, error)` (`WHERE request_nonce = ? AND request_confirmed <> 0 AND request_consumed = 0`: `request_consumed = 1, request_abandoned = 1`; only abort, an expiry action and operator supersession call it: the request's write committed but its wait will never be satisfied, so the phase gives it up and proceeds); `ConfirmSelfServeRequest(id, nonce string, now time.Time) (bool, error)` (`WHERE request_nonce = ? AND request_consumed = 0 AND request_voided = 0 AND request_confirmed = 0`: `request_confirmed = now`; false → the caller re-reads the row: confirmed already → **resume** (success), voided or consumed → stop; for kinds `stage` **and `maintain`** also `stage_request = request_proposed`, for `precheck` also `precheck_request = request_proposed`); `VoidSelfServeRequest(id, nonce string) error` (`WHERE request_nonce = ? AND request_consumed = 0 AND request_confirmed = 0`: `request_consumed = 1, request_voided = 1`; a confirmed request is never voided; the phase function decides again); `AdvanceSelfServe(id, from, to, reason string, step, window time.Time, nonce string, now time.Time) (bool, error)`: one UPDATE with the predicate `phase = ? AND (abort_requested_at = 0 OR ? IN ('reverting','done','opened','health','finishing')) AND (? = '' OR request_nonce = ?)` that sets the phase, `phase_started_at`, the deadlines, `expiry_pending = 0, expiry_phase = ''` when `to <> expiry_phase`, and, when a nonce is given, additionally requires `request_nonce = ? AND request_confirmed <> 0 AND request_consumed = 0` and sets `request_consumed = 1` (**consumption and the advance are one statement**), and inserts the phase event in the same transaction (`event_seq = event_seq + 1`); a stale worker holding another nonce, or the same nonce after a void, cannot confirm or consume. |
| +- Table `self_serve_moves (id TEXT PRIMARY KEY, account TEXT, app TEXT, host TEXT, incarnation INTEGER, from_region TEXT, to_region TEXT, participants TEXT, display_name TEXT, requested_by TEXT, phase TEXT, phase_started_at INTEGER, step_deadline INTEGER, window_deadline INTEGER, expiry_pending INTEGER, expiry_phase TEXT, expiry_kind TEXT, expiry_deadline INTEGER, staging_started_at INTEGER, request_issued INTEGER, request_kind TEXT, request_phase TEXT, request_nonce TEXT, request_expect_seq INTEGER, request_proposed INTEGER, request_confirmed INTEGER, request_consumed INTEGER, request_voided INTEGER, request_abandoned INTEGER, request_expect_inc INTEGER, request_deadline INTEGER, cut_requested_at INTEGER, phase_step TEXT, stage_request INTEGER, precheck_request INTEGER, rounds INTEGER, estimate_seconds INTEGER, estimate_terms TEXT, reason TEXT, health_attempts INTEGER, health_consecutive_ok INTEGER, health_first_ok_at INTEGER, health_last_probe_at INTEGER, health_last_error TEXT, finish_gate TEXT, notified_seq INTEGER, event_seq INTEGER, outcome TEXT, outcome_reason TEXT, warning TEXT, abort_requested_at INTEGER, source_reopened_at INTEGER, abort_finished_at INTEGER, effects_done INTEGER, release_pending INTEGER, open_record_pending INTEGER, created_at INTEGER, updated_at INTEGER)`; `CREATE UNIQUE INDEX IF NOT EXISTS self_serve_moves_open ON self_serve_moves(account, app, host) WHERE outcome = ''`. |
| +- **The request lifecycle** (one request at a time per row): `IssueSelfServeRequest(id, kind, phase string, pr persistedRequest, now time.Time) (issued bool, existing *persistedRequest, err error)` (`WHERE phase = ? AND outcome = '' AND (request_consumed <> 0 OR request_issued = 0)`: sets `request_issued = now, request_kind, request_phase, request_nonce, request_expect_inc, request_expect_seq, request_proposed, request_deadline = the issuing phase's step deadline, request_confirmed = 0, request_consumed = 0, request_voided = 0`; not issued → `existing` is the outstanding request, whatever its kind and phase); `ConsumeSelfServeRequest(id, nonce, nextStep string) (bool, error)` (`WHERE request_nonce = ? AND request_confirmed <> 0 AND request_consumed = 0`: `request_consumed = 1, phase_step = nextStep` in one statement; the standalone consume for a same-phase sequence: the substep advances exactly when the previous request is consumed, so a crash between them cannot lose the position); `AbandonSelfServeRequest(id, nonce string) (bool, error)` (`WHERE request_nonce = ? AND request_confirmed <> 0 AND request_consumed = 0`: `request_consumed = 1, request_abandoned = 1`; only abort, an expiry action and operator supersession call it: the request's write committed but its wait will never be satisfied, so the phase gives it up and proceeds); `ConfirmSelfServeRequest(id, nonce string, now time.Time) (bool, error)` (`WHERE request_nonce = ? AND request_consumed = 0 AND request_voided = 0 AND request_confirmed = 0`: `request_confirmed = now`; false → the caller re-reads the row: confirmed already → **resume** (success), voided or consumed → stop; for kinds `stage` **and `maintain`** also `stage_request = request_proposed`, for `precheck` also `precheck_request = request_proposed`); `VoidSelfServeRequest(id, nonce string) error` (`WHERE request_nonce = ? AND request_consumed = 0 AND request_confirmed = 0`: `request_consumed = 1, request_voided = 1`; a confirmed request is never voided; the phase function decides again); `AdvanceSelfServe(id, from, to, reason string, step, window time.Time, nonce string, now time.Time) (bool, error)`: one UPDATE with the predicate `phase = ? AND (abort_requested_at = 0 OR ? IN ('reverting','done','opened','health','finishing')) AND (? = '' OR request_nonce = ?)` that sets the phase, `phase_started_at`, the deadlines, `expiry_pending = 0, expiry_phase = ''` when `to <> expiry_phase`, and, when a nonce is given, additionally requires `request_nonce = ? AND request_confirmed <> 0 AND request_consumed = 0` and sets `request_consumed = 1` (**consumption and the advance are one statement**), and inserts the phase event in the same transaction (`event_seq = event_seq + 1`); a stale worker holding another nonce, or the same nonce after a void, cannot confirm or consume. |
| - Other store methods: `AdmitSelfServeMove(m SelfServeMove) (admitted bool, open *SelfServeMove, err error)`; `SelfServeMove(id string) (*SelfServeMove, error)`; `SelfServeMoveForHost(acct, app, host string) (open, terminal *SelfServeMove, err error)` (the open row, and the newest terminal row); `OpenSelfServeMoves() ([]SelfServeMove, error)` (`outcome = '' OR release_pending = 1 OR effects_done = 0 OR open_record_pending = 1`, **ordered by `created_at, id`**: the pass steps rows in that order; **per-box FIFO**: a row in `reserving` may reserve only when no older open row (by `created_at, id`) that shares any participant `BoxID` is still in `requested` or `reserving`; otherwise it waits with `reason` "waiting for an earlier move on this box"; rows with disjoint participants never wait on each other); `MarkSelfServeCut(id string, now time.Time, copyDeadline time.Time) (bool, error)` (`WHERE phase = 'ready' AND abort_requested_at = 0 AND cut_requested_at = 0 AND expiry_pending = 0`: a confirmation expiry already committed wins, and the cut door re-reads and answers 409 "the move expired. prepare again"; sets `cut_requested_at = now`, **`window_deadline = 0`** (the confirmation deadline is retired by the cut that won) and `step_deadline = copyDeadline` (the cut's copy wait) in the same UPDATE; the cut door's only write); the `ready` expiry rule applies only while `cut_requested_at = 0`; `SetSelfServeEstimate`, `RecordSelfServeHealth(id string, probeAt time.Time, ok bool, errText string) error` (`health_attempts + 1`; `consecutive_ok = CASE ok THEN consecutive_ok + 1 ELSE 0`; `first_ok_at` kept once set), `SetSelfServeFinishGate`, `SetSelfServeWarning`, `SetSelfServeNotifiedSeq(id string, seq uint64) error`, `SetSelfServeExpiry(id, phase string, kind string, sawDeadline time.Time, pending bool) (bool, error)` (`kind` is `window` or `step`; the UPDATE also records `expiry_kind = kind, expiry_deadline = sawDeadline`, and the expiry **action** later re-reads the row and runs only when `expiry_pending`, `expiry_phase`, `expiry_kind` and `expiry_deadline` still match what it is acting on (an expiry whose identity changed is dropped); the UPDATE's predicate is `phase = ? AND (window_deadline = ? for window, step_deadline = ? for step) AND (kind <> 'window' OR cut_requested_at = 0)`, so a worker whose snapshot predates a successful cut cannot record the obsolete confirmation expiry (`TestStaleExpiryLosesToTheCut`: a worker paused across the cut at T-1 resumes at T+1 and its `window` expiry is refused; the copy's `step` expiry after the cut is accepted; **expiry first**: the window expiry commits at T+ε, then the cut door → 409 expired, and the abort runs; a restart between the committed expiry and the cut → the same); `SetSelfServeExpiry` returning false → the worker re-reads the row and decides again`, `RequestSelfServeAbort(id string, phases []string, now time.Time) (bool, error)` (`WHERE phase IN (...) AND abort_requested_at = 0`), `SetSelfServeReopened`, `SetSelfServeAbortFinished`, `FinishSelfServeMove(id, outcome, reason, warning string, now time.Time) error` (`release_pending = 1`, `effects_done = 0`), `SetSelfServeEffectsDone(id string) error`, `ClearSelfServeRelease(id string) error`, `SetSelfServeOpenRecordPending(id string, pending bool) error`. |
| - Table `self_serve_events (id TEXT, seq INTEGER, phase TEXT, account TEXT, app TEXT, host TEXT, at INTEGER, reason TEXT, detail TEXT, emitted INTEGER, PRIMARY KEY (id, seq))`: `InsertSelfServeEventTx(tx *sql.Tx, e SelfServeEvent) error` (allocates `seq` from the row's `event_seq` in the same transaction; used by `AdvanceSelfServe` and by every refusal, fact decision, lane probe, health probe and timing); `UnemittedSelfServeEvents(limit int) ([]SelfServeEvent, error)`; `DeliverSelfServeEvent(ctx context.Context, e SelfServeEvent, deliver func(tx *sql.Tx) error) error` (one transaction: first `UPDATE self_serve_events SET emitted = 1 WHERE id = ? AND seq = ? AND emitted = 0`; zero rows → commit nothing and return (another worker delivered it); else `deliver(tx)` inserts the audit row through `auditActivityTx(tx, ...)`; a failure rolls both back). The terminal event is inserted by `FinishSelfServeMove` in the transaction that sets the outcome, so a crash between finalisation steps never allocates a second terminal event. Table `self_serve_timings (id TEXT, mark TEXT, account TEXT, app TEXT, host TEXT, at INTEGER, bytes INTEGER, PRIMARY KEY (id, mark))`. |
| - Phases: `requested`, `reserving`, `queued`, `reports`, `staging`, `prechecks`, `ready`, `restage`, `barrier`, `stopping`, `sealing`, `landing`, `landed`, `open-uncertain`, `opened`, `health`, `finishing`, `reverting`, `done`. |
| @@ -163,7 +163,7 @@ Builds on plan 1 (`2026-10-22-self-serve-moves-1-lifetimes-and-order.md`, rev 13 |
| **Interfaces:** |
| - `type persistedRequest struct { Kind, Phase, Nonce string; ExpectInc, ExpectSeq uint64; Proposed time.Time; Confirmed, Consumed, Voided bool }`; `func (s *Server) selfServeApply(ctx context.Context, t *leaseToken, row *SelfServeMove, req request, proposed *time.Time, mut func(at time.Time, rc provision.Receipt) func(*provision.Record) error) (*provision.Record, error)` with `type request struct { Kind, Phase string }`: (1) the judged record → `pr.ExpectInc = rec.Incarnation`, `pr.ExpectSeq = rec.RecordSeq`; `pr.Proposed = *proposed` when given (the open intent) else `max(now, latest stamp on the record + 1 ns)`; `pr.Nonce = random`; `IssueSelfServeRequest(id, kind, phase, pr, now)`: not issued and `existing == nil` → `errRequestRefused` (the row left the phase or ended; the step ends); `existing.Kind != req.Kind || existing.Phase != req.Phase` → return `errRequestForeign` (this caller never interprets another request; the step ends and reconciliation decides next tick); `existing.Confirmed && !existing.Consumed` → **resume**: return the record (the caller continues its wait and the notification catch-up); otherwise `pr = *existing` and go to (3); (2) `provision.SetRecordMoveAt(ctx', s, acct, app, host, provision.Expect{Incarnation: pr.ExpectInc, Seq: pr.ExpectSeq}, mut(pr.Proposed, Receipt{pr.Nonce}))` with the lease-bounded context and the write guard: **one CAS attempt against the record at that incarnation and sequence; no fresh-read retry inside**; (3) **confirmation** from a fresh judged read, whatever (2) returned: `rec` carries the receipt (`SelfServe.LastRequest == pr.Nonce`, or `LastMove.Request == pr.Nonce` for a clearing write) → `ConfirmSelfServeRequest(id, nonce, now)` (**the receipt alone confirms**; a box that has already advanced the record past the written state (seal → sealed, flip → landed) does not change that; the `confirmed` callback is dropped and phase progress is reconciliation's business) (false → re-read: already confirmed → resume; voided → stop); else `rec.Incarnation == pr.ExpectInc && rec.RecordSeq == pr.ExpectSeq` (unchanged) → the write did not land: repeat (2) (idempotent replay of the same bound write; bounded by the lease); else (the record advanced without the receipt: another writer, an operator) → `VoidSelfServeRequest(id, nonce)` and return `errRequestVoided`; the bound write makes this safe: a PUT transmitted against `ExpectSeq` cannot commit on a record that has moved past it; after a confirmed write: `bumpSerialErr` and, on success, `SetSelfServeNotifiedSeq(id, rec.RecordSeq)`; a failed bump leaves `notified_seq` behind and every later tick bumps again while `rec.RecordSeq != notified_seq`. A crash anywhere resumes at (3) on the next tick from the persisted request. |
| - `type provision.Expect struct { Incarnation, Seq uint64 }`; `provision.SetRecordMoveAt(ctx, s, acct, app, host string, expect Expect, mut) (*provision.Record, error)`: reads the judged record; `rec.Incarnation != expect.Incarnation || rec.RecordSeq != expect.Seq` → `ErrCASConflict` without writing; otherwise one `WriteRecord` against the observed ETag; a CAS conflict → `ErrCASConflict` (no retry). `SetRecordMove` (today's, with its retry) is no longer used by any self-serve or operator move writer. |
| -- `selfServeStep`: (1) the row's deadlines: an expired `StepDeadline`, or an expired `WindowDeadline` while `cut_requested_at = 0`, → `SetSelfServeExpiry(id, phase, true)` (no action yet); (2) the judged record; (3) **reconcile forward** by the compatible sets (Task 4): an unconfirmed request → step (3) of `selfServeApply` first, **except** when `expiry_pending` is set for the issuing phase and the record is at or before `offline-landed`: then the record is observed once; the receipt present → confirmed (the write won before the expiry, the expiry is post-write); absent → `fenceRecord` (plan 1; so the outstanding bound PUT can never land) and **the fenced observation is classified**: the receipt present in it → the write won during the fence: confirm and proceed forward (an open that committed is reconciled as opened, never aborted); absent → `VoidSelfServeRequest` and the expiry action runs (abort), **never a replay** (`TestExpiredOpenIsNeverReplayed`); then the row's phase not in the record's set → `AdvanceSelfServe` to the set's first phase (without the abort predicate for `opened`), which clears a foreign expiry; (4) the expiry action for an `expiry_pending` whose `expiry_phase == phase`: the record at or before `offline-landed` → abort; committed open → the blocked action (page); unreadable → `open-uncertain`; `health` → warning, page, `finishing`; (5) the abort step when requested and the record at or before `offline-landed`; (6) the phase function. Every wait for a report is for `Request`/`StagedRequest == request_proposed` of the **confirmed** request; **supersession by stamp**: each tick the waiting phase also reads the record's stamp for the awaited field (`StagedAt` for `stage`/`maintain`, `PrecheckAt` for `precheck`); when it is **later** than `request_proposed` while the receipt is unchanged, an operator round has superseded the job's (`OfflineRestage`/`OfflinePrecheck` from the operator doors move the stamp without a receipt), so the job `AbandonSelfServeRequest`s its request and, on the next step, issues a fresh bound round (the abandon is one UPDATE; a crash after it finds no outstanding request and issues; a crash before it repeats the comparison); the job never adopts an operator round as its own (`TestOperatorRoundSupersedesTheJobsWait`: a confirmed restage at S1, an operator stage to S2, the stager reports S2 only → the job abandons S1 and issues S3, the cut proceeds; the same in `staging/round` and `ready/round`; a crash between the abandon and the issue); the report is consumed by the `AdvanceSelfServe` that leaves the waiting phase, with the nonce, **or**, inside a phase that writes more than once (`stopping`: start then seal; `finishing`: gate 1 then gate 2; `staging`: round after round; `prechecks`: the precheck then the estimate; `ready`: each maintenance round), by `ConsumeSelfServeRequest(id, nonce)` once its wait is satisfied, immediately before the next request is issued, and **the phase function dispatches on `phase_step`**: `stopping` = `start` → `seal`; `finishing` = `gate1` → `gate2`; `prechecks` = `precheck` → `estimate`; `reverting` = `abort` → `finish`; `staging` and `ready` = `round` (repeated); a phase entered by `AdvanceSelfServe` from the **job's own** progression starts at its first substep (`phase_step` set by the advance); a phase entered by **reconciliation** starts at the substep the record dictates (below); a restart resumes the substep recorded, never the phase's first write (`TestSubstepsResumeAfterEveryCrash`: a crash after consuming start and before issuing seal re-enters at `seal`; a crash with the estimate request outstanding re-enters at `estimate` and resumes it; the same for gate 1 → gate 2 and abort → finish). **Abandonment** is for **leaving a wait**, never for retaining it: when abort is requested, when an expiry whose action is **abort** fires, or when an operator transition supersedes the phase, a confirmed request whose wait is unsatisfied is `AbandonSelfServeRequest`ed before the abort or the reconciliation step issues its own request; an expiry whose action is page-and-retain (`opened`, `finishing`, `reverting`, `open-uncertain`) or warn-and-finish (`health`) abandons nothing and the wait continues (`TestPageOnlyExpiriesKeepTheWait`: the open request confirmed, `opened` missing at the deadline, the page sent, then the target reports opened and the nonce-bearing advance succeeds). By phase: `requested`/`reserving` resolve or fence an issued queue request before finalising or releasing; `queued`/`reports` abandon a superseded wait, a queued abort clears and finalises atomically; `staging`/`prechecks`/`ready` may abandon on cancel or aborting expiry; `restage`/`barrier`/`stopping`/`sealing`/`landing`/pre-open `landed` may abandon on aborting expiry after resolving the uncertain write; `open-uncertain` abandons nothing; a confirmed clearing write is consumed with the terminal outcome (`TestAbandonedRequestsNeverStrandAbort`: a confirmed stage request whose copy never reports, then cancel → the abort request issues; a confirmed precheck whose box died, then the deadline → abort issues). **Clearing requests**: `OfflineFinish`, `OfflineAbortAt` and the abort path's finish are confirmed from `LastMove.Request`; the finaliser's `FinishSelfServeMove` consumes that confirmed request in the same transaction as the outcome, so a crash between confirmation and finalisation resumes at the finaliser, not at a re-issue. |
| +- `selfServeStep`: (1) the row's deadlines: an expired `StepDeadline`, or an expired `WindowDeadline` while `cut_requested_at = 0`, → `SetSelfServeExpiry(id, phase, true)` (no action yet); (2) the judged record; (3) **reconcile forward** by the compatible sets (Task 4): an unconfirmed request → step (3) of `selfServeApply` first, **except** when the request's own `request_deadline` has passed (whatever the row's current phase: `landed`, `open-uncertain` or any other) and the record is at or before `offline-landed`: then the record is observed once; the receipt present → confirmed (the write won before the expiry, the expiry is post-write); absent → `fenceRecord` (plan 1; so the outstanding bound PUT can never land) and **the fenced observation is classified**: the receipt present in it → the write won during the fence: confirm and proceed forward (an open that committed is reconciled as opened, never aborted); absent → `VoidSelfServeRequest` and the expiry action runs (abort), **never a replay** (`TestExpiredOpenIsNeverReplayed`); then the row's phase not in the record's set → `AdvanceSelfServe` to the set's first phase (without the abort predicate for `opened`), which clears a foreign expiry; (4) the expiry action for an `expiry_pending` whose `expiry_phase == phase`: the record at or before `offline-landed` → abort; committed open → the blocked action (page); unreadable → `open-uncertain`; `health` → warning, page, `finishing`; (5) the abort step when requested and the record at or before `offline-landed`; (6) the phase function. Every wait for a report is for `Request`/`StagedRequest == request_proposed` of the **confirmed** request; **supersession by stamp**: each tick the waiting phase also reads the record's stamp for the awaited field (`StagedAt` for `stage`/`maintain`, `PrecheckAt` for `precheck`); when it is **later** than `request_proposed` while the receipt is unchanged, an operator round has superseded the job's (`OfflineRestage`/`OfflinePrecheck` from the operator doors move the stamp without a receipt), so the job `AbandonSelfServeRequest`s its request and, on the next step, issues a fresh bound round (the abandon is one UPDATE; a crash after it finds no outstanding request and issues; a crash before it repeats the comparison); the job never adopts an operator round as its own (`TestOperatorRoundSupersedesTheJobsWait`: a confirmed restage at S1, an operator stage to S2, the stager reports S2 only → the job abandons S1 and issues S3, the cut proceeds; the same in `staging/round` and `ready/round`; a crash between the abandon and the issue); the report is consumed by the `AdvanceSelfServe` that leaves the waiting phase, with the nonce, **or**, inside a phase that writes more than once (`stopping`: start then seal; `finishing`: gate 1 then gate 2; `staging`: round after round; `prechecks`: the precheck then the estimate; `ready`: each maintenance round), by `ConsumeSelfServeRequest(id, nonce)` once its wait is satisfied, immediately before the next request is issued, and **the phase function dispatches on `phase_step`**: `stopping` = `start` → `seal`; `finishing` = `gate1` → `gate2`; `prechecks` = `precheck` → `estimate`; `reverting` = `abort` → `finish`; `staging` and `ready` = `round` (repeated); a phase entered by `AdvanceSelfServe` from the **job's own** progression starts at its first substep (`phase_step` set by the advance); a phase entered by **reconciliation** starts at the substep the record dictates (below); a restart resumes the substep recorded, never the phase's first write (`TestSubstepsResumeAfterEveryCrash`: a crash after consuming start and before issuing seal re-enters at `seal`; a crash with the estimate request outstanding re-enters at `estimate` and resumes it; the same for gate 1 → gate 2 and abort → finish). **Abandonment** is for **leaving a wait**, never for retaining it: when abort is requested, when an expiry whose action is **abort** fires, or when an operator transition supersedes the phase, a confirmed request whose wait is unsatisfied is `AbandonSelfServeRequest`ed before the abort or the reconciliation step issues its own request; an expiry whose action is page-and-retain (`opened`, `finishing`, `reverting`, `open-uncertain`) or warn-and-finish (`health`) abandons nothing and the wait continues (`TestPageOnlyExpiriesKeepTheWait`: the open request confirmed, `opened` missing at the deadline, the page sent, then the target reports opened and the nonce-bearing advance succeeds). By phase: `requested`/`reserving` resolve or fence an issued queue request before finalising or releasing; `queued`/`reports` abandon a superseded wait, a queued abort clears and finalises atomically; `staging`/`prechecks`/`ready` may abandon on cancel or aborting expiry; `restage`/`barrier`/`stopping`/`sealing`/`landing`/pre-open `landed` may abandon on aborting expiry after resolving the uncertain write; `open-uncertain` abandons nothing; a confirmed clearing write is consumed with the terminal outcome (`TestAbandonedRequestsNeverStrandAbort`: a confirmed stage request whose copy never reports, then cancel → the abort request issues; a confirmed precheck whose box died, then the deadline → abort issues). **Clearing requests**: `OfflineFinish`, `OfflineAbortAt` and the abort path's finish are confirmed from `LastMove.Request`; the finaliser's `FinishSelfServeMove` consumes that confirmed request in the same transaction as the outcome, so a crash between confirmation and finalisation resumes at the finaliser, not at a re-issue. |
| - **Reconciliation and recovery table** (the record is the truth): |
| | row phase | record | action | |
| @@ -201,7 +201,7 @@ Builds on plan 1 (`2026-10-22-self-serve-moves-1-lifetimes-and-order.md`, rev 13 |
| - Test: `internal/e2e/selfserve/cut_test.go`, `internal/console/offlinereturn_test.go` |
| **Interfaces:** |
| -- Phases and deadlines (`est` the row's terms): `restage`: substep `issue` → `selfServeApply(stage, OfflineRestage)` then the matching `staged`; substep `adopted` → straight to `barrier`; `max(5 min, 2 × est.Copy)`; expiry → abort → `barrier`. `barrier`: `selfServeApply(precheck, OfflinePrecheck)`; both participants' reports passing the decisions and the book comparison with zero deferrals; `max(5 min, 2 × est.Copy)`; a failed fact or expiry → abort → `stopping`. `stopping`: `offlineRecordChecks` + `offlineRegionChecks(..., stage: false, requireIDs: true, "")` on the fresh record, then `selfServeApply(start, OfflineStartStaged, )`; then the seal check with participants → `selfServeApply(seal, OfflineSeal, )`; `max(5 min, 2 × est.Stop)`; expiry → abort. `sealing`: the sealer's `sealed`; `max(10 min, 3 × est.Copy)`; expiry → abort. `landing`: `offlineFlipCheck` → `selfServeApply(flip, OfflineFlip, )`; the pinned target's `landed`; `max(10 min, 3 × est.Land)`; expiry → abort. `landed`: `offlineOpenCheck` → write the open record (create-only, `OpenedAt = proposed`; `open_record_pending = 1` until it exists) → `selfServeApply(open, OfflineOpenTransition, )`; a refusal or `max(3 min, 2 × est.Open)` **while the record is `offline-landed`** → abort; an unconfirmable write (the record unreadable) → `open-uncertain`. `open-uncertain`: re-read each tick: `offline-open` → `opened`; `offline-landed` → `landed` (the deadline continues); 5 min unreadable → page, stay. `opened`: the target's `opened`; `max(3 min, 2 × est.Open)`; expiry → blocked: page, retry. `health`: a probe when `now - HealthLastProbeAt >= 30 s` (`probeTarget`: the target box's `EIP` from its edges file, TLS with the host as SNI and `Host`, `GET <ReadyPathOrDefault>?carlos-probe=<random>`, `Cache-Control: no-cache`, no redirects, 30 s; OK when 2xx and `x-carlos-version` equals the sealer report's `Release`; plan 3 adds `X-Carlos-Probe`/`X-Carlos-Served-By`); `RecordSelfServeHealth`; `HealthConsecutiveOK >= 2` → `finishing`; expiry `max(3 min, 2 × est.Promote) + 30 s` → `SetSelfServeWarning("the account did not answer on <region> within <n> minutes")`, page, `finishing`. `finishing`: gate 1 (`offlineFinishCheck` opened + every pinned source parked) → `selfServeApply(finish1, OfflineFinishingTransition)`, `FinishGate = lifted`; gate 2 (lifted + every pinned source gone) → `selfServeApply(finish, OfflineFinish, )`; 10 min each → blocked: page, retry. `reverting`: `OfflineAbort` once (or `OfflineAbortAt` from queued); `SourceReopenedAt` when (no tombstone, or the sealer's `reverting`/`reverted` with `TombstoneLifted`) **and** every pinned source's `reverted`; `AbortFinishedAt` when `offlineStepFor` at `offline-reverting` passes → `OfflineFinish`; **the target is never required**: a target that stopped reporting after landing does not block the source's reopening; the abort's `Detail` records "target <label> did not report; its staged copy is left for cleanup"; 10 min → blocked: page, retry. |
| +- Phases and deadlines (`est` the row's terms): `restage`: substep `issue` → `selfServeApply(stage, OfflineRestage)` then the matching `staged`; substep `adopted` → straight to `barrier`; `max(5 min, 2 × est.Copy)`; expiry → abort → `barrier`. `barrier`: `selfServeApply(precheck, OfflinePrecheck)`; both participants' reports passing the decisions and the book comparison with zero deferrals; `max(5 min, 2 × est.Copy)`; a failed fact or expiry → abort → `stopping`. `stopping`: `offlineRecordChecks` + `offlineRegionChecks(..., stage: false, requireIDs: true, "")` on the fresh record, then `selfServeApply(start, OfflineStartStaged, )`; then the seal check with participants → `selfServeApply(seal, OfflineSeal, )`; `max(5 min, 2 × est.Stop)`; expiry → abort. `sealing`: the sealer's `sealed`; `max(10 min, 3 × est.Copy)`; expiry → abort. `landing`: `offlineFlipCheck` → `selfServeApply(flip, OfflineFlip, )`; the pinned target's `landed`; `max(10 min, 3 × est.Land)`; expiry → abort. `landed`: `offlineOpenCheck` → write the open record (create-only, `OpenedAt = proposed`; `open_record_pending = 1` until it exists) → `selfServeApply(open, OfflineOpenTransition, )`; a refusal or `max(3 min, 2 × est.Open)` **while the record is `offline-landed`** → abort; an unconfirmable write (the record unreadable) → `open-uncertain`. `open-uncertain`: re-read each tick: `offline-open` → `opened`; `offline-landed` with the open request's `request_deadline` not passed → `landed`; with it passed → the no-replay rule of Task 7 (fence, classify: the receipt present → `opened`; absent → void and abort); 5 min unreadable → page, stay (`TestOpenUncertainNeverReplaysAnExpiredOpen`: enter `open-uncertain` with the PUT uncommitted, restart after the deadline, reads recover with the record landed → fenced, voided, aborted; the PUT racing the fence and winning → confirmed, `opened`). `opened`: the target's `opened`; `max(3 min, 2 × est.Open)`; expiry → blocked: page, retry. `health`: a probe when `now - HealthLastProbeAt >= 30 s` (`probeTarget`: the target box's `EIP` from its edges file, TLS with the host as SNI and `Host`, `GET <ReadyPathOrDefault>?carlos-probe=<random>`, `Cache-Control: no-cache`, no redirects, 30 s; OK when 2xx and `x-carlos-version` equals the sealer report's `Release`; plan 3 adds `X-Carlos-Probe`/`X-Carlos-Served-By`); `RecordSelfServeHealth`; `HealthConsecutiveOK >= 2` → `finishing`; expiry `max(3 min, 2 × est.Promote) + 30 s` → `SetSelfServeWarning("the account did not answer on <region> within <n> minutes")`, page, `finishing`. `finishing`: gate 1 (`offlineFinishCheck` opened + every pinned source parked) → `selfServeApply(finish1, OfflineFinishingTransition)`, `FinishGate = lifted`; gate 2 (lifted + every pinned source gone) → `selfServeApply(finish, OfflineFinish, )`; 10 min each → blocked: page, retry. `reverting`: `OfflineAbort` once (or `OfflineAbortAt` from queued); `SourceReopenedAt` when (no tombstone, or the sealer's `reverting`/`reverted` with `TombstoneLifted`) **and** every pinned source's `reverted`; `AbortFinishedAt` when `offlineStepFor` at `offline-reverting` passes → `OfflineFinish`; **the target is never required**: a target that stopped reporting after landing does not block the source's reopening; the abort's `Detail` records "target <label> did not report; its staged copy is left for cleanup"; 10 min → blocked: page, retry. |
| - **The finaliser** `selfServeFinalise(ctx, t, row, outcome, reason, warning)`: (1) `FinishSelfServeMove` (outcome; `release_pending = 1`, `effects_done = 0`; consumes a confirmed clearing request and inserts the terminal event, one transaction); (2) the terminal effects: the timings (Task 9), the terminal event, `bumpSerialErr`; `SetSelfServeEffectsDone`; (3) `releaseBoxes(move, row.ID)`; (4) `ClearSelfServeRelease`. A crash between any two leaves the row in `OpenSelfServeMoves` and the next tick resumes at the first unfinished step. The operator doors that clear a self-serve move's record route through it. |
| - **The operator return handshake (§7.5):** (1) `POST .../move/offline/return {"region", "return_of"}` (operator-only, `offlineGate`): the open record at `OpenRecordKey(host, return_of)` must exist with `To == rec.Region`, `From == region`, `Incarnation == rec.Incarnation`, and the record carries no move; else 409 "no completed move <id> from <region> to <rec.Region> for this instance"; writes `control/offline-moves/<host>/<return_of>/return/<requested_at unix nanos>-<console id>-<8 random hex>.json` `{"requested_at", "region", "console", "key"}` create-only (the **request key** is the request's identity; a new call makes a new request; the current request is the key that sorts **last** by `(requested_at, key)`; older ones are ignored) and answers 202 "return requested; the target box will report its leftovers". (2) The moves pass of every box in `region` lists `control/offline-moves/<host>/*/return/` for hosts whose open record names `From == its region` and, when the open record carries `Participants`, whose `Source` includes its own `BoxID` **and** (when pinned) its own `InstanceID` (an id-less forward keeps **`BoxID` membership**: the box must be a forward source participant by `BoxID`, only the instance comparison is dropped; the region rule applies solely to a forward **without** `Participants`); a box whose `BoxID` matches but whose `InstanceID` differs does nothing; it reads the newest request, fills `Leftovers` against the open record (`OpenedAt`, `Layout`, `MoveID`) and writes `control/offline-moves/<host>/<return_of>/reports/return-ready-<box id>-<request key>.json` **create-only** (an immutable report per request: `Op = return_of`, role `target`, phase `ready`, `Request = requested_at`, `RequestKey = <the request key>` (a new `OfflineReport` field), `BoxID/InstanceID`), without touching the record; a create-only conflict means this request was already answered; a newer request gets its own report; nothing is ever overwritten, so no ordering between writers matters. (3) The operator's `stage` (or `start`) door with `return_of`: `readReturnReport(ctx, host, returnOf)` reads the report keyed by the current request's key for the selected target box, which must exist, within 10 minutes, from a box whose `BoxID` **and `InstanceID`** match a forward source participant (or, **only for a forward without `Participants`**, a box in `From`; with `Participants` the predicate is `participantMatches(p, report)`: `p.BoxID == report.BoxID && (p.InstanceID == "" || p.InstanceID == report.InstanceID)`, the same function discovery uses) **and** equal the target `offlineSelect` chose for the return (the report authorises returning onto that box alone; a report from another forward source does not); the open record's `Incarnation` must equal the current record's **again here** and inside the guarded write (`OfflineStage`/`OfflineStart` with `returnOf` refuse with `ErrMoveConflict` "the instance was recreated since that move" when `rec.Incarnation != open.Incarnation`, the open record being passed to the mutation); then `targetReadyDecision` in return mode accepts **exactly**: `Lock` present with owner `moves.TombstoneOwner(return_of)` and lifted; `InstancesPrefix.Objects` (current versions, `lock.json` excluded) all with `LastModified < OpenedAt` (any newer → `NewerObjects`, refused); `DataPaths["/data/instances/<host>"]` absent or present-and-empty; for `Layout == nested` also `DataPaths["/data/.instances/<host>"]` absent or present-and-empty, and the mount view likewise; for `flat` the nested pair must be absent; park directories: each present one must be `<return_of>/source/` or `<return_of>/sealer/` (absence allowed: cleanup may have removed them; any other → refused); `LiveLink`/`ProgramsFile` naming the channel build; anything else → the ordinary refusal. Then `OfflineStage(..., returnOf)`/`OfflineStart(..., returnOf)` carry it on the record. `type moves.ReturnLeftovers struct { ForwardID string \`json:"forward_id"\`; LockOwner string \`json:"lock_owner"\`; LockLifted bool \`json:"lock_lifted"\`; ParkDirs []string \`json:"park_dirs"\`; EmptyNested []string \`json:"empty_nested"\`; NewerObjects []string \`json:"newer_objects,omitempty"\`; Layout string \`json:"layout"\` }`. |
| - Paging: `alerting.Page` through `MovePageAttempt`/`SetMovePageAttempt` (seconds encoding, unchanged) with keys `selfserve:<id>:<condition>` every 10 minutes for: `opened` past its deadline; a health failure; `finishing`/`reverting` past 10 minutes; `open-uncertain` unreadable past 5 minutes; an open record still pending after 5 minutes; a reservation the reclaim rule cannot reclaim older than the grace period (the per-tick scan, also at start); a lane probe failing on a configured pair; the attempt is recorded **before** the page is sent, as the existing paging does. |