amadan
docs: region HA stage B plan rev 5.2, Astra round 7
Finishes round 6's finding 3. A completed relocation is now finished from the claim's own record on every pass rather than from the row's hold. The order is discharge, hold release, owned-fence lift, then the mark last, and every step is idempotent, so a failed mark CAS, a lift that never ran, or a restart after the hold release all resume on the next pass (ruling 80, tested through RelocationRuntime.once). The discharge barrier is one function called by both RelocationStartGuard and the shared StartGuard, so a successor move, delete, purge or placement waits for it exactly as a relocation does (ruling 81). The StartGuard line is named change N13 and claim ask C12 on #236; until C12 is confirmed, relocation stays off. Each task was recompiled and retested in order in a throwaway copy, and the plan's code was extracted back out and matched the checked commits exactly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+435
−87
docs/superpowers/plans/2026-10-10-region-ha-1b-next-stage-b.md+435 −87
diff --git a/docs/superpowers/plans/2026-10-10-region-ha-1b-next-stage-b.md b/docs/superpowers/plans/2026-10-10-region-ha-1b-next-stage-b.md| index 7bba0ab83..33a65168f 100644 |
| --- a/docs/superpowers/plans/2026-10-10-region-ha-1b-next-stage-b.md |
| +++ b/docs/superpowers/plans/2026-10-10-region-ha-1b-next-stage-b.md |
| @@ -1,4 +1,4 @@ |
| -# Region HA 1b-next Stage B: Relocation Wired Into the Claim Assembly Implementation Plan (rev 5.1) |
| +# Region HA 1b-next Stage B: Relocation Wired Into the Claim Assembly Implementation Plan (rev 5.2) |
| > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
| @@ -8,14 +8,23 @@ |
| **Tech Stack:** Go 1.26, module `github.com/carlosframework/platform`; `release.Store`; `hostclaim`, `hostactors`, `hostfence`, `hostguard`, `hostexec`, `hostbox`, `hostrelocate`, `hostfiles`, `genselect`, `moves`, `lease`, `activator`, `registry`, `appconfig`, `edgesfile`, `instancekey`, `release`. |
| -**Spec:** `docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md` (rev 5.4 with §24): §5.3, §5.8, §6, §8. The 1b-next plan's rulings 1 to 27 stand; this plan's rulings continue the numbering at 28. Claim Plan 11 (`docs/superpowers/plans/2026-10-09-host-lifecycle-claim-11-region-ha-integration.md` on `origin/design-replica-ownership`, "Region HA integration and rollout", items 1 to 7) and the #236 answers (R1 to R10, C1 to C5, M1 to M3, J2, J3) are the contracts. Rev 5.1's code was compiled and run on this branch at b9a753f0 with `internal/instancekey` from `instance-key-build` Task 1 as built (e8487c3a; its API is unchanged since); file:line cites are to this branch. |
| +**Spec:** `docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md` (rev 5.4 with §24): §5.3, §5.8, §6, §8. The 1b-next plan's rulings 1 to 27 stand; this plan's rulings continue the numbering at 28. Claim Plan 11 (`docs/superpowers/plans/2026-10-09-host-lifecycle-claim-11-region-ha-integration.md` on `origin/design-replica-ownership`, "Region HA integration and rollout", items 1 to 7) and the #236 answers (R1 to R10, C1 to C5, M1 to M3, J2, J3) are the contracts. Rev 5.2's code was compiled and run on this branch at b9a753f0 with `internal/instancekey` from `instance-key-build` Task 1 as built (e8487c3a; its API is unchanged since); file:line cites are to this branch. |
| **Rev 5 in one paragraph.** Rev 5 is the build-ready consolidation of rev 4.2 against the pre-flight scan (`.superpowers/sdd/2026-10-10-region-ha-1b-next-stage-b/preflight-scan.md`, K1 to K25 and D1 to D9). Every task's code was written out, compiled and run task by task in a throwaway copy, and each task's text below is generated from its own commit there: its Files, its tests, its `-run` filter and its `git add` match its code. No task depends on a later task. Nothing in a task refers to a revision section; the revision layers (rev 2 to 4.2) are folded into the tasks, and their disposition tables are kept below as history. Every test has a body that asserts behaviour, and every function has its body. C10 is committed in Task 9 as region HA code behind the activation gate. Four promised tests are withdrawn with reasons (R5-2, R5-4), and one rev 1 change is withdrawn (R5-1); they are listed under "Unresolved in rev 5". |
| **Rev 5.1 in one paragraph.** Astra's round 6 on rev 5 found four P2s and no P1, accepted R5-1, R5-3 (with Task 10b mandatory before activation) and R5-4, and judged R5-2's replacement coverage insufficient for a successor relocation. Rev 5.1 fixes each finding the way the review directs: the probe stop's activator fence is owned and checked under the activator's lock (findings 1 and 2, rulings 75 and 76), a successor cannot start from a box that has not discharged its completed relocation (finding 3, ruling 77), and the landing conversion runs under the worker's own checks (finding 4, ruling 78). Task 10b is named and mandatory before activation (ruling 79). Every task was rebuilt and rerun in order (the record below). |
| +**Rev 5.2 in one paragraph.** Astra's round 7 closed round 6's findings 1, 2 and 4 and left finding 3 with two P2s. Rev 5.2 fixes both. The completed relocation is finished from the claim's record on every pass, never from the row, with the mark last and every step idempotent (ruling 80). The discharge barrier is one function that both the shared `StartGuard` and the relocation start guard call, so a successor move, delete, purge or placement waits for it exactly as a relocation does (ruling 81; named change N13, claim ask C12). Every task was rebuilt and rerun in order. |
| + |
| --- |
| +## Rev 5.2: Astra round 7 |
| + |
| +| # | Finding | Fix in rev 5.2 | Where | |
| +|---|---|---|---| |
| +| 1 (P2) | `relocationReleaseHold` released the hold before the lift and the mark, and the pass found completions only through a row still held, so a failed mark CAS, a lost race or a crash after the release made the retry unreachable, blocked successors for good and could leave admission fenced. | Ruling 80. `RelocationRuntime.finishCompletion` replaces `releaseCompletedHold`: every pass, for every claim whose serving op records a completion on this box that is not yet `Discharged`, it calls `Finish` with that op, reading the claim afresh after the pass's steps. The runtime no longer reads the registry. The order is: while the row is held for the op, discharge (C10), then release the hold; then lift the owned fence; then the mark, last. Every step is idempotent and the record stays unmarked until the end, so a failure or a crash at any point resumes on the next pass. A lift that never ran is retried, and after a restart there is no fence to lift. Test through `RelocationRuntime.once`: `TestRelocatePassFinishesACompletionFromItsRecord`, which covers one pass, a failed discharge, a failed mark CAS (the next pass marks with the row already released), a lift that never ran after the hold went, and a restart after the hold release. Each case ends discharged exactly once, unheld, unfenced, marked, with the barrier open and the op's journals swept. | 9 (comment), 10 | |
| +| 2 (P2) | The barrier was only in `RelocationStartGuard`; a B to C move through `BeginMove` and `StartGuard` could start before B's discharge. | Ruling 81. One function, `hostclaim.completionBarrier(c, fencedSource)`, is called by `RelocationStartGuard` (with a failover's fenced source) and by `StartGuard` (N13), so moves, deletes, purges and placements wait exactly as relocations do. A completion box on `lost` holds nothing. `StartGuard` is claim-owned: the change is one line, named N13, and asked as **C12** on #236. Fallback: relocation stays off until C12 is confirmed (activation gate). Test: `TestMoveSuccessorWaitsForTheCompletedDischarge` (A to B through every real transition; `BeginAdmittedMove` and `BeginDelete` from B refused with the claim unchanged; a lost B holds nothing; after the mark the real move start CAS succeeds). | 8a | |
| + |
| ## Rev 5.1: Astra round 6 |
| | # | Finding | Fix in rev 5.1 | Where | |
| @@ -101,21 +110,21 @@ The labels and their order are rev 4.2's (8a, 8b, 8c, 8d, 9, 10, 11). What moved |
| | The lease-region test | 8d (`relocation_rev2_test.go`) | 10 | |
| | The hold-only reconcile change | 9 | withdrawn (R5-1) | |
| -## Per-task compile and test record (rev 5.1) |
| +## Per-task compile and test record (rev 5.2) |
| -Each task was committed in a throwaway copy of this branch (code as at b9a753f0; the branch has only plan commits since), on top of `internal/instancekey` from `instance-key-build` Task 1 as built (e8487c3a), in the order 8a, 8b, 8c, 8d, 9, 10, and checked at its own commit with `GOFLAGS=-buildvcs=false`, niced, and the socket tests run outside the sandbox with `TMPDIR` in a private `/tmp/k.XXXX`. Each task's code in this plan was then extracted from the text into a second copy and found identical, file for file, to the commit that was checked. |
| +Each task was committed in a throwaway copy of this branch (code as at b9a753f0; the branch has only plan commits since), on top of `internal/instancekey` from `instance-key-build` Task 1 as built (e8487c3a), in the order 8a, 8b, 8c, 8d, 9, 10 (rev 5.2), and checked at its own commit with `GOFLAGS=-buildvcs=false`, niced, and the socket tests run outside the sandbox with `TMPDIR` in a private `/tmp/k.XXXX`. Each task's code in this plan was then extracted from the text into a second copy and found identical, file for file, to the commit that was checked. |
| | Task | `go build ./...` | `go vet` (touched packages) | `gofmt -l` | Tests at the task's commit | |
| |---|---|---|---|---| |
| | IK-1 (precondition) | clean | clean: instancekey | empty | `instancekey` passes in full | |
| -| 8a | clean | clean: host, hostclaim, hostrelocate, activator | empty | `hostclaim` and `hostrelocate` pass in full; the activator's fence and maintenance tests (the owned fence's three among them) pass; host's release, census, sweep and participation tests and the Plan 9 and Plan 11 regressions pass | |
| +| 8a | clean | clean: host, hostclaim, hostrelocate, activator, hostops | empty | `hostclaim`, `hostrelocate` and `hostops` (whose purge and placement starts pass `StartGuard`) pass in full; the activator's fence and maintenance tests (the owned fence's three among them) pass; host's release, census, sweep and participation tests and the Plan 9 and Plan 11 regressions pass | |
| | 8b | clean | clean: host, hostrelocate | empty | `hostrelocate` passes in full; the build and verifier tests pass | |
| | 8c | clean | clean: host, hostbox, hostfiles, registry, provision | empty | `registry` and `hostfiles` pass in full; hostbox's adapter and census tests, provision's reconcile and route tests and host's install tests pass | |
| | 8d | clean | clean: host | empty | the owner, wake, attempt and root tests pass, with the moves probe-owner, Plan 9 D5, Plan 11 and claim-launch regressions | |
| | 9 | clean | clean: host, hostbox | empty | hostbox's reconcile tests and host's probe-stop (restart, failed marker write), hold-release, successor, discharge, handover, sweep, Plan 9, Plan 11 and handover regressions pass | |
| -| 10 | clean | clean: host, hostrelocate, edgesfile, console | empty | `hostrelocate` and `edgesfile` pass in full; the console's edges, eligibility, placement, region, executor and claim-runtime tests pass; 79 region HA host tests pass with the Plan 9, Plan 11, claim-launch, moves and handover regressions | |
| +| 10 | clean | clean: host, hostrelocate, edgesfile, console | empty | `hostrelocate` and `edgesfile` pass in full; the console's edges, eligibility, placement, region, executor and claim-runtime tests pass; 80 region HA host tests pass with the Plan 9, Plan 11, claim-launch, moves and handover regressions | |
| -At Task 10 the whole `internal/activator` (142s), `internal/hostclaim`, `internal/hostrelocate`, `internal/hostbox` and `internal/host` (45s) packages also pass, once, outside the sandbox; the console's whole package passed at rev 5 and rev 5.1 does not touch it. At Task 10, `GOOS=darwin`, `GOOS=windows` and `GOARCH=arm64` builds of `./...` are clean, and so is `go vet` of host, hostrelocate, hostclaim and edgesfile for all three and of activator for darwin. Two windows vets fail at b9a753f0 already and this plan does not touch either cause: `./internal/hostbox/` (`plan9_fix2_minors_test.go:100`, `undefined: plan9D1Probe`) and `./internal/activator/` (`activator_test.go:4528`, `undefined: unix.Mkfifo`). |
| +At Task 10 the whole `internal/hostclaim`, `internal/hostops`, `internal/hostrelocate`, `internal/hostbox` and `internal/host` (47s) packages also pass, once, outside the sandbox (rev 5.2). The whole `internal/activator` package passed at rev 5.1 and rev 5.2 does not touch it; the console's passed at rev 5 and neither revision since touches it. At Task 10, `GOOS=darwin`, `GOOS=windows` and `GOARCH=arm64` builds of `./...` are clean, and so is `go vet` of host, hostrelocate, hostclaim, hostops and edgesfile for all three (and, at rev 5.1, of activator for darwin). Two windows vets fail at b9a753f0 already and this plan does not touch either cause: `./internal/hostbox/` (`plan9_fix2_minors_test.go:100`, `undefined: plan9D1Probe`) and `./internal/activator/` (`activator_test.go:4528`, `undefined: unix.Mkfifo`). |
| ## Disposition history (rev 2 to rev 4.2) |
| @@ -197,6 +206,10 @@ Astra's round 1 (`CODEX-PLAN-REGION-HA-STAGE-B-R1.md`) raised 10 P1s and 4 P2s a |
| > **C10 (rev 4).** A CC9 discharge of a region HA relocation participation by its persisted binding and exact completed probe evidence. In every `productionClaimLocal` cleanup path (the slot's close, `prepareWith`'s close of a retained slot, and `clearUntracked` after a restart), before any ordinary cleanup, read the claim. If it now serves on the binding's box, its op journals `completed_relocation` for exactly the binding's op, its holder is that record's stopped probe session (session, stamp, generation, writer box, incarnation equal to the binding's), and root's completed `writer-session-<S>` receipt with the recorded fence id, phase and digest is in this box's guard journal and verifies, then the participation is discharged: no stop is requested and `ClearStale` follows. Otherwise main's cleanup runs unchanged. Region HA guarantees the precondition: it releases the row's hold and the activator fence only after this discharge has cleared the slot and the record, so no serving admission can move the holder first. Any error while making this check, a failed claim read included, ends the cleanup with that error: it never falls through to main's ordinary cleanup, so no serving-shaped stop or `BeginFence` runs, and the next pass retries. |
| **Rev 5 (rule 5 of the consolidation):** the code is committed in Task 9 as region HA code (`claimparticipation_completed.go`, and `cleanup` taking the binding at its call sites), behind the activation gate. The claim session still confirms it on #236; the flags stay off until it does, and if it is refused the commit is reverted. |
| +- **C12** (rev 5.2; **code committed in Task 8a as N13**): the completion barrier in `StartGuard`. Fallback: relocation stays off until it is confirmed (activation gate). The text for #236: |
| + |
| +> **C12 (region HA rev 5.2).** One line in `hostclaim.StartGuard` (ops.go): after its kind check, `completionBarrier(c, "")`. While a serving op journals `completed_relocation` without `discharged`, the box the relocation completed on still holds that op's participation slot, so main's `Prepare` for any successor operation's participation there answers `ErrLost`. Every start from serving that goes through `StartGuard` (`BeginMove`/`BeginAdmittedMove`, `BeginDelete`, `hostops.BeginPurge`, `hostops.BeginPlacement`) is then refused with `ErrPendingChore` until that box marks the completion discharged (`MarkCompletedDischarged`, written by the box after C10's discharge, its hold release and its fence lift). A completion box on `lost` holds nothing, so the barrier does not apply to it. `completionBarrier` is region HA's (relocation_probestop.go) and is also the barrier `RelocationStartGuard` uses, so there is one implementation. It changes nothing for a claim that has never completed a relocation. Please confirm, or name the guard you would rather it sat in. |
| + |
| - **C11** (rev 3): `claimLaunchAuthority` pins a relocating launch to `install_stamp` only for P's opening and validates a later attempt at the admitted writer's stamp through root's attempt reader. Fallback: one launchable attempt per target (ruling 58). |
| ### Moves asks |
| @@ -219,6 +232,7 @@ Each is minimal and changes no claim semantics; anything that would is an ask in |
| | N9 | `internal/host/claimcensus.go` | `relocate` census-opaque | Region HA's self-verifying journals, swept by op | 8a | |
| | N11 | `internal/host/claimcensus.go` | `moves/held-stage` and `moves/held-staged` census-opaque | M-K25's fallback; moves' relay files, validated by root before any effect | 8c | |
| | N12 | `internal/host/claimparticipation.go` | `productionClaimLocal.lockRoot`, a test seam: locks under a test directory and no update-floor check; empty in production | Production is unchanged; it lets tests open main's real controllers through `Prepare` | 8a | |
| +| N13 | `internal/hostclaim/ops.go` | `StartGuard` calls `completionBarrier(c, "")` after its kind check | It changes claim semantics for successors of a completed relocation, so it is also asked as **C12**; unreachable for a claim that never completed a relocation | 8a | |
| N7 and N8 were never used; N10 was withdrawn in rev 4.1. C10's code (Task 9) is the claim ask's content, committed as region HA code, not a named change. |
| @@ -299,6 +313,7 @@ Each is modelled as a dependency with a fallback. None is assumed. Rev 2 added C |
| | **C6** | claim | A kind-scoped CC9 discharge for a relocation target the op excluded (`replaceTargetSlot` removes it from participants, relocation.go:1091) while its unopened install and held row remain. | **Rev 3, ruling 63: two safe outcomes.** If the slot was captured at the target's own install stamp, main's `CheckLocal` finds the complete installation and cleanup succeeds, so `ClearStale` removes the denial while the inert, held, never-reactivated installation waits for its `park-local`. If it was captured earlier, the census refuses the row and the denial stands until `park-local`. No writer can start there either way. Rev 5 tests the first outcome (Task 8a, the op ending without the box); the second is main's census, unchanged. | rulings 42, 63 | |
| | ~~C7~~ | claim | **Withdrawn in rev 3 (ruling 56).** `ColdHandover.Run` accepts the probe stop's exact receipt as prior beside a validated cold serving successor without matching op ids; the agent supplies it through N6. Nothing is re-stopped or relabelled. | none | ruling 56 | |
| | **C10** | claim | A CC9 discharge of a relocation participation keyed by the persisted binding and the exact completed probe evidence, first in every cleanup path, the slot's close and `clearUntracked` alike (text under "Claim asks"). **Rev 5 commits its code in Task 9** as region HA code; the claim session confirms it on #236. | **An activation gate.** The flags stay off until the claim confirms C10. If it is refused, Task 9's C10 commit is reverted and region HA releases nothing (ruling 66): the target stays held and unserved, alerting once; nothing unsafe is admitted. | rulings 56, 66, 70, Task 9 | |
| +| **C12** | claim | `StartGuard` passes the completion barrier (text under "Claim asks"). **Rev 5.2 commits it in Task 8a** (N13). | Relocation stays off until C12 is confirmed on #236; if it is refused, N13 is reverted and the guard it names takes the one-line call instead. | 8a; activation | |
| | **IK-1** | instance-key plan, Task 1 (`instance-key-build` Task 1 as built, e8487c3a) | `internal/instancekey` on main (`ParseEnv`, `Effective`, `Digest`, `Binding`, `Adoption`, `AdoptionRecord`, `ParseAdoption`, `Binds`, `VerifyDelivered`, `AdoptionItem`, `AdoptionSuffix`, `SourcePlatform`, `SourceConfig`). Rev 5 compiled against e8487c3a's package. | Stage B's pinner, install and flush import it, so stage B merges after it; until then relocation is not built. | rulings 55, 68 | |
| | **O6** | instance-key plan rev 3 (claim Plan 9 Group D's launcher) | Claim launches bind a control socket and mount the adoption verb, with `CARLOS_INSTANCE_KEY_SOURCE` (instance-key ruling 23). | **An explicit refusal.** An app whose key is not its own `instance.key` cannot report adoption on a claim box, so its relocation refuses (`stuck`). It is not HA-supported until O6. | ruling 55, checklist item 9 | |
| | **O7** | instance-key plan rev 3 (moves) | Moves' journalled artifact lists retire the flat `<host>.instance-key-adoption.json` (instance-key ruling 24). O6 does not ship before O7. | None needed before O6: no record exists anywhere until O6 ships. | ruling 55 | |
| @@ -372,9 +387,11 @@ Rulings 28 to 41 are rev 1's; 42 to 54 rev 2's; 55 to 64 rev 3's; 65 to 69 rev 4 |
| 74. **Every task compiles and passes its tests at its own commit** (rev 5). Code lives in the earliest task that uses it; a later task may extend an earlier task's file, never the reverse. *Cost if wrong:* none. |
| 75. **The probe stop's fence is owned** (Astra round 6 findings 1 and 2; amends 71). `relocationStopProbe` persists `{host, session, owner}` in `relocate/ops/<op>/probe-fence.json` before asking the activator, with a fresh random owner unless one is persisted for this host and session. `activator.FenceAndStopAs` installs only on an unfenced host, or meets again exactly the fence it installed under that owner at that generation, deciding under the lock that installs; any other fence is `ErrFenceHeld`, never replaced. `LiftFenceAs` lifts only that fence. A failed marker write installs nothing; a restart empties the activator, so a surviving owner matches no fence. *Cost if wrong:* a probe stop waits behind another operation's fence until ruling 67's bound. |
| 76. **The target ends a completed relocation by marking it discharged** (Astra round 6 findings 2 and 3; extends 66). `relocationReleaseHold`: while the row is held for the op, the holder must still be the probe session, then main's discharge (C10), then `registry.ReleaseHold`; then `LiftFenceAs` under the persisted owner; then `hostclaim.MarkCompletedDischarged`. Once the row is no longer held for the op, a replay only lifts and marks; a missing row is refused. The journal sweep keeps a completed op's directory until the mark. *Cost if wrong:* a replay after a crash between the release and the mark repeats one lift. |
| -77. **No successor relocation starts from an undischarged completion** (Astra round 6 finding 3). `RelocationStartGuard` refuses with `ErrPendingChore` while the serving op's `CompletedRelocation` is not `Discharged`, unless the start is a failover citing the completion box's fence (a dead box's slot binds nothing on the new target). *Cost if wrong:* a drain from a just-completed target waits one pass. |
| +77. **No successor relocation starts from an undischarged completion** (Astra round 6 finding 3; one barrier for every start since ruling 81). `RelocationStartGuard` refuses with `ErrPendingChore` while the serving op's `CompletedRelocation` is not `Discharged`, unless the start is a failover citing the completion box's fence (a dead box's slot binds nothing on the new target). *Cost if wrong:* a drain from a just-completed target waits one pass. |
| 78. **The landing conversion is the worker's, under Step's checks** (Astra round 6 finding 4). `hostrelocate.Worker.ConvertLanding` checks this box's current roster entry (enrollment, `active`, region, no termination fence) as `Step` does, and converts only a landing into its own region. *Cost if wrong:* none. |
| 79. **Task 10b is mandatory before activation** (Astra round 6 on R5-3). The source's row is held at its fence's close by Task 10b; relocation is not switched on anywhere until it has landed and its tests pass. *Cost if wrong:* none while dormant. |
| +80. **A completion is finished from its record, the mark last** (Astra round 7 finding 1; amends 76). The pass hands `Finish` every serving claim whose `completed_relocation` names this box and is not `Discharged`, on every pass and independent of the row. `relocationReleaseHold`: while the row is held for the op, the holder must be the probe session, then the discharge, then `ReleaseHold`; then `LiftFenceAs` under the persisted owner; then `MarkCompletedDischarged`. Once the row is not held for the op, the discharge has happened (only this function releases a completed op's hold), so a replay skips it and lifts and marks. The journal sweep keeps the op's journals, with the fence owner, until the mark. *Cost if wrong:* if the discharge could be undone after the hold went, a replay would mark a box whose slot is not discharged and a successor's `Prepare` would answer `ErrLost`, which is the stuck state the mark prevents. Nothing in main releases a relocating hold, so this would take a new writer of `hold_op`. |
| +81. **One discharge barrier for every start** (Astra round 7 finding 2; extends 77). `completionBarrier` is called by `RelocationStartGuard` and by `StartGuard` (N13, claim ask C12), so a move, delete, purge, placement or relocation from serving waits for the mark. The exemptions are a completion box on `lost` and a failover citing that box's fence. *Cost if wrong:* an operator's move or delete of a host that has just relocated waits one pass of the target's finish; if the target cannot finish (its discharge refused), every start waits until it can or the box is fenced. |
| ## L21: where the platform-provisioned `CARLOS_INSTANCE_KEY` belongs |
| @@ -389,7 +406,7 @@ Stage B's cost without the L21 plan is unchanged from rev 1: a host whose app mi |
| ## Global Constraints |
| - **Dormant.** Nothing changes on a box or console where `CARLOS_REGION_HA` is unset or not true (`regionHAEnabled`, boxrecord.go:25). The relocation owners open only with it on (claimrelocation.go:53) and with moves' C8 provider bound (ruling 53); root installs its relocating attempt reader only with it on (N2); the passes start only with it and `CARLOS_HOST_CLAIM` on; the console's chooser only with `Config.RegionHA`. |
| -- **Activation gate (claim §1.1, L18).** Relocation does not merge until Task 11's joint round passes on the two-box model, and is not switched on until the claim confirms C10, C8 with O1 is registered, Task 10b has landed (R5-3, ruling 79), and the two-box run passes on a dev box (checklist item 12). |
| +- **Activation gate (claim §1.1, L18).** Relocation does not merge until Task 11's joint round passes on the two-box model, and is not switched on until the claim confirms C10 and C12, C8 with O1 is registered, Task 10b has landed (R5-3, ruling 79), and the two-box run passes on a dev box (checklist item 12). |
| - **One implementation per concept; main's wins.** No second scanner, receipt verifier, roster reader, fence writer, inventory, retirement, marker publisher, participation controller or WorkSource. The probe is main's attempt path; the attempt journal is region HA's only because `OperationAttempts` is per kind. The install policy is region HA's only until C9. |
| - **Absence is never evidence.** A fence, receipt, pin, manifest, env item or inventory that cannot be read is an error. |
| - **No region HA file under a managed root names a host** (ruling 44): journals under `<DataDir>/relocate/ops/<op>/`, positions under `<DataDir>/relocate/release/<op>.json`, the build's scratch removed before the install. |
| @@ -453,7 +470,7 @@ Non-test files only; each task lists its tests. |
| ### Task 8a: Main's participation owner, the release providers, and the probe-stop primitives |
| -Region HA reads main's one participation controller per box instead of opening its own (ruling 42), and the release runs in claim Plan 11's order (ruling 28). The claim's probe-stop primitives, the participant's preflight gate and probe-stop branch, and the worker's bounds land here too, because `RelocationEffects` implements `hostrelocate.Effects` in this task and every later task builds on those types (K1, K2, K5: rev 4.2 had them in Tasks 8d and 9). Stage A's controller set and its two tests are deleted. The journals are census-opaque (N9) and swept by op (ruling 61), release positions are keyed by op with one op-id check (`validOpID`, D5c), and the flush publishes the releasing writer's adoption item alone (O5). N12 is a test seam in main's participation owner. Rev 5.1 adds the activator's owned fence (ruling 75), the completed relocation's discharge mark and the start guard that waits for it (ruling 77), and keeps a completed op's journals until that mark (ruling 76). |
| +Region HA reads main's one participation controller per box instead of opening its own (ruling 42), and the release runs in claim Plan 11's order (ruling 28). The claim's probe-stop primitives, the participant's preflight gate and probe-stop branch, and the worker's bounds land here too, because `RelocationEffects` implements `hostrelocate.Effects` in this task and every later task builds on those types (K1, K2, K5: rev 4.2 had them in Tasks 8d and 9). Stage A's controller set and its two tests are deleted. The journals are census-opaque (N9) and swept by op (ruling 61), release positions are keyed by op with one op-id check (`validOpID`, D5c), and the flush publishes the releasing writer's adoption item alone (O5). N12 is a test seam in main's participation owner. Rev 5.1 adds the activator's owned fence (ruling 75), the completed relocation's discharge mark and the one barrier both start guards call (rulings 77 and 81; `StartGuard`'s call is N13, claim ask C12), and keeps a completed op's journals until that mark (ruling 76). |
| Order notes: |
| @@ -463,18 +480,18 @@ Order notes: |
| **Files:** |
| - Create: `internal/activator/fenceowner_test.go`, `internal/host/claimparticipation_relocation.go`, `internal/host/relocationassembly.go`, `internal/host/relocationcensus_test.go`, `internal/host/relocationflush_adoption_test.go`, `internal/host/relocationparticipation_linux_test.go`, `internal/host/relocationsweep.go`, `internal/host/relocationsweep_test.go`, `internal/hostclaim/relocation_probestop.go`, `internal/hostclaim/relocation_probestop_test.go`, `internal/hostrelocate/participant_gate_test.go`, `internal/hostrelocate/worker_bounds_test.go` |
| -- Modify: `internal/activator/activator.go` (`fenceOwners`), `internal/activator/maintenance.go` (`FenceAndStopAs`, `LiftFenceAs`), `internal/host/claimcensus.go` (N9), `internal/host/claimparticipation.go` (N12 only), `internal/host/relocationeffects.go`, `internal/host/relocationeffects_test.go`, `internal/host/relocationflush.go`, `internal/hostclaim/relocation.go`, `internal/hostclaim/relocation_test.go` (each `CompleteRelocation` preceded by `RecordProbeStop`), `internal/hostrelocate/participant.go`, `internal/hostrelocate/participant_test.go` (the fake's `Finish` marks the completion discharged, as production's hold release does), `internal/hostrelocate/worker.go`, `internal/hostrelocate/worker_test.go` (`TestWorkerDrivesADrain` waits for the probe stop) |
| +- Modify: `internal/activator/activator.go` (`fenceOwners`), `internal/activator/maintenance.go` (`FenceAndStopAs`, `LiftFenceAs`), `internal/host/claimcensus.go` (N9), `internal/host/claimparticipation.go` (N12 only), `internal/host/relocationeffects.go`, `internal/host/relocationeffects_test.go`, `internal/host/relocationflush.go`, `internal/hostclaim/ops.go` (N13: `StartGuard` calls `completionBarrier`), `internal/hostclaim/relocation.go`, `internal/hostclaim/relocation_test.go` (each `CompleteRelocation` preceded by `RecordProbeStop`), `internal/hostrelocate/participant.go`, `internal/hostrelocate/participant_test.go` (the fake's `Finish` marks the completion discharged, as production's hold release does), `internal/hostrelocate/worker.go`, `internal/hostrelocate/worker_test.go` (`TestWorkerDrivesADrain` waits for the probe stop) |
| - Delete: `internal/host/relocationcontrollers_linux_test.go`, `internal/host/relocationlifecycle_linux_test.go` (they test stage A's controller set, which this task deletes, ruling 42) |
| **Interfaces:** |
| - Consumes: main's `productionClaimLocal` (`Prepare`, `prepareWith`, `slots`, `claimParticipation`: claimparticipation.go); `execlease.State`, `ErrLost`, `ErrInactive`, `InspectLocal`; `ClaimAssembly` as `Build` binds it, `localStopIdentity`, `ClaimWakeAssembly.Producers`, `get`, `Roles` (`Preflight`, `PreflightKind`); `ClaimAssembly.WritersStopped`; `CollectClaimStop`; `StopRequest.Recover`; `ReleaseObligation`; `ReleaseFlushFiles`; `genselect.Select`, `Plan.MaxLevel0TXID`; `activator.Classify`; `provision.InstanceDir`; hostclaim's `requireRelocation`, `readJournal`, `journalJSON`, `isHex64`, `ToInstalled`, `MarkStuck`, `RelocationStartGuard`; the activator's fence map and `fenceAndStop` (maintenance.go); `instancekey` (IK-1: `Binding`, `ParseAdoption`, `Binds`, `AdoptionItem`, `AdoptionSuffix`). |
| -- Produces: `hostclaim.ProbeStop`, `RecordProbeStop`, `Probe.At`, `CompletedRelocation`, `JournalCompleted`, `ReadCompletedRelocation`, `ToInstalledAt`, `CompleteRelocation`'s probe-stop gate and `released_move`; `CompletedRelocation.Discharged`, `MarkCompletedDischarged`, `RelocationStartGuard`'s refusal of a start from an undischarged completion (ruling 77); `activator.(*Service).FenceAndStopAs`, `LiftFenceAs` (ruling 75); `hostrelocate.Effects.Preflight` and `EndProbe`, `ErrLaterAttemptUnlaunchable`, `Participant.sameInstallation`, `CertifiedFor`, the worker's probe-stop wait and its `installed` and passed-probe bounds; `(*productionClaimLocal).Acknowledged` (N1), the N12 seam (`lockRoot`, `lockDir`, `updateFloor`); `RelocationActivator`, `RelocationAssemblyConfig`, `(ClaimAssembly) RelocationEffects`, `relocationPreflight`, `relocationPathsFor`, `relocationHostPaths`, `relocationFlushDB`; the rewritten `RelocationEffects`; `validOpID`; `sweepRelocationJournals`, which keeps a completed op's journals until its discharge mark (ruling 76). |
| +- Produces: `hostclaim.ProbeStop`, `RecordProbeStop`, `Probe.At`, `CompletedRelocation`, `JournalCompleted`, `ReadCompletedRelocation`, `ToInstalledAt`, `CompleteRelocation`'s probe-stop gate and `released_move`; `CompletedRelocation.Discharged`, `MarkCompletedDischarged`, `completionBarrier`, called by `RelocationStartGuard` and `StartGuard` (rulings 77, 81; N13); `activator.(*Service).FenceAndStopAs`, `LiftFenceAs` (ruling 75); `hostrelocate.Effects.Preflight` and `EndProbe`, `ErrLaterAttemptUnlaunchable`, `Participant.sameInstallation`, `CertifiedFor`, the worker's probe-stop wait and its `installed` and passed-probe bounds; `(*productionClaimLocal).Acknowledged` (N1), the N12 seam (`lockRoot`, `lockDir`, `updateFloor`); `RelocationActivator`, `RelocationAssemblyConfig`, `(ClaimAssembly) RelocationEffects`, `relocationPreflight`, `relocationPathsFor`, `relocationHostPaths`, `relocationFlushDB`; the rewritten `RelocationEffects`; `validOpID`; `sweepRelocationJournals`, which keeps a completed op's journals until its discharge mark (ruling 76). |
| - [ ] **Step 1: Write the failing tests** |
| The release tests drive `RelocationEffects.Release` over recording providers (`effectsFixture`). `TestRelocationAcknowledgesThroughMainsParticipation` runs main's real `productionClaimLocal` in agent mode over real hostexec controllers, a signed roster and the real census, with its locks under the N12 seam. It is Linux-only and runs inside the sandbox. |
| -New tests: `TestFenceAndStopAsMeetsOnlyItsOwnFence`, `TestAPlainFenceTakesOwnershipAway`, `TestLiftFenceAsLiftsOnlyTheOwnersFence`, `TestRelocationJournalsOfACompletedOpStayUntilItsDischarge`, `TestRelocationSuccessorWaitsForTheCompletedDischarge`, `TestCensusInitializesWithOldRelocationJournals`, `TestReleaseFlushPublishesOnlyTheReleasingWritersAdoption`, `TestRelocationAcknowledgesThroughMainsParticipation`, `TestRelocationJournalsAreSweptWhenTheirOpEnds`, `TestCompleteRelocationRequiresTheProbeStop`, `TestCompleteRelocationJournalsReleasedMoveForAConvertedLanding`, `TestToInstalledAtJournalsItsStart`, `TestParticipantCountsNoAttemptWhileRootRefusesTheKind`, `TestParticipantFailsAProbeRootCannotRelaunch`, `TestParticipantRecordsItsReceiptAfterAnEffectWroteTheClaim`, `TestParticipantEndsNoProbeWithoutItsLease`, `TestParticipantRecordsNoProbeStopWhileRootRefuses`, `TestWorkerBoundsAPassedProbeAwaitingItsStop`, `TestWorkerBoundsAnUntimedPassedProbeFromProbingSince`, `TestWorkerBoundsAnInstallThatNeverProbes`, `TestWorkerBoundsARefusedProbe`, `TestReleaseStopsWritersAndFlushesBeforeItQueues`, `TestReleaseNeverReturnsBeforeRootsReceipt`, `TestReleaseAfterARestartRecoversTheRecordedOwner`, `TestEnsureLocalRepairsOnlyUnderTheActiveStampBeforePOpens`. |
| +New tests: `TestFenceAndStopAsMeetsOnlyItsOwnFence`, `TestAPlainFenceTakesOwnershipAway`, `TestLiftFenceAsLiftsOnlyTheOwnersFence`, `TestRelocationJournalsOfACompletedOpStayUntilItsDischarge`, `TestRelocationSuccessorWaitsForTheCompletedDischarge`, `TestMoveSuccessorWaitsForTheCompletedDischarge`, `TestCensusInitializesWithOldRelocationJournals`, `TestReleaseFlushPublishesOnlyTheReleasingWritersAdoption`, `TestRelocationAcknowledgesThroughMainsParticipation`, `TestRelocationJournalsAreSweptWhenTheirOpEnds`, `TestCompleteRelocationRequiresTheProbeStop`, `TestCompleteRelocationJournalsReleasedMoveForAConvertedLanding`, `TestToInstalledAtJournalsItsStart`, `TestParticipantCountsNoAttemptWhileRootRefusesTheKind`, `TestParticipantFailsAProbeRootCannotRelaunch`, `TestParticipantRecordsItsReceiptAfterAnEffectWroteTheClaim`, `TestParticipantEndsNoProbeWithoutItsLease`, `TestParticipantRecordsNoProbeStopWhileRootRefuses`, `TestWorkerBoundsAPassedProbeAwaitingItsStop`, `TestWorkerBoundsAnUntimedPassedProbeFromProbingSince`, `TestWorkerBoundsAnInstallThatNeverProbes`, `TestWorkerBoundsARefusedProbe`, `TestReleaseStopsWritersAndFlushesBeforeItQueues`, `TestReleaseNeverReturnsBeforeRootsReceipt`, `TestReleaseAfterARestartRecoversTheRecordedOwner`, `TestEnsureLocalRepairsOnlyUnderTheActiveStampBeforePOpens`. |
| `internal/host/relocationcensus_test.go` (new): |
| @@ -1243,16 +1260,7 @@ func TestRelocationSuccessorWaitsForTheCompletedDischarge(t *testing.T) { |
| SourceBox: relTarget, SourceEnrollment: relEnrollment(relTarget), TargetBox: relSpare, TargetEnrollment: relEnrollment(relSpare), |
| DirectoryRevision: 9, AdmissionID: "adm-2", Fence: fence} |
| } |
| - completed := func(t *testing.T) (release.Store, Observed) { |
| - t.Helper() |
| - s, o := probedDrainIn(t) |
| - o = step(t, s, o, RecordProbeStop("rel-1", stopOf(o))) |
| - o = step(t, s, o, CompleteRelocation("rel-1", counter())) |
| - if done, ok := ReadCompletedRelocation(o.Claim); !ok || done.Discharged || o.Claim.Op.Exec != relTarget { |
| - t.Fatalf("A to B did not complete undischarged: %+v %v", done, ok) |
| - } |
| - return s, o |
| - } |
| + completed := completedDrain |
| t.Run("a drain from B waits for B's discharge", func(t *testing.T) { |
| s, o := completed(t) |
| for _, reason := range []string{ReasonDrain, ReasonRebalance} { |
| @@ -1299,6 +1307,52 @@ func TestRelocationSuccessorWaitsForTheCompletedDischarge(t *testing.T) { |
| } |
| }) |
| } |
| + |
| +// completedDrain is A to B through every real transition, completed and |
| +// not yet discharged on B. |
| +func completedDrain(t *testing.T) (release.Store, Observed) { |
| + t.Helper() |
| + s, o := probedDrainIn(t) |
| + o = step(t, s, o, RecordProbeStop("rel-1", stopOf(o))) |
| + o = step(t, s, o, CompleteRelocation("rel-1", counter())) |
| + if done, ok := ReadCompletedRelocation(o.Claim); !ok || done.Discharged || o.Claim.Op.Exec != relTarget { |
| + t.Fatalf("A to B did not complete undischarged: %+v %v", done, ok) |
| + } |
| + return s, o |
| +} |
| + |
| +// Astra round 7, finding 2: the discharge barrier is StartGuard's too, so |
| +// a move from B (and a delete) after A to B completes is refused, with the |
| +// claim unchanged, until B marks the completion discharged; then the real |
| +// move start CAS succeeds. A completion box on lost holds nothing. |
| +func TestMoveSuccessorWaitsForTheCompletedDischarge(t *testing.T) { |
| + ctx := context.Background() |
| + s, o := completedDrain(t) |
| + move := BeginAdmittedMove("mv-1", "console", relTarget, relSpare, "uk", relObserved.Add(time.Hour), |
| + MoveAdmission{AppID: "app-1", AdmissionID: "mv-1", SourceEnrollment: relEnrollment(relTarget), TargetEnrollment: relEnrollment(relSpare)}) |
| + if _, err := Update(ctx, s, o, move); !errors.Is(err, ErrPendingChore) { |
| + t.Fatalf("a move started before B's discharge: %v", err) |
| + } |
| + if _, err := Update(ctx, s, o, BeginDelete("del-1", "console", "etag-1", relObserved.Add(time.Hour), []string{"ie"}, []string{relTarget})); !errors.Is(err, ErrPendingChore) { |
| + t.Fatalf("a delete started before B's discharge: %v", err) |
| + } |
| + if now, err := Read(ctx, s, o.Claim.Host); err != nil || now.ETag != o.ETag { |
| + t.Fatalf("a refused start wrote the claim: %v", err) |
| + } |
| + lost := copyOf(t, o.Claim) |
| + lost.Lost = append(lost.Lost, relTarget) |
| + if err := StartGuard(lost, []string{"uk"}, []string{relSpare}); err != nil { |
| + t.Fatalf("a lost completion box still holds the barrier: %v", err) |
| + } |
| + o = step(t, s, o, MarkCompletedDischarged("rel-1")) |
| + next, err := Update(ctx, s, o, move) |
| + if err != nil { |
| + t.Fatalf("the move after the discharge: %v", err) |
| + } |
| + if next.Claim.Op.Kind != KindMoving || next.Claim.Op.ID != "mv-1" || next.Claim.Op.Exec != relTarget { |
| + t.Fatalf("move op %+v", next.Claim.Op) |
| + } |
| +} |
| ``` |
| `internal/activator/fenceowner_test.go` (new): |
| @@ -2514,7 +2568,7 @@ Expected: FAIL to compile: `undefined: hostclaim.ProbeStop`, `a.FenceAndStopAs u |
| - [ ] **Step 3: The claim's probe-stop primitives** |
| -Ruling 49 step 4, ruling 56's record, ruling 60's `installed_at`, ruling 67's `Probe.At`, and ruling 77's discharge mark and start guard. |
| +Ruling 49 step 4, ruling 56's record, ruling 60's `installed_at`, ruling 67's `Probe.At`, and ruling 77's discharge mark with ruling 81's one barrier, which `StartGuard` calls too (N13). |
| `internal/hostclaim/relocation.go`: |
| @@ -2536,12 +2590,12 @@ Ruling 49 step 4, ruling 56's record, ruling 60's `installed_at`, ruling 67's `P |
| if c.Holder == nil || c.Op.Kind != KindServing || c.Op.ID != "" { |
| return fmt.Errorf("%w: kind %q", ErrWrongState, c.Op.Kind) |
| } |
| -+ // A completed relocation whose box has not yet discharged its probe's |
| -+ // participation (C10, MarkCompletedDischarged) keeps that box's slot |
| -+ // for the old op, so a successor from it would never be prepared. A |
| -+ // failover from a fenced source does not need the source's slot. |
| -+ if done, ok := ReadCompletedRelocation(c); ok && !done.Discharged && !(sourceFenced && source == done.Stop.Box) { |
| -+ return fmt.Errorf("%w: relocation %s is not yet discharged on %s", ErrPendingChore, done.Op, done.Stop.Box) |
| ++ fenced := "" |
| ++ if sourceFenced { |
| ++ fenced = source |
| ++ } |
| ++ if err := completionBarrier(c, fenced); err != nil { |
| ++ return err |
| + } |
| for _, o := range c.Obligations { |
| switch { |
| @@ -2590,6 +2644,7 @@ package hostclaim |
| import ( |
| "fmt" |
| + "slices" |
| "time" |
| ) |
| @@ -2670,6 +2725,23 @@ func ReadCompletedRelocation(c *Claim) (CompletedRelocation, bool) { |
| return r, true |
| } |
| +// completionBarrier is region HA ruling 77, the one barrier every start |
| +// from serving passes (StartGuard for moves, deletes, purges and |
| +// placements; RelocationStartGuard for relocations): while the serving |
| +// op's completed relocation is not marked discharged, the box it completed |
| +// on still holds that op's participation slot, and main's Prepare for any |
| +// successor operation would answer ErrLost, so nothing starts. A |
| +// completion box on lost, or the one the start itself cites as fenced |
| +// (fencedSource, a failover's source), holds nothing a successor needs: a |
| +// dead box's slot binds nothing on another box. |
| +func completionBarrier(c *Claim, fencedSource string) error { |
| + done, ok := ReadCompletedRelocation(c) |
| + if !ok || done.Discharged || slices.Contains(c.Lost, done.Stop.Box) || (fencedSource != "" && fencedSource == done.Stop.Box) { |
| + return nil |
| + } |
| + return fmt.Errorf("%w: relocation %s is not yet discharged on %s", ErrPendingChore, done.Op, done.Stop.Box) |
| +} |
| + |
| // MarkCompletedDischarged records, on the serving op, that the box the |
| // relocation opID completed on has discharged its participation. It |
| // changes nothing else; a repeat is ErrAlreadyRecorded. A serving op with |
| @@ -2709,6 +2781,29 @@ func ToInstalledAt(opID string, a PreparedAuthority, now time.Time) func(*Claim) |
| } |
| ``` |
| +`internal/hostclaim/ops.go`: |
| + |
| +```diff |
| +--- a/internal/hostclaim/ops.go |
| ++++ b/internal/hostclaim/ops.go |
| +@@ -22,10 +22,15 @@ func RequireOp(id string) func(*Claim) error { |
| + |
| + // StartGuard is §4: one operation at a time, started from serving, and |
| + // CC11: never over a chore in a region or box the operation will write. |
| ++// Region HA N13 (ruling 77): nor while a completed relocation's box has |
| ++// not discharged its participation (completionBarrier). |
| + func StartGuard(c *Claim, regions, boxes []string) error { |
| + if c.Holder == nil || c.Op.Kind != KindServing { |
| + return fmt.Errorf("%w: kind %q", ErrWrongState, c.Op.Kind) |
| + } |
| ++ if err := completionBarrier(c, ""); err != nil { |
| ++ return err |
| ++ } |
| + inR, inB := set(regions), set(boxes) |
| + for _, o := range c.Obligations { |
| + if (o.Region != "" && inR[o.Region]) || (o.Box != "" && inB[o.Box]) { |
| +``` |
| + |
| The activator's owned fence (ruling 75). `FenceAndStopAs` is `FenceAndStopIfUnfenced` that also meets again exactly the fence its owner installed at that generation; a plain install or lift clears the owner. |
| `internal/activator/maintenance.go`: |
| @@ -4035,7 +4130,7 @@ func sweepRelocationJournals(ctx context.Context, dataDir string, claims release |
| Run: |
| ```bash |
| -nice -n 19 go test ./internal/hostclaim/ ./internal/hostrelocate/ -count=1 |
| +nice -n 19 go test ./internal/hostclaim/ ./internal/hostrelocate/ ./internal/hostops/ -count=1 |
| nice -n 19 go test ./internal/activator/ -count=1 -run 'TestFenceAndStopAs|TestAPlainFence|TestLiftFenceAs|TestFenceAndStopIfUnfenced|TestFenced|TestFenceGeneration|TestMaintenance' |
| TMPDIR=$(mktemp -d /tmp/k.XXXX) nice -n 19 go test ./internal/host/ -count=1 -run '^(TestRelease|TestRelocationRelease|TestEnsureLocal|TestVerifyReceipt|TestCensusInitializesWithOld|TestRelocationJournals|TestRelocationAcknowledges|TestPlan9|TestPlan11|TestClaimCensus)' |
| ``` |
| @@ -4046,8 +4141,8 @@ Expected: PASS. Run outside the sandbox (the `TestPlan9`/`TestPlan11` regression |
| ```bash |
| git rm internal/host/relocationcontrollers_linux_test.go internal/host/relocationlifecycle_linux_test.go |
| -git add internal/activator/fenceowner_test.go internal/activator/activator.go internal/activator/maintenance.go internal/host/claimparticipation_relocation.go internal/host/relocationassembly.go internal/host/relocationcensus_test.go internal/host/relocationflush_adoption_test.go internal/host/relocationparticipation_linux_test.go internal/host/relocationsweep.go internal/host/relocationsweep_test.go internal/hostclaim/relocation_probestop.go internal/hostclaim/relocation_probestop_test.go internal/hostrelocate/participant_gate_test.go internal/hostrelocate/worker_bounds_test.go internal/host/claimcensus.go internal/host/claimparticipation.go internal/host/relocationeffects.go internal/host/relocationeffects_test.go internal/host/relocationflush.go internal/hostclaim/relocation.go internal/hostclaim/relocation_test.go internal/hostrelocate/participant.go internal/hostrelocate/participant_test.go internal/hostrelocate/worker.go internal/hostrelocate/worker_test.go |
| -git commit -m "host: relocation reads main's participation owner and releases in Plan 11's order; hostclaim: a relocation completes only after its passed probe is stopped with root's receipt, and no successor starts before its target discharges it; activator: owned fences |
| +git add internal/activator/fenceowner_test.go internal/activator/activator.go internal/activator/maintenance.go internal/host/claimparticipation_relocation.go internal/host/relocationassembly.go internal/host/relocationcensus_test.go internal/host/relocationflush_adoption_test.go internal/host/relocationparticipation_linux_test.go internal/host/relocationsweep.go internal/host/relocationsweep_test.go internal/hostclaim/relocation_probestop.go internal/hostclaim/relocation_probestop_test.go internal/hostrelocate/participant_gate_test.go internal/hostrelocate/worker_bounds_test.go internal/host/claimcensus.go internal/host/claimparticipation.go internal/host/relocationeffects.go internal/host/relocationeffects_test.go internal/host/relocationflush.go internal/hostclaim/ops.go internal/hostclaim/relocation.go internal/hostclaim/relocation_test.go internal/hostrelocate/participant.go internal/hostrelocate/participant_test.go internal/hostrelocate/worker.go internal/hostrelocate/worker_test.go |
| +git commit -m "host: relocation reads main's participation owner and releases in Plan 11's order; hostclaim: a relocation completes only after its passed probe is stopped with root's receipt, and no operation starts from it before its target discharges it; activator: owned fences |
| Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" |
| ``` |
| @@ -11702,7 +11797,7 @@ func (h relocationCompletedHandover) After() func(context.Context, hostclaim.Obs |
| "github.com/carlosframework/platform/internal/hostrelocate" |
| "github.com/carlosframework/platform/internal/hostwrite" |
| "github.com/carlosframework/platform/internal/lease" |
| -@@ -211,3 +213,201 @@ func (a ClaimAssembly) relocationAdmit(cfg RelocationAssemblyConfig) func(contex |
| +@@ -211,3 +213,203 @@ func (a ClaimAssembly) relocationAdmit(cfg RelocationAssemblyConfig) func(contex |
| } |
| } |
| } |
| @@ -11830,21 +11925,23 @@ func (h relocationCompletedHandover) After() func(context.Context, hostclaim.Obs |
| + return rec, true, nil |
| +} |
| + |
| -+// relocationReleaseHold is ruling 49 step 5 and ruling 76: once the claim |
| -+// serves on this box under kind serving, main's participation is |
| -+// discharged (C10), the row's relocating hold is released, the activator |
| -+// fence the probe stop installed under its persisted owner is lifted, and |
| -+// the serving op's completed relocation is marked discharged, which is |
| -+// what lets a successor relocation start from this box |
| -+// (hostclaim.RelocationStartGuard). Ruling 66: while the row is held for |
| -+// the op, the holder must still be the stopped probe session and the |
| -+// discharge succeeds before anything is released; a refusal keeps the |
| -+// hold and the fence and is retried next pass. Once the row is no longer |
| -+// held for the op, the discharge has happened (nothing else releases a |
| -+// relocating hold for a completed op), and a replay only lifts and marks. |
| -+// A missing row is refused: absence is never evidence of the discharge. |
| -+// Anything else leaves everything: an excluded box's row is its |
| -+// park-local's. |
| ++// relocationReleaseHold finishes a completed relocation on the box it |
| ++// completed on (ruling 49 step 5, rulings 76 and 80), in this order: while |
| ++// the row is held for the op, the holder must still be the stopped probe |
| ++// session, main's participation is discharged (C10) and the hold is |
| ++// released (ruling 66: nothing is released before the discharge); then |
| ++// the activator fence the probe stop installed under its persisted owner |
| ++// is lifted; last, the serving op's completed relocation is marked |
| ++// discharged, which is what lets any successor start (the completion |
| ++// barrier, ruling 77). The pass calls it for every serving claim whose |
| ++// completion record names this box and is not yet marked, never from the |
| ++// row (ruling 80), and every step is idempotent, so a failure or a crash |
| ++// at any point leaves the record unmarked and the next pass resumes where |
| ++// it stopped. Once the row is no longer held for the op, the discharge has |
| ++// happened (nothing else releases a relocating hold for a completed op), |
| ++// and a replay only lifts and marks. A missing row is refused: absence is |
| ++// never evidence of the discharge. Anything else leaves everything: an |
| ++// excluded box's row is its park-local's. |
| +func (a ClaimAssembly) relocationReleaseHold(cfg RelocationAssemblyConfig) func(context.Context, string, string) error { |
| + return func(ctx context.Context, host, op string) error { |
| + if cfg.Registry == nil || cfg.Activator == nil || cfg.Discharge == nil { |
| @@ -12015,23 +12112,23 @@ Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" |
| ### Task 10: The pass behind `CARLOS_REGION_HA`, the shared edges read, and the rollout checklist |
| -Every box with the claim and region HA on, and moves' C8 provider bound, replays its region's relocations: per host, the participant, then the worker, then this box's completed hold, so a passed probe completes in one pass (ruling 49); a landing whose target is fenced is converted when its move committed (J3); the journals of ended operations are swept (ruling 61). The pass starts no relocation: `Worker.Start` is the operator's and spec 3's placer's, so the worker's `Admission` and `Census` refuse here (D8: rev 1's `relocationAppAdmission`, `localBoxReport` and `hostfloor.Read` belong to that starter, not to this pass). |
| +Every box with the claim and region HA on, and moves' C8 provider bound, replays its region's relocations: per host, the participant, then the worker, then the completion this box has not yet finished, found from the claim's record (ruling 80), so a passed probe completes in one pass (ruling 49); a landing whose target is fenced is converted when its move committed (J3); the journals of ended operations are swept (ruling 61). The pass starts no relocation: `Worker.Start` is the operator's and spec 3's placer's, so the worker's `Admission` and `Census` refuse here (D8: rev 1's `relocationAppAdmission`, `localBoxReport` and `hostfloor.Read` belong to that starter, not to this pass). |
| The agent registers the source with the roster directory the claim's owners read, and opens the claim assembly before the wiring so the pass binds to the same one (`ensureProductionClaimAssembly`). N5 is in `openClaimRelocationOwners`, inside its region HA gate, before the source opens (K17). `relocationAssemblyConfig` takes the discharge and the opt root (K16). `Worker.Relocatable` is `relocationRelocatable`, an adapter over `relocationRecord` (K24). The chooser's fingerprint is the box's own (`edgesFileSecretsFP`). The console's live edges read moves to `internal/edgesfile` unchanged (`ReadSnapshot`, `ReadLiveSnapshot`, `Snapshot.BoxChoice`, `LiveSnapshot`), and the console reads through aliases. |
| **Files:** |
| -- Create: `internal/console/claim_runtime_chooser_test.go`, `internal/edgesfile/snapshot.go`, `internal/edgesfile/snapshot_test.go`, `internal/host/relocatepass.go`, `internal/host/relocatepass_test.go`, `internal/host/relocationlanding.go`, `internal/host/relocationlanding_test.go`, `internal/hostrelocate/pin_versioned_test.go` |
| +- Create: `internal/console/claim_runtime_chooser_test.go`, `internal/edgesfile/snapshot.go`, `internal/edgesfile/snapshot_test.go`, `internal/host/relocatepass.go`, `internal/host/relocatepass_test.go`, `internal/host/relocationfinish_unix_test.go`, `internal/host/relocationlanding.go`, `internal/host/relocationlanding_test.go`, `internal/hostrelocate/pin_versioned_test.go` |
| - Modify: `internal/console/apiinstances.go`, `internal/console/boxchoice.go`, `internal/console/boxselect.go`, `internal/console/claim_runtime.go` (`Executors`), `internal/console/instancesplacement.go`, `internal/console/regioncandidates.go`, `internal/console/regioncandidates_test.go` (`snap.EvidenceComplete()`), `internal/console/regiondefault.go`, `internal/host/agent.go` (register the source; start the pass), `internal/host/claimparticipation_relocation.go`, `internal/host/claimrelocation.go` (N5), `internal/hostrelocate/worker.go`, `internal/hostrelocate/worker_bounds_test.go` |
| **Interfaces:** |
| - Consumes: `hostrelocate.Worker`, `Participant`, `RegionChooser`, `StorePinner`; `hostfence.ClaimFenceAuthority`; `StopReceiptReleases`, `RelocationReleaseReceipts`; `hostops.ExecutorChooser`, `ClaimLoads`; `release.Channel`, `NewS3Store`, `BumpSerial`, `ParsePublicKey`; `lease.Acquire`, `NewVersionedS3`; `genselect.Select`; `provision.GetRecord`, `MoveState`, `MoveCommitted`; `moves.CommitScopes`; `hostclaim.ConvertCommittedLanding`, `CommittedLandingGuard`; `boxvolumes.List`, `Discharge`; `hostfloor.ReadEnrollment`; `appconfig.ParsePrivateKey`; `edgesFileSecretsFP`, `MovesRecordsFrom`; `ensureProductionClaimAssembly`, `claimEntry`; `status.LoadBoxesComplete`, `BoxStale`. |
| -- Produces: `relocatePassEnabled`, `RelocationRuntime` (`Run`, `once`, `releaseCompletedHold`, `alert`), `relocationStepper`, `relocationStores`, `openRelocationStores`, `relocationLeases`, `relocationAssemblyConfig`, `channelRelease`, `volumeDischarge`, `relocationRelocatable`, `relocationStartRefused`, `relocationRuntimeInputs`, `productionRelocationRuntimeInputs`, `NewRelocationRuntime`, `wakeDispatch`, `convertFencedLanding`, `registerRelocationSource`, `startRelocationPass`; `(*productionClaimLocal).boundAssembly` (N1); `CommittedLandingRecords`; `hostrelocate.Worker.ConvertLanding` (ruling 78), `newestFenceObservation` and the no-target bound; `edgesfile.PlacementView`, `Snapshot`, `ReadSnapshot`, `ReadLiveSnapshot`, `LiveSnapshot`; `ProductionAuthority.Executors` from the console's chooser. |
| +- Produces: `relocatePassEnabled`, `RelocationRuntime` (`Run`, `once`, `finishCompletion`, `alert`; no registry), `relocationStepper`, `relocationStores`, `openRelocationStores`, `relocationLeases`, `relocationAssemblyConfig`, `channelRelease`, `volumeDischarge`, `relocationRelocatable`, `relocationStartRefused`, `relocationRuntimeInputs`, `productionRelocationRuntimeInputs`, `NewRelocationRuntime`, `wakeDispatch`, `convertFencedLanding`, `registerRelocationSource`, `startRelocationPass`; `(*productionClaimLocal).boundAssembly` (N1); `CommittedLandingRecords`; `hostrelocate.Worker.ConvertLanding` (ruling 78), `newestFenceObservation` and the no-target bound; `edgesfile.PlacementView`, `Snapshot`, `ReadSnapshot`, `ReadLiveSnapshot`, `LiveSnapshot`; `ProductionAuthority.Executors` from the console's chooser. |
| - [ ] **Step 1: Write the failing tests** |
| -The pass tests drive `once` over a DirStore control bucket with recording steppers; the steps test holds a landing claim beside the relocating ones and asserts only it is offered to `Convert` and nothing steps it. `TestRelocatePassConvertsAFencedCommittedLanding` runs the production conversion over a real landing claim, a signed roster, a termination fence and the authoritative record, committed and not; `TestRelocatePassConversionRefusesAnUnauthorizedWorker` runs it as a revoked, retired, fenced, wrongly enrolled or other-region worker and finds the claim unwritten each time (ruling 78). `TestRelocationReleaseIsTheChannelPointer` ships and promotes real releases with a generated root key. `TestRelocationPinnerSelectsOverTheVersionedBucket` (hostrelocate) runs the pinner over `genselect.Select` and a versioned bucket holding a marker, a snapshot and a level-0 file. Rev 1's `TestRelocatePassStaysOffWhileThePreflightRefuses` is not a pass test in rev 5: the preflight is the participant's (Task 8a, `TestParticipantCountsNoAttemptWhileRootRefusesTheKind`). Finding 12's test (`TestRelocationLeasesSignForTheCloudRegion`) is folded into `TestRelocationRuntimeOpensADataBucketStore` (K6). |
| +The pass tests drive `once` over a DirStore control bucket with recording steppers; `TestRelocatePassFinishesACompletionFromItsRecord` drives it over the production `Finish` (`relocationReleaseHold`) with a real registry and the real activator's owned fence, through a failed discharge, a failed mark CAS, a lift that never ran and a restart (ruling 80); the steps test holds a landing claim beside the relocating ones and asserts only it is offered to `Convert` and nothing steps it. `TestRelocatePassConvertsAFencedCommittedLanding` runs the production conversion over a real landing claim, a signed roster, a termination fence and the authoritative record, committed and not; `TestRelocatePassConversionRefusesAnUnauthorizedWorker` runs it as a revoked, retired, fenced, wrongly enrolled or other-region worker and finds the claim unwritten each time (ruling 78). `TestRelocationReleaseIsTheChannelPointer` ships and promotes real releases with a generated root key. `TestRelocationPinnerSelectsOverTheVersionedBucket` (hostrelocate) runs the pinner over `genselect.Select` and a versioned bucket holding a marker, a snapshot and a level-0 file. Rev 1's `TestRelocatePassStaysOffWhileThePreflightRefuses` is not a pass test in rev 5: the preflight is the participant's (Task 8a, `TestParticipantCountsNoAttemptWhileRootRefusesTheKind`). Finding 12's test (`TestRelocationLeasesSignForTheCloudRegion`) is folded into `TestRelocationRuntimeOpensADataBucketStore` (K6). |
| -New tests: `TestCreateDoorUsesTheChooserOnlyWithRegionHAOn`, `TestLiveSnapshotJoinsEdgesFilesWithLiveness`, `TestRelocatePassIsOffUnlessBothFlagsAreOn`, `TestRelocatePassStepsEveryRelocationAndSweeps`, `TestRelocatePassAlertsOnceWhileAConversionWaitsForATarget`, `TestRelocatePassStopsCompletesAndReleasesInOnePass`, `TestCommittedLandingRecordsAdmitOnlyACommittedMove`, `TestRelocationReleaseIsTheChannelPointer`, `TestDischargeIsUnqualifiedWithoutEnrolledAt`, `TestRelocationRuntimeOpensADataBucketStore`, `TestRelocationBindsWithARegisteredMovesSource`, `TestRelocatePassConvertsAFencedCommittedLanding`, `TestRelocatePassConversionRefusesAnUnauthorizedWorker`, `TestRelocationPinnerSelectsOverTheVersionedBucket`, `TestWorkerBoundsAConversionWaitingForATarget`. |
| +New tests: `TestCreateDoorUsesTheChooserOnlyWithRegionHAOn`, `TestLiveSnapshotJoinsEdgesFilesWithLiveness`, `TestRelocatePassIsOffUnlessBothFlagsAreOn`, `TestRelocatePassStepsEveryRelocationAndSweeps`, `TestRelocatePassAlertsOnceWhileAConversionWaitsForATarget`, `TestRelocatePassStopsCompletesAndReleasesInOnePass`, `TestRelocatePassFinishesACompletionFromItsRecord`, `TestCommittedLandingRecordsAdmitOnlyACommittedMove`, `TestRelocationReleaseIsTheChannelPointer`, `TestDischargeIsUnqualifiedWithoutEnrolledAt`, `TestRelocationRuntimeOpensADataBucketStore`, `TestRelocationBindsWithARegisteredMovesSource`, `TestRelocatePassConvertsAFencedCommittedLanding`, `TestRelocatePassConversionRefusesAnUnauthorizedWorker`, `TestRelocationPinnerSelectsOverTheVersionedBucket`, `TestWorkerBoundsAConversionWaitingForATarget`. |
| `internal/console/claim_runtime_chooser_test.go` (new): |
| @@ -12148,6 +12245,7 @@ import ( |
| "context" |
| "crypto/ed25519" |
| "crypto/rand" |
| + "encoding/json" |
| "errors" |
| "os" |
| "path/filepath" |
| @@ -12313,16 +12411,14 @@ func TestRelocatePassStopsCompletesAndReleasesInOnePass(t *testing.T) { |
| ctx := t.Context() |
| store := release.NewDirStore(t.TempDir()) |
| passClaim(t, store, "a.example", relocatingOp("rel-a")) |
| - reg, err := registry.Open(filepath.Join(t.TempDir(), "registry.db")) |
| - if err != nil { |
| - t.Fatal(err) |
| - } |
| - defer reg.Close() |
| - if err := reg.Upsert(registry.Route{Host: "a.example", Status: "relocating", HoldOp: "rel-a", Socket: "/run/a.sock"}); err != nil { |
| - t.Fatal(err) |
| - } |
| + // Ruling 80: completions are found from the claim's record. One this |
| + // box finished already, and one another box finishes, are left alone. |
| + passClaim(t, store, "b.example", hostclaim.Op{}) |
| + completeOn(t, store, "b.example", "rel-b", "dst", true) |
| + passClaim(t, store, "c.example", hostclaim.Op{}) |
| + completeOn(t, store, "c.example", "rel-c", "spa", false) |
| var order []string |
| - r := &RelocationRuntime{Worker: stepRecorder{name: "worker", order: &order}, Participant: stepRecorder{name: "participant", order: &order}, Box: "dst", Registry: reg, |
| + r := &RelocationRuntime{Worker: completingWorker{order: &order, store: store}, Participant: stepRecorder{name: "participant", order: &order}, Box: "dst", |
| Finish: func(_ context.Context, host, op string) error { |
| order = append(order, "finish:"+host+":"+op) |
| return nil |
| @@ -12333,6 +12429,48 @@ func TestRelocatePassStopsCompletesAndReleasesInOnePass(t *testing.T) { |
| } |
| } |
| +// completeOn writes host's serving op with op's completed relocation on |
| +// box, marked discharged or not. |
| +func completeOn(t *testing.T, store release.Store, host, op, box string, discharged bool) hostclaim.Observed { |
| + t.Helper() |
| + ctx := t.Context() |
| + obs, err := hostclaim.Read(ctx, store, host) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + done, _ := json.Marshal(hostclaim.CompletedRelocation{Op: op, Stop: hostclaim.ProbeStop{Box: box, Session: obs.Claim.Holder.Session, Receipt: "fence-1", ReceiptSHA256: strings.Repeat("a", 64)}, |
| + Phase: hostclaim.PhaseRelocatingProbing, Discharged: discharged}) |
| + obs, err = hostclaim.Update(ctx, store, obs, func(c *hostclaim.Claim) error { |
| + c.Op = hostclaim.Op{Kind: hostclaim.KindServing, Exec: box, Participants: []string{box}, Journal: map[string]string{hostclaim.JournalCompleted: string(done)}} |
| + return nil |
| + }) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + return obs |
| +} |
| + |
| +// completingWorker is the worker as the pass sees it when its Step makes |
| +// the serving CAS: the claim then serves on dst with the completion. |
| +type completingWorker struct { |
| + order *[]string |
| + store release.Store |
| +} |
| + |
| +func (w completingWorker) Step(ctx context.Context, host string) (hostclaim.Observed, error) { |
| + *w.order = append(*w.order, "worker:"+host) |
| + obs, err := hostclaim.Read(ctx, w.store, host) |
| + if err != nil { |
| + return obs, err |
| + } |
| + done, _ := json.Marshal(hostclaim.CompletedRelocation{Op: obs.Claim.Op.ID, Stop: hostclaim.ProbeStop{Box: "dst", Session: obs.Claim.Holder.Session, Receipt: "fence-1", ReceiptSHA256: strings.Repeat("a", 64)}, |
| + Phase: hostclaim.PhaseRelocatingProbing}) |
| + return hostclaim.Update(ctx, w.store, obs, func(c *hostclaim.Claim) error { |
| + c.Op = hostclaim.Op{Kind: hostclaim.KindServing, Exec: "dst", Participants: []string{"dst"}, Journal: map[string]string{hostclaim.JournalCompleted: string(done)}} |
| + return nil |
| + }) |
| +} |
| + |
| // J3: the guard admits a conversion only for the move the claim's op |
| // names, committed for this incarnation with no rollback; anything else is |
| // moves' case (ErrWrongState). |
| @@ -12521,6 +12659,206 @@ func TestRelocationBindsWithARegisteredMovesSource(t *testing.T) { |
| } |
| ``` |
| +`internal/host/relocationfinish_unix_test.go` (new): |
| + |
| +```go |
| +//go:build unix |
| + |
| +package host |
| + |
| +import ( |
| + "context" |
| + "encoding/json" |
| + "errors" |
| + "os" |
| + "path/filepath" |
| + "strings" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/activator" |
| + "github.com/carlosframework/platform/internal/hostbox" |
| + "github.com/carlosframework/platform/internal/hostclaim" |
| + "github.com/carlosframework/platform/internal/registry" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +// failingClaimPut is a control bucket whose next fail writes of one claim |
| +// key fail: a lost or failed claim CAS. |
| +type failingClaimPut struct { |
| + release.Store |
| + key string |
| + fail int |
| +} |
| + |
| +func (s *failingClaimPut) Put(ctx context.Context, key string, body []byte, ifMatch string) (string, error) { |
| + if key == s.key && s.fail > 0 { |
| + s.fail-- |
| + return "", errors.New("the claim CAS failed") |
| + } |
| + return s.Store.Put(ctx, key, body, ifMatch) |
| +} |
| + |
| +// finishWorld is a host that serves on dst after relocation rel-1 |
| +// completed there, undischarged: its row held for rel-1, the probe stop's |
| +// owned fence on the activator, and the pass of dst with the production |
| +// Finish (relocationReleaseHold). |
| +type finishWorld struct { |
| + t *testing.T |
| + store *failingClaimPut |
| + reg *registry.Registry |
| + act *stopProbeActivator |
| + order []string |
| + discharged int |
| + discharge error |
| + runtime *RelocationRuntime |
| + dataDir string |
| +} |
| + |
| +const finishHost = "shop.example" |
| + |
| +func newFinishWorld(t *testing.T) *finishWorld { |
| + t.Helper() |
| + ctx := t.Context() |
| + w := &finishWorld{t: t, dataDir: t.TempDir()} |
| + created := time.Date(2026, 10, 7, 0, 0, 0, 0, time.UTC) |
| + inc, err := hostclaim.Incarnation("acct", "app", finishHost, created) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + done, _ := json.Marshal(hostclaim.CompletedRelocation{Op: "rel-1", Stop: hostclaim.ProbeStop{Box: "dst", Session: 12, Receipt: "fence-1", ReceiptSHA256: strings.Repeat("a", 64)}, Stamp: 9, Phase: hostclaim.PhaseRelocatingProbing, Generation: "P1"}) |
| + c := &hostclaim.Claim{V: 2, Gen: 31, Host: finishHost, |
| + Holder: &hostclaim.Holder{Family: "instance", Key: "apps/acct/app/instances/" + finishHost + ".json", Account: "acct", App: "app", Incarnation: inc, CreatedAt: created, |
| + Location: "instances/" + finishHost, Stamp: 9, Session: 12, Generation: "P1", WriterBox: "dst", Admissions: []hostclaim.AdmissionRecord{{Session: 12, Box: "dst", Generation: "P1", Mode: "cold", Region: "ie"}}}, |
| + Op: hostclaim.Op{Kind: hostclaim.KindServing, Exec: "dst", Participants: []string{"dst"}, Journal: map[string]string{hostclaim.JournalCompleted: string(done)}}} |
| + key, _ := hostclaim.Key(finishHost) |
| + w.store = &failingClaimPut{Store: release.NewDirStore(t.TempDir()), key: key} |
| + raw, _ := json.Marshal(c) |
| + if _, err := w.store.Put(ctx, key, raw, ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + if w.reg, err = registry.Open(filepath.Join(t.TempDir(), "registry.db")); err != nil { |
| + t.Fatal(err) |
| + } |
| + t.Cleanup(func() { w.reg.Close() }) |
| + if err := w.reg.Upsert(registry.Route{Host: finishHost, Status: "relocating", HoldOp: "rel-1", Socket: "/run/shop.sock"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + marker, err := relocationProbeFencePath(w.dataDir, "rel-1") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + owner, err := relocationProbeFenceOwner(marker, finishHost, 12) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + w.act = newStopProbeActivator(&w.order) |
| + fctx, cancel := context.WithTimeout(ctx, time.Second) |
| + defer cancel() |
| + if _, err := w.act.svc.FenceAndStopAs(fctx, finishHost, 12, owner); err != nil && !errors.Is(err, activator.ErrStopUnproven) { |
| + t.Fatal(err) |
| + } |
| + a := ClaimAssembly{Claims: w.store, stopLocal: func(context.Context) (claimStopLocal, error) { |
| + return claimStopLocal{Actor: hostbox.Actor{Box: "dst"}}, nil |
| + }} |
| + cfg := RelocationAssemblyConfig{Registry: w.reg, Activator: w.act, DataDir: w.dataDir, Discharge: func(context.Context, hostclaim.Observed) error { |
| + w.discharged++ |
| + return w.discharge |
| + }} |
| + var steps []string |
| + w.runtime = &RelocationRuntime{Worker: stepRecorder{name: "worker", order: &steps}, Participant: stepRecorder{name: "participant", order: &steps}, Box: "dst", |
| + Finish: a.relocationReleaseHold(cfg), DataDir: w.dataDir} |
| + return w |
| +} |
| + |
| +func (w *finishWorld) pass() { w.runtime.once(w.t.Context(), w.store) } |
| + |
| +func (w *finishWorld) held() bool { |
| + row, err := w.reg.Get(finishHost) |
| + return err == nil && row != nil && row.Status == "relocating" && row.HoldOp == "rel-1" |
| +} |
| + |
| +// marked reads the record, and whether a successor (a move from dst) may |
| +// start: the completion barrier. |
| +func (w *finishWorld) marked() (bool, error) { |
| + obs, err := hostclaim.Read(w.t.Context(), w.store, finishHost) |
| + if err != nil { |
| + w.t.Fatal(err) |
| + } |
| + done, ok := hostclaim.ReadCompletedRelocation(obs.Claim) |
| + return ok && done.Discharged, hostclaim.StartGuard(obs.Claim, []string{"uk"}, []string{"spa"}) |
| +} |
| + |
| +// done asserts the finished state: discharged once in all, the hold |
| +// released, no fence, the completion marked so a successor may start, and |
| +// the op's journals swept by the same pass. |
| +func (w *finishWorld) done(discharges int) { |
| + w.t.Helper() |
| + ok, barrier := w.marked() |
| + if w.discharged != discharges || w.held() || w.act.svc.Fenced(finishHost) || !ok || barrier != nil { |
| + w.t.Fatalf("discharged %d (want %d) held=%v fenced=%v marked=%v barrier=%v", w.discharged, discharges, w.held(), w.act.svc.Fenced(finishHost), ok, barrier) |
| + } |
| + if _, err := os.Stat(filepath.Join(w.dataDir, "relocate", "ops", "rel-1")); !errors.Is(err, os.ErrNotExist) { |
| + w.t.Fatalf("the finished op's journals survived: %v", err) |
| + } |
| +} |
| + |
| +// Astra round 7, finding 1 (ruling 80): a completion is finished from the |
| +// claim's record on every pass, whatever the row says, and every step |
| +// resumes after a failure or a crash at any point, through the pass. |
| +func TestRelocatePassFinishesACompletionFromItsRecord(t *testing.T) { |
| + t.Run("in one pass", func(t *testing.T) { |
| + w := newFinishWorld(t) |
| + w.pass() |
| + w.done(1) |
| + w.pass() |
| + w.done(1) |
| + }) |
| + t.Run("a failed discharge keeps everything, the next pass finishes", func(t *testing.T) { |
| + w := newFinishWorld(t) |
| + w.discharge = errors.New("the participation slot is still open") |
| + w.pass() |
| + if ok, barrier := w.marked(); w.discharged != 1 || !w.held() || w.act.svc.FenceGeneration(finishHost) != 12 || ok || !errors.Is(barrier, hostclaim.ErrPendingChore) { |
| + t.Fatalf("discharged %d held=%v fence=%d marked=%v barrier=%v", w.discharged, w.held(), w.act.svc.FenceGeneration(finishHost), ok, barrier) |
| + } |
| + w.discharge = nil |
| + w.pass() |
| + w.done(2) |
| + }) |
| + t.Run("the mark's CAS fails, the next pass marks", func(t *testing.T) { |
| + w := newFinishWorld(t) |
| + w.store.fail = 1 |
| + w.pass() |
| + if ok, barrier := w.marked(); w.discharged != 1 || w.held() || w.act.svc.Fenced(finishHost) || ok || !errors.Is(barrier, hostclaim.ErrPendingChore) { |
| + t.Fatalf("discharged %d held=%v fenced=%v marked=%v barrier=%v", w.discharged, w.held(), w.act.svc.Fenced(finishHost), ok, barrier) |
| + } |
| + // The row is no longer held: the record alone brings it back. |
| + w.pass() |
| + w.done(1) |
| + }) |
| + t.Run("the lift never ran after the hold went, the next pass lifts", func(t *testing.T) { |
| + w := newFinishWorld(t) |
| + // An earlier call discharged and released the hold, then failed. |
| + w.discharged = 1 |
| + if _, err := w.reg.ReleaseHold(finishHost, "rel-1"); err != nil { |
| + t.Fatal(err) |
| + } |
| + w.pass() |
| + w.done(1) |
| + }) |
| + t.Run("a restart after the hold release", func(t *testing.T) { |
| + w := newFinishWorld(t) |
| + w.discharged = 1 |
| + if _, err := w.reg.ReleaseHold(finishHost, "rel-1"); err != nil { |
| + t.Fatal(err) |
| + } |
| + w.act.svc = &activator.Service{} // the restart: the fence went with the process |
| + w.pass() |
| + w.done(1) |
| + }) |
| +} |
| +``` |
| + |
| `internal/host/relocationlanding_test.go` (new): |
| ```go |
| @@ -13648,14 +13986,14 @@ func relocatePassEnabled(getenv func(string) string) bool { |
| type RelocationRuntime struct { |
| Worker relocationStepper |
| Participant relocationStepper |
| - Box string // this box, for alerts |
| + Box string // this box: its alerts, and the completions it finishes |
| // Convert is J3 for a landing claim (convertFencedLanding); Finish is |
| - // RelocationEffects.Finish for a row still held relocating. |
| - Convert func(context.Context, release.Store, hostclaim.Observed) error |
| - Finish func(ctx context.Context, host, op string) error |
| - Registry *registry.Registry |
| - DataDir string // the managed data root: relocate/ journals (N9) |
| - alerted sync.Map // host\x00cause: one alert per host and cause (rulings 35, 36) |
| + // RelocationEffects.Finish for a completion this box has not yet |
| + // marked discharged (ruling 80). |
| + Convert func(context.Context, release.Store, hostclaim.Observed) error |
| + Finish func(ctx context.Context, host, op string) error |
| + DataDir string // the managed data root: relocate/ journals (N9) |
| + alerted sync.Map // host\x00cause: one alert per host and cause (rulings 35, 36) |
| } |
| // relocationStepper is hostrelocate.Worker's and Participant's Step. |
| @@ -13885,7 +14223,7 @@ func NewRelocationRuntime(ctx context.Context, getenv func(string) string, store |
| Self: termfence.WrittenBy{Box: in.Enrollment.Box, Enrollment: in.Enrollment.Enrollment}, |
| Region: stores.Region, Controller: newClaimID(), NewID: newClaimID, Now: time.Now} |
| participant := hostrelocate.Participant{Claims: store, Box: in.Enrollment.Box, Enrollment: in.Enrollment.Enrollment, Effects: effects, Now: time.Now} |
| - return &RelocationRuntime{Worker: worker, Participant: participant, Box: in.Enrollment.Box, Convert: convertFencedLanding(worker, store), Finish: effects.Finish, Registry: reg, DataDir: cfg.DataDir}, nil |
| + return &RelocationRuntime{Worker: worker, Participant: participant, Box: in.Enrollment.Box, Convert: convertFencedLanding(worker, store), Finish: effects.Finish, DataDir: cfg.DataDir}, nil |
| } |
| // wakeDispatch is the wake's owner dispatcher, as Build bound it. |
| @@ -13917,8 +14255,9 @@ func (r *RelocationRuntime) Run(ctx context.Context, store release.Store, serial |
| // once is one pass. Per host, in this order, so a passed probe completes |
| // in one pass (ruling 49): the participant (its probe stop), the worker |
| -// (the serving CAS), then this box's completed hold. A landing whose |
| -// target is fenced is converted when its move committed (J3). |
| +// (the serving CAS), then the completion this box has not yet finished, |
| +// found from the claim's record (ruling 80). A landing whose target is |
| +// fenced is converted when its move committed (J3). |
| func (r *RelocationRuntime) once(ctx context.Context, store release.Store) { |
| keys, err := store.List(ctx, "control/hosts/") |
| if err != nil { |
| @@ -13949,7 +14288,7 @@ func (r *RelocationRuntime) once(ctx context.Context, store release.Store) { |
| r.alert(host, "worker", err) |
| } |
| } |
| - r.releaseCompletedHold(ctx, host) |
| + r.finishCompletion(ctx, store, host) |
| } |
| // Ruling 61: journals of operations that are over go, once per pass. |
| if r.DataDir != "" { |
| @@ -13959,19 +14298,27 @@ func (r *RelocationRuntime) once(ctx context.Context, store release.Store) { |
| } |
| } |
| -// releaseCompletedHold finds this box's row still held relocating and |
| -// hands it to Finish, which releases it only once the claim serves here |
| -// and main's participation is discharged (ruling 66); otherwise nothing. |
| -func (r *RelocationRuntime) releaseCompletedHold(ctx context.Context, host string) { |
| - if r.Registry == nil || r.Finish == nil { |
| +// finishCompletion is ruling 80: a serving claim whose completed |
| +// relocation names this box and is not yet marked discharged is handed to |
| +// Finish (the hold release, relocationReleaseHold) on every pass, read |
| +// afresh after this pass's steps so a serving CAS made just now is |
| +// finished in the same pass. The work is found from the claim's record, |
| +// never from the row: a row already released, a lift that never ran or a |
| +// mark whose CAS failed is retried until the mark is written. |
| +func (r *RelocationRuntime) finishCompletion(ctx context.Context, store release.Store, host string) { |
| + if r.Finish == nil { |
| + return |
| + } |
| + obs, err := hostclaim.Read(ctx, store, host) |
| + if err != nil || obs.Absent { |
| return |
| } |
| - row, err := r.Registry.Get(host) |
| - if err != nil || row == nil || row.Status != "relocating" || row.HoldOp == "" { |
| + done, ok := hostclaim.ReadCompletedRelocation(obs.Claim) |
| + if !ok || done.Discharged || done.Stop.Box != r.Box { |
| return |
| } |
| - if err := r.Finish(ctx, host, row.HoldOp); err != nil { |
| - r.alert(host, "release-hold", err) |
| + if err := r.Finish(ctx, host, done.Op); err != nil { |
| + r.alert(host, "finish", err) |
| } |
| } |
| @@ -14207,7 +14554,7 @@ Expected: PASS. Run outside the sandbox (the `TestPlan9`/`TestPlan11` regression |
| - [ ] **Step 8: Commit** |
| ```bash |
| -git add internal/console/claim_runtime_chooser_test.go internal/edgesfile/snapshot.go internal/edgesfile/snapshot_test.go internal/host/relocatepass.go internal/host/relocatepass_test.go internal/host/relocationlanding.go internal/host/relocationlanding_test.go internal/hostrelocate/pin_versioned_test.go internal/console/apiinstances.go internal/console/boxchoice.go internal/console/boxselect.go internal/console/claim_runtime.go internal/console/instancesplacement.go internal/console/regioncandidates.go internal/console/regioncandidates_test.go internal/console/regiondefault.go internal/host/agent.go internal/host/claimparticipation_relocation.go internal/host/claimrelocation.go internal/hostrelocate/worker.go internal/hostrelocate/worker_bounds_test.go |
| +git add internal/console/claim_runtime_chooser_test.go internal/edgesfile/snapshot.go internal/edgesfile/snapshot_test.go internal/host/relocatepass.go internal/host/relocatepass_test.go internal/host/relocationfinish_unix_test.go internal/host/relocationlanding.go internal/host/relocationlanding_test.go internal/hostrelocate/pin_versioned_test.go internal/console/apiinstances.go internal/console/boxchoice.go internal/console/boxselect.go internal/console/claim_runtime.go internal/console/instancesplacement.go internal/console/regioncandidates.go internal/console/regioncandidates_test.go internal/console/regiondefault.go internal/host/agent.go internal/host/claimparticipation_relocation.go internal/host/claimrelocation.go internal/hostrelocate/worker.go internal/hostrelocate/worker_bounds_test.go |
| git commit -m "host: with the claim and region HA on and moves' files provider bound, every box replays its region's relocations one pass per host and converts a fenced committed landing; edgesfile: the live placement read the console and the pass share |
| Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" |
| @@ -14271,7 +14618,7 @@ Unchanged from the 1b-next plan's Task 11 (L13558-13586), with the model at `ori |
| | What | Waits for | |
| |---|---| |
| | Drains (`reason: drain`, `rebalance`) | C1 | |
| -| Switching relocation on anywhere | C10 confirmed on #236, C8 with O1 registered, O2′ wired, Task 10b landed (R5-3), the two-box run on dev (checklist item 12), Task 11 | |
| +| Switching relocation on anywhere | C10 and C12 confirmed on #236, C8 with O1 registered, O2′ wired, Task 10b landed (R5-3), the two-box run on dev (checklist item 12), Task 11 | |
| | More than one probe attempt per target | C11 | |
| | A shorter denial for an excluded target | C6 | |
| | The install's three actions under the claim's own Policy | C9 | |
| @@ -14284,17 +14631,18 @@ Unchanged from the 1b-next plan's Task 11 (L13558-13586), with the model at `ori |
| ## What cannot be planned yet |
| -- **C10's answer.** Its code is committed (Task 9); the claim confirms or refuses it on #236. |
| +- **C10's and C12's answers.** Their code is committed (Tasks 9 and 8a); the claim confirms or refuses each on #236. |
| - **The two-box end-to-end run** (R5-4) needs a two-box harness and an activator test service; until one exists it is checklist item 12 and Task 11's joint round. |
| - **W1, W2 and W4** are observable only on a box. |
| -## Unresolved in rev 5.1 |
| +## Unresolved in rev 5.2 |
| - **R5-2.** Rev 3's ported lifecycle property (ruling 63) stays withdrawn. The successor case Astra named is now prevented (ruling 77) and tested through the real start transitions; other interleavings the property explored are covered only by the per-hop tests. |
| - **R5-3.** Accepted while dormant; Task 10b is written and landed before activation (ruling 79). |
| - **R5-4.** Accepted as an explicit activation gate (checklist item 12), with no claim that the per-hop tests are equivalent integration coverage. |
| - **M-K25** is a new moves ask; N11 is its committed fallback. |
| - **C10** is committed as region HA code; the claim session's confirmation is outstanding. |
| +- **C12** (N13, `StartGuard`'s completion barrier) is committed in Task 8a; the claim session's confirmation is outstanding. |
| ## Self-review |