carlos / platform Public

docs: region HA plan 1b-next rev 14 closes the controller lifecycle class with a property test

Astra round 13, two P2s, both accepted. A controller that loses its lease
is now joined and its scoped cleanup repeated, then a fresh controller
opens for the same operation without a restart; denial stays durable
throughout. A failed clear forgets a participation only if the hostexec
record is verifiably absent or another binding's, never on ErrState.
Because this was the sixth round on this class, the rev adds a property
test over real hostexec controllers, a fault-injecting store and a test
clock: 200 seeded event sequences, invariants (a) to (e) checked after
every step. Removing each of ten guards fails a seed. One guard survives
by design: Open repeats the cleanup itself. Building the test found
three more faults, now fixed: Open waits on cleanup without a context;
sweeping a superseded record waited out the pass; and a real-kernel
test was flaky under shared-process state. Dry-run in task order on
3407d114.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

claude 2 weeks ago 0d8979ec8fec5ae11816bde104e5be27266efe7a parent 9295bf9

1 file changed, +770 −83

docs/superpowers/plans/2026-10-07-region-ha-1b-next-relocation.md +770 −83

diff --git a/docs/superpowers/plans/2026-10-07-region-ha-1b-next-relocation.md b/docs/superpowers/plans/2026-10-07-region-ha-1b-next-relocation.md

index cfccb5e44..1542777a0 100644

--- a/docs/superpowers/plans/2026-10-07-region-ha-1b-next-relocation.md

+++ b/docs/superpowers/plans/2026-10-07-region-ha-1b-next-relocation.md

@@ -1,15 +1,50 @@

-# Region HA 1b-next: Relocation, Row Cases, Create Choice and Fences Implementation Plan (rev 13)

+# Region HA 1b-next: Relocation, Row Cases, Create Choice and Fences Implementation Plan (rev 14)

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Build what region HA spec 1 had to leave until the host lifecycle claim was on main: the `relocating` operation within one region, released by the source's stop evidence (drain) or by its termination fence (failover); reconcile's row cases for it; create's once-only choice of executor box with `boxchoice.Choose`; and every claim CAS that consumes a termination fence. All of it dormant behind `CARLOS_REGION_HA`, and none of it mergeable until the joint two-box simulation round passes (L18).

-**Architecture:** Two stages (ruling 24). **Stage A, build now:** Tasks 1 to 7 and 9a, buildable on main today and dry-run on every revision: the claim library's relocation transitions and `released` coverage (`internal/hostclaim`), create's executor selection (`internal/hostclaim`, `internal/hostops`), the row and wake decisions (`internal/hostbox`), the writers-stopped observation (`internal/stopevidence`), the regional worker and the participant driver (new `internal/hostrelocate`), and the box-side release adapter with its receipt verifier (`internal/host`). **Stage B, gated:** Tasks 8a, 8b, 8c, 9 and 10 bind to claim Plan 6b and moves Plans 1 and 2, none merged. They keep their design and contracts here; their code is rewritten against the real 6b code after it merges and re-reviewed then (the "Stage B re-plan" step before Task 8a lists every obligation rounds 1 to 12 placed on them). What those need and no plan declares is a numbered request, never an assumption. The last task is the merge gate. The worker only reads termination fences: `hostfence.TerminationWorker` writes them, and `hostactors.Directory` is the only roster reader.

+**Architecture:** Two stages (ruling 24). **Stage A, build now:** Tasks 1 to 7 and 9a, buildable on main today and dry-run on every revision: the claim library's relocation transitions and `released` coverage (`internal/hostclaim`), create's executor selection (`internal/hostclaim`, `internal/hostops`), the row and wake decisions (`internal/hostbox`), the writers-stopped observation (`internal/stopevidence`), the regional worker and the participant driver (new `internal/hostrelocate`), and the box-side release adapter with its receipt verifier (`internal/host`). **Stage B, gated:** Tasks 8a, 8b, 8c, 9 and 10 bind to claim Plan 6b and moves Plans 1 and 2, none merged. They keep their design and contracts here; their code is rewritten against the real 6b code after it merges and re-reviewed then (the "Stage B re-plan" step before Task 8a lists every obligation rounds 1 to 13 placed on them). What those need and no plan declares is a numbered request, never an assumption. The last task is the merge gate. The worker only reads termination fences: `hostfence.TerminationWorker` writes them, and `hostactors.Directory` is the only roster reader.

**Tech Stack:** Go 1.26, module `github.com/carlosframework/platform`; `release.Store` (create-only and ETag compare-and-swap puts); the claim library (`hostclaim`), roster (`hostactors`), fences (`hostfence`, `termfence`), box choice (`boxchoice`), files (`hostfiles`), generation selection (`genselect`), stop evidence (`stopevidence`, `internal/host` stop receipts), execution controller (`hostexec`).

**Spec:** `docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md`, rev 5.4 with §24 (rulings H1 to H19 final). Read §0, §2, §5.2, §5.3, §5.8, §5.9, §5.10, §9 (L1 to L20, accepted) and §24 first. Then the claim spec rev 6.30 on `origin/design-replica-ownership` (`docs/superpowers/specs/2026-10-02-host-lifecycle-claim-design.md`): §1.1's activation gate, §4 ("Same-region relocation (L1)" and the relocation exception to CC11), §5.1 and P5-D2/P5-D6 (the create intent), §6.1 ("Release before full stop (L10)"), §6.4a item 2 (`released`), §7.3 D2 ("Automatic termination (L2, L4)", "The fence's journal reference") and D4 ("Automatic loss persistence"). Then moves rev 7.22 on `origin/design-moves-any-region` §7.6 ("One install routine, two sources", L16 and L17) and §7.10 (the pre-flip conversion). The contracts note `docs/superpowers/notes/2026-10-07-region-ha-contracts.md` gives `hostfiles` v2, `CopyManifest`, `Publish` and `boxchoice`. Plan 1b (`docs/superpowers/plans/2026-10-06-region-ha-1b-claim-readers.md`) lists in "Waits for the claim" what this plan now builds.

+## Rev 14: round 13 dispositions

+Astra's round 13 (`CODEX-PLAN-REGION-HA-1BN-R13.md`) resolved both findings from round 12 and raised two P2s, both in the execution-controller lifecycle. Both are accepted; none is declined. This is the sixth round in a row to find a controller cleanup or recovery interleaving (rounds 2, 4, 5, 6, 12 and 13). So rev 14 also closes the whole class with a property test over real hostexec controllers, not only the two examples.

+The changes were dry-run on a clean 3407d114 with stage A applied in task order, building, vetting and running each task's tests after it, plus vet for darwin and windows and the cross-builds. The property test ran its 200 seeds niced, in about three minutes. Each lifecycle guard was then removed in turn (mutation check); the results are below.

+|---|---|---|---|---|---|

+| 1 (P2) | A | One failed lease put makes hostexec latch the controller lost (controller.go, `run`: `loseLocked(putErr)`), and a lost controller never becomes ready again. Rev 13's `Ready` reported the error and kept the controller, `Step` reopens only a missing one, and `Sweep` keeps a running operation's controller. A transient storage fault in `installing` therefore stalled the relocation until the agent restarted. | accepted | `RelocationControllers.live`, used by `Ready` and `Acknowledge`, recovers within the same operation. If the open controller has lost its lease (`Lost()` is closed, newly on `RelocationController`), it is joined (`Close`, bounded by the pass) and the scoped cleanup is repeated. The object is then forgotten, but its journal entry is kept. `Ready` reports `ErrNoController`, and the participant's `Acknowledge` opens a fresh controller with fresh authority and its own lease. The claim keeps the first acknowledgement, as after a restart (rev 2, finding 10). The record on disk stays `lost` until the fresh controller replaces it, so denial is durable throughout. A failed join or cleanup keeps the lost controller, and the next pass repeats both. | Task 7 (`TestRelocationAcknowledgeRetainsTheControllerUnderTheAgentLifetime`: lost lease, cleanup failing, then recovered and reopened, no restart); the property test |

+| 2 (P2) | A | `ClearStale` returns `ErrState` for an absent or invalid claim (controller.go:619) and for an unreadable or mismatched local record, as well as for a record that names another binding. Rev 13's `finish` treated every `ErrState` as replacement and forgot the participation while this binding's record stayed. | accepted | `finish` no longer reads anything into the error. It asks the record itself (`recordIsMine`: `<DataDir>/.hostexec/<host>.json`, compared on box, enrollment, incarnation and operation, as `ClearStale` compares them). The participation is forgotten only when the record is absent or names another binding. An unreadable record, or this binding's record, keeps the participation and reports the error. `RelocationControllers.DataDir` is new; production sets it to the assembly's `Guard.DataDir`, and with it empty nothing is ever forgotten on a failed clear. | Task 7; the property test (claim read failure, absent claim, invalid local record, then restart and recovery) |

+| C1 | A | The class. | added | `TestRelocationControllerLifecycleProperties` (`internal/host/relocationlifecycle_linux_test.go`) runs over real hostexec controllers, a store that injects faults, and a clock the test drives. Each seed runs 30 random events: lease-renewal failure, a close that times out (a cleanup that blocks), cleanup failure, `ClearStale` failing for each of its causes (claim unreadable, claim absent, invalid local record, a successor that lists this box), agent restart, a successor starting before the sweep, completion, and recovery of the storage. Participant passes are the real `Participant.Step` and `Sweep`. After every step it asserts: (a) no clear without a successful scoped cleanup earlier in the same call; (b) no participation forgotten while its record exists; (c) at most one controller per host; (d) no pass past its deadline. Then (e), liveness: once faults stop, within eight passes the running relocation is ready again; after completion every record is cleared, no controller remains, and a new relocation opens. 200 seeds by default, 20 with `-short`. | Task 7 |

+| C2 | A | Found while building C1. `hostexec.Open` waits for its Cleanup hook without a context (`err = <-cleaned`), so rev 13's `openBounded`, which waited for `Open` to return, did not bound a pass whose cleanup blocks. | fixed | `openBounded` runs the open apart and returns when the pass's context ends. A controller that opens too late is closed, never kept. | Task 7; invariant (d) |

+| C3 | A | Found while building C1. After a handover, sweeping the predecessor called `ClearStale` while the successor's live controller held the host's lock, which waited out the whole pass. | fixed | With no controller object, `finish` reads the record first. If it is verifiably not this binding's (absent, or the successor's), the participation is forgotten without a clear: nothing of its own is left to clear. The handover no longer forgets superseded entries itself; the next sweep does, through that check. | Task 7 (`TestASuccessorRelocationRetiresItsPredecessorsController` now sweeps A after the handover) |

+| C4 | A | Found by the in-order dry run. `TestProcessDescriptorsTellsSharedFromUnsharedThreadsByKernelIdentity` (rev 7) failed about once in five full-package runs. Main's `TestFDHoldersUnsharedThread` can run its unshare on the leader thread, which then keeps a private table for the rest of the process, so no thread shares the leader's. | fixed | The real-kernel test re-executes the test binary and runs in a fresh process. Its guards still fail it (the `KCMP_FILES` check was rechecked). | Task 4 |

+Mutation check: each lifecycle guard was removed in turn, and the property test was run with `-failfast`.

+| Guard removed | Result |

+|---|---|

+| `finish` repeating the scoped cleanup before a clear | killed, seed 1: invariant (a) |

+| `finish` forgetting only a verified other binding | killed, seed 3: invariant (b) |

+| `live` recovering a lost controller | killed, seed 13: invariant (e) (the relocation never recovers) |

+| `openBounded` | killed: invariant (d) (a pass never returns) |

+| the journal written before the open | killed, seed 3: invariant (b) |

+| the handover before a successor opens | killed, seed 53: invariant (e) |

+| `finish` closing before the clear | killed, seed 2: invariant (e) (never clean) |

+| `recordIsMine` comparing the binding | killed, seed 3: invariant (b) |

+| a handover treating hostexec's `ErrInactive` as superseded | killed, seed 7: invariant (e) |

+| the restart path verifying the record before forgetting | killed, seed 3: invariant (b) |

+| `live` repeating the scoped cleanup before the reopen | survived 200 seeds |

+The survivor is defence in depth, not a load-bearing guard. `hostexec.Open` runs the controller's Cleanup hook, which is the same scoped `relocationCleanup` (Task 8a), and fails if the cleanup fails. So no fresh controller, and no fresh authority, exists without a successful cleanup, and invariant (a) is about clears, which recovery never makes. The explicit cleanup is kept so that a failure surfaces at the pass that notices the loss.

+Astra's readings of rulings 1 to 24 stand. Rev 13's statement that the predecessor's entry is removed "once the successor's open succeeds" is replaced by C3.

## Rev 13: round 12 dispositions

Astra's round 12 (`CODEX-PLAN-REGION-HA-1BN-R12.md`) resolved the file-origin finding and raised two P2s. Both are accepted; none is declined. Finding 1 is stage A code. It was dry-run on a clean 3407d114 with stage A applied in task order, building and running each task's tests after it, plus vet for darwin and windows and the cross-builds. Four new guards were removed in turn, and each removal failed its test (listed in the Self-review). Finding 2 is a stage B contract and has no code.

@@ -278,7 +313,7 @@ Left out, with where it goes:

- **Go environment.** `GOFLAGS=-buildvcs=false`. Leave `GOCACHE` at its default; if the sandbox refuses it, `GOCACHE=$TMPDIR/gocache` for that command. Socket tests need `TMPDIR=/tmp/k` (`mkdir -p /tmp/k`). Run tests niced (`nice -n 19`), targeted with `-run`; never the full `-race` suite, and no simulation sweeps. Scratch goes under `$TMPDIR`.

- **Gate before each commit:** `go build ./... && go vet ./<pkgs>/ && gofmt -l <dirs>` (empty), the task's targeted tests, then `GOOS=darwin go build ./... && GOOS=windows go build ./...`. (`GOOS=windows go vet ./internal/...` fails on main's own test files, `release/versions_test.go`, `appconfig/syncexec_test.go` and `activator/activator_test.go`; vet the task's packages instead.)

- Commit messages: lowercase `area:` prefix and a sentence stating the behaviour, then a blank line and `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`. Work on a branch cut from `origin/main` (suggested `region-ha-1b-next`). Merging is Paul's call, through `amadan branch merge`, and only after Task 11.

-- **Dry run.** Tasks 1 to 7 were applied in order to `origin/main` at 3407d114, built, vetted, cross-built for darwin, windows and linux/arm64, and their tests run; nineteen guards were each removed in turn and their test watched to fail (rev 1), then rev 2's twenty-five, rev 3's eleven, rev 4's eight, rev 5's seven, rev 6's eleven, rev 7's three, rev 8's two, rev 9's two, rev 10's one, rev 11's four, rev 12's four and rev 13's four (listed in the Self-review); from rev 10 on, stage A is also applied one task at a time in plan order, building and testing after each. Then the code was removed; only this document is committed.

+- **Dry run.** Tasks 1 to 7 were applied in order to `origin/main` at 3407d114, built, vetted, cross-built for darwin, windows and linux/arm64, and their tests run; nineteen guards were each removed in turn and their test watched to fail (rev 1), then rev 2's twenty-five, rev 3's eleven, rev 4's eight, rev 5's seven, rev 6's eleven, rev 7's three, rev 8's two, rev 9's two, rev 10's one, rev 11's four, rev 12's four, rev 13's four and rev 14's ten killed mutations of the lifecycle property (listed in the Self-review); from rev 10 on, stage A is also applied one task at a time in plan order, building and testing after each. Then the code was removed; only this document is committed.

## Review Focus

@@ -4166,10 +4201,12 @@ package stopevidence

import (

"errors"

"os"

+ "os/exec"

"path/filepath"

"runtime"

"slices"

"strconv"

+ "strings"

"testing"

"golang.org/x/sys/unix"

@@ -4453,6 +4490,19 @@ func threadWith(t *testing.T, unshare bool, act func() error) (tid int, release

// description or by reopening it independently there, and is reported

// with FD -1, never as the leader's.

func TestProcessDescriptorsTellsSharedFromUnsharedThreadsByKernelIdentity(t *testing.T) {

+ // The kernel facts are this process's own threads, so the test runs in

+ // a fresh copy of the test binary: another test in the package

+ // (main's TestFDHoldersUnsharedThread) may have unshared the leader

+ // thread's table for good, and then no thread shares the leader's.

+ if os.Getenv("STOPEVIDENCE_KCMP_CHILD") != "1" {

+ cmd := exec.Command(os.Args[0], "-test.run=^TestProcessDescriptorsTellsSharedFromUnsharedThreadsByKernelIdentity$", "-test.count=1", "-test.v")

+ cmd.Env = append(os.Environ(), "STOPEVIDENCE_KCMP_CHILD=1")

+ out, err := cmd.CombinedOutput()

+ if err != nil || !strings.Contains(string(out), "--- PASS: TestProcessDescriptorsTellsSharedFromUnsharedThreadsByKernelIdentity") {

+ t.Fatalf("the fresh process: %v\n%s", err, out)

+ }

+ return

+ }

db := filepath.Join(t.TempDir(), "data.db")

f, err := os.OpenFile(db, os.O_RDWR|os.O_CREATE, 0o600)

if err != nil {

@@ -7465,7 +7515,7 @@ Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"

**Files:**

- Create: `internal/host/relocationreleases.go`, `internal/host/relocationeffects.go`, `internal/host/relocationflush.go`, `internal/hostfiles/platform.go`

-- Test: `internal/host/relocationreleases_test.go`, `internal/host/relocationeffects_test.go`, `internal/host/relocationcontrollers_linux_test.go`

+- Test: `internal/host/relocationreleases_test.go`, `internal/host/relocationeffects_test.go`, `internal/host/relocationcontrollers_linux_test.go`, `internal/host/relocationlifecycle_linux_test.go` (rev 14)

**Interfaces:**

- Consumes: Tasks 1, 4 and 6; main's `StopReceipt`, `StopRequest`, `StopResult`, `StopDrainer`, `DrainAcknowledgement`, `mustMarshalStopReceipt`, `ErrClaimStopUnavailable`, `ErrClaimStopReceipt`, `hostbox.Actor`, `hostexec.Binding`, `hostexec.State`, `hostfiles.Open`, `ExportWake`, `StageWake`, `StagePlatform`, `SyncDir`, `Commit`, `ReadPlatform`, `registry.Registry`, `hostexec.ClearStale`.

@@ -7600,6 +7650,14 @@ func (c *readyController) Clear(context.Context) error {

return nil

}

+func (c *readyController) Lost() <-chan struct{} {

+ ch := make(chan struct{})

+ if c.lost != nil {

+ close(ch)

+ }

+ return ch

func (c *readyController) Ready(context.Context) (execctl.State, error) {

if c.lost != nil {

return execctl.State{}, c.lost

@@ -7940,11 +7998,26 @@ func TestRelocationAcknowledgeRetainsTheControllerUnderTheAgentLifetime(t *testi

if err := e.Ready(context.Background(), obs); err != nil {

t.Fatal(err)

}

- // A lost lease is reported before any effect.

+ // A lost lease is recovered without a restart (round 13 finding 1):

+ // Ready joins the lost controller and repeats the scoped cleanup,

+ // reports no controller, and the participant's Acknowledge opens a

+ // fresh one; a cleanup that fails keeps the lost one for the next pass.

opened[0].lost = execctl.ErrLost

- if err := e.Ready(context.Background(), obs); !errors.Is(err, execctl.ErrLost) {

+ cleanupErr := errors.New("cleanup: a process would not stop")

+ e.Controllers.Cleanup = func(context.Context, execctl.Binding) error { return cleanupErr }

+ if err := e.Ready(context.Background(), obs); !errors.Is(err, cleanupErr) || !opened[0].closed {

+ t.Fatalf("lost controller, cleanup failing: %v closed=%v", err, opened[0].closed)

+ }

+ e.Controllers.Cleanup = func(context.Context, execctl.Binding) error { return nil }

+ if err := e.Ready(context.Background(), obs); !errors.Is(err, hostrelocate.ErrNoController) {

t.Fatalf("lost controller: %v", err)

}

+ if _, err := e.Acknowledge(context.Background(), obs); err != nil || len(opened) != 2 {

+ t.Fatalf("recovery: %v opened=%d", err, len(opened))

+ }

+ if err := e.Ready(context.Background(), obs); err != nil {

+ t.Fatalf("after recovery: %v", err)

+ }

// An agent restart has no retained controller: Ready says so, and

// Acknowledge opens a fresh one.

restarted := *e

@@ -7952,7 +8025,7 @@ func TestRelocationAcknowledgeRetainsTheControllerUnderTheAgentLifetime(t *testi

if err := restarted.Ready(context.Background(), obs); !errors.Is(err, hostrelocate.ErrNoController) {

t.Fatalf("after a restart: %v", err)

}

- if _, err := restarted.Acknowledge(context.Background(), obs); err != nil || len(opened) != 2 {

+ if _, err := restarted.Acknowledge(context.Background(), obs); err != nil || len(opened) != 3 {

t.Fatalf("reopen: %v opened=%d", err, len(opened))

}

// Finishing a participation closes its controller and forgets it; a

@@ -7964,21 +8037,21 @@ func TestRelocationAcknowledgeRetainsTheControllerUnderTheAgentLifetime(t *testi

// sweep can finish it; then it is forgotten.

cut, cancel := context.WithCancel(context.Background())

cancel()

- if err := restarted.Finish(cut, "shop.example", "rel-1"); err == nil || opened[1].closed || len(kept(&restarted)) != 1 {

- t.Fatalf("a cut-short close forgot the controller: %v closed=%v retained=%v", err, opened[1].closed, kept(&restarted))

+ if err := restarted.Finish(cut, "shop.example", "rel-1"); err == nil || opened[2].closed || len(kept(&restarted)) != 1 {

+ t.Fatalf("a cut-short close forgot the controller: %v closed=%v retained=%v", err, opened[2].closed, kept(&restarted))

}

- if err := restarted.Finish(context.Background(), "shop.example", "rel-1"); err != nil || !opened[1].closed || !opened[1].cleared || len(kept(&restarted)) != 0 {

- t.Fatalf("finish: %v closed=%v cleared=%v retained=%v", err, opened[1].closed, opened[1].cleared, kept(&restarted))

+ if err := restarted.Finish(context.Background(), "shop.example", "rel-1"); err != nil || !opened[2].closed || !opened[2].cleared || len(kept(&restarted)) != 0 {

+ t.Fatalf("finish: %v closed=%v cleared=%v retained=%v", err, opened[2].closed, opened[2].cleared, kept(&restarted))

}

if err := restarted.Ready(context.Background(), obs); !errors.Is(err, hostrelocate.ErrNoController) {

t.Fatalf("after finish: %v", err)

}

obs.Claim.Op.ID = "rel-2"

- if _, err := restarted.Acknowledge(context.Background(), obs); err != nil || len(opened) != 3 {

+ if _, err := restarted.Acknowledge(context.Background(), obs); err != nil || len(opened) != 4 {

t.Fatalf("the next relocation: %v opened=%d", err, len(opened))

}

- if err := restarted.Controllers.Close(context.Background()); err != nil || !opened[2].closed {

- t.Fatalf("close: %v closed=%v", err, opened[2].closed)

+ if err := restarted.Controllers.Close(context.Background()); err != nil || !opened[3].closed {

+ t.Fatalf("close: %v closed=%v", err, opened[3].closed)

}

@@ -8550,7 +8623,7 @@ func TestFinishClearsTheDurableRecordSoUpdatesAreEligible(t *testing.T) {

CloseAdmission: func() {}, CancelWork: func() {}, Cleanup: scoped}

journal := filepath.Join(dataDir, "relocate", "participations")

controllers := func() *RelocationControllers {

- return &RelocationControllers{Journal: journal,

+ return &RelocationControllers{Journal: journal, DataDir: dataDir,

Cleanup: func(ctx context.Context, _ execctl.Binding) error { return scoped(ctx) },

Clear: func(ctx context.Context, b execctl.Binding) error {

c := cfg

@@ -8673,7 +8746,7 @@ func TestASuccessorRelocationRetiresItsPredecessorsController(t *testing.T) {

t.Cleanup(stop)

e, _, _, _ := effectsFixture(t)

e.Lifetime = lifetime

- e.Controllers = &RelocationControllers{Journal: filepath.Join(dataDir, "relocate", "participations"),

+ e.Controllers = &RelocationControllers{Journal: filepath.Join(dataDir, "relocate", "participations"), DataDir: dataDir,

Cleanup: func(context.Context, execctl.Binding) error { return nil },

Clear: func(ctx context.Context, b execctl.Binding) error {

c := cfg

@@ -8707,6 +8780,11 @@ func TestASuccessorRelocationRetiresItsPredecessorsController(t *testing.T) {

if ack.Op != "rel-2" {

t.Fatalf("ack: %+v", ack)

}

+ // A's journal entry stays until a sweep verifies the record is B's

+ // (rev 14), then it goes; B's stays.

+ if err := e.Finish(pass, host, "rel-1"); err != nil {

+ t.Fatalf("sweeping A after the handover: %v", err)

+ }

got, err := e.Retained()

if err != nil || len(got) != 1 || got[0].Op != "rel-2" {

t.Fatalf("retained after the handover: %+v %v", got, err)

@@ -8752,6 +8830,533 @@ func TestASuccessorRelocationRetiresItsPredecessorsController(t *testing.T) {

}

```

+`internal/host/relocationlifecycle_linux_test.go` (rev 14: the controller lifecycle as a property over real hostexec controllers; 200 seeds, `-short` 20):

+```go

+//go:build linux

+package host

+import (

+ "context"

+ "encoding/json"

+ "errors"

+ "fmt"

+ "math/rand"

+ "os"

+ "path/filepath"

+ "slices"

+ "strings"

+ "sync"

+ "testing"

+ "time"

+ "github.com/carlosframework/platform/internal/hostbox"

+ "github.com/carlosframework/platform/internal/hostclaim"

+ execctl "github.com/carlosframework/platform/internal/hostexec"

+ "github.com/carlosframework/platform/internal/hostguard"

+ "github.com/carlosframework/platform/internal/hostrelocate"

+ "github.com/carlosframework/platform/internal/release"

+// lifeSeeds is the property test's seed count (rev 14: 200 by default).

+// Run it niced; it is not a sweep.

+const lifeSeeds = 200

+// lifeStore is the bucket with faults: lease puts that fail (a renewal

+// that cannot reach the store) and claim reads that fail or find nothing.

+type lifeStore struct {

+ release.Store

+ mu sync.Mutex

+ failLease bool

+ claimErr error

+ claimKey string

+func (s *lifeStore) Put(ctx context.Context, key string, b []byte, etag string) (string, error) {

+ s.mu.Lock()

+ fail := s.failLease && strings.Contains(key, "/exec/")

+ s.mu.Unlock()

+ if fail {

+ return "", errors.New("lease put: store unreachable")

+ }

+ return s.Store.Put(ctx, key, b, etag)

+func (s *lifeStore) Get(ctx context.Context, key string) ([]byte, string, error) {

+ s.mu.Lock()

+ err := s.claimErr

+ s.mu.Unlock()

+ if err != nil && key == s.claimKey {

+ return nil, "", err

+ }

+ return s.Store.Get(ctx, key)

+func (s *lifeStore) DeleteIfMatch(ctx context.Context, key, etag string) error {

+ return release.DeleteIfMatch(ctx, s.Store, key, etag)

+// lifeClock is hostexec's clock under the test's hand: every After fires

+// on tick, so a renewal happens exactly when a step says so.

+type lifeClock struct {

+ mu sync.Mutex

+ wall time.Time

+ boot int64

+ waiters []chan time.Time

+func (c *lifeClock) Now() (time.Time, int64, error) {

+ c.mu.Lock()

+ defer c.mu.Unlock()

+ return c.wall, c.boot, nil

+func (c *lifeClock) After(time.Duration) <-chan time.Time {

+ ch := make(chan time.Time, 1)

+ c.mu.Lock()

+ c.waiters = append(c.waiters, ch)

+ c.mu.Unlock()

+ return ch

+func (c *lifeClock) tick() {

+ c.mu.Lock()

+ c.wall, c.boot = c.wall.Add(time.Second), c.boot+int64(time.Second)

+ ws := c.waiters

+ c.waiters = nil

+ now := c.wall

+ c.mu.Unlock()

+ for _, w := range ws {

+ w <- now

+ }

+// lifeWorld is one box ("src") taking part in a host's relocations over

+// real hostexec controllers, with the faults a seed injects.

+type lifeWorld struct {

+ t *testing.T

+ ctx context.Context

+ inner release.Store

+ store *lifeStore

+ clock *lifeClock

+ dataDir string

+ host string

+ cfg execctl.Config

+ e *RelocationEffects

+ p hostrelocate.Participant

+ ops int

+ mu sync.Mutex

+ cleanErr bool // both cleanups fail

+ block chan struct{} // non-nil: both cleanups wait for it (a close that times out)

+ cleanOK map[string]bool

+ saved []byte // a record the seed corrupted, to restore on recovery

+ violated []string

+const lifeDeadline = 400 * time.Millisecond

+func newLifeWorld(t *testing.T) *lifeWorld {

+ t.Helper()

+ ctx := context.Background()

+ w := &lifeWorld{t: t, ctx: ctx, host: "shop.bab.oncarlos.com", clock: &lifeClock{wall: time.Date(2026, 10, 7, 9, 0, 0, 0, time.UTC), boot: int64(time.Hour)}, cleanOK: map[string]bool{}}

+ w.dataDir = t.TempDir()

+ for _, name := range []string{"hostguard", ".hostexec", "locks", "exec-locks"} {

+ if err := os.Mkdir(filepath.Join(w.dataDir, name), 0o770); err != nil {

+ t.Fatal(err)

+ }

+ w.inner = release.NewDirStore(t.TempDir())

+ key, _ := hostclaim.Key(w.host)

+ w.store = &lifeStore{Store: w.inner, claimKey: key}

+ created := time.Date(2026, 10, 4, 1, 0, 0, 0, time.UTC)

+ nh := hostclaim.NewHolder{Family: "instance", Key: "apps/bab/jam/instances/" + w.host + ".json", Account: "bab", App: "jam", Host: w.host, Region: "ie", RecordRegion: "ie", CreatedAt: created}

+ body := []byte(`{"host":"` + w.host + `","account":"bab","app":"jam","region":"ie","created_at":"2026-10-04T01:00:00Z"}`)

+ w.claim(hostclaim.BeginCreateOn(nh, body, "c1", "console", created, hostclaim.ExecutorSelection{Box: "src", Enrollment: strings.Repeat("a", 32)}))

+ w.claim(hostclaim.CompleteCreate("c1", "src"))

+ w.claim(hostclaim.AdmitWriter(hostclaim.Admission{Box: "src", Mode: "cold", NewGeneration: "g1", Region: "ie", KeepStamp: true}))

+ w.cfg = execctl.Config{Store: w.store, Guard: hostguard.Guard{DataDir: w.dataDir, LockDir: filepath.Join(w.dataDir, "locks"), Claims: hostguard.StoreReader{Store: w.store}},

+ Directory: activeDirectory{}, Clock: w.clock, LockDir: filepath.Join(w.dataDir, "exec-locks"),

+ CloseAdmission: func() {}, CancelWork: func() {}, Cleanup: func(context.Context) error { return w.cleanup(context.Background(), "") }}

+ lifetime, stop := context.WithCancel(ctx)

+ t.Cleanup(stop)

+ w.e = &RelocationEffects{Lifetime: lifetime,

+ Identity: func(context.Context) (RelocationIdentity, error) {

+ return RelocationIdentity{Actor: hostbox.Actor{Box: "src", Enrollment: strings.Repeat("a", 32), Revision: 4}, BootID: "boot", ControllerID: "ctl"}, nil

+ },

+ Controller: func(ctx context.Context, b execctl.Binding) (RelocationController, error) {

+ c := w.cfg

+ c.Binding = b

+ opened, err := OpenRelocationController(ctx, c)

+ if err != nil {

+ return nil, err

+ }

+ return &lifeController{RelocationController: opened, w: w, op: b.Op}, nil

+ }}

+ w.e.Controllers = w.controllers()

+ w.p = hostrelocate.Participant{Claims: w.store, Box: "src", Enrollment: strings.Repeat("a", 32), Effects: w.e, Now: func() time.Time { return w.clock.wall }}

+ return w

+// lifeController checks invariant (a) at every clear of a live

+// controller's record.

+type lifeController struct {

+ RelocationController

+ w *lifeWorld

+ op string

+func (c *lifeController) Clear(ctx context.Context) error {

+ c.w.clearing(c.op)

+ return c.RelocationController.Clear(ctx)

+func (w *lifeWorld) controllers() *RelocationControllers {

+ return &RelocationControllers{Journal: filepath.Join(w.dataDir, "relocate", "participations"), DataDir: w.dataDir,

+ Cleanup: func(ctx context.Context, b execctl.Binding) error { return w.cleanup(ctx, b.Op) },

+ Clear: func(ctx context.Context, b execctl.Binding) error {

+ w.clearing(b.Op)

+ c := w.cfg

+ c.Binding = b

+ return execctl.ClearStale(ctx, c)

+ }}

+// cleanup is both cleanups: hostexec's hook (op "") and the scoped one.

+func (w *lifeWorld) cleanup(ctx context.Context, op string) error {

+ w.mu.Lock()

+ block, fail := w.block, w.cleanErr

+ if op != "" {

+ w.cleanOK[op] = false

+ }

+ w.mu.Unlock()

+ if block != nil {

+ select {

+ case <-block:

+ case <-ctx.Done():

+ return ctx.Err()

+ }

+ if fail {

+ return errors.New("cleanup: a partial install would not go")

+ }

+ if op != "" {

+ w.mu.Lock()

+ w.cleanOK[op] = true

+ w.mu.Unlock()

+ }

+ return nil

+// clearing is invariant (a): a record is cleared only straight after a

+// scoped cleanup of the same participation succeeded, in the same call.

+func (w *lifeWorld) clearing(op string) {

+ w.mu.Lock()

+ defer w.mu.Unlock()

+ if !w.cleanOK[op] {

+ w.violated = append(w.violated, "(a) clear of "+op+" without a successful scoped cleanup first")

+ }

+ w.cleanOK[op] = false

+func (w *lifeWorld) claim(tr func(*hostclaim.Claim) error) {

+ w.t.Helper()

+ o, err := hostclaim.Read(w.ctx, w.inner, w.host)

+ if err != nil {

+ w.t.Fatal(err)

+ }

+ if _, err := hostclaim.Update(w.ctx, w.inner, o, tr); err != nil {

+ w.t.Fatal(err)

+ }

+func (w *lifeWorld) current() *hostclaim.Claim {

+ w.t.Helper()

+ o, err := hostclaim.Read(w.ctx, w.inner, w.host)

+ if err != nil {

+ w.t.Fatal(err)

+ }

+ return o.Claim

+// startRelocation starts the next relocation, src taking part (as source

+// or as target), whatever the claim was doing.

+func (w *lifeWorld) startRelocation() {

+ w.ops++

+ id := fmt.Sprintf("rel-%d", w.ops)

+ from, to := "src", "dst"

+ if w.current().Op.Exec == "dst" {

+ from, to = "dst", "src"

+ }

+ w.claim(func(c *hostclaim.Claim) error {

+ c.Op = hostclaim.Op{Kind: hostclaim.KindRelocating, ID: id, Phase: hostclaim.PhaseStarting, Exec: from, Participants: []string{from, to},

+ Intent: map[string]string{"region": "ie", "source_box": from, "target_box": to}}

+ return nil

+ })

+func (w *lifeWorld) complete() {

+ if c := w.current(); c.Op.Kind == hostclaim.KindRelocating {

+ box := c.Op.Intent["target_box"]

+ w.claim(func(c *hostclaim.Claim) error {

+ c.Op = hostclaim.Op{Kind: hostclaim.KindServing, Exec: box, Participants: []string{box}}

+ return nil

+ })

+ }

+// pass is one participant pass, Step then Sweep as the agent runs them,

+// under a deadline; (d) is that it returns within it.

+func (w *lifeWorld) pass() {

+ w.mu.Lock()

+ w.cleanOK = map[string]bool{} // "the same call": nothing carries over from an earlier pass

+ w.mu.Unlock()

+ done := make(chan struct{})

+ start := time.Now()

+ go func() {

+ defer close(done)

+ ctx, cancel := context.WithTimeout(w.ctx, lifeDeadline)

+ defer cancel()

+ _, _ = w.p.Step(ctx, w.host)

+ _ = w.p.Sweep(ctx)

+ }()

+ select {

+ case <-done:

+ if took := time.Since(start); took > lifeDeadline+time.Second {

+ w.violate(fmt.Sprintf("(d) a pass took %v past its %v deadline", took, lifeDeadline))

+ }

+ case <-time.After(lifeDeadline + 3*time.Second):

+ w.t.Fatalf("(d) a pass never returned within %v", lifeDeadline+3*time.Second)

+ }

+func (w *lifeWorld) violate(s string) {

+ w.mu.Lock()

+ w.violated = append(w.violated, s)

+ w.mu.Unlock()

+// renew fires hostexec's timers so every controller renews (or loses).

+func (w *lifeWorld) renew() {

+ w.clock.tick()

+ time.Sleep(20 * time.Millisecond)

+func (w *lifeWorld) record() (execctl.State, bool, bool) {

+ raw, err := os.ReadFile(filepath.Join(w.dataDir, ".hostexec", w.host+".json"))

+ if errors.Is(err, os.ErrNotExist) {

+ return execctl.State{}, false, true

+ }

+ var st execctl.State

+ if err != nil || json.Unmarshal(raw, &st) != nil {

+ return st, true, false

+ }

+ return st, true, true

+func (w *lifeWorld) open() []RelocationController {

+ r := w.e.Controllers

+ r.mu.Lock()

+ defer r.mu.Unlock()

+ var out []RelocationController

+ for _, c := range r.open {

+ out = append(out, c)

+ }

+ return out

+// check is invariants (a) to (c) after a step.

+func (w *lifeWorld) check(step string) {

+ st, exists, readable := w.record()

+ if exists && readable {

+ retained, _ := w.e.Controllers.Retained()

+ if !slices.ContainsFunc(retained, func(p hostrelocate.Participation) bool { return p.Host == w.host && p.Op == st.Op }) {

+ w.violate(fmt.Sprintf("(b) after %s: %s's record exists but its participation is forgotten (retained %+v)", step, st.Op, retained))

+ }

+ if n := len(w.open()); n > 1 {

+ w.violate(fmt.Sprintf("(c) after %s: %d controllers for one host", step, n))

+ }

+func (w *lifeWorld) restart() {

+ w.recoverCleanup()

+ // The kernel releases a dead process's locks. In-process the old

+ // controllers are joined in the background: hostexec keeps retrying a

+ // "lost" write until it is durable, so with an invalid local record a

+ // join waits for the storage to recover.

+ for _, c := range w.open() {

+ go func() { _ = c.Close(context.Background()) }()

+ }

+ w.e.Controllers = w.controllers()

+func (w *lifeWorld) recoverCleanup() {

+ w.mu.Lock()

+ if w.block != nil {

+ close(w.block)

+ w.block = nil

+ }

+ w.cleanErr = false

+ w.mu.Unlock()

+func (w *lifeWorld) recoverAll() {

+ w.recoverCleanup()

+ w.store.mu.Lock()

+ w.store.failLease, w.store.claimErr = false, nil

+ w.store.mu.Unlock()

+ if w.saved != nil {

+ path := filepath.Join(w.dataDir, ".hostexec", w.host+".json")

+ if raw, err := os.ReadFile(path); err == nil && string(raw) == "{" {

+ _ = os.WriteFile(path, w.saved, 0o660)

+ }

+ w.saved = nil

+ }

+func (w *lifeWorld) event(r *rand.Rand) string {

+ switch n := r.Intn(14); n {

+ case 0, 1, 2, 3:

+ w.pass()

+ return "pass"

+ case 4:

+ w.renew()

+ return "renew"

+ case 5:

+ w.store.mu.Lock()

+ w.store.failLease = true

+ w.store.mu.Unlock()

+ w.renew()

+ return "lease renewal fails"

+ case 6:

+ w.mu.Lock()

+ if w.block == nil {

+ w.block = make(chan struct{})

+ }

+ w.mu.Unlock()

+ return "cleanup blocks (close times out)"

+ case 7:

+ w.mu.Lock()

+ w.cleanErr = true

+ w.mu.Unlock()

+ return "cleanup fails"

+ case 8:

+ w.store.mu.Lock()

+ if r.Intn(2) == 0 {

+ w.store.claimErr = errors.New("claim read: store unreachable")

+ } else {

+ w.store.claimErr = os.ErrNotExist // ClearStale: ErrState for an absent claim

+ }

+ w.store.mu.Unlock()

+ return "claim reads fail"

+ case 9:

+ path := filepath.Join(w.dataDir, ".hostexec", w.host+".json")

+ if raw, err := os.ReadFile(path); err == nil && w.saved == nil {

+ w.saved = raw

+ _ = os.WriteFile(path, []byte("{"), 0o660) // ClearStale: an invalid local record

+ }

+ return "local record invalid"

+ case 10:

+ w.restart()

+ return "agent restart"

+ case 11:

+ w.complete()

+ w.startRelocation() // the successor starts before this box's sweep

+ return "completion, successor before the sweep"

+ case 12:

+ w.complete()

+ return "completion"

+ default:

+ w.recoverAll()

+ return "storage recovers"

+ }

+// TestRelocationControllerLifecycleProperties drives seeded random

+// sequences of lease-renewal failures, close timeouts, cleanup failures,

+// ClearStale failures (claim unreadable, claim absent, invalid local

+// record, a successor that lists this box), agent restarts, successors

+// before the sweep, completions and recovery, over real hostexec

+// controllers, and checks after every step: (a) no clear without a

+// successful scoped cleanup in the same call; (b) no participation

+// forgotten while its record exists; (c) at most one controller per host;

+// (d) no pass past its deadline; and (e) once faults stop, within eight

+// passes the current relocation is ready again, then after completion

+// every record is cleared and no controller remains, and a new relocation

+// opens.

+func TestRelocationControllerLifecycleProperties(t *testing.T) {

+ seeds := lifeSeeds

+ if testing.Short() {

+ seeds = 20

+ }

+ for seed := 1; seed <= seeds; seed++ {

+ t.Run(fmt.Sprint(seed), func(t *testing.T) {

+ r := rand.New(rand.NewSource(int64(seed)))

+ w := newLifeWorld(t)

+ defer w.recoverAll()

+ w.startRelocation()

+ var trace []string

+ for i := 0; i < 30; i++ {

+ ev := w.event(r)

+ trace = append(trace, ev)

+ w.check(ev)

+ if len(w.violated) > 0 {

+ t.Fatalf("seed %d: %v\ntrace: %v", seed, w.violated, trace)

+ }

+ // (e) Faults stop.

+ w.recoverAll()

+ trace = append(trace, "faults stop")

+ ready := func() bool {

+ c := w.current()

+ if c.Op.Kind != hostclaim.KindRelocating || !slices.Contains(c.Op.Participants, "src") || c.Op.Journal["exec_ack.src"] == "" {

+ return c.Op.Kind != hostclaim.KindRelocating

+ }

+ obs, err := hostclaim.Read(w.ctx, w.inner, w.host)

+ return err == nil && w.e.Ready(w.ctx, obs) == nil

+ }

+ ok := false

+ for i := 0; i < 8 && !ok; i++ {

+ w.renew()

+ w.pass()

+ w.check("recovery pass")

+ ok = ready()

+ }

+ if !ok || len(w.violated) > 0 {

+ t.Fatalf("seed %d: the running relocation did not recover within 8 passes: %v\ntrace: %v", seed, w.violated, trace)

+ }

+ w.complete()

+ clean := false

+ for i := 0; i < 8 && !clean; i++ {

+ w.renew()

+ w.pass()

+ w.check("sweep pass")

+ _, exists, _ := w.record()

+ retained, err := w.e.Controllers.Retained()

+ clean = !exists && len(retained) == 0 && err == nil && len(w.open()) == 0

+ }

+ if !clean || len(w.violated) > 0 {

+ st, exists, _ := w.record()

+ retained, _ := w.e.Controllers.Retained()

+ t.Fatalf("seed %d: not clean within 8 passes after completion: record %v %+v retained %+v open %d %v\ntrace: %v", seed, exists, st, retained, len(w.open()), w.violated, trace)

+ }

+ w.startRelocation()

+ for i := 0; i < 3; i++ {

+ w.pass()

+ }

+ w.check("new relocation")

+ if !ready() || len(w.open()) != 1 || len(w.violated) > 0 {

+ t.Fatalf("seed %d: a new relocation did not open: open %d %v\ntrace: %v", seed, len(w.open()), w.violated, trace)

+ }

+ for _, c := range w.open() {

+ _ = c.Close(context.Background())

+ }

+ })

+ }

+```

`internal/host/relocationeffects.go`:

```go

@@ -8792,6 +9397,10 @@ type RelocationController interface {

Ready(context.Context) (execctl.State, error)

Close(context.Context) error

Clear(context.Context) error

+ // Lost is closed once the controller has lost its lease for good

+ // (hostexec latches the first failure: a failed renewal, a lost

+ // authority, a clock fault).

+ Lost() <-chan struct{}

}

// participation is one journaled participation whose controller may have

@@ -8835,6 +9444,12 @@ type RelocationControllers struct {

// Clear removes a journaled binding's durable record after a restart

// (hostexec.ClearStale with the assembly's config for the binding).

Clear func(context.Context, execctl.Binding) error

+ // DataDir is hostexec's data directory (Guard.DataDir), whose

+ // .hostexec/<host>.json is the durable record. When a clear fails,

+ // the participation is forgotten only if that record is verifiably

+ // not this binding's: absent, or naming another binding (rev 14).

+ // Empty: never verifiable, so a failed clear always keeps it.

+ DataDir string

// bindings remembers each open controller's binding.

bindings map[string]execctl.Binding

}

@@ -9009,17 +9624,33 @@ func (r *RelocationControllers) finish(ctx context.Context, host, op string, suc

if ok {

clear = func() error { return c.Clear(ctx) }

}

+ if !ok {

+ // No controller object: when the record is verifiably not this

+ // binding's (gone, or a successor's that holds the host's lock),

+ // there is nothing to clear, and ClearStale would only wait out

+ // the pass on the successor's lock.

+ if mine, verr := r.recordIsMine(host, b); verr == nil && !mine {

+ return r.forget(host, op)

+ }

superseded := false

- switch err := clear(); {

- case err == nil:

- case errors.Is(err, execctl.ErrState):

- // The host's record names another binding (a successor opened

- // over it, or another operation's): this participation has no

- // record left to clear.

- case successor && errors.Is(err, execctl.ErrInactive):

- superseded = true // the lock is free; the journal stays until the successor opens

- default:

- return err

+ if err := clear(); err != nil {

+ // Whatever the error, the participation goes only if the record

+ // itself shows it is not this binding's (round 13 finding 2:

+ // ClearStale's ErrState also means an absent claim or an invalid

+ // local record, and those leave this binding's record in place).

+ mine, verr := r.recordIsMine(host, b)

+ switch {

+ case verr != nil:

+ return errors.Join(err, verr)

+ case !mine:

+ // Verified: no record, or another binding's (a successor

+ // opened over it).

+ case successor && errors.Is(err, execctl.ErrInactive):

+ superseded = true // the lock is free; the journal stays until the successor opens

+ default:

+ return err

+ }

}

r.mu.Lock()

if ok && r.open[key] == c {

@@ -9033,6 +9664,68 @@ func (r *RelocationControllers) finish(ctx context.Context, host, op string, suc

return r.forget(host, op)

}

+// recordIsMine reads hostexec's durable record for host and reports

+// whether it is b's binding (box, enrollment, host, incarnation,

+// operation: what ClearStale matches). An absent record is not b's. A

+// record that cannot be read or decoded is an error: never a reason to

+// forget.

+func (r *RelocationControllers) recordIsMine(host string, b execctl.Binding) (bool, error) {

+ if r.DataDir == "" {

+ return false, fmt.Errorf("%w: no hostexec data directory to verify the record", ErrClaimStopUnavailable)

+ }

+ raw, err := os.ReadFile(filepath.Join(r.DataDir, ".hostexec", host+".json"))

+ if errors.Is(err, os.ErrNotExist) {

+ return false, nil

+ }

+ if err != nil {

+ return false, err

+ }

+ var st execctl.State

+ if err := json.Unmarshal(raw, &st); err != nil || st.V != 1 || st.Host != host || st.Box == "" || st.Op == "" {

+ return false, fmt.Errorf("%w: hostexec record for %s is not readable", execctl.ErrState, host)

+ }

+ return st.Box == b.Box && st.Enrollment == b.Enrollment && st.Incarnation == b.Incarnation && st.Op == b.Op, nil

+// live returns the open controller for {host, op}, retiring it first if it

+// has lost its lease (round 13 finding 1): hostexec latches the first

+// failure, so a lost controller never becomes ready again. Retiring joins

+// it (Close, bounded by ctx), repeats the scoped cleanup, and forgets the

+// object but keeps the journal: the record on disk stays "lost" (durable

+// denial) until a fresh controller, opened with fresh authority and its

+// own lease, replaces it. ok false: none open (or just retired).

+func (r *RelocationControllers) live(ctx context.Context, host, op string) (RelocationController, bool, error) {

+ key := controllerKey(host, op)

+ r.mu.Lock()

+ c, ok := r.open[key]

+ b := r.bindings[key]

+ r.mu.Unlock()

+ if !ok {

+ return nil, false, nil

+ }

+ select {

+ case <-c.Lost():

+ default:

+ return c, true, nil

+ }

+ if r.Cleanup == nil {

+ return nil, false, ErrClaimStopUnavailable

+ }

+ if err := c.Close(ctx); err != nil {

+ return nil, false, err // joined later; the entry stays

+ }

+ if err := r.Cleanup(ctx, b); err != nil {

+ return nil, false, err // the entry stays: the next pass repeats both

+ }

+ r.mu.Lock()

+ if r.open[key] == c {

+ delete(r.open, key)

+ delete(r.bindings, key)

+ }

+ r.mu.Unlock()

+ return nil, false, nil

// forget removes one participation's journal entry.

func (r *RelocationControllers) forget(host, op string) error {

if r.Journal != "" {

@@ -9047,27 +9740,23 @@ func (r *RelocationControllers) forget(host, op string) error {

// before op's controller opens (round 12 finding 1): hostexec keeps one

// controller per host, and a predecessor whose sweep has not yet run still

// holds the host's lifetime lock, so the successor's Open would wait on it

-// for as long as the agent lives. Each predecessor is closed and its

-// scoped cleanup repeated under the pass's context (bounded); one whose

-// record hostexec will not clear because the successor lists this box is

-// returned as superseded, its journal entry removed only once the

-// successor's controller has opened over its record.

-func (r *RelocationControllers) handover(ctx context.Context, host, op string) ([]string, error) {

+// for the whole pass. Each predecessor is closed and its scoped cleanup

+// repeated under the pass's context (bounded). One whose record hostexec

+// will not clear because the successor lists this box (errSuperseded)

+// keeps its journal entry: the successor's Open replaces the record under

+// the freed lock, and a later sweep forgets the entry once the record is

+// verifiably the successor's (rev 14).

+func (r *RelocationControllers) handover(ctx context.Context, host, op string) error {

retained, _ := r.Retained() // a damaged entry is Sweep's to report

- var superseded []string

for _, p := range retained {

if p.Host != host || p.Op == op {

continue

}

- switch err := r.finish(ctx, host, p.Op, true); {

- case err == nil:

- case errors.Is(err, errSuperseded):

- superseded = append(superseded, p.Op)

- default:

- return nil, fmt.Errorf("relocation: retiring %s's controller for %s before %s: %w", p.Op, host, op, err)

+ if err := r.finish(ctx, host, p.Op, true); err != nil && !errors.Is(err, errSuperseded) {

+ return fmt.Errorf("relocation: retiring %s's controller for %s before %s: %w", p.Op, host, op, err)

}

- return superseded, nil

+ return nil

}

// writeFileSynced writes path whole: a synced temporary file renamed into

@@ -9190,7 +9879,10 @@ func (e *RelocationEffects) Acknowledge(ctx context.Context, obs hostclaim.Obser

return hostclaim.ExecutionAcknowledgement{}, ErrClaimStopUnavailable

}

c := obs.Claim

- ctl, ok := e.Controllers.get(c.Host, c.Op.ID)

+ ctl, ok, err := e.Controllers.live(ctx, c.Host, c.Op.ID)

+ if err != nil {

+ return hostclaim.ExecutionAcknowledgement{}, err

+ }

if !ok {

id, err := e.Identity(ctx)

if err != nil {

@@ -9205,8 +9897,7 @@ func (e *RelocationEffects) Acknowledge(ctx context.Context, obs hostclaim.Obser

}

// A predecessor this box has not swept yet still holds the host's

// lifetime lock: retire it first, under this pass's context.

- superseded, err := e.Controllers.handover(ctx, c.Host, c.Op.ID)

- if err != nil {

+ if err := e.Controllers.handover(ctx, c.Host, c.Op.ID); err != nil {

return hostclaim.ExecutionAcknowledgement{}, err

}

opened, err := openBounded(ctx, e.Lifetime, func(octx context.Context) (RelocationController, error) { return e.Controller(octx, b) })

@@ -9216,12 +9907,6 @@ func (e *RelocationEffects) Acknowledge(ctx context.Context, obs hostclaim.Obser

if ctl, ok = e.Controllers.put(c.Host, c.Op.ID, b, opened); !ok {

_ = opened.Close(ctx) // another pass opened one first

}

- for _, old := range superseded {

- // The successor's Open replaced the predecessor's record.

- if err := e.Controllers.forget(c.Host, old); err != nil {

- return hostclaim.ExecutionAcknowledgement{}, err

- }

}

st, err := ctl.Ready(ctx)

if err != nil {

@@ -9237,38 +9922,34 @@ func (e *RelocationEffects) Acknowledge(ctx context.Context, obs hostclaim.Obser

// opened in the same instant is closed, never kept with a cancelled life.

func openBounded(ctx, lifetime context.Context, open func(context.Context) (RelocationController, error)) (RelocationController, error) {

octx, cancel := context.WithCancel(lifetime)

- var mu sync.Mutex

- opened, cancelled := false, false

- done := make(chan struct{})

+ type result struct {

+ c RelocationController

+ err error

+ }

+ ch := make(chan result, 1)

+ // hostexec.Open waits on its Cleanup hook without a context, so the

+ // open runs apart and the pass waits for it no longer than ctx.

go func() {

- select {

- case <-ctx.Done():

- mu.Lock()

- if !opened {

- cancelled = true

- cancel()

- }

- mu.Unlock()

- case <-done:

- }

+ c, err := open(octx)

+ ch <- result{c, err}

}()

- c, err := open(octx)

- mu.Lock()

- opened = true

- wasCancelled := cancelled

- mu.Unlock()

- close(done)

- if wasCancelled {

- if c != nil {

- _ = c.Close(context.Background())

- }

- return nil, errors.Join(ctx.Err(), err)

- }

- if err != nil {

+ select {

+ case res := <-ch:

+ if res.err != nil {

+ cancel()

+ return nil, res.err

+ }

+ context.AfterFunc(lifetime, cancel) // octx ends only with lifetime

+ return res.c, nil

+ case <-ctx.Done():

cancel()

- return nil, err

+ go func() {

+ if res := <-ch; res.c != nil {

+ _ = res.c.Close(context.Background()) // opened too late: never kept

+ }

+ }()

+ return nil, ctx.Err()

}

- return c, nil // octx is cancelled only with lifetime

}

// Ready checks the retained controller for the operation is active now.

@@ -9276,11 +9957,14 @@ func (e *RelocationEffects) Ready(ctx context.Context, obs hostclaim.Observed) e

if e.Controllers == nil || obs.Claim == nil {

return ErrClaimStopUnavailable

}

- ctl, ok := e.Controllers.get(obs.Claim.Host, obs.Claim.Op.ID)

+ ctl, ok, err := e.Controllers.live(ctx, obs.Claim.Host, obs.Claim.Op.ID)

+ if err != nil {

+ return err

+ }

if !ok {

- return hostrelocate.ErrNoController

+ return hostrelocate.ErrNoController // the participant reopens one (Acknowledge)

}

- _, err := ctl.Ready(ctx)

+ _, err = ctl.Ready(ctx)

return err

}

@@ -9465,6 +10149,8 @@ type execController struct {

func (c *execController) Clear(ctx context.Context) error { return execctl.ClearStale(ctx, c.cfg) }

+func (c *execController) Lost() <-chan struct{} { return c.Controller.Lost() }

// OpenRelocationController opens hostexec's controller for cfg and wraps

// it with its durable clear.

func OpenRelocationController(ctx context.Context, cfg execctl.Config) (RelocationController, error) {

@@ -9478,13 +10164,13 @@ func OpenRelocationController(ctx context.Context, cfg execctl.Config) (Relocati

- [ ] **Step 5: Run the tests**

+Run: `nice -n 19 go test ./internal/host/ -run 'StopReceiptReleases|Relocation|ReleasePositions|ReleaseFlushFiles|ClaimStop|Finish|ARestartFinds|ASuccessor' -count=1 && nice -n 19 go test ./internal/hostfiles/ -count=1` (the lifecycle property, `TestRelocationControllerLifecycleProperties`, runs its 200 seeds in about three minutes niced; `-short` runs 20)

Expected: PASS.

- [ ] **Step 6: Commit**

```bash

-git add internal/host/relocationreleases.go internal/host/relocationreleases_test.go internal/host/relocationeffects.go internal/host/relocationeffects_test.go internal/host/relocationcontrollers_linux_test.go internal/host/relocationflush.go internal/hostfiles/platform.go

+git add internal/host/relocationreleases.go internal/host/relocationreleases_test.go internal/host/relocationeffects.go internal/host/relocationeffects_test.go internal/host/relocationcontrollers_linux_test.go internal/host/relocationlifecycle_linux_test.go internal/host/relocationflush.go internal/hostfiles/platform.go

git commit -m "host: a release is proved by its full stop receipt; it stops the writers, flushes files, wake state and env, records its positions and only then drains

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"

@@ -9995,7 +10681,7 @@ func (a ClaimAssembly) RelocationEffects(cfg RelocationAssemblyConfig) *Relocati

// and Finish clears only straight after Cleanup succeeds. The

// re-plan authenticates the binding against the verified actor

// before relocationCleanup runs (stage B re-plan table).

- Controllers: &RelocationControllers{Journal: filepath.Join(cfg.DataDir, "relocate", "participations"),

+ Controllers: &RelocationControllers{Journal: filepath.Join(cfg.DataDir, "relocate", "participations"), DataDir: cfg.Exec.Guard.DataDir,

Cleanup: func(ctx context.Context, b execctl.Binding) error { return a.relocationCleanup(ctx, cfg, b.Host, b.Op) },

Clear: func(ctx context.Context, b execctl.Binding) error { c := cfg.Exec; c.Binding = b; return execctl.ClearStale(ctx, c) }},

Identity: func(ctx context.Context) (RelocationIdentity, error) {

@@ -11680,3 +12366,4 @@ Expected: exit 0. Exit 2 is pending: wait and read again. Exit 1: read the log a

- **Rev 11 guards**, each removed in turn and its test watched to fail: the console chooser's live-edges reader (an empty snapshot fails on the box identities); its claim-loads reader (an empty map fails on the counts); `ProductionAuthority.SelectExecutor` consulting its chooser (a fixed box-a fails); `regionHAEnabled` honouring the parsed value (any parseable value as on fails on `0`). Four in all.

- **Rev 12 guards**, each removed in turn and its test watched to fail: the pin's referenced origins (the copy refuses g1's object with `ErrOrigin`); the refusal of a reference outside the incarnation's files (a foreign reference is pinned); `preparedOpened` reading `prepared_opened` (P is forgotten with its admission); the opening admission journaling it. Four in all.

- **Rev 13 guards**, each removed in turn and its test watched to fail: `Acknowledge`'s handover (B's acknowledgement waits out the pass); `openBounded` (the open under the agent's lifetime never returns: the test's own timeout); the superseded case for hostexec's `ErrInactive` (the handover fails); removing the superseded journal entry once B opens (A is still retained). Four in all.

+- **Rev 14 guards**, removed one at a time under `TestRelocationControllerLifecycleProperties` with 200 seeds (the table in the Rev 14 dispositions): ten killed, each with its seed and invariant. One, the extra scoped cleanup before a recovery reopen, survives as defence in depth, because `hostexec.Open` runs the same cleanup and refuses on its failure. Rev 13's guard "removing the superseded journal entry once B opens" is gone with that code (C3).