carlos / platform Public

branch:

Clone

git clone https://amadan.net/carlos/platform

Plain git — no account needed to clone.

Download

Download this file

Zero-downtime deploys: flag-readiness Implementation Plan

> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Close the code items amadan #272 owes before CARLOS_SWAP=overlap may be set on any box, and turn the claims-on gaps into one documented limitation.

Architecture: Every fix lands in the dormant zero-downtime code on main (anchored on origin/main 5da1cc9b). The failure book (internal/rollout/failures.go) gains a persisted sequence, write-ahead intents and schema 2. The planner and scheduler honour hold-ran markers everywhere a good build could run. Declined boxes get an exec-time gate and process signed retries. Host retries become console-signed root envelopes. Claims-on boxes decline to arm.

Tech Stack: Go 1.x, stdlib only. Tests: go test, the rollout simulation (internal/rollout/sim_test.go) and its mutation script.

Spec: /home/paulca/amadan.net/carlos/platform-design-zero-downtime-deploys/docs/superpowers/specs/2026-10-07-zero-downtime-deploys-design.md (rev 17, binding). Inputs: docs/superpowers/plans/CODEX-FINDINGS-ZDD-IMPL-5*.md, CODEX-FINDINGS-ZDD-IMPL-6*.md, amadan discussion #272.

Global Constraints

Flag off is unchanged. With CARLOS_SWAP unset, behaviour stays as on origin/main, with exactly four authorized exceptions: Task 13's bucket retention also pins GoodVersion (it can only keep more, never delete more); Task 15's F6 change (flag-off by nature); Task 15's /run sweep (flag-independent cleanup); Task 19's recreation of a cgroup that took cgroup.kill (dev-box kernel defect). Every new hook defaults to nil and is set only on an armed or declined box. A changed shared helper (for example configAckFunc) keeps its flag-off behaviour; new behaviour goes on the armed path only.
Wire shapes are additive (AGENTS.md). Restart-scope entries gain an optional signed field. The status rollout report gains per-host held_epoch (Task 8) and transient (Task 17), both omitempty; the console's instance-list rollout view passes them through.
The failure book file (<dataDir>/swap-failures.json) is written as "schema": 2 from Task 5 on. Readers accept 1 and 2. A schema-1 file must read unchanged.
Lock order (internal/rollout/manager.go:98): goodMu, then activator.Service.mu, then Manager.mu, then scheduler.mu, then FailureBook.mu. No disk write under activator.Service.mu or scheduler.mu.
Copy. No new member- or developer-facing text without a // TODO-COPY(<id>) comment on its line and a row in copy-review/strings.json (Task 17). Reuse approved text first: internal/rolloutwords/words.go, rollout.HeldDownRefusal, host.refuseNoSwaps, host.refuseTexts, the approved record docs/superpowers/plans/2026-10-07-zero-downtime-deploys-6-copy.json. No em dashes in any new string. Agent log lines are operator text and need no review.
Commands. Sandboxed packages (internal/rollout, internal/release, internal/provision, internal/rolloutwords, internal/status): export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; nice -n 19 go test ./internal/rollout/ -run 'Name' -count=1. Every other package (activator, host, console, cgroup, instancectl, root package) runs unsandboxed (dangerouslyDisableSandbox: true) with a short temp root: mkdir -p /tmp/k && cd /home/paulca/amadan.net/carlos/platform/.claude/worktrees/zdd-flag-readiness && TMPDIR=/tmp/k GOFLAGS='-mod=mod -buildvcs=false' GOPROXY=off nice -n 19 go test ./internal/host/ -run 'Name' -count=1. Run tests serially. Never go test ./... inside a task. gofmt -l on touched files must be empty.
Commits: plain git commit -m in the worktree (identity is configured there). End messages with Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>.
Simulation gate. Tasks 3, 4, 5 and 7 touch scheduler or gate logic: run nice -n 19 internal/rollout/testdata/rolloutsim/mutate.sh 300 (sandboxed exports as above) after committing, and paste its verdict lines. Task 16 adds mutants and runs the one 3000-seed sweep for the revision.
Integrity: reports paste real command output only.

Review Focus

A failed write of the failure book followed by an agent restart (disk full, then a crash). Expected: no build that failed, and no hold build that ran, is eligible again after the restart. Task 5 pins it with TestAFailedSaveThenRestartKeepsTheFailure.
An older agent binary started over a schema-2 book. Expected: it refuses managed starts loudly rather than erase markers. Task 5 pins the reader side with TestASchema2BookIsUnreadableToASchema1Reader (an old-reader stub).
A box with CARLOS_HOST_CLAIM set and CARLOS_SWAP=overlap. Expected: it declines with host-claims-on, runs today's paths, and the deploy watch URL-watches it like any declined box. Task 1 pins it.
A retry entry replayed from an old log, forged unsigned by a box, or naming another build. Expected: ignored with a log line. Task 8 pins all three.
A late outcome of an older wake or attempt arriving after a newer retry, marker or good write. Expected: it never undoes the newer state. Task 4 pins the two sequences Astra gave (N6, retry ordering).

Rulings recorded by this plan

Ruling (Paul's, 2026-10-11): claims-on boxes do not get swaps. Arming declines on any box with host claims enabled (CARLOS_HOST_CLAIM non-empty), reason code host-claims-on. The four claims-on gaps from CODEX-FINDINGS-ZDD-IMPL-5-MERGE.md stay open as the future feature "claims-on swaps" (#272).
Ruling: "refuse to arm" means decline (today's paths, failure records honoured), not refuseManaged (every managed wake refused). Why: refusing would make the flag an outage switch on claims boxes; declining runs nothing new. Cost if wrong: a claims box with the flag set serves on today's stop-then-start paths instead of refusing, which is the flag-off behaviour.
Ruling: host retries are signed in the console (release.SignRootEnvelope with the console's root key), verified on the box (release.VerifyRootEnvelope), and an unsigned or invalid retry is ignored on every box. The grant names account, app, host, build and the restriction epoch the box published for that host (Task 8): any change to the host's restriction since the console read it (a new failure, a re-mark) voids the grant, and the epoch is persisted, so a replay after a restart clears nothing new. A 24-hour issue window is a backstop. Cost if wrong: a member whose retry raced a second failure has to retry again.
Ruling: an unresolved write-ahead intent found at startup is a failure of its build on that instance (held down in hold mode). Cost if wrong: an agent crash in the middle of a swap halts that build's cohort on the box until carlos promote --retry or a signed host retry.
Ruling: the failure book is always written as schema 2. Cost if wrong: downgrading an agent with the flag on leaves that box refusing managed starts until the file is upgraded or removed.
Ruling (replaces the plan-time pointer read, plan review #7): explicit rollback evidence clears the marker. The pointer pass (armed applyPointers, declined Task 10) clears a host's hold-ran marker when the route's channel pointer, signature-verified, names the route's installed good build with PromotedAt after the marker's At. carlos rollback re-signs the pointer, so it qualifies; root's automatic sibling repoint does not. While a marker stands, the good build never boots in place of another build, at plan or at exec, whatever is installed. Cost if wrong: a wake in the seconds between carlos rollback and the next pointer pass is refused once with the approved "roll back or retry" text.
Ruling: deferred (not code defects on the flag-on path): round-3 §5 coverage-only items (config delivery order/removal coverage, shutdown join tests, blocked-publication tests, admitting CoalesceRecycle coverage, concurrency-one/two-cohort coverage), reproducible simulation replay, and every real-box check (D1, kernel suite, real-unit checks, flagship probe). The status waking/stopping race is NOT deferred: Task 17.

---

Task 1: Claims-on boxes decline to arm

Corrections (plan review round 1; these override the text below):

overlapEnv takes a string (internal/host/rollout_test.go:58): call overlapEnv("").

Files:

Modify: internal/host/rollout.go:25-37 (code), :142-153 (swapArming)
Modify: internal/rolloutwords/words.go:83-121 (unavailablePhrases, UnavailableDeclines)
Modify: internal/status/snapshot.go:261-262 (comment lists the decline codes)
Test: internal/host/rollout_test.go, internal/rolloutwords/words_test.go

Interfaces:

Produces: const rolloutHostClaims = "host-claims-on" in package host. rolloutwords.Unavailable("host-claims-on") returns the approved unavailableDeclined, and UnavailableDeclines("host-claims-on") is true, so the deploy watch URL-watches a claims box exactly as a declined one.

[ ] Step 1: Write the failing tests

// internal/host/rollout_test.go
func TestSwapArmingDeclinesWhenHostClaimsAreOn(t *testing.T) {
	resetRollout(t)
	var buf bytes.Buffer
	log.SetOutput(&buf)
	t.Cleanup(func() { log.SetOutput(os.Stderr) })
	// No cgroups either: the claims check must come first, so the code
	// says claims, not delegation.
	eb := &activator.ExecBackend{Cfg: activator.Config{HostWrites: hostwrite.Config{Enabled: true}}}
	if swapArming(overlapEnv(nil), eb) {
		t.Fatal("armed a box with host claims on")
	}
	if got := rolloutUnarmed(); got != rolloutHostClaims {
		t.Fatalf("decline code %q, want %q", got, rolloutHostClaims)
	}
	if !strings.Contains(buf.String(), "REFUSING CARLOS_SWAP=overlap (host-claims-on)") {
		t.Fatalf("no operator line naming the claims refusal: %q", buf.String())
	}
	r := rolloutStatusProbe()
	if r == nil || r.Unavailable != rolloutHostClaims {
		t.Fatalf("status %+v: want Unavailable %q", r, rolloutHostClaims)
	}
}

func TestSwapArmingIgnoresClaimsWithTheFlagOff(t *testing.T) {
	resetRollout(t)
	eb := &activator.ExecBackend{Cfg: activator.Config{HostWrites: hostwrite.Config{Enabled: true}}}
	if swapArming(noEnv, eb) || rolloutUnarmed() != "" {
		t.Fatal("flag off: a claims box must say nothing")
	}
}

// internal/rolloutwords/words_test.go
func TestHostClaimsDeclineReadsAsADecline(t *testing.T) {
	if Unavailable("host-claims-on") != unavailableDeclined || !UnavailableDeclines("host-claims-on") || UnavailableRefuses("host-claims-on") {
		t.Fatal("host-claims-on must read and act as a decline")
	}
}

overlapEnv takes the public key the existing tests pass; pass whatever its signature needs (nil or a test key). Use the imports the file already has plus hostwrite.

[ ] Step 2: Run to verify they fail

Run (unsandboxed): TMPDIR=/tmp/k GOFLAGS='-mod=mod -buildvcs=false' GOPROXY=off nice -n 19 go test ./internal/host/ -run 'TestSwapArming' -count=1 Run (sandboxed): nice -n 19 go test ./internal/rolloutwords/ -run TestHostClaims -count=1 Expected: FAIL (rolloutHostClaims undefined; the words test fails).

[ ] Step 3: Implement

// internal/host/rollout.go, in the const block beside rolloutNoDelegation
	// The flag is on but host claims are on: claims-backed launches cannot
	// yet carry a swap's recovery choice, its execution evidence, the
	// generations file or the good build (#272, "claims-on swaps"), so the
	// box declines and stays on today's paths.
	rolloutHostClaims = "host-claims-on"

// internal/host/rollout.go, swapArming, first check after rollout.Enabled
	if eb != nil && eb.Cfg.HostWrites.Enabled {
		log.Printf("agent: REFUSING %s=overlap (%s): host claims are on, and claims-backed launches cannot yet keep a swap's "+
			"recovery choice, execution evidence, generations file or good build, so this box stays on today's stop-then-wake paths",
			rollout.SwapEnv, rolloutHostClaims)
		declineRollout(rolloutHostClaims)
		return false
	}

// internal/rolloutwords/words.go
	"host-claims-on":          unavailableDeclined,
...
	case "no-cgroup-delegation", "no-generation-control", "generations-off", "host-claims-on":

Update the comment at internal/status/snapshot.go:261-262 to list "host-claims-on".

[ ] Step 4: Run to verify they pass, plus the packages' existing rollout tests: -run 'Rollout|Swap|Declin' in host and the whole rolloutwords package.

[ ] Step 5: Commit zdd-readiness: claims-on boxes decline CARLOS_SWAP=overlap (host-claims-on)

---

Task 2: The deploy watch never drops a fleet or polls an unread host

Corrections (plan review round 1; these override the text below):

Seed the provision test the way TestListRecords does (CreateRecord, internal/provision/provision_test.go:413); DirStore.Put returns two values (internal/release/store.go:402). There is no putTestRecord.
An absent host must also stop being polled if it was already in modeURL: the poll loop (deploywatch.go:317) checks only mode and urlLive. Give watchedHost an absent bool, set it in the new case !ok: arm (cleared whenever the host is listed again), and make the poll loop skip absent hosts. Add TestWatchRolloutStopsPollingAURLHostThatDropsOffTheList: a host first listed with no report on a complete join (so it is URL-watched), then missing from every later list; after the transition, zero further origin requests.

Files:

Modify: internal/provision/provision.go:2610-2657 (ListRecords), and the census pin internal/provision/plan9_b8_census_test.go:849 if TestPlan9B8RecordWriteCensus fails on the moved body
Modify: internal/console/apiinstances.go:630-636 (handleAPIInstanceList)
Modify: deploywatch.go:474-475 (rolloutWatch.apply), and pendingDetail (~593) only if the absent-host line needs it
Test: internal/provision/provision_test.go, internal/console/rolloutfleet_test.go, deploywatch_test.go

Interfaces:

Produces: provision.ListRecordsComplete(ctx, s release.Store, acct, app string, max int) (recs []Record, truncated bool, skipped []string, err error). ListRecords becomes a wrapper that drops skipped; no other caller changes.

[ ] Step 1: Write the failing tests

// internal/provision/provision_test.go
func TestListRecordsCompleteNamesTheSkippedHost(t *testing.T) {
	ctx := context.Background()
	s := release.NewDirStore(t.TempDir()) // the store TestListRecords uses; match it
	putTestRecord(t, s, "acct", "app", "a.example.com") // use TestListRecords' own seeding helper
	if err := s.Put(ctx, RecordKey("acct", "app", "b.example.com"), []byte("not json"), ""); err != nil {
		t.Fatal(err)
	}
	recs, truncated, skipped, err := ListRecordsComplete(ctx, s, "acct", "app", 10)
	if err != nil || truncated || len(recs) != 1 || len(skipped) != 1 || skipped[0] != "b.example.com" {
		t.Fatalf("recs %d truncated %v skipped %v err %v", len(recs), truncated, skipped, err)
	}
}

// internal/console/rolloutfleet_test.go (helpers: placedRecord, writeFleetHeldBox, listRollout)
// Astra plan 6 R3: an unreadable record must never drop its fleet from the
// completeness check. The record's host sits on fleet/pi, whose only
// snapshot is unreadable too.
func TestAnUnreadableRecordKeepsEveryFleetInTheCompletenessCheck(t *testing.T) {
	// 1. placedRecord a readable host on the platform and a second host on fleet/pi.
	// 2. register fleet pi (writeFleetHeldBox or UpsertFleet, as the neighbouring tests do).
	// 3. overwrite the pi host's record with "{not json" at provision.RecordKey(acct, app, host).
	// 4. put "{not json" at fleets/pi/status/boxes/broken.json.
	// 5. GET the instance list: want "rollout_incomplete": true.
}

Write the test body with the file's own helpers, following TestFleetBoxHoldShowsOnTheListPageAndRetryDoor; the five numbered steps are the arrangement and the assertion.

// deploywatch_test.go (helpers: scripted, sweep, noURL, testWatch)
// A target missing from a complete, untruncated list is not evidence that it
// uses the old path: it is never polled, and the timeout names it.
func TestWatchRolloutNeverPollsATargetMissingFromACompleteList(t *testing.T) {
	// targets: a (on a reporting box) and b; every sweep lists only a, with a
	// followed report still settling, incomplete=false, truncated=false.
	// noURL(t) fails the test on any origin request.
	// Want: a timeout error naming b, and zero origin requests.
}

Model the deploywatch test on TestWatchRolloutIncompleteJoinIsNotNoReports (deploywatch_test.go:708).

[ ] Step 2: Run to verify they fail (provision sandboxed; console and root package unsandboxed with TMPDIR=/tmp/k).

[ ] Step 3: Implement

// internal/provision/provision.go
func ListRecords(ctx context.Context, s release.Store, acct, app string, max int) (recs []Record, truncated bool, err error) {
	recs, truncated, _, err = ListRecordsComplete(ctx, s, acct, app, max)
	return recs, truncated, err
}

// ListRecordsComplete is ListRecords that also names each host whose record
// could not be read (a store error, a malformed body, a body that disagrees
// with its path). A caller that must know whether the list is the whole app
// (the deploy watch's completeness, Astra plan 6 R3) reads skipped.
func ListRecordsComplete(ctx context.Context, s release.Store, acct, app string, max int) (recs []Record, truncated bool, skipped []string, err error) {
	// ListRecords' body moved here unchanged, with the skip recorded:
	//	rec, err := GetRecord(ctx, s, acct, app, host)
	//	if err != nil {
	//		skipped = append(skipped, host)
	//		continue
	//	}
}

// internal/console/apiinstances.go, handleAPIInstanceList
	recs, truncated, skipped, err := provision.ListRecordsComplete(ctx, s.cfg.Releases, acct.Sqid, app, instanceListMax)
	...
	relevant := recordFleets(recs)
	if len(skipped) > 0 {
		// A skipped record's placement is unknown, so every fleet the account
		// has, and the platform, counts toward completeness (nil means all).
		log.Printf("console: api instances list %s/%s: %d record(s) unreadable %v; every fleet counts toward the rollout join",
			acct.Sqid, app, len(skipped), skipped)
		relevant = nil
	}
	join, joinComplete := s.loadRolloutJoin(ctx, acct.Sqid, time.Now(), relevant)

// deploywatch.go, rolloutWatch.apply: replace `case !ok && l.truncated:`
		case !ok:
			// Absent from this list: cut short, or a record the console could
			// not read. Neither says the host uses the old path, so it keeps
			// its mode and state and is never moved to URL watching here
			// (Astra plan 6 R3).
			h.unread = !l.truncated

Check pendingDetail: an absent host on an untruncated list now has unread true. If watchPendingUnread's approved text does not fit "missing from the list", reuse watchPendingUnlisted by leaving unread false instead, and say which in the report; do not add new text.

Update apply's doc comment bullets (deploywatch.go:430-446) to match.

[ ] Step 4: Run to verify they pass, plus -run 'TestListRecords|TestPlan9B8' (provision), -run 'Rollout|Fleet|InstanceList' (console), -run 'TestWatch|TestDeployWatch' (root).

[ ] Step 5: Commit zdd-readiness: an unreadable record keeps every fleet in the watch's completeness check, and is never URL-polled

---

Task 3: Hold-ran markers honour records, workers and rollback provenance (N5, N7, ruling 205)

Corrections (plan review round 1; these override the text below):

Rollback provenance moves out of the planner (plan review #7). Drop wakeInputs.RolledBack and the plan-time pointer read. Instead: - decideHoldRan's installed == rt.GoodVersion arm refuses with HeldDownRefusal(mark.Build) whenever a marker stands. - planCheck's good-build arm refuses the good build whenever a marker stands, whatever rt.Version is: delete && rt.Version != rt.GoodVersion (wakeplan.go:317). No I/O in the check. - applyPointers (manager.go:418) also runs for an app with a marker on any of its hosts (not only one with records). For each such host whose route installs its good build (rt.Version == rt.GoodVersion), a pointer for the route's channel naming rt.GoodVersion with PromotedAt after mark.At clears the marker (ClearHoldRan) and logs it. An installed-build change seen by observe on a marked host makes that app's pointer read due at once (reset m.pointerRead[k]). - Test: TestASignedRollbackClearsTheMarkerAndTheGoodBuildBoots replaces TestAnInstalledGoodBuildBootsOverAMarkerOnlyAfterASignedRollback: before the pointer, the plan is refused and p.Check("v1") (from any earlier plan) refuses; after h.pointer("stable", "v1", later) and h.pass(), the marker is gone and the plan boots v1. An older pointer clears nothing. The harness pointer is an unsigned fixture (fake_test.go:352); signature verification is release.Channel's, already tested in internal/release.
The book's clock: give FailureBook a now func() time.Time (nil means time.Now), and set h.book.now = h.clock in newHarness so At follows the harness clock.

Files:

Modify: internal/rollout/failures.go:62-66 (HoldRan gains At), :197-221 (markLocked/MarkHoldRan stamp At)
Modify: internal/rollout/wakeplan.go:31-40 (wakeInputs), :127-152 (decideHoldRan), :254-283 (planRow), :300-329 (planCheck)
Modify: internal/rollout/worker.go:21-30 (executeWorker)
Modify: internal/rollout/holdran_seq_test.go:58-71, internal/rollout/sim_test.go:~826-840 (remove the N7 exemption)
Test: internal/rollout/holdran_test.go, internal/rollout/worker_test.go

Interfaces:

Produces: HoldRan{Account, App, Build string; At time.Time} (At is json:"at,omitempty", set by the book from its clock, time.Now unless a test injects one). wakeInputs.RolledBack func(good string, since time.Time) bool.

Rules (spec §6.4, §8.3, §8.4; Astra R3 N5, N7, ruling 205):

A marked build is planned only when nothing holds it for the instance: held down refuses (as today), and a record that Blocks it while the instance has a good build refuses too, with HeldDownRefusal(mark.Build) (approved text; no new copy). Without a good build the §8.5 rule stands: a record alone refuses nothing.
The installed good build boots over a marker only on explicit rollback evidence: the route's channel pointer, signature-verified (Deps.Pointer), names rt.GoodVersion with PromotedAt after mark.At. carlos rollback re-signs the pointer (internal/release/release.go:1252, signPointer stamps a fresh PromotedAt), so a member's rollback passes. Root's automatic sibling repoint (internal/release/rollback.go:96) does not. No pointer, an unreadable one, or an older one refuses with HeldDownRefusal(mark.Build).
A scheduled worker replacement carries no fallback while a marker of the same app stands on the host.

[ ] Step 1: Write the failing tests

// internal/rollout/holdran_test.go
// N7: a marker never turns into permission to run a build a sibling's
// failure record holds. The instance has a good build, so it stays down.
func TestAMarkedBuildStillHonoursASiblingsFailureRecord(t *testing.T) {
	ctx := context.Background()
	h := newHarness(t, "h", "s")
	h.stage("v2")
	h.install("v2", "h", "s")
	h.setGood("h", "v1") // the good-build helper the file's other tests use
	h.markHoldRan("h", "v2")
	h.recordFailure("v2", "s") // a sibling's plain failure record of v2
	h.sleepHost("h")
	h.wakeHost("h")
	p := h.m.Plan(ctx, h.rows.get("h"))
	if p.Refuse != HeldDownRefusal("v2") {
		t.Fatalf("plan %+v: want the hold-down refusal for v2", p)
	}
}

// Ruling 205: the installed good build boots over a marker only after a
// signed rollback newer than the marker.
func TestAnInstalledGoodBuildBootsOverAMarkerOnlyAfterASignedRollback(t *testing.T) {
	ctx := context.Background()
	h := newHarness(t, "h")
	h.setGood("h", "v1")
	h.markHoldRan("h", "v2") // stamped at the harness clock
	h.install("v1", "h")     // root repointed h to its good build
	h.sleepHost("h")
	h.wakeHost("h")
	if p := h.m.Plan(ctx, h.rows.get("h")); p.Refuse != HeldDownRefusal("v2") {
		t.Fatalf("no rollback pointer: plan %+v, want refused", p)
	}
	h.advance(time.Minute)
	h.pointer("stable", "v1", h.clock())
	if p := h.m.Plan(ctx, h.rows.get("h")); p.Refuse != "" || p.Boot.Version != "v1" {
		t.Fatalf("after a signed rollback: plan %+v, want v1", p)
	}
}

// internal/rollout/worker_test.go
// N5: a worker replacement on a host with a hold-ran marker carries no
// fallback onto the good build.
func TestAWorkerReplacementCarriesNoFallbackWhileAMarkerStands(t *testing.T) {
	// Arrange as the file's existing fallback test does (a worker host with
	// good v1, an overlap adoption of v3), then mark the host hold-ran on v2
	// before the pass. Want: h.act.replacements()[0].fallback == nil.
}

If the harness lacks setGood, markHoldRan or recordFailure, add them to fake_test.go as thin wrappers (h.rows.put with GoodVersion/GoodDigest; h.book.MarkHoldRan; h.book.Record).

Change TestASupersededHoldThatRanKeepsTheInstanceOffItsGoodBuild (holdran_seq_test.go:58-71): with s's record of v2 standing and h holding good v1, h's plan must now be Refuse == HeldDownRefusal("v2"), and its comment must say the instance stays down until a retry or a newer build (N7).

[ ] Step 2: Run to verify they fail: nice -n 19 go test ./internal/rollout/ -run 'Marker|HoldRan|Superseded|Worker' -count=1

[ ] Step 3: Implement

// wakeplan.go, decideHoldRan
	switch {
	case installed == mark.Build:
		if hasGood && in.Book.Blocks(acct, app, installed, host, false) {
			// N7: the marker stops a fallback; it never lifts a record.
			return activator.WakePlan{Refuse: HeldDownRefusal(mark.Build)}, nil
		}
	case hasGood && installed == rt.GoodVersion:
		if in.RolledBack == nil || !in.RolledBack(rt.GoodVersion, mark.At) {
			// Ruling 205: only a signed rollback newer than the marker is an
			// explicit recovery; root's sibling repoint is not.
			return activator.WakePlan{Refuse: HeldDownRefusal(mark.Build)}, nil
		}
		return recovery, nil
	case hasGood && in.Book.Blocks(acct, app, installed, host, false):
		return activator.WakePlan{Refuse: HeldDownRefusal(mark.Build)}, nil
	}

// wakeplan.go, planRow: wakeInputs gains
		RolledBack: func(good string, since time.Time) bool {
			if m.d.Pointer == nil {
				return false
			}
			ptr, err := m.d.Pointer(ctx, acct, app, routeChannel(rt))
			return err == nil && ptr != nil && ptr.Version == good && ptr.PromotedAt.After(since)
		},

// wakeplan.go, planCheck, the marker arm
			case build == mark.Build:
				if m.d.Book.HeldDown(acct, app, build, host) || (hasGood && m.d.Book.Blocks(acct, app, build, host, false)) {
					return held(build)
				}
				return nil

// worker.go, executeWorker
	acct := release.RouteAccount(rt)
	if _, marked := holdRanFor(m.d.Book, acct, rt.App, rt.Host); !marked {
		if g := rt.Good(); g.Version != "" && g.Digest != "" && g.Version != a.Build {
			fallback = &release.StagedIdentity{...as today...}
		}
	}

Rewrite decideHoldRan's and planCheck's doc comments to state the three rules above.

In sim_test.go, delete the N7 exemption (&& build != s.holdRan[h.name] in checkBoot, ~840) and make checkWakeRefused (~826-832) expect a refusal when the marked build is held for a host with a good build. Model the rollback rule if the simulation installs the good build over a marker; otherwise say so in the report.

[ ] Step 4: Run the rollout package (nice -n 19 go test ./internal/rollout/ -count=1) and nice -n 19 internal/rollout/testdata/rolloutsim/mutate.sh 300; paste both.

[ ] Step 5: Commit zdd-readiness: hold-ran markers keep records, block worker fallback, and need a signed rollback (N5, N7, ruling 205)

---

Task 4: A book sequence orders marker clears and retries (N6, retry ordering)

Corrections (plan review round 1; these override the text below):

A stale outcome suppresses the whole failure transition (plan review #3). Record returns a disposition, type RecordResult int with Recorded, Merged, Unchanged, PreRetry. failCohort (queue.go:~480-500) halts and supersedes only for Recorded/Merged; for PreRetry it neither halts the cohort nor supersedes attempts. Add Manager-level tests through done() and OnGen: after RetryHost (and, separately, after a pointer clear), a delayed failure of a pre-retry attempt and of a pre-retry wake leave the cohort unhalted, the retried host's queued adoption admitted, and no running attempt's SchedRev changed.
Markers are per execution, not per build (plan review #4). HoldRan gains Exec string (json:"exec,omitempty"): the execution's identity, "a<SchedRev>" for an attempt and "w<token>" for a planned wake (the same IDs Task 5's intents use). markLocked keeps Seq/At only when Exec matches the standing marker (a repeated report of one execution); a different execution of the same build gets a fresh Seq = ++seq and At. Extend the N6 regression: marker for B (exec a1), snapshot C, B runs again (exec a2, re-marked), C's good write clears nothing.

Files:

Modify: internal/rollout/failures.go (sequence, HoldRan.Seq, clears, Record)
Modify: internal/rollout/manager.go:892-1003 (served, writeGood, writeGoodLocked, goodRecorded), :515-553 (RetryHost), :418-466 (applyPointers), :581-587 (newWakeToken), :732-741 (Bootstrap)
Modify: internal/rollout/queue.go:34-50 (dispatch.BookSeq), internal/rollout/scheduler.go:96-287 (execute, done), internal/rollout/wakeplan.go:186-246, 362-477 (Plan, OnGen, buildFailed)
Test: internal/rollout/failures_test.go, new internal/rollout/bookseq_test.go

Interfaces:

Produces: - func (b *FailureBook) Snapshot() uint64: the book's sequence plus one. Taken before a generation can execute. Zero means "no snapshot" and clears nothing. - HoldRan.Seq uint64 (json:"seq,omitempty"), set from the book's sequence when the marker is written. - func (b *FailureBook) ClearHoldRanBefore(host string, snap uint64) (bool, error): drops the marker only if snap != 0 && marker.Seq < snap. - FailureRecord.Seq uint64 (json:"-"): the snapshot the failing execution started under; zero when unknown. - The book's sequence is persisted (failureFile.Seq, json:"seq,omitempty"), so snapshots after a restart exceed every marker before it. - Manager.served(host string, g activator.Gen, snap uint64) and writeGood(host string, g activator.Gen, snap uint64).

Rules:

N6. A good write clears only markers set before its generation's execution began. Attempts take d.BookSeq = Book.Snapshot() in execute immediately before Act.Swap or executeWorker. Planned wakes take a snapshot in Plan right after newWakeToken, kept per token (Manager.wakeSnap map[uint64]uint64, dropped when that token's terminal event arrives, when forget drops the host, and when a newer plan for the host replaces an entry with no event). Bootstrap passes zero.
Retry ordering. RetryHost records a host clear at ++seq, and ClearOnPointer records an app-wide clear of the build at ++seq (in memory, map[[4]string]uint64 keyed by account, app, build and host, "" for app-wide; Prune drops entries for builds no route installs). Record(r) with r.Seq != 0 && r.Seq <= clear for its host or app-wide key began before that retry: it creates no record, leaves Retried alone, adds no HeldDown. It still sets a requested hold-ran marker, because the marker is a fact about the database, not a failure.
A marker written by markLocked gets Seq = ++b.seq and At = b.now().

[ ] Step 1: Write the failing tests

// internal/rollout/bookseq_test.go
// N6, Astra's sequence: A's good write was decided before hold B ran on the
// host; B's marker is newer, and A's write must not clear it.
func TestAnOlderGoodWriteNeverClearsANewerMarker(t *testing.T) {
	b := &FailureBook{}
	snapA := b.Snapshot()
	if err := b.MarkHoldRan("h", HoldRan{Account: "bab", App: "blog", Build: "v2"}); err != nil {
		t.Fatal(err)
	}
	if cleared, _ := b.ClearHoldRanBefore("h", snapA); cleared {
		t.Fatal("a write decided before the marker cleared it")
	}
	if cleared, _ := b.ClearHoldRanBefore("h", b.Snapshot()); !cleared {
		t.Fatal("a write decided after the marker must clear it")
	}
}

func TestAZeroSnapshotClearsNothing(t *testing.T) {
	b := &FailureBook{}
	_ = b.MarkHoldRan("h", HoldRan{Account: "bab", App: "blog", Build: "v2"})
	if cleared, _ := b.ClearHoldRanBefore("h", 0); cleared {
		t.Fatal("no snapshot must clear nothing")
	}
}

// Retry ordering: a failure of an attempt that began before the host's retry
// never takes the retry back.
func TestAPreRetryFailureLeavesTheRetryStanding(t *testing.T) {
	b := &FailureBook{}
	before := b.Snapshot()
	b.Record(FailureRecord{Account: "bab", App: "blog", Build: "v2", Host: "s", Seq: before})
	if ok, _ := b.RetryHost("bab", "blog", "v2", "h"); !ok {
		t.Fatal("retry")
	}
	b.Record(FailureRecord{Account: "bab", App: "blog", Build: "v2", Host: "h", Seq: before, HeldDown: []string{"h"}})
	if b.Blocks("bab", "blog", "v2", "h", false) || b.HeldDown("bab", "blog", "v2", "h") {
		t.Fatal("a pre-retry failure undid the retry")
	}
	b.Record(FailureRecord{Account: "bab", App: "blog", Build: "v2", Host: "h", Seq: b.Snapshot()})
	if !b.Blocks("bab", "blog", "v2", "h", false) {
		t.Fatal("a failure after the retry must stand")
	}
}

func TestAPreRetryFailureNeverRecreatesAClearedRecord(t *testing.T) {
	b := &FailureBook{}
	before := b.Snapshot()
	b.Record(FailureRecord{Account: "bab", App: "blog", Build: "v2", Host: "s", Seq: before})
	if ok, _ := b.ClearOnPointer("bab", "blog", "v2", time.Now().Add(time.Hour)); !ok {
		t.Fatal("clear")
	}
	b.Record(FailureRecord{Account: "bab", App: "blog", Build: "v2", Host: "t", Seq: before, MarkHoldRan: true, Mode: "hold"})
	if _, held := b.Lookup("bab", "blog", "v2"); held {
		t.Fatal("a pre-retry failure recreated the record")
	}
	if _, marked := b.HoldRanOn("t"); !marked {
		t.Fatal("the marker is a fact about t's database and must still be set")
	}
}

func TestTheSequenceSurvivesAReopen(t *testing.T) {
	dd := t.TempDir()
	b, _ := OpenFailureBook(dd)
	_ = b.MarkHoldRan("h", HoldRan{Account: "bab", App: "blog", Build: "v2"})
	b2, err := OpenFailureBook(dd)
	if err != nil {
		t.Fatal(err)
	}
	if cleared, _ := b2.ClearHoldRanBefore("h", b2.Snapshot()); !cleared {
		t.Fatal("after a reopen a fresh snapshot must exceed the old marker")
	}
}

Add one Manager-level regression in bookseq_test.go for N6 through OnGen: plan a first wake of v1 on h (snapshot taken), mark h hold-ran on v2 through a scheduled hold attempt that is superseded (as holdran_seq_test.go does), then deliver v1's GenSettled with the first plan's token; assert h.book.HoldRanOn("h") still names v2.

[ ] Step 2: Run to verify they fail: -run 'Snapshot|Marker|PreRetry|Sequence|OlderGood'

[ ] Step 3: Implement the rules above. Key code:

// failures.go
type FailureBook struct {
	// ...existing fields...
	seq     uint64               // persisted; every marker and clear takes ++seq
	cleared map[[4]string]uint64 // in memory: retry clears, host "" for app-wide
	now     func() time.Time     // nil: time.Now
}

func (b *FailureBook) Snapshot() uint64 {
	b.mu.Lock()
	defer b.mu.Unlock()
	return b.seq + 1
}

func (b *FailureBook) preRetryLocked(r FailureRecord) bool {
	if r.Seq == 0 {
		return false
	}
	return r.Seq <= b.cleared[[4]string{r.Account, r.App, r.Build, r.Host}] ||
		r.Seq <= b.cleared[[4]string{r.Account, r.App, r.Build, ""}]
}

In Record: after the marker step, if b.preRetryLocked(r) { return false, save-or-retry }. markLocked sets h.Seq = b.bumpLocked() and h.At = b.nowLocked(); compare old and new markers on Account/App/Build only, so re-marking the same build keeps its first Seq and At. goodRecorded(host, ver, snap) calls ClearHoldRanBefore. RetryHost keeps its unconditional ClearHoldRan (the retry is the explicit recovery).

[ ] Step 4: Run the rollout package and mutate.sh 300; paste both.

[ ] Step 5: Commit zdd-readiness: a persisted book sequence orders marker clears and pre-retry outcomes (N6, retry ordering)

---

Task 5: Failure state survives a failed write and a restart (F5, schema 2)

Corrections (plan review round 1; these override the text below):

Never end an intent on !Owns (plan review #1). A failed wake deletes its activator entry before its OnGen is delivered (activator.go:1453, :1549), so dropUnownedWakes must not call End. Instead, every planned first wake ends with exactly one token-carrying terminal event: in internal/activator, the planned-wake exits that today send no event (refused at the lease, abandoned at the exec, refused by planCheck) emit GenCancelled{Executed: false, Token: plan.Token}; only for a plan with a Token and only when OnGen is set, so flag-off is untouched. End runs on the token's first event that settles the planned build's fate: GenSettled, GenFailed, GenFellBack, GenCancelled, and GenHealthy with FirstWake == false (the verdict floor booted the good build and the planned build never ran, activator/wakeplan.go:667). Begin's replacement of an older wake intent for the same host stays (the outcome fence keeps a newer plan from starting before the older wake's outcome is delivered); say so in its comment, with the fence's anchor.
Persist the effective mode (plan review #2). Swap turns an overlap attempt on a foreign-layout row into a hold (activator/swap.go:264, a.foreignRow). Add EffectiveMode(host string, m activator.SwapMode) activator.SwapMode to the rollout Activator interface (*activator.Service implements it with foreignRow; the fake returns m unless a test sets it), and write the intent's Mode from it. Check the planned-wake path for the same override and do likewise. Add TestAForeignLayoutHoldCrashReopensHeldDown: an overlap attempt on a foreign row writes a hold intent; reopening the book with that intent unresolved holds the host down and marks it.
The marker a reconciled hold intent sets carries Exec = the intent's ID (Task 4).

Files:

Modify: internal/rollout/failures.go (intents, schema 2, reconciliation)
Modify: internal/rollout/scheduler.go:96-129 (execute), :133-287 (done)
Modify: internal/rollout/wakeplan.go:186-246 (Plan), :362-426 (OnGen), internal/rollout/manager.go:638-661 (dropUnownedWakes)
Modify: internal/host/rollout.go:315-320 (armRollout), internal/host/rolloutdeclined.go:68-84 (honourDeclinedFailures)
Test: internal/rollout/failures_test.go, new internal/rollout/intent_test.go, internal/host/rollout_test.go

Interfaces:

Produces: - type Intent struct { Account, App, Build, Mode, ID string } (JSON keys lower-case), stored in failureFile.Intents map[string]Intent by host. - func (b *FailureBook) Begin(host string, in Intent) error: writes the intent and saves; on a save error the intent is removed from memory and the error returned, and the caller must not execute. - func (b *FailureBook) End(host, id string) error: removes the host's intent if its ID matches, and saves. - func (b *FailureBook) Reconciled() int: how many intents OpenFailureBook turned into records. - func (b *FailureBook) Flush() error: saves now (used after reconciliation).

Rules (Astra R1 #5, R3 §5 item 2):

Write-ahead. An adoption attempt (d.Class == classAdopt) calls Begin(host, Intent{..., ID: "a" + strconv.FormatUint(d.SchedRev, 10)}) in execute after resolve and before Swap/executeWorker. A planned first wake (plan.FirstWake && plan.Boot.Version != "" && plan.Refuse == "") calls Begin(host, Intent{..., ID: "w" + strconv.FormatUint(tok, 10)}) at the end of Plan (no activator lock is held there). A failed Begin refuses: the attempt finishes SwapAbandoned with Executed: false and reason "failure records could not be written"; the wake is refused with the approved text "this box cannot start instances right now: its record of failed builds cannot be read" (the same string as host.refuseTexts[rolloutBookUnreadable]; share it, do not copy-edit it).
End runs after the outcome's own book write (Record/MarkHoldRan), never before: attempts in done(), wakes on the token's terminal OnGen event (GenSettled, GenFailed, GenFellBack, GenCancelled), on dropUnownedWakes for the dropped entry's token, and in Begin itself when a newer wake intent replaces an older wake intent for the host. A Begin that finds an attempt intent ("a...") for the host returns an error.
Reconcile at open. Each intent left in the file becomes a failure record for its build on that host (Cause release, Mode the intent's, HeldDown and a hold-ran marker when Mode == "hold", PromotedAt and At the open time, Reason a string reasonWord maps to an existing word; check internal/rollout/events.go). Intents are then deleted in memory and the book is marked dirty.
Schema 2. saveLocked writes Schema: 2; OpenFailureBook accepts 1 and 2 and errors on anything else.
armRollout: after OpenFailureBook, if book.Reconciled() > 0, log the count and Flush(); a flush error is refuseManaged(svc, rolloutBookUnreadable). honourDeclinedFailures: the reconciled records are honoured; a flush error is logged.

[ ] Step 1: Write the failing tests

// internal/rollout/intent_test.go
// A save fails by making the data dir read-only (WriteFileAtomic writes its
// temp file beside the target). Running as root defeats it: the tests skip.

// Review Focus 1: B's failure is held only in memory because the save
// failed; the agent restarts. The intent written before B ran is still on
// disk and comes back as B's failure.
func TestAFailedSaveThenRestartKeepsTheFailure(t *testing.T) {
	dd := t.TempDir()
	b, _ := OpenFailureBook(dd)
	if err := b.Begin("h", Intent{Account: "bab", App: "blog", Build: "v2", Mode: "hold", ID: "a7"}); err != nil {
		t.Fatal(err)
	}
	if err := os.Chmod(dd, 0o500); err != nil {
		t.Fatal(err)
	}
	t.Cleanup(func() { os.Chmod(dd, 0o700) })
	if _, err := b.Record(FailureRecord{Account: "bab", App: "blog", Build: "v2", Host: "h", Mode: "hold", HeldDown: []string{"h"}}); err == nil {
		t.Skip("the save did not fail (running as root?)")
	}
	_ = b.End("h", "a7") // fails too: the intent stays on disk
	os.Chmod(dd, 0o700)
	b2, err := OpenFailureBook(dd)
	if err != nil {
		t.Fatal(err)
	}
	if b2.Reconciled() != 1 || !b2.HeldDown("bab", "blog", "v2", "h") {
		t.Fatalf("after restart: reconciled %d, held down %v", b2.Reconciled(), b2.HeldDown("bab", "blog", "v2", "h"))
	}
	if m, ok := b2.HoldRanOn("h"); !ok || m.Build != "v2" {
		t.Fatal("a hold intent must leave its marker")
	}
}

func TestABeginThatCannotSaveRefusesAndLeavesNothing(t *testing.T) {
	dd := t.TempDir()
	b, _ := OpenFailureBook(dd)
	os.Chmod(dd, 0o500)
	t.Cleanup(func() { os.Chmod(dd, 0o700) })
	if err := b.Begin("h", Intent{Account: "bab", App: "blog", Build: "v2", Mode: "overlap", ID: "a1"}); err == nil {
		t.Skip("the save did not fail")
	}
	os.Chmod(dd, 0o700)
	b2, _ := OpenFailureBook(dd)
	if b2.Reconciled() != 0 {
		t.Fatal("a refused begin left an intent")
	}
}

func TestASchema1BookReadsAndIsWrittenAsSchema2(t *testing.T) {
	dd := t.TempDir()
	raw := `{"schema":1,"records":[{"account":"bab","app":"blog","build":"v2","host":"h","cause":"release","mode":"overlap","reason":"x","promoted_at":"2026-10-01T00:00:00Z","at":"2026-10-01T00:00:00Z"}]}`
	os.WriteFile(filepath.Join(dd, FailureFile), []byte(raw), 0o644)
	b, err := OpenFailureBook(dd)
	if err != nil || !b.Blocks("bab", "blog", "v2", "x", false) {
		t.Fatalf("schema 1: %v", err)
	}
	if err := b.Flush(); err != nil {
		t.Fatal(err)
	}
	out, _ := os.ReadFile(filepath.Join(dd, FailureFile))
	if !strings.Contains(string(out), `"schema": 2`) {
		t.Fatalf("written as %s", out)
	}
}

// Review Focus 2: what an older agent sees. Its reader refused schema 2.
func TestASchema2BookIsUnreadableToASchema1Reader(t *testing.T) {
	var f struct{ Schema int `json:"schema"` }
	dd := t.TempDir()
	b, _ := OpenFailureBook(dd)
	_ = b.Flush()
	raw, _ := os.ReadFile(filepath.Join(dd, FailureFile))
	json.Unmarshal(raw, &f)
	if f.Schema == 1 {
		t.Fatal("an old reader would accept the file and drop markers and intents")
	}
}

Also add, in intent_test.go, Manager-level tests: (a) an adoption whose Begin fails never reaches Act.Swap (h.act.snapshot() shows no call) and is retried after the abandon deferral; (b) a first-wake plan whose Begin fails is refused with the shared text; (c) a planned wake's intent is ended by its GenSettled and by dropUnownedWakes. In internal/host/rollout_test.go: an armed box whose book holds an intent reconciles it, and a flush failure refuses managed starts (rolloutUnavailable() == rolloutBookUnreadable).

[ ] Step 2: Run to verify they fail.

[ ] Step 3: Implement the rules. Begin/End never run under scheduler.mu or activator.Service.mu. Reword the dirty comment (failures.go:89-92) and Record's comment (:155-163) to say what now protects a failed save: the intent on disk.

[ ] Step 4: Run the rollout package, the host tests -run 'Rollout|Declin|Book' (unsandboxed), and mutate.sh 300; paste all three.

[ ] Step 5: Commit zdd-readiness: write-ahead intents and schema 2 keep failures across a failed save and a restart (F5)

---

Task 6: An uncertain start in hold mode is a failure (ruling 206)

Corrections (plan review round 1; these override the text below):

Fallback needs termination evidence (plan review #9). A nested start error can wrap both errStartedThenExited and a failed termination proof (backend_exec.go:1709), and spawnPlanned refuses a fallback when proc == nil (activator/wakeplan.go:393). So in overlap, errStartedThenExited is a failure of the boot build (w.failed), and the fallback runs only when the generation's termination is proven (no genStartUnproven in the error chain, or the activator's own proof succeeded). Otherwise the wake records the failure (GenFailed), serves nothing, and keeps the lease/containment as an unproven wake does today. Test both cleanup outcomes.

Files:

Modify: internal/activator/backend_exec.go:~1702-1725 (the markStartUncertain sites; give the inv == "" case its own sentinel errStartedThenExited)
Modify: internal/activator/swaphold.go:~190-205 (holdResult)
Modify: internal/activator/wakeplan.go:~355-390 (spawnPlanned)
Test: internal/activator/swaphold_test.go or the file with holdResult tests; internal/activator/wakeplan_test.go

Interfaces:

Produces: var errStartedThenExited = errors.New("generation exited before it was observed") wrapped by markStartUncertain for the inv == "" case, so errors.Is finds both.

Rules (Astra R3 ruling 206): a nested start that root accepted and whose outcome is uncertain may have run its build.

Hold swap: holdResult returns SwapDown (held down by done(), cohort halted) when werr != nil, the proc is nil and startMayHaveRun(nil, werr) is true. It never returns abandoned with Executed: true for such a start, so the abandon deferral cannot re-run the build.
Planned wake: spawnPlanned treats an uncertain start as a failure of the boot build (w.failed) in hold mode always, and in overlap mode only for errStartedThenExited. Other uncertain overlap starts keep today's GenCancelled{Executed: true}.

[ ] Step 1: Write the failing tests against the nested fakes the existing uncertain-start tests use (search markStartUncertain in internal/activator/*_test.go): a hold swap whose nested start returns markStartUncertain(errStartedThenExited) yields SwapDown with Executed true; one returning another uncertain error also yields SwapDown; a hold planned wake with an uncertain start emits GenFailed (not GenCancelled); an overlap planned wake with errStartedThenExited emits GenFellBack when it has a fallback, and with another uncertain error still emits GenCancelled{Executed: true}.

[ ] Step 2: Run to verify they fail (unsandboxed, TMPDIR=/tmp/k, -run 'Uncertain').

[ ] Step 3: Implement.

[ ] Step 4: Run -run 'Uncertain|Hold|Planned|Swap' in activator; paste.

[ ] Step 5: Commit zdd-readiness: an uncertain nested start is a hold failure, never a re-runnable abandon (ruling 206)

---

Task 7: Execution admission: worker gate, managed revalidation, uncaptured recycle

Files:

Modify: internal/activator/backend_exec.go:~2113-2160 (StartWorkerStaged), internal/activator/workerbuild.go:~51-118 (startWorker, runBeforeExec), :~160-186 (recycleWorkerLocked)
Modify: internal/rollout/scheduler.go:96-103 (execute)
Test: internal/activator/exec_gate_test.go, internal/rollout/manager_test.go

Rules (Astra R3 §5 item 4, ruling 206 "park worker launch hook"):

Worker gate after preparation. runBeforeExec keeps only the generations-file publication. The caller's ownership gate rides the context (withExecGate(ctx, gate)), and StartWorkerStaged spawns with b.spawnInCgroupGated(rt, build, b.launchGate(ctx, rt.Host)), so the gate runs after cgroup preparation and before every start attempt, as the instance path does (cgroup.go:~354/367).
Managed revalidation. execute: after reading the row, if !Managed(*rt) || release.RouteAccount(*rt) != s.key.acct || rt.App != s.key.app the attempt finishes SwapAbandoned with Executed: false and reason "route no longer managed", before resolve.
Uncaptured recycle. In recycleWorkerLocked, when a.CoalesceRecycle != nil (an armed or refusing box) and the worker has no captured build (e.workerServing.StagedPath == ""), it never respawns from the live link: with a live process (stopOld != nil) the recycle is skipped and the process keeps running; with none, the entry stays down and the agent logs once per host. Today's behaviour stays when CoalesceRecycle is nil.

[ ] Step 1: Write the failing tests: (a) in exec_gate_test.go, modelled on the instance tests at :419, a worker start whose gate starts refusing during cgroup preparation never execs (the fake cgroup's prepare hook flips the gate); (b) a worker start retry after a failed PID move re-asks the gate; (c) in manager_test.go, a queued host whose row becomes a unit route before dispatch gets no Swap call and its outcome is abandoned; (d) an armed box's recycle of a worker with no captured build makes no live-link spawn.

[ ] Step 2: Run to verify they fail.

[ ] Step 3: Implement.

[ ] Step 4: Run activator -run 'Gate|Worker|Recycle' (unsandboxed), rollout package, mutate.sh 300; paste.

[ ] Step 5: Commit zdd-readiness: worker gates after cgroup preparation, managed revalidation at dispatch, no uncaptured live-link recycle

---

Task 8: Host retries are signed by the console

Corrections (plan review round 1; these override the text below):

Bind the grant to the restriction's epoch, not to a watermark (plan review #13). Drop restartMemory.retryAt. HostRetryGrant gains Epoch uint64 \json:"epoch"\`. The epoch of a host's restriction is a persisted book sequence number: FailureRecord gains Epochs map[string]uint64 \json:"epochs,omitempty"\ (per host: ++seq each time Record changes that host's standing in the record, and the record-wide entry under key "" each time the record is created or merged), and a marker's epoch is its Seq (Task 4). func (b *FailureBook) RestrictionEpoch(acct, app, build, host string) uint64 returns the larger of the record's "" and host entries, or the marker's Seq when the marker names build`.
Status publishes it. status.RolloutHost gains HeldEpoch uint64 \json:"held_epoch,omitempty"\`, set beside HeldBy (internal/rollout/status.go:74-93). The console's rollout join and retryHostBuild (internal/console/apiretry.go:41-58) return (build string, epoch uint64)`, and the door signs both. An absent epoch (an older box) is 0; the console still signs it, and the box refuses a 0 epoch.
Which build a grant clears (plan review #5). Manager.RetryHost(host, build string, epoch uint64) bool clears (a) the record of build for the host, when one holds the host, and (b) the host's marker, when it names build; each only if RestrictionEpoch(..., build, host) == epoch. A grant naming the marker's build B while the slot wants A clears the marker and nothing else. Drop the wantBuild equality rule in the body below.
The box's verification keeps the 24-hour issue window (HostRetryGrantTTL = 24 * time.Hour) as a backstop.
Tests replace the replay test: a grant whose epoch predates a second failure of the same build on the same host clears nothing; reopening the book and replaying a used grant clears nothing new.

Files:

Create: internal/release/hostretry.go, internal/release/hostretry_test.go
Modify: internal/release/restartscope.go:107-110 (RestartScope.Signed), :161-175 (ValidRestartScope), AppendRestartScope if it compares scopes
Modify: internal/console/apiretry.go:61-71 (requestHostRetry), :76-122, and the browser twin internal/console/browserretry.go
Modify: internal/host/restartscope.go:~179-200 (retryRequests returns the signed field), internal/host/restart.go:~838-863 (verify before retryHost), internal/host/agent.go:~2296-2306 (restartMem.pub, retryHost signature)
Modify: internal/rollout/manager.go:515-553 (RetryHost takes the build)
Test: internal/console/apiretry_test.go (or the existing retry test file), internal/host/restart_test.go (or the retry tests' file), internal/rollout/manager_test.go

Interfaces:

Produces:

// internal/release/hostretry.go
package release

// HostRetryGrant is the console's signed `carlos instances retry <host>`
// (spec §8.4, §11): a box clears a host's failure hold only on a grant the
// deployment root signed, naming that account, app, host and build.
type HostRetryGrant struct {
	V        int    `json:"v"`
	Account  string `json:"account"`
	App      string `json:"app"`
	Host     string `json:"host"`
	Build    string `json:"build"`
	IssuedAt string `json:"issued_at"` // RFC 3339, UTC, whole seconds
}

// HostRetryGrantTTL is how long a box accepts a grant after it was issued.
const HostRetryGrantTTL = time.Hour

var ErrHostRetryGrant = errors.New("host retry grant: not valid for this request")

func SignHostRetry(sg Signer, acct, app, host, build string, now time.Time) (string, error) {
	g := HostRetryGrant{V: 1, Account: acct, App: app, Host: host, Build: build,
		IssuedAt: now.UTC().Truncate(time.Second).Format(time.RFC3339)}
	wire, err := SignRootEnvelope(sg, g)
	return string(wire), err
}

// VerifyHostRetry checks wire against the root key and the entry it rode in:
// same account, app and host, issued no later than now plus a minute of skew
// and no earlier than now minus HostRetryGrantTTL.
func VerifyHostRetry(pub ed25519.PublicKey, wire, acct, app, host string, now time.Time) (HostRetryGrant, error) {
	var g HostRetryGrant
	if err := VerifyRootEnvelope(pub, []byte(wire), &g); err != nil {
		return HostRetryGrant{}, err
	}
	at, err := time.Parse(time.RFC3339, g.IssuedAt)
	if err != nil || g.V != 1 || g.Account != acct || g.App != app || g.Host != host || g.Build == "" ||
		at.After(now.Add(time.Minute)) || at.Before(now.Add(-HostRetryGrantTTL)) {
		return HostRetryGrant{}, ErrHostRetryGrant
	}
	return g, nil
}

- RestartScope.Signed string \json:"signed,omitempty"\`: only a retry scope may carry it; ValidRestartScope refuses it on other kinds and above MaxRootEnvelopeBytes. - Manager.RetryHost(host, build string) bool: clears only when build is the build the host's queue slot wants (h.wantBuild) or, with no slot build, the build a record holds the host on; a mismatch clears nothing (and leaves the marker). - restartMemory.pub ed25519.PublicKey; restartMemory.retryHost func(host, build string) bool; restartMemory.retryAt map[string]time.Time (newest accepted IssuedAt` per host).

Rules:

Console. requestHostRetry(ctx, acct, app, host, build) signs with release.SignHostRetry(release.RootSigner(s.cfg.Key), ...) and appends RestartScope{Kind: retry, Value: host, Signed: wire}. With s.cfg.Key == nil the door answers 503 and appends nothing. For its text, reuse the read-only console's existing 503 wording for promote (internal/console/server.go:~4135) if it reads right for a retry; otherwise add a draft with // TODO-COPY(zdd-readiness.retry-no-key).
Box. Every retry entry is verified with mem.pub against the entry's account, app and host at the box's clock. Unsigned, unverifiable, expired, not newer than retryAt[host], or a build that does not match: logged (agent: retry requested for %s, dropped: %s) and skipped. A box with no public key drops every retry. Verified: retryAt[host] = IssuedAt, then retryHost(host, g.Build).
Old entries stay readable; a new box reading an old console's unsigned entry drops it.

[ ] Step 1: Write the failing tests: release (sign/verify round trip; wrong host, wrong app, expired, future, tampered payload, scoped signer refused); console (the API and browser doors append an entry whose signed verifies for the held build; a console with no key answers 503 and appends nothing); host (Review Focus 4: an unsigned entry, a replay of the same grant under a new entry id, and a grant for another build each leave RetryHost uncalled; a valid grant calls it once with the grant's build); rollout (RetryHost("h", "v9") with the slot wanting v2 clears nothing).

[ ] Step 2: Run to verify they fail.

[ ] Step 3: Implement. Keep the restart-scope id/mark logic as is.

[ ] Step 4: Run release, rollout (sandboxed), console and host -run 'Retry|RestartScope' (unsandboxed); paste.

[ ] Step 5: Commit zdd-readiness: host retries are root-signed in the console and verified on the box (Q4, spec §11)

---

Task 9: Declined boxes gate the actual launch and honour markers

Corrections (plan review round 1; these override the text below):

Every declined launch mechanism is gated against the identity it actually starts (plan review #6): flat instance (launchGate), nested instance (runExecGate, backend_exec.go:1682), worker (StartWorkerStaged, onto == nil), and a claims-backed wake (after the claim snapshot resolves the live link, claim_snapshot.go:87, before the root request; Task 1 makes a claims box decline, so this is a declined box's path). Each passes the version it will start.
No flat read/exec race. When LiveExecGate is set, the flat live-link spawn resolves the link once (release.ResolveLive), gates that version, and execs the resolved staged path instead of the link. With the gate nil (every other box) the spawn is byte-for-byte today's. Drop the "Residual" sentence in rule 1.
Markers on a declined box: the installed good build is refused while a marker stands (as in the body); Task 10's pointer pass clears the marker on a signed rollback, as the armed box does (Task 3), so HeldDownRefusal's "roll back or retry" stays true on a declined box.

Files:

Modify: internal/activator/genbackend.go:~65-114 (launchGate), internal/activator/backend_exec.go:~1465-1500 (live-link spawn), :~2113-2150 (StartWorkerStaged, onto == nil), internal/activator/activator.go:~2641 (convergeWorkerLocked), internal/activator/workerbuild.go:~160-186 (recycleWorkerLocked converge branch)
Modify: internal/host/rolloutdeclined.go (failureHolds: markers, the gate, activation on markers or records)
Test: internal/host/rolloutdeclined_test.go, internal/activator/exec_gate_test.go

Interfaces:

Produces: ExecBackend.LiveExecGate func(rt registry.Route, build string) error (nil everywhere except a declined box with records or markers). It is asked (a) in launchGate when the context carries no gate, with the version read from the live link immediately before the start; (b) in StartWorkerStaged with onto == nil, with the resolved id.Version, before the spawn; (c) in convergeWorkerLocked and the converge branch of recycleWorkerLocked with the live link's version, before the old process is stopped: a refusal leaves the worker running its current build (a recycle of a dead worker respawns its captured build when it has one).
failureHolds.heldBuild(rt) also answers for markers (rule below), and failureHolds.gate(rt, build) error returns errors.New(rollout.HeldDownRefusal(b)) or errors.New(refuseNoSwaps) when build is held for rt.

Rules (Astra R1 #6, R2/R3 declined residuals; spec §6.4, §8.4):

Exec-time check. The plan-time refusal stays; the gate repeats it at the real launch with the build actually being started. Residual, recorded in the ledger: a flat instance execs the live-link path, so a root repoint landing between the gate's read and exec is not caught.
Markers on a declined box. With a marker m for the route's account and app: the marked build may start unless held down, or blocked by a record while the route has a good build; any other build a record blocks is refused; the installed good build is refused (a declined box has no pointer read in this path; recovery is a signed retry, Task 10, or a newer build); any other build starts.
honourDeclinedFailures installs the planner and the gate when the book holds records or markers.

[ ] Step 1: Write the failing tests: (a) TestDeclinedBoxRefusesAHeldBuildAtLaunch: plan with the live link on the good build, repoint to the held build before the start, assert no exec (use TestDeclinedBoxKeepsAHeldBuildOffARealActivator, rolloutdeclined_test.go:314, as the template); (b) a declined worker whose live link moves to a held build is not stopped by the sweep's converge and keeps its process; (c) a declined worker that crashes with the live link on a held build respawns its captured build, or stays down with none; (d) markers: a declined box with only a marker refuses the installed good build and the marked build when a record blocks it; (e) flag off: LiveExecGate is nil and no leg changes (legsAreTodays).

[ ] Step 2: Run to verify they fail (unsandboxed).

[ ] Step 3: Implement. LiveExecGate is consulted under the activator's lock in (c); it must only read memory and the registry row and the link, as heldBuild does today.

[ ] Step 4: Run host -run 'Declin|Rollout' and activator -run 'Gate|Worker|Converge'; paste.

[ ] Step 5: Commit zdd-readiness: a declined box gates the actual launch, workers included, and honours hold-ran markers

---

Task 10: Declined boxes process retries

Corrections (plan review round 1; these override the text below):

A declined box publishes its holds, and the console lets them be retried (plan review #5). rolloutStatusProbe's declined report (internal/host/rollout.go:481-490) gains Hosts entries for each managed route failureHolds holds: HeldBy (the held build), HeldEpoch (RestrictionEpoch, Task 8), Attempt status.AttemptDown when held down or when a marker refuses the installed build, else status.AttemptFailed. retryHostBuild (internal/console/apiretry.go:50) accepts a report whose Unavailable is a decline code (rolloutwords.UnavailableDeclines) when the host has a HeldBy; it still refuses every other Unavailable. Add an API-to-box test: a declined box's held host gets a signed grant from the door, and the declined box's restart leg clears it.
Retry clearing follows Task 8's epoch rule (retryHost(host, build, epoch)).
The pointer pass also clears a marker on a signed rollback (Task 3's rule), and runs for apps with markers as well as records.

Files:

Modify: internal/host/rolloutdeclined.go (failureHolds.retryHost, failureHolds.applyPointers; the comment that says the book is never written)
Modify: internal/host/agent.go:~2259-2306, ~2390-2396 (fastConvergeLoop: set restartMem.retryHost for a declined box; run the pointer pass)
Test: internal/host/rolloutdeclined_test.go

Interfaces:

Consumes: Task 8's restartMemory.retryHost func(host, build string) bool and signed-entry verification; Task 4's FailureBook.RetryHost, ClearOnPointer, ClearHoldRan.
Produces: func (h *failureHolds) retryHost(host, build string) bool and func (h *failureHolds) applyPointers(ctx context.Context, read func(ctx context.Context, acct, app, channel string) (*release.ChannelPointer, error), routes []registry.Route).

Rules (Astra R2 ruling 194 "startup-only records"):

A verified signed retry for a host clears that host from the record for build (RetryHost) and its marker (ClearHoldRan), and saves.
At most every 30 s (the armed pointerReadEvery), for each app with a standing record, the declined box reads each followed channel's signed pointer (release.Channel with the agent's store and key, both already open in fastConvergeLoop) and clears a record whose build the pointer names with a newer PromotedAt (ClearOnPointer). A read error is logged and skipped. No store or no key: no pointer pass, logged once.
Records and markers now change in the running process, so heldBuild reads the live book (it does today); no restart is needed.

[ ] Step 1: Write the failing tests: a declined box with a hold-down of v2 on h wakes h onto v2 after a verified signed retry for (h, v2) (planner returns today's plan); after a pointer naming v2 with a newer PromotedAt the record is gone; an older pointer clears nothing; a retry for another build clears nothing.

[ ] Step 2–4: run to fail, implement, run -run 'Declin|Retry' (unsandboxed); paste.

[ ] Step 5: Commit zdd-readiness: declined boxes process signed retries and promote --retry pointers

---

Task 11: Bounded reads and acknowledgement writes

Corrections (plan review round 1; these override the text below):

Ack order is fixed at the transition (plan review #10). configTrack.activate (called under m.mu) hands back the ack together with a per-host sequence taken under m.mu; the writer (Manager.acks) keeps the newest pending ack per host (latest sequence wins, older ones are dropped, never written after a newer one), submission never blocks, and one drain goroutine writes. When the manager's context ends, the drain finishes the write in progress and stops.
Flag off keeps configAckFunc as it is (plan review #11): put the 10 s bound in the armed path only (the writer wraps each write in context.WithTimeout), not in configAckFunc.
A graceful stop joins the scheduler (Task 5 review; spec §8.6 "Shutdown cancels and joins the whole operation set"). Add func (m *Manager) Close(ctx context.Context): it stops admitting attempts, then waits (bounded by ctx) for every in-flight execute goroutine to finish its done() (including the book write and the intent End) and for every in-flight OnGen call to return, then stops the ack writer as above. internal/host/agent.go calls it right after svc.Shutdown(ctx) (~line 732), with the same 60 s context, when a manager is armed. Test: an attempt whose Swap is cancelled by shutdown has its outcome and End on disk when Close returns (no intent left in the file), so a planned agent restart mid-swap reconciles nothing.

Files:

Modify: internal/rollout/policy.go:~60-117 (policyCache)
Modify: internal/rollout/manager.go:418-466, 555-568, 831-887, 1017-1057 (applyPointers, failedUnder, resolve/stagedDemand, activationDone, configSlept), internal/rollout/scheduler.go:46-50, 113-117, internal/rollout/wakeplan.go:254-283
Modify: internal/host/agent.go:~1166-1173 (configAckFunc)
Test: internal/rollout/policy_test.go, internal/rollout/manager_test.go

Interfaces:

Produces: const readBound = 5 * time.Second (rollout); failedUnder(ctx context.Context, rt registry.Route, build string) time.Time; Manager.acks (an ordered asynchronous writer: one drain goroutine, FIFO, no lock held while writing).

Rules (Astra R1 Flag triage T3/T8/T10, R3 §5 items 6-7):

Policy cache. Each fill takes a per-key generation before load; the result is stored only if no newer fill has stored since. A failed read stamps failedAt and is not retried for policyRetryAfter = 5 * time.Second; meanwhile get answers the last value, or hold for an app never read. Missing (an empty manifest from appconfig.Get) is the defaults; an error is never the defaults.
Bounds. policies.get's load, Deps.Pointer and Deps.Demand run under context.WithTimeout(ctx, readBound). failedUnder takes the caller's context: the wake's in planRow, the attempt's in execute.
Acks. activationDone and configSlept hand write to m.acks instead of calling it, so neither a scheduler slot (done() calls configActivated before finish) nor the scheduler loop waits on the store. configAckFunc bounds its store write with a 10 s timeout.

[ ] Step 1: Write the failing tests: (a) an older slow fill finishing after a newer one does not overwrite it (gate the loader on channels); (b) a failing loader is called at most once within policyRetryAfter; (c) a blocked ack writer does not stop done() from freeing the slot (the next attempt is admitted while the writer blocks); (d) a blocked pointer read in failedUnder returns zero once the wake's context is cancelled.

[ ] Step 2–4: run to fail, implement, run the rollout package and host -run 'ConfigAck|Rollout'; paste.

[ ] Step 5: Commit zdd-readiness: versioned policy fills, bounded store reads, acks off the scheduler's path

---

Task 12: Workers record clean one-shots and wait out a fallback; halts are always announced

Files:

Modify: internal/activator/workerswap.go:~173-192 (SwapResult.CleanExit), internal/activator/activator.go:~1209-1222, ~2484-2562 (WakeWorker budget)
Modify: internal/rollout/scheduler.go:181-194 (done), internal/rollout/manager.go (a good write for a clean exit), internal/rollout/wakeplan.go:469-476 (buildFailed with no scheduler)
Test: internal/activator/workerswap_test.go (or the worker test file), internal/rollout/worker_test.go, internal/rollout/wakeplan_test.go

Interfaces:

Produces: activator.SwapResult.CleanExit bool: the replacement's build exited cleanly as a one-shot (spec §6.5 success).

Rules:

One-shot good (spec §6.5, §8.5): done() for SwapDone with res.CleanExit writes the good build from res.Gen without acknowledged (the attempt still holds the host's slot, so no newer generation can have completed). Snapshot rules from Task 4 apply (d.BookSeq).
WakeWorker budget: while the entry is a first wake, WakeWorker's wait extends once to arrival + firstWakeBudget, as request waits do (activator.go:~1209-1222).
Halt event: buildFailed with no scheduler emits EvCohortHalted with the approved text when Record reports the record new, as cohortFailed does.

[ ] Step 1: Write the failing tests for each rule (a parked one-shot replacement records good; a first-wake worker whose fallback takes 40 s returns success to WakeWorker; a first-wake failure on a host with no scheduler emits the halt event).

[ ] Step 2–4: run to fail, implement, run activator -run 'Worker' and the rollout package; paste.

[ ] Step 5: Commit zdd-readiness: clean one-shot replacements record good, WakeWorker waits a fallback out, halts are announced

---

Task 13: Fallback artifacts stay, and a hold build is published before its incumbent stops

Corrections (plan review round 1; these override the text below):

Publish outside a.mu (plan review #10). persistGenerations takes genMu then a.mu (generations_file.go:238), and swapHold's admit body runs under a.mu (swaphold.go:107). So: register the pending generation under a.mu, release it, persist, re-acquire a.mu, and re-run the full admission checks (ownership, newest, the entry's state) before beginStopLocked. Any check that fails withdraws the pending entry and abandons with Executed: false.
The retention change is an authorized flag-off exception (Global Constraints).

Files:

Modify: internal/host/retention.go:41-46
Modify: internal/activator/swaphold.go:~110-135, ~245-255, internal/activator/generations_file.go:~151-181
Test: internal/host/retention_test.go, internal/activator/swaphold_test.go (or the hold tests' file)

Rules (Astra R1 T12, R3 §5 item 9):

Bucket retention pins rt.GoodVersion beside Version and PrevVersion.
swapHold registers the hold build's generation as pending (as startWorker does with genPending) and persists the generations file before beginStopLocked stops the incumbent; a failed persist abandons the swap (Executed: false) with the incumbent still serving. The pending entry is dropped when the hold withdraws.

[ ] Step 1: Write the failing tests: retention keeps a good build no route installs; a hold swap whose generations write fails leaves the incumbent running and returns abandoned; the generations file names the hold build before the incumbent's stop begins (record the order in the fakes).

[ ] Step 2–4: run to fail, implement, run host -run Retention and activator -run 'Hold|Generations'; paste.

[ ] Step 5: Commit zdd-readiness: retention pins the good build; a hold build is published before its incumbent stops

---

Task 14: The stage relay gives up, and a populated legacy parent is refused

Flag blocker confirmed on the dev box (#272, 2026-10-11, arm64, kernel 6.12.105, systemd 257, real-unit check (d)): a generation started under a legacy carlos-exec@<host> parent that still holds a process starts silently with NO memory limit, while the strict path refuses. The guard must fail closed and be reported: the generation start is refused while the host's parent holds processes directly, with an agent log line naming the host and the pid count. Unit-test it with a fake cgroupfs, and add a kernel-gated test (skipped unless the env the existing kernel tests use, e.g. CARLOS_CLAIM_SPAWN_KERNEL=1 with cgroup.Setup, is present) that puts a process directly in a delegated parent and asserts the generation start is refused and creates no g* child.

Corrections (plan review round 1; these override the text below):

Refuse direct parent members whatever children exist (plan review #12). ensureParent (internal/cgroup/gen.go:127) returns an error now (its signature changes; update every caller, including StartChildGated at :168). It reads the parent's own cgroup.procs (direct members only, not cgroup.events' descendant population) and returns ErrLegacyParentPopulated whenever that file lists a process, with or without g* children. Replace the "one with a g1 child still passes" test with TestAPopulatedParentWithAStaleEmptyGenerationIsRefused; keep a test that an empty parent with live g* children passes.

Files:

Modify: internal/host/stagerelay.go (pendingHead gains since time.Time; relayGiveUp = time.Hour)
Modify: internal/cgroup/gen.go:127-146 (ensureParent), and its caller's error path in internal/activator (abandon before exec)
Test: internal/host/stagerelay_test.go, internal/cgroup/gen_test.go

Rules (Astra R1 T15 and T14 L146):

A relay head that root has neither installed nor refused for relayGiveUp is dropped with a log line; the next blocked pass notes it again.
ensureParent reads the parent's own cgroup.procs; when it holds a process and no generation child exists, it returns ErrLegacyParentPopulated and creates no child. The generation start fails before exec (the swap abandons with Executed: false; a wake fails as a start failure before exec).

[ ] Step 1: Write the failing tests (a relay head older than relayGiveUp stops touching the marker; a fake cgroupfs parent with a pid in cgroup.procs and no g* child makes ensureParent return ErrLegacyParentPopulated; one with a g1 child still passes).

[ ] Step 2–4: run to fail, implement, run host -run Relay, cgroup -run Parent (unsandboxed); paste.

[ ] Step 5: Commit zdd-readiness: the stage relay gives up after an hour; a populated legacy parent refuses generation placement

---

Task 15: F6 and leftover executable files in /run

Files:

Modify: internal/activator/opset.go:~244-285 (genChildEmpty)
Modify: internal/instancectl/gen.go:~259-300 (sweepGenLeftovers)
Test: internal/activator/opset_test.go, internal/instancectl/gen_test.go

Rules (Astra IMPL-3 #F6 CODEX-FINDINGS-ZDD-IMPL-3.md:44-49; IMPL-2-R2 :45):

F6 (flag off): when KillGen answers errNoGenContainment for a flat exec row's retained failed wake, genChildEmpty requires the host cgroup proven empty (hostKiller.KillHost) before the wake counts as stopped. Before writing it, establish what else can live in carlos-exec@<host> at that moment (co-spawned sidecar members, another live entry). If a live process of another entry can be there, use the non-killing Cleanup.Populated() check instead of KillHost, so the wake stays retained (and its lease kept) until the cgroup empties on its own. If neither is safe, stop and report BLOCKED with what you found; the item is then listed as owed.
/run sweep: sweepGenLeftovers also walks ExeRoot for generation exe directories (<iid>@<n>, parsed with the names helpers) that have no drop-in, are not starting (genStarting), whose unit systemd reports inactive or failed and whose cgroup is empty, and removes them under genMu. Host directories (no @) are never touched.

[ ] Step 1: Write the failing tests (F6: a flag-off retained wake whose leader exited is not released while the host cgroup is populated, and is released once it is empty; sweep: an exe-only leftover is removed, a starting one and a host dir are kept).

[ ] Step 2–4: run to fail, implement, run activator -run 'Unproven|Retained|GenChild' and instancectl -run Sweep (unsandboxed); paste.

[ ] Step 5: Commit zdd-readiness: F6 proves the host cgroup empty for a retained flag-off wake; /run exe leftovers are swept

---

Task 16: Simulation: new invariants, new mutants, one 3000-seed sweep

Files:

Modify: internal/rollout/sim_test.go, internal/rollout/testdata/rolloutsim/mutate.sh:58-63
Modify: the marked lines in internal/rollout/wakeplan.go, failures.go, scheduler.go (add // rolloutsim:Mn markers)

Rules:

Add mutants for this plan's invariants, each marked on exactly one line, each with the invariant that must catch it: - M7 (N7): the marked-build Blocks check in decideHoldRan disabled (if false && hasGood && in.Book.Blocks(). Caught by I1 (no instance boots a build a record holds for it) now that the exemption is gone. - M8 (N6): ClearHoldRanBefore's sequence test disabled. Needs an invariant that a good write decided before a hold build ran never clears that build's marker; add it to the simulation as I5 if the simulation models good writes racing marks, otherwise make M8's kill a named unit test (rule 2). - M9 (retry ordering): preRetryLocked returns false. Invariant: a retry is never undone by an outcome that began before it (I6). - M10 (F5): a failed Begin does not refuse. Invariant: after an injected save failure and a simulated restart, no failed or held build boots on an instance it failed on (I7). Add a fault-injecting book path to the simulation if needed. - M11 (plan review #3): failCohort halts on a PreRetry disposition. Invariant I6 (a retry is never undone) must cover the cohort halt, not only the book. - M12 (plan review #4): markLocked keeps the old Seq for a different execution of the same build. Caught by I5 or by the extended N6 unit test (rule 2).
If a mutant's invariant cannot be modelled in the simulation, extend mutate.sh so a mutant line may name its own test (-run) instead of TestSimInvariants, keep the integrity rules, and say so in the script's header.
Run uptime first (back off above load 12), then once: nice -n 19 internal/rollout/testdata/rolloutsim/mutate.sh 3000 with the sandboxed exports. Want a PASSED baseline and CAUGHT for M1 to M12. Paste the verdict lines.

[ ] Steps: add invariants and mutants, run mutate.sh 300 until clean, commit, run the 3000 sweep, commit any fixes and the evidence in the report.
[ ] Commit zdd-readiness: simulation invariants and mutants for markers, retries and intents

---

Task 17: A host that is starting or stopping is never counted done

Files:

Modify: internal/status/snapshot.go (RolloutHost.Transient), internal/rollout/status.go:~40-95 (Manager.Status)
Modify: the console's rollout view struct and its join (internal/console/apiinstances.go, search apiRollout / the rollout view the list returns), the CLI's apiRollout and classifyRollout (deploywatch.go:~182)
Test: internal/rollout/status_test.go, internal/console/rolloutjoin_test.go, deploywatch_test.go

Interfaces:

Produces: status.RolloutHost.Transient bool \json:"transient,omitempty"\`: the activator owns the host (Owns) but has no current generation for it (Current false), so its process is starting or stopping and the cached serving build says nothing settled. The console view carries it as transient`.

Rules (Astra R3 §5 item 10; plan review #8): spec §9.1 counts a sleeping instance as "installed, awaiting first wake"; a host caught mid-wake or mid-stop is neither asleep nor awake.

Manager.Status asks Owns and Current for each host with no manager or scheduler lock held (the lock order on Manager), and for a transient host reports Transient: true, Awake: false, and no Serving from the stale cache.
classifyRollout returns pending for a transient host, so the watch neither counts it done as asleep nor as swapped.

[ ] Step 1: Write the failing tests: Status reports a host the fake activator owns with no current generation as transient with no serving build; the console list carries "transient": true; the watch keeps a transient host pending through a sweep and completes once it reports asleep or swapped.
[ ] Step 2–4: run to fail, implement, run rollout -run Status, console -run Rollout, root -run TestWatch; paste.
[ ] Step 5: Commit zdd-readiness: a host starting or stopping is reported transient and never counted done

---

Task 18: Roll notes, the owed list, and the copy index

Files:

Modify: docs/superpowers/plans/2026-10-07-zero-downtime-deploys-6-roll-notes.md ("Before Gate E: owed", the claims reply, a new "Claims-on boxes" note)
Create: copy-review/strings.json (only if any TODO-COPY marker exists on the branch)

[ ] Step 1: git grep -n 'TODO-COPY' and build copy-review/strings.json in the copy-review skill's format (id, section, label, text, context, notes), one row per marker, the id matching the marker. If there are none, write none and say so.
[ ] Step 2: Update the roll notes: which #272 items this branch closes (task commit refs filled in by the controller), the claims-on limitation and its reason code, the retry-signing roll order (boxes before consoles; unsigned retries are ignored), the schema-2 downgrade note, and the real-box checks unchanged (D1, the kernel suite, real-unit checks, the flagship probe).
[ ] Step 3: Roll note from #272: platform-infrastructure/scripts/update-platform-binary.sh does not restart carlos-instance-control; use carlos system update (main's system apply restarts it, 0bb57a31), or restart it by hand after the old script.
[ ] Step 4: Commit zdd-readiness: roll notes and copy index

---

Task 19: A killed cgroup is recreated before anything respawns into it

(Added 2026-10-11 from the dev-box run on #272. Execute it before Task 15, which uses KillHost.)

Flag blocker confirmed on the dev box (arm64, kernel 6.12.105, systemd 257): once cgroup.kill has been written to a cgroup, every later clone3(CLONE_INTO_CGROUP) into that same cgroup is SIGKILLed at once. Recreating the cgroup fixes it. TestClaimContainmentCleanupRespawn (internal/activator/claimspawn_kernel_linux_test.go:23) reproduces it 3 of 3 on the box. Exposed: ExecBackend.KillSidecar (internal/activator/genbackend.go:340, the role-sidecar child) and ExecBackend.KillHost (:431, carlos-exec@<host>), which kill a cgroup and keep it. Generation kills remove their cgroup and are safe.

Files:

Modify: internal/cgroup/cleanup_linux.go:~282 (Cleanup.KillAndEmpty) and/or the cgroup Manager's start path (internal/cgroup/cgroup.go:~295-370), internal/cgroup/cleanup_other.go if a signature changes
Modify: internal/activator/genbackend.go:340-370, 431-449 if the fix lives there
Test: internal/cgroup/*_test.go (fake cgroupfs), the kernel-gated internal/activator/claimspawn_kernel_linux_test.go

Rules:

After a cgroup.kill on a cgroup we keep (the host cgroup, a role child), nothing is ever cloned into that same directory again. Before the next spawn into it, the emptied cgroup (and any empty child directories left under it) is removed and created fresh. Choose the place, removal right after the proven-empty kill or recreation at the next start, and say why. Either way, the recreated cgroup gets the same limits and controller settings it had (the ones the start path applies when it creates the cgroup), and it stays under the same parent, so the parent's accounting is unchanged.
Removal must not race a concurrent start into the same cgroup. Use whatever lock the start path already holds for that host, or add one, and document it.
A removal that fails (busy, EBUSY on a cgroup that is somehow repopulated) is an error: the kill is not proven, and the next start refuses rather than clone into a killed cgroup.
Flag off: KillHost is reached with the flag off only through Task 15's F6 path. The new behaviour applies there too; it is an authorized exception (it only replaces a cgroup that was just proven empty).

[ ] Step 1: Write the failing tests: a fake-cgroupfs test that a kill of a kept cgroup followed by a start creates a fresh directory (new inode, or a recorded remove-then-create) with the limit files written again; a test that a failed removal makes the next start refuse; and make TestClaimContainmentCleanupRespawn's kernel-gated path (unchanged gate, CARLOS_CLAIM_SPAWN_KERNEL=1) exercise the real KillHost/KillSidecar then respawn sequence if it does not already. The kernel test cannot run on this build box; say so, and leave it for the dev-box rerun.
[ ] Step 2–4: run to fail, implement, run cgroup and activator -run 'Kill|Respawn|Cleanup|Sidecar' (unsandboxed); paste.
[ ] Step 5: Commit zdd-readiness: a cgroup that took cgroup.kill is recreated before anything respawns into it