carlos / platform Public

Host floor guard: fix these before anyone runs guard-arm (Opus review of Plans 8/8b) open

On branch feat/hostclaim-floor

The Opus authority review (2026-10-07) found Plans 8 and 8b safe to merge as inert: 0 critical, 5 important, 8 minor. I-3 (a wrapped not-found meant the console report was never created on S3) and I-5 (whole binaries read into memory) are fixed in febfa4f0. The rest do nothing while no box is armed, but each must be fixed before carlos system guard-arm runs anywhere.

Arming blockers

  • I-1: an armed box cannot start edge, recover or apply until the floor is published. guard-exec treats an absent remote floor as a refusal (guardexec.go:36-48, hostfloor/remote.go:30-38). Fix: when local mode is armed and both remote objects are absent, accept the local policy. Keep refusing only a half-written pair, a different digest or a malformed object.
  • I-2: the edge drop-in's Requires=carlos-system-recover.service makes every in-apply edge restart wait about 30 s for recover's lock wait (guardarm.go:142, systemapply.go:316-345). That breaks the zero-downtime pickup. Fix: run recover's checks inside guard-exec edge and drop Requires=, or have --recover return immediately when a live apply holds the lock.
  • I-4: a held activation reservation blocks rollback and re-promote, and CancelActivation has no caller (hostactivation.go:107-118, :196-205). Fix: an operator-only cancel route, and ReleaseActivation excluding fenced or quarantined actors per D7.

Minor

  • M-2: fence records are unsigned and written_by is self-asserted. Require an active, unfenced writer. IAM must allow only box roles to put to control/box-fences/, and nothing may delete there.
  • M-3: console reports are unauthenticated. Refuse to overwrite a fresh report from a different process_id.
  • M-4: nothing produces ConsoleProcesses or RetirementReceiptSHA256 yet, so compute-bound consoles and readmission always refuse. That fails closed.
  • M-5: setting CARLOS_CONSOLE_ID on a root-key console enables root-signed ReleaseActivation writes even with the flag off.
  • M-6: flag-off promote and rollback add floor reads and a CAS window. Map ErrCASConflict to 409 on rollback, and consider one retry.
  • M-7: staging is kept after a failed run that keeps its journal.
  • M-8: reserveHostUpdate should check roster.FloorSHA256 == floor.Digest and Publication == "published".

Full report: $TMPDIR/own/opus-8b/review.md on DOCS-T1 (platform-1e's session).

The Opus re-review after merging main (region HA 1b) is safe as inert. M1-1 (a torn region HA marker wedged apply and recover, and the decision was read before the lock) is fixed in the commit after d905565d. New arming blockers to add to the list above:

  • M1-2: nothing creates <DataDir>/.hostexec, but the armed path requires it (hostfloor/participation.go:13-23 via CheckControllersIdle). Every guarded update would be refused.
  • M1-3: guard-arm should refuse while any update-begun.json* exists (guardarm.go:79-81).
  • M1-4: a late participation refusal on the armed path makes endGuard record completion while the box is still on its old image, releasing the reservation. Record completion only when the running image is the target or the transaction rolled back.
  • M1-5: ValidateTerminationFence does not require the writer to be active and unfenced (carried over from M-2).

Rollout checklist additions from the Opus review of claim Plan 5's C5/C7 slice (feat/hostclaim-console ad4d749b, safe to merge as inert). These matter at #232 and claim activation, not before:

  • I2: hostactors.BindCensusKey (censuskey.go) writes the roster without Replace's checkMembership gate, and binds to non-active boxes. Once any key is bound, binaries older than this commit fail strict decoding of control/host-actors.json. Before it gets a caller: gate it on publication and on an active enrollment, and ship it behind the release floor, or move census keys into their own signed object.
  • M5: the claim holder's acq field is strict-decoded. The first claim carrying acq breaks every older flag-on binary for that host. Put it in the floor ordering before #232 writes it.
  • M7: ConvertMovingToRelocating leaves the one-box-region precondition (§1.1) to PreFlipGuard. State it in the guard contract when the caller lands.

Activation blockers from claim Plan 6b (box assembly), from the Opus re-review at fb3ffd98 (safe to merge as inert, 0 Important). All are flag-on only:

  • 6b-N1: registry/claimtransaction.go:47-70. When ROLLBACK or the NORMAL restore fails, the pooled connection is discarded and the replacement opens without main's pragmas: busy_timeout 0 and synchronous FULL. Put the pragmas in the DSN, or re-apply them through a connector hook.
  • 6b-N2: bootstrap.go:375 and systemapply.go:1176-1210. On an enabled box, an unrelated odd line in host.env makes bootstrap and system apply render the flag-off control unit and try-restart it. On a parse error, leave the installed unit alone, as adopt does.
  • 6b-N3: claimconstruction.go:138-152 and hostbox/journalpath.go. The O_NOFOLLOW check is followed by a lazy sql.Open, so the journal open can still be raced. Hand SQLite a /proc/self/fd path, or hold the checked descriptor. Also from claim Plan 5 Tasks 7-9 (Opus, safe as inert):
  • 5-I3: every gated SQL write is its own exclusive app operation, and S3 calls hold the console's single SQLite connection. Ingest writes should join a shared registration, permits should be cached per transaction, and S3 calls should not hold the connection.
  • 5-M4: the release floor must list the new strict-decoded fields: discoverybound, discoveryboxes and requestlimitns.
  • 5-M7: flag-on response shapes (JSON 202 on browser doors, raw error text in 409/503).
  • 5-M10: host-census report puts need no grant; confirm the readers verify signatures, or require a grant.

Activation notes from the Opus review of the Plan 6b x Plan 5 Tasks 7-9 integration (feat/hostclaim-console 348d2132: safe as inert, 0 Important). All four are flag-on only:

  • R1: stop receipts are byte-exact across versions. Add requestlimitns, region, claimetag and observedboot_id to the release floor (with 5-M4's list). Require every participant at or above this build before activation, and forbid downgrade once claims are on, or bump the receipt version.
  • R2: a reset-wake-retry registration's lifecycle grant allows any same-app claim transition (lifecycle_grant.go:169-177). Tie it to a ResetWakeRetry-shaped transition. Not exploitable today.
  • R3: cfg.WakeRetryReset is not constructed in cmd/console (the enabled reset route answers 503), and no ClaimOwnerProvider is registered. Wire both before activation.
  • R4: the agent and adopt stop coordinators have no signer. Confirm that only the root instance-control unit issues stop receipts in the activation topology.

Paul's rulings, 2026-10-08 (given in the platform-1e session). He accepted all six claim Plan 9 decisions as recommended. The first rollout scope is Carloku.

  1. B4 (5-I3), SQL authority: shared ingest registrations plus transaction-scoped permits; no S3 call while holding the SQLite connection; unsettled transactions stay uncertain until proven.
  2. B7 (6b-N3), journals: a root-owned journal service and namespace.
  3. B3 (R3), remote reset: a typed request and receipt through the signed bucket control plane.
  4. A3 (M-2), fence writers: dedicated box-writer roles; the module default stays empty.
  5. A1, pending controller: a root recovery operation that proves the old controller dead, then settles.
  6. C2, order: choose the floor, arm every box, publish once all report ready, then the flag. Each milestone is a separate approval. Plan 9 rev 5 turns these into complete tasks.

A dependency for claim activation, from the carlos run (#259) and seeded-instances (#260, #261) specs on branch design-run-and-seeded-instances (rev 2, 999fc5c2). Both refuse on a box with claim control armed, because RoleLaunch.Start runs no tenant code yet (rolelaunch_linux.go:23-28) and instancectl's legacy routes answer 503 (server.go:840). Once the claim is active, they stay refused until it has two pieces:

  1. A cmd role in the reservation protocol, beside main, seed and sidecar. The cmd unit's producer must be settled before the generation is published (run spec §3.1).
  2. A seed install on cold admission (seeded spec §3.4): - select the remote baseline first; - if there is no baseline and the record carries an unplaced seed, purge the local state and install the snapshot as the admitted generation's content; - publish the generation before the seed is marked placed.

Separately, any controld conversions not finished by activation also wait on item 2.

Paul approved J4, 2026-10-08: moves' declare-lost goes through hostresolve with a typed per-kind owner. hostresolve keeps the challenge, quarantine and loss records; moves supplies the lost participant and what the declaration releases. This is claim Plan 10, with J2 (recovery probes for reverting/restoring) and J17 (the console move grant also covers the resolve door's --retry and declare-lost writes).

carlos run Plan 1 (box foundation) is built on branch carlos-run (plan docs/superpowers/plans/2026-10-08-carlos-run-1-box-foundation.md). For claim activation, the run refusals are now concrete:

  • the box refuses with activator.ErrCmdClaim;
  • the control service's legacy /cmd routes answer 503 (claimAwareLegacy);
  • the console refuses with RUN-C42 (Plan 2).

To lift them, the claim's launch protocol needs three things:

  1. cmd added to the reservation protocol beside main, seed and sidecar.
  2. The cmd unit's producer settled before a generation is published.
  3. A stop that settles the cmd role before replication closes. finishStop does this today, and the claim path needs the same ordering.

Activation blocker from the final reviews of claim Plan 9. Opus and Astra both found it safe to merge as inert.

  • 9-CG (Critical, flag-on): root accepts the edge's supplied Launch.Groups without deriving the permitted cgroup parent from the claim tuple. A compromised authenticated edge can make root's recovery write cgroup.kill on an unrelated cgroup. Assigned to claim Plan 11 (R10): root derives and captures the containment parent and revalidates before any privileged action.
  • 9-MASK (Minor): migration mask recovery must bind to the mask object's identity. Separately, Opus re-reported P-1, a local root escalation on main today (flag-independent): root's registry chown followed symlinks in carlos-owned /data. It is fixed on fix/registry-chown-nofollow and merges first.

Activation blocker status, 2026-10-10.

Closed:

  • 9-CG: fixed by claim Plan 11, on main as 93ef4b50. Root derives and captures the cgroup containment parent itself. The R10 root launch is bound to the host's app executable.
  • 9-MASK: fixed in Plan 11 rev 4; mask recovery is bound to the mask object's identity.
  • P-1 and the rest of #267: on main (29dab2d8, 2d3ff963, 75618480), plus moves' floor gate (00fdf541). Root never reads AWS config the service user could write, and root registry work runs in a privilege-dropped child.

Ready, not merged:

  • Claim Plan 10, moves recovery and grants: at 1208e627 on feat/hostclaim-moves-recovery. Opus and Astra findings are closed and CI is green. The merge waits on Paul's copy review of its 50 draft strings. Two of those are pending moves copy rows (HC5-N2, HC5-N3) that keep moves.DefaultEnabled false.

Still open for activation:

  • the cmd role in the reservation protocol, settling before publication, and a stop that settles cmd before replication closes (carlos run, #259);
  • the seed install on cold admission (#260/#261). Both wait on the carlos-run and seeded-instances branches landing.
  • Moves' Plan 5 owner alignment before the console ClaimMoveRecovery policy is wired. A test in Plan 10 enforces this.

Flag-off deltas so far, for the rollout note:

  • Plan 11: legacy RoleRun start and stop Lstat the claim launch artifacts.
  • Plan 11: in scanFDs, a descriptor that vanishes now reads as closed.
  • #267: root scrubs AWS file, profile, endpoint and trust-store environment variables at start, and adds IMDS to NO_PROXY.

Claim Plan 10, moves recovery and grants, is on main as 44c1b6d7. Paul approved its copy unchanged, including moves rows HC5-N2 and HC5-N3. Claim Plans 1 through 11 are now all on main, inert behind CARLOSHOSTCLAIM. Still open for activation: carlos run's cmd role (#259), the seed install on cold admission (#260/#261), and moves' Plan 5 owner alignment before the console ClaimMoveRecovery policy is wired. The dev ColdRestore check (platform-d8) passed; its WAL-leftover fix is on fix-restore-wal-leftovers. The flag-on dev run (real carlos-instance@ launch, strict kernel and isolation witness) has not been done yet.

Claim status, 2026-10-12. Claim Plans 1 to 13 are all on main, inert behind CARLOSHOSTCLAIM:

  • Plan 12, moves integration completion: 1b5802fb.
  • Plan 13, home-region naming: a0806e4c.
  • Plan 13 follow-ups: 2ad5412c. Paul approved every user-facing string. Each plan had Opus and Astra reviews.

New activation constraints from Plan 13 (also in docs/host-claim-restore-runbook.md):

  1. Every reader of control/host-actors.json must be on a Plan 13 build before the home designation (home_region) is published. Older binaries reject a roster that carries it.
  2. Then publish the designation, which is write-once with claims on.
  3. Then enable CARLOSHOSTCLAIM. Without a designation, a home box runs named-only and logs it.

Still open for activation:

  • carlos run's cmd role (#259) and the seed install (#260/#261), waiting on those branches;
  • moves Plan 5 Tasks 8/9b and Plan 4 Tasks 9/10 (platform-d8);
  • C8, the files-manifest provider (moves);
  • a dev-box run with the flag on: real carlos-instance@ launch, strict kernel and isolation witness;
  • then the release and the Carloku roll, with Paul's approval at each step.

One more claim activation blocker, raised by moves (C8 registration). Establish runs on every claim-on serving wake, but today the files-manifest provider registers only alongside the moves owners. So a box with CARLOSHOSTCLAIM on and moves off refuses every serving wake. The fix: moves builds the C8 provider (ruling M1) plus a RegisterProductionClaimFiles seam that registers it under CARLOSHOSTCLAIM alone, and the claim side reviews it. The claim can't be switched on anywhere until that lands.

Activation checklist addition (from moves Task 7b, the root held release under the claim). In claim mode, instance-control reads CARLOSRELEASEPUBKEY from /etc/carlos/host.env, and root needs it to verify held releases. A box without it refuses every held release, so landings under the claim can never publish. Check that every box's host.env carries CARLOSRELEASEPUBKEY before enabling CARLOSHOSTCLAIM.

New activation blocker: the new thread on root trusting service-writable /data/hostguard and /data/healthy records. Root's claim paths (Plan 11 launch, the held release, adoption) trust agent-writable guard and health state. It must be fixed before CARLOSHOSTCLAIM is enabled anywhere.

Two activation items (2026-10-12):

  1. Files-manifest backfill, a claim-side step owned by me. Under the claim, a host whose chain has no files manifest and no never-published evidence refuses every serving wake (hostfiles.ErrNoEvidence, moves Task 6 C8 provider). Every host that predates the files publisher therefore needs a one-time manifest backfill from its current live files (hostfiles NewFromCopy), on the claim's adoption path, before CARLOSHOSTCLAIM is enabled on its box. Ruled: backfill, not evidence carried on serving claims.
  2. Plan 14 (#280), approved by Paul in full: - claim-only: root-owned fence and installation records, a root lease broker, and an offline migration; - live, without the claim: root-owned D3 health history (old hashes not imported, so the first rollback per app after the roll waits for a fresh observation or a signed proof), a root release-denial ledger, and moves-floor enforcement on both boot selectors.

Activation checklist (moves Task 8): with CARLOSMOVESPHASE1 on, CARLOSSECRETSKEY must be usable on every box. Otherwise the moves wake source can't open and the claim refuses every wake on that box, serving included. That's fail-closed on purpose, because a host may be mid-move.

Claim Plan 15, durable admitted-generation provenance, is on main as 01edf901, inert. The chain starts empty, and retained admissions are never evidence. Activation requirement: every existing host's files must go through the #250 files backfill into an admitted prefix before CARLOSHOSTCLAIM_ORIGINS and moves' Proven(set) are relied on. Until then, those hosts refuse to establish.

Activation order for files provenance (claim-side, I own the backfill):

  1. Run the files backfill for every host on a box: re-copy its live files, platform, env and wake objects into an admitted prefix, and record the admission through the claim so Plan 15's chain proves it.
  2. Only then enable CARLOSHOSTCLAIM_ORIGINS anywhere C8 serves. On boxes without the backfill, moves' Proven(set) refuses every origin, by design (the chain starts empty). The backfill writer is claim work that hasn't been built yet.

Files seed (#250 C8 backfill): Paul's rulings, 2026-10-13

The first plan draft proposed a separate box-wide maintenance job, with workloads stopped until activation. Opus reviewed it and found a smaller design that meets the same requirements. Paul chose:

  1. Timing. Seed every host inside claim activation's existing offline window (Plan 9 milestone 4), with tenants already stopped and both flags set. That costs no extra downtime, and no box sits stopped between seeding and activation. "Before CARLOSHOSTCLAIM is enabled on that box" now reads as "before anything on that box serves under the claim".
  2. Database. The seeding admission is an ordinary, uncertified cold admission. Writes past the remote baseline are declared lost, as at any first claimed wake. Nothing is lost in practice, because the runbook's drained, replicated stop comes first. No certified-predecessor transition and no new trust.
  3. Skipped files. Use the release flush's rule: too-large and non-regular files are recorded as skipped and listed, and unreadable or torn files refuse that host.
  4. Operator path. A box-local step in the activation runbook, safe to rerun, with no console API, remote job or resume verb. A host that fails to seed refuses wakes (ErrNoEvidence) until a rerun succeeds. The box's other hosts serve, and the operator records which hosts are blocked.

Controller rulings (internal):

  • The independent claim writer remains Plan 15's activation-wide gate. It is not a deliverable of the seed.
  • Census bootstrap belongs to Plan 14 Part 3.
  • The preflight covers current manifest versions only, runs with claims off, and reports in plain text.
  • The name is "files seed", distinct from the existing Backfill in #232.

Files seed: controller rulings after the Opus plan review (2026-10-13)

These rulings apply Paul's four rulings above. None widens trust. Paul can override any of them.

  1. Verification. Retiring a serving host's old database on the same box becomes a dependency of the seed. Today the seed's restore leaves a <db>.gen marker, and the next cold restore on that box refuses (restore.go:59-65, claimretire_local.go:190), which would break every seeded host's first wake. The second cold start as verification is replaced by a read-only replay of the proof checks that Establish runs, plus a readback of every referenced object by exact version.
  2. Stragglers. This is what makes ruling 4 true. A serving wake for a host with no files manifest is refused before admission and before any retry attempt is spent. It is a claims-on change to the serving path only. A straggler therefore never exhausts its wake retries, and a rerun of the seed is never blocked.
  3. Failure after commit. A host whose manifest committed but whose verification failed may serve, because its committed manifest is ordinary C8 evidence. Moves are blocked for it until it is repaired. The runbook names how to repair it. No hold is promised that doesn't exist.
  4. Missing files directory. The seed refuses that host rather than seeding an empty tree. An empty tree would erase the host's files under the claim.

Files seed: Paul's rulings after the round-2 Opus review (2026-10-13)

  1. Serving split from moves. The claimed serving path, C8 Materialize included, is built whenever CARLOSHOSTCLAIM is on, independent of CARLOSMOVESPHASE1. The seed and milestone 4's cold-wake check then work with moves still off.
  2. Seed verification. Each seeded host is verified once, before moves are activated, and there is a repair step. Moves keep their existing per-object sha256 checks. There is no extra full readback inside the cut deadline.

Files seed, round 4 rulings (2026-10-14)

  1. Paul: the old local database on serving retirement is parked, then reaped after 7 days. Park it on the same mount as the source; on nested hosts that means inside the /data/instances bind mount, so the rename never crosses mounts (EXDEV). A reaper deletes parked databases older than 7 days. The reaper is root-verified: it only deletes what the retirement parked, identified by root-recorded identity, never a path read from the registry. The disk cost is one database per retirement for a week.
  2. Controller: a crashed seed's retry attempt is settled by the rerun itself. There is no startup recovery on claim boxes. A rerun finds and settles the previous attempt before starting, so the runbook never needs a console reset for this.

Files seed, round 5 rulings, Paul, 2026-10-14. These supersede round 4's "one database per retirement for a week".

  1. Parked databases. Every claimed wake retires the previous generation's local database, so parking must be bounded. - Keep only the latest parked database per host, for up to 7 days. A new park for a host replaces that host's previous park. - Under disk pressure, the root reaper may delete parks early, oldest first, to keep the box above its disk floor (#214). - The reaper remains root-verified: it deletes only what retirement parked.
  2. The legacy database at the first seed is parked once, under the same rule.

Files seed: Paul's ruling, 2026-10-14 (after the round-5 review, which said READY TO BUILD)

Only a generation whose tenant actually started can replace a host's park slot. The parked copy is always the last database a tenant really wrote, and the legacy park from the first seed keeps its full 7 days. A retired generation that never launched (a failed retry, or the seed's own restored copy) is discarded, and it never displaces the park.

Added to the claims activation list from #272 (taken over from the zero-downtime session; Plan 14 owns it).

Blocker: stale host artefacts make re-installation refuse.

  • Leftover consumed/<host> request files, .<host>.lock files and other hosts' records under root's state make the empty-installation census refuse re-installation.
  • Instance-key ruling 24 keeps that census strict and retires host records with the installation. Only root can remove those files, so the fix lies in root's claim lifecycle and carry fence, not in the agent.

Owner: Plan 14 (platform-1e). Scheduled for Part 3 alongside G (census bootstrap). If it's small enough, it goes into E3's census-only retirement.

Must hold before host claims are switched on on any box:

  • A host's retirement removes or retires its consumed request, lock and record atomically with the installation's retirement.
  • Re-installation on a box that once had claims passes the empty-installation census.
  • A root-runner test covers install → retire → re-install.

Scope check for claim activation on Carloku (2026-10-15). The full table is in the controller's scope-check-result.md, from an Astra read-only pass.

Paul's decisions:

  • Positive worker support: deferred past activation. A read-only inventory of the flagship found no instance on any box using workers, sidecars, seeds or carlos run.
  • Named-unit and unit-backed routes: now REQUIRED before activation. eu-west-1 holds 139 of the 141 flagship instances. Among them are 8 named-unit routes (console.service and seven woodstar-* units) and 1 backing=unit route, and the claim cold wake refuses all of these today. This is on track E's list as item 4a.
  • #272 stale-artefact re-installation: deferred past activation. It refuses, so nothing is reused unsafely.

Deferred within existing rulings, because moves stay off:

  • files seed: Tasks 4 and 4R (readback and repair are a gate for moves) and Task 5;
  • the 0R/0A capacity and rehabilitation extras;
  • root-relayed relocation (R-E2a);
  • the moves Plan 4/5 tasks;
  • the moves secrets key.

Dropped: J's whole-registry and fixture conversion.

Still required:

  • E2: D3 verifier (R-E2b now covers every VerifyLaunches caller), containment, timers, installed-launch acceptance; then E3 and E4;
  • Part 3: F2-F4, G1/G2 and H2, plus J1-J4 as narrowed. Part 3 started today on feat/claim-root-trusted-state-p3;
  • the serving core of the files seed;
  • C8 registration;
  • the dev claim-on run;
  • release, floor and IAM certification;
  • the Plan 13 reader → home → enable order;
  • CARLOSRELEASEPUBKEY on boxes.

Claim activation scope, NARROWED (Paul's decision, 2026-10-16). BINDING on every builder.

Source: minimal-boundary-result.md (Astra). Paul chose to narrow the activation gate, and to activate in cohorts: dev and boxes without named units first, the flagship eu-west-1 console box last. This supersedes the parts of the spec rev 6.34 / Plan 14 activation gate listed below as FREEZE or DROP. Those items are finished AFTER activation, not before it.

The boundary that must hold before activation (KEEP)

  1. Root independently authorizes every enabled claim effect (launch, stop completion, cold handover, retirement, publication and replay): the host, installation, session, operation and authenticated controller are bound. Fences, retirement decisions and lease uncertainty are held outside agent control and rechecked at execution and receipt replay.
  2. The whole execution configuration is authorized: executable bytes, the effective unit plus its drop-ins, privileged arguments and environment, identity, mounts, capabilities and containment. Every restart route uses the same reservation boundary.
  3. Root derives and revalidates destructive targets. It stops the recorded containment, retires the recorded installation, and removes only exact root-recorded artifacts. Pending and running generation pins stay independent of registry deletion. (Destructive covers deletion, wrong-target stop, overwrite, rename, chown, mounts and publication.)
  4. A verified, recoverable migration: admission exclusion; drain and stop existing work; old-session denies retained; a real installation census; no imported legacy "allow".
  5. Boxes survive ordinary lifecycle events: seed, cold wake, stop, second wake, reboot, and signed repair or update. Device numbers are boot-local. Floors keep a usable repair executor. Nothing turns routine operation into a permanent, box-wide trap.
  6. Proven at the real boundaries, with hostile service-UID inputs, substitution, durability failures at AUTHORITY TRANSITIONS (fence before effect, reservation before launch, retirement intent before mutation, permission and receipt before success), and permitted positive cases through production constructors. No-census boxes keep exactly main's behaviour.

Per-track decisions

| Work | Decision | |---|---| | 4a named units plus console always-running (boot, adoption, restart, recover) | KEEP, but it gates only the eu-west-1 cohort, not the first activation | | Unit containment compatibility with the REAL console and woodstar services | KEEP (for eu-west-1) | | Containment population on the selected boxes | KEEP | | Device audit of active launch, inventory, retirement and unit bindings | KEEP. Other families: FREEZE | | E3: canonical attribution and the inputs that authorize claim effects; retry and recovery for root denials already shipped | KEEP | | E3: remaining failure/intent producers; exhaustive served/build/completion history | FREEZE. But sticky root state must never block legitimate operation: either complete its delivery and settlement, or gate the sticky state off | | E4: protection of claim-bound units on BOTH convergence paths (fixed root configuration, or refusal) | KEEP | | E4: the full dynamic route --grant workflow | FREEZE | | J4: the five E4 slots as mandatory metadata | DROP | | J4: exhaustive mapping of about 1,100 authority rows | FREEZE | | H2: production boundary witnesses for enabled effects and their shared recovery | KEEP | | H2: the whole matrix as one indivisible gate | FREEZE | | Floors: enforcement plus a usable boot, update and recovery path | KEEP. Further moves and seeds floor machinery: FREEZE | | Historical-record conversion for records that cannot exist at first activation | FREEZE (explicit refusal is enough) | | Files seed: the reboot race (fixed in run 31), FS-R11 (done in run 31), and unverifiable retained parks blocking later retirement | KEEP (the last one is an availability trap) | | Files seed: full readback, repair, sizing and automatic rehabilitation | FREEZE | | #250: C8 provider, admitted provenance, Plan 15 writer, scoped root guard, claim-on launch/isolation/two cold wakes, reader→home→enable order, release key, IAM, scoped rollout | KEEP | | #250: cmd role, workers, sidecars, seeded cold install; moves Plans 4/5; moves secrets; #272 | FREEZE | | Whole-registry conversion; fixtures for unchanged legacy families | DROP |

Census and J4 under the narrowed gate

  • The runtime installation census stays.
  • The source-read census test (TestPlan14RootReadCensus) stays as it is: it freezes changes.
  • J4 (TestPlan14RootInputInventoryCoverage) must check ONLY the families of the ENABLED effects listed in KEEP, plus the exemption list. Every other authority row is reported (t.Log, plus a generated report file), not failed. Do not weaken the check for the enabled families.

Risks to resolve before shipping

  • No named-unit replacement before always-running startup works on the target box.
  • The unit sandbox must be proven against the real console and woodstar services before eu-west-1.
  • No sticky E3 state without delivery and settlement.
  • Retained parks must not block retirement.

Dev cohort-1 activation runbook, draft for review. Branch claim-dev-cohort1-runbook (86faf921), file docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md. Box: carlos-dev-ie-b (leo-climbing); carlos-dev-uk (dennis-stenciling) is an optional second region. Nothing was changed on dev; the prechecks were read-only (SSM, S3 and IAM reads), run 2026-10-16 07:20-07:45Z.

Prechecks today: release pubkey PASS (present on all three dev boxes, same fingerprint). IAM/actor FAIL. Roster readers PASS for Plan 13 (all v0.27.3). Leo quiet PASS. No named-unit routes PASS on registry, records and unit files, but FAIL on the bucket app tree (below). No /var/lib/carlos-claims PASS. Switches PASS (CARLOS_SWAP=overlap, claim switches unset).

Blockers:

  1. No runner on main or the Plan 14 branches creates the roster, invites and enrolls a box, installs host-evidence.key, publishes the floor or publishes home_region. Dev has no control/host-actors.json.
  2. The dev deployment bucket has an empty app segment: 286 objects under apps/bab//errors/leo-climbing/, newest 06:46Z today. hostops.Apps returns ErrIdentity on it, so Part 3's claimActivationVerified and the migration census refuse on every box using that bucket.
  3. Every unit the offline window masks lives in /etc/systemd/system. Runtime masks in /run lose to /etc (the same thing #272 round 3 saw), so verifyOfflineRootMigration's LoadState=masked check can't pass as written.
  4. Leo's role carlos-dev-host-b lacks the Plan 2 version actions. The dev deployment bucket has no bucket policy, and there is no fence-writer role.
  5. Parts 2 and 3 are unmerged, and v0.27.3 has neither them nor the files seed.

Open questions for Paul: (1) wakecheck: proposed it leaves leo via CARLOS_POOLS=cohort1, with a leo-only pooled acceptance app, because both home boxes mint it and DNS sends its traffic to sophie. (2) Include dev-uk? (3) Must flagship be rolled, claims off, before dev's window?

Open questions for platform-1e: (4) the runner for blocker 1; (5) the masking method for blocker 3; (6) which installer renders the claim-on instance-control unit while adopt is masked; (7) is the dev console a claim actor (reset-wake-retry), given that would arm sophie; (8) dev's home_region name, ie?; (9) who removes apps/bab//, which code writes it, and should hostops.Apps skip an empty segment?; (10) the Plan 15 independent-writer boundary on dev, since leo's role can put to control/hosts/; (11) box roles in eu-west-1 can read /carlos/release-key from SSM; (12) after a full rollback, how do claim-era writes reach the claims-off restore path?

Paul's decisions (2026-10-16), after the activation tooling audit. The audit answers the dev runbook's questions 4-12, with code evidence.

1. Dev cohort 1 is a MECHANICS TRIAL.

  • carlos-dev-ie-b (leo) only;
  • a disposable acceptance app;
  • rollback is "stop and hold" (no data-preserving export yet);
  • the independent claim writer (Plan 15) and signing-key rotation are deferred FOR THIS TRIAL ONLY.

Production cohorts still need every gate: the independent writer, a data-preserving rollback export, key isolation and rotation, and IAM exclusivity.

2. Being built now on feat/claim-activation-tools, cut from the integrated Plan 14 branch:

  • the activation runner over the existing hostactors libraries (roster, enrolment, floor before BindCensusKey, home_region ie, RequestLimitNS);
  • protected installation of the evidence key and fence credentials;
  • a persistent-mask transaction for the offline window, with verifyOfflineRootMigration kept strict;
  • a narrow claim-on instance-control unit installer;
  • dev console enrolment as a claim actor (its box keeps claims off).

3. Security finding. Box roles on the flagship and dev can read /carlos/* SSM, which includes /carlos/release-key, the deployment root signing key. An isolation and rotation plan is being written for Paul to review. Nothing is applied without his approval.

4. The empty-app-name writer (apps/bab//) is fixed by platform-d8 on fix-empty-app-key; its census is ruled. The cleanup waits on the roll and Paul's approval.

5. Plan 14 integrated is still in whole-branch review. Round 3's fixes are in, and the production-wired lifecycle end-to-end test is being built. Its prototype exposed a provisioning refusal that is under investigation as a possible production defect.

Cohort-1 runbook updated for Paul's 2026-10-16 decisions: claim-dev-cohort1-runbook at 358f5278, same file. It is now a mechanics trial on leo only, and dev-uk is kept as a later option. Acceptance runs on a disposable app, claimtrial. I kept the CARLOS_POOLS=cohort1 plan (new step 5.0, its own go): without a pool, sophie would mint the disposable app too, and the pool also takes wakecheck off leo. Failure now means "Stop and hold": stop the edge, the timers and paths, and instance-control with its socket, then keep everything as found for diagnosis. The section lists what keeps running (the OS, SSM, sophie and dennis) and what must not be touched (host.env, /var/lib/carlos-claims, the guard and keys, binaries, /data, unit files and masks, the claim's S3 objects; no reboot, reset-failed, reruns or stable-channel roll). The independent writer and key rotation are listed as accepted trial gaps, to be closed before any non-trial cohort. Steps 3b, 3c, 4b and 5 now point at platform-1e's activation runner and persistent-mask installer (both pending), and those include console enrollment. Blocker 2 records fix-empty-app-key, with the apps/bab// cleanup after the fix reaches leo, under Paul's approval. Still open: for Paul, whether to remove the three proof* routes, approval of the pool change, and whether flagship must roll before dev's window. For platform-1e: whether the runner also renders the claim-on instance-control unit, the home_region name (ie?), whether hostops.Apps should skip an empty segment, and how a later teardown treats claim-era writes.

Cohort-1 trial runbook, revision 3: claim-dev-cohort1-runbook at 313562f8. Steps 3 to 7 now quote platform-1e's activation tools exactly, from activation-tools-report.md run 2. That covers the carlos system claims roster and enrollment phases, console enrollment and runtime, the evidence key, fence credentials, install-control, the offline-begin/offline-seed/offline-restore transaction and the reset API call. The tools are on the unmerged branch feat/claim-activation-tools at 0e81652a, under authority review. They replace blocker 1 and blocker 3's runtime masks, and the approved seed strings move to an appendix as reference. The release-key finding is marked resolved. I re-checked it read-only today with policy reads and simulate-principal-policy, without reading or decrypting any parameter: leo and sophie get explicitDeny from deny-release-key-reads; dev-uk is denied for the eu-west-1 key, and no copy of the key exists in eu-west-2. I did not check the flagship half (carloku-infrastructure#11). Paul's answers are recorded. The proof* routes are removing (2026-10-16), and the apps/bab// cleanup waits for the e8e961dd release and Paul's approval. Step 5.0 is approved, to apply near the trial after coordinating with platform-d8. Flagship need not roll before dev's window. Stop and hold now follows the tools' report: no reboot while held until a persistent hold procedure is approved. Open items for platform-1e: whether hostops.Apps should skip an empty app segment, how a later teardown treats writes made under the claim, and that hold procedure.

Blocker 2's source is gone (2026-10-16). The three app-less proof.* routes on leo (carlos-dev-ie-b) were removed at 15:46Z.

  • Origin: they were hand-added on leo with carlos add on 2026-08-23/24 (provisioned=0, no console record).
  • Removal: carlos remove on the box. Only those three rows changed, and wakecheck is byte-identical. The edge reloaded on its own, with no restart and 0 failed units. The hosts now get a 404 "unknown host".
  • The writes have stopped. The last object under apps/bab// was written at 15:46:30Z, and it holds the final pre-removal 502s. None have arrived since, and the count has held at 282 through 15:56Z.
  • Recreating them only needs carlos add --account bab --kind instance --addr 127.0.0.1:8099 <host> for each host.
  • Not touched: the existing apps/bab// objects. Their cleanup still waits for the e8e961dd release and Paul's approval. I also left the old certificate copies under control/certs/proof*.

Plan 14 Parts 2 and 3 plus the claim activation tools are MERGED to main as 2606537 (2026-10-17, Paul's OK). Claims stay OFF everywhere: with no census, every box runs main's exact code.

  • Reviews: whole-branch rounds 1-5, then a narrow re-check ending "MERGE: approve".
  • Tests: the production-wired lifecycle root E2E (real guard-armed edge, an active sibling, three crash points, reboots) found and fixed about 33 production defects. The final root suite passed 120 plain tests and 15 reboot pairs.
  • Census: ruled (census-ruling-plan14-part2, -part3 and -activation-tools).
  • Tools: carlos system claims (inspect, run, evidence-key, fence-credentials, console-config, console-report, install-control) is on main. The automated offline window was removed; leo's window uses the files seed's approved runbook with manual masking.

Next, each step with Paul's approval:

  1. a release and roll, claims off, dev first;
  2. the leo (carlos-dev-ie-b) mechanics trial per platform-d8's runbook.

Before production cohorts: the independent claim writer (Plan 15), a data-preserving rollback export, signing-key rotation and console isolation, and the console-box named-unit work (track E 4a).

Cohort-1 trial runbook refreshed against main (26065378): claim-dev-cohort1-runbook at 64824ffd, with origin/main merged in. The runbook now points at main's carlos system claims (inspect, run, evidence-key, fence-credentials, console-config, console-report, install-control); I checked the flags against main's code and help text. Step 8 is the attended manual window from activation-tools-report run 11, with masks that persist (each of the twelve /etc unit files is moved aside and /dev/null linked in its place) and install-control --dry-run as the verifyOfflineRootMigration check. The approved seed strings are back in the window steps, verbatim, and stop-and-hold is rewritten for the manual window. The order is now: release and roll with claims off (dev first), then the apps/bab// cleanup, IAM, the approved pool and disposable app, and then the leo trial step by step. I re-ran the prechecks read-only today. Leo has only wakecheck (the proof routes are gone). Every box is still on v0.27.3, which lacks 26065378 and e8e961dd. The release-key explicitDeny holds for leo and sophie. Leo's role still lacks the version actions, and dev still has no bucket policy or fence role. 267 objects remain under apps/bab//, with nothing new written since 2026-10-16 15:46. The twelve window units are root-owned files in the expected states. jq is not installed on any dev box, yet steps 5 to 7 use it there. Open for platform-1e: should hostops.Apps skip an empty segment; how a later teardown treats writes made under the claim; whether install-control needs a second run after journal-migrate (run 2 said yes, run 11 runs it once); and jq on the boxes.

Cohort-1 step 5 preparation, nothing run or created: claim-dev-cohort1-runbook at c29518e2.

  • Step 5.0 publishes box identity records with carlos system box publish-record (v0.28.1, dev only), run through systemd-run as the agent user. Leo and sophie each get their own Paul's go, and a rerun on each box is the boxid.Verifier check.
  • A trusted signing host is proposed in docs/superpowers/notes/claim-trial-inputs/signing-host.md: a throwaway carlos-claim-signer instance on the root-runner pattern (t4g.small arm64, the dev Debian AMI, SSM only, egress 443 only). It gets its own role that may read /carlos/release-key; the box-role deny doesn't touch it, and a simulation says allowed. The key goes into a 0600 tmpfs file, and jq and the carlos CLI are checked by sha256. Teardown doubles as the rollback, and the box helpers move to DOCS-T1.
  • Draft inputs are in docs/superpowers/notes/claim-trial-inputs/, with a TO-CONFIRM list.

New blocker 5, for platform-1e: step 6's guard-arm needs an armed policy (the floor plus a root-signed moves-protocol authorization) and a signed v0.28.1 system identity. On main, nothing signs that authorization without publishing it, and PublishFloor needs every actor armed first. PublishSystemIdentity has no production caller, and the dev bucket holds no system identity for any release. Sophie's console also needs a root-owned executable path with a matching .prev for guard-arm --console.

Cohort-1 runbook d8f56ebd (claim-dev-cohort1-runbook): drafted the builder findings from the claim-step6-artifacts @ c38a4ca9 review for Paul after v0.28.2. New step 5.0a (its own go, outside any roll: digests on for bab, then a fresh promote of a console stamped version.Build=v0.28.2, with writers-before-readers checks and a rollback). Step 6 is rewritten around sign-policy, publish-identity --out and install-console at /opt/carlos/system/console/console. Also: publish-identity's verbatim inputs and sources, the signer role gains the apps/bab/console reads and the one identity write, and a trusted console build attestation is owed before production cohorts. Docs only; nothing was run on any box.

Claim-activation blocker from design-console-invite-delivery (census-ruling-console-invite-delivery R5, platform-1e).

The branch gives the console its own email sender for invites (internal/console/consolemail.go). It runs mail.EnsureDomain and mail.VerifyDomain for ops account bab, referrer app console-mail, against the raw release store. With CARLOS_HOST_CLAIM on, that goes around the claim mail gate (s.claimMailStore()).

Routing it through the gate as built would fail. console-mail is not a registered app, so claimMailAdmitter returns hostops.ErrIdentity.

Not a merge blocker: HostClaim is off by default, and ruling 6 holds. But before any console that sends mail runs under claims, one of these must land:

  • register console-mail as a platform identity the admitter knows, so the console mailer goes through the gate; or
  • refuse to start the console mailer when HostClaim is on.

Until then, a console under claims must not have console mail enabled. Concretely: do not set CARLOS_CONSOLE_MAIL_FROM, and with CARLOS_MAIL_PLATFORM_SUFFIX set, no mailer is built only if CARLOS_MAIL_REGION is also unset. platform-1e is keeping console mail off on the dev cohort-1 trial's console (sophie).

Correction to my last reply. Console mail is on by default wherever CARLOS_MAIL_REGION and CARLOS_MAIL_PLATFORM_SUFFIX are both set: the From defaults to noreply@console-mail.bab.<suffix>. So keeping it off by configuration means giving up tenant mail too.

I'm adding the second option on the branch before merge instead: cmd/console will not build the console mailer when HostClaim is on, and will log why. A claims console then has no console mail until the gate route exists. Removing that refusal is the activation item for this thread.

Dev cohort-1 signing session 2 (v0.28.4), 2026-10-19: stopped at 6.2. The session is closed and CloudTrail is clean. The runbook record is claim-dev-cohort1-runbook @ 1ddc6185.

Passed:

  • 4: opened the signer.
  • 5: refresh, rev 5.
  • 6.1: sign-policy and publish-identity. control/system/router/releases/v0.28.4.json was published at 09:17:58Z and is permanent.
  • 6.1a: install-channel --policy --identity on leo and sophie. Both now have carlos and carlos.prev at v0.28.4.

Stopped at 6.2: guard-arm on leo returned hostfloor: authenticated floor required and wrote nothing.

  • Cause: guard-arm's OpenProtected requires root-owned parent directories, but /opt/carlos is carlos:carlos on both dev boxes. The source is chown -R carlos:carlos /data /opt/carlos in platform-infrastructure dev/host-init.sh:10, and flagship's host-init has the same line.
  • Session 1: its 6.2 refused with the same message, and the ownership check runs first. That refusal was probably this too.
  • Inputs: a local replay of guard-arm's input checks against f3bf1214 passes.

Held:

  • roster rev 5;
  • no floor;
  • the v0.28.4 identity;
  • the v0.28.4 policy and identity staged on both boxes.

Teardown: the key was shredded and the signer terminated. The exchange, role, profile and SG are deleted. CloudTrail shows one release-key read, from the signer.

Ruling needed from platform-1e and Paul: change host-init ownership on the boxes, or change guard-arm's rule. Then a third signing session with Paul's go. That session can reuse the published identity and skip 6.1a.

Rollback slot after session 2. Step 6.1a put the v0.28.4 bytes (736826e4) in both /opt/carlos/carlos and carlos.prev on leo and sophie. Neither box can roll back to v0.28.3 from its own slot.

The recovery path is to re-fetch from the channel. Promote v0.28.3 on that box's system channel (stable for leo, canary for sophie) with carlos system update. The box's update path then downloads and verifies bootstrap/router/v0.28.3/carlos-linux-arm64 (6154afb4…). Never copy a binary in by hand.

This is recorded in the runbook's "Stop and hold" section (claim-dev-cohort1-runbook @ 349b4acc). Everything else stays held while platform-1e's /opt/carlos ownership check runs and Paul rules.

Claim trial: overnight plan, Option B (platform-1e, 2026-10-20 00:30Z). Paul asked for a Fable review and the fastest robust path to ship by morning, run unattended with the moves session.

Fable's review is at after236/fable-plan-review.md on DOCS-T1. Verdict: rev 3 of the cohort design is right in general but far too big for tonight. Two facts remove most of it for the dev trial:

  • the cohort is empty: claimtrial is deleted before the window;
  • the dev console stays claims-off through the window, so the record freeze is never read.

The trial then needs ONE new signer phase, backfill-done (collecting to done). It refuses any census report that lists an installation.

Overnight (D1, the reversible default): platform-1e builds backfill-done, its tests, an empty-leo root-runner witness, census rulings and an Astra review, on branch claim-empty-cohort-done (from the parked claim-step8-readiness). platform-d8 does read-only prechecks, runbook delta v2 and the window handoff blocks. Nothing writes to a box, bucket or release channel.

Paul's morning decisions (recommended answers):

  • D2: delete claimtrial through the ordinary door with the console off, and move its and wakecheck's leftovers aside on leo. Yes.
  • D3: cut v0.28.5 dev-only. Yes.
  • D4: signing session 6: P-B2 grants, publish-identity v0.28.5, collect, done. Yes.
  • D5: accept four trial assumptions. Yes. - excluded-box completeness and stability, by read-only precheck; - #312 handled by ordering and retry; - pool-less creates refused while the dev console is in claim mode; - claimtrial2 can't be deleted until sophie's cohort.
  • D6: gos for the window to done, for step 9, for B4 plus acceptance, and for 10.5 and 10.6. Yes.
  • D7: copy review of the branch strings plus the one new refusal. It gates the merge.
  • D8: lift the moves freeze for the offline operator move only, after a dev rehearsal. Tony's three flagship moves, one at a time, on Paul's go.
  • D9: after acceptance, switch the dev console's claim mode off again. Yes.
  • D10: roll v0.28.5 through stable, which also moves dennis. Yes.
  • D11: if apps/bab// isn't empty, clean it before the window. Yes.

There are about four hours of morning steps, listed in Fable's section 5.

Update to D2 (platform-1e, 2026-10-20 01:00Z): d8's read-only precheck found that every dev instance record (claimtrial, wakecheck, rollcheck-0261) was created claims-off and has no acquisition. backfill-collect refuses records like that, and nothing on main can stamp one; only claim-mode creates mint it. The ruling is a zero-host plan.

D2 (Paul, morning): delete all three dev records through the ordinary door with the console claims-off, remove their stale rows (including wakecheck's 4 routes on sophie), and move the leftovers aside. wakecheck, the standing dev wake check, is recreated after the trial once the console is claims-off again (D9). rollcheck-0261's owner is being asked.

D2 detail: rollcheck-0261 is the v0.26.1 roll-check probe from 2026-10-13, a static app in account bab. It's deleted under D2 along with claimtrial and wakecheck.

D2 addition: the offline-move rehearsal app runs before the window, a new claims-off app moving from sophie to dennis. B1 also deletes it, so 8.1b still expects 0 hosts.

Copy approved (Paul, 2026-10-20): the 10 strings for claim-empty-cohort-done, verbatim. Every one matches the branch's code and docs. D7 is done. The branch still needs Astra's review, which is running, and the harness run, which platform-d8 is building, before it merges.