amadan
docs: cohort-1 runbook: R1 ruled, undo before 8.2's host.env edit; window waits for v0.28.5
Paul ruled R1: a window that stops before 8.2's host.env edit is fully undone (units and edge restored, enablement restored, barrier lifted); from that edit on it is stop-and-hold. The window package's section 5 gains case B1 with the exact commands (checks, move the units back from held and remove the /dev/null links, daemon-reload, restore enablement, start the edge and triggers, re-authorize 80 and 443), and step 8 and "Stop and hold" say the same. S1, S2 and journal-migrate's chmod are being fixed by platform-1e for v0.28.5; the window waits for that release and Paul's go.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+90
−12
docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md+90 −12
CI cancelled — superseded by a newer push — run details
CI log
=== gate === cd launchctx && go vet ./... && go test ./... -count=1 ok amadan.net/carlos/platform.git/launchctx 0.014s ? amadan.net/carlos/platform.git/launchctx/controlname [no test files] ? amadan.net/carlos/platform.git/launchctx/wireid [no test files] go vet ./... gofmt -l .: clean go test ./... -count=1 -timeout 30m amadan: job cancelled — a newer push superseded it
diff --git a/docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md b/docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md| index fe7dbf274..9d6834992 100644 |
| --- a/docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md |
| +++ b/docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md |
| @@ -8,6 +8,8 @@ Prepared 2026-10-19 for platform-1e, who takes it to Paul. Every check below was |
| **Verdict: not ready for a go.** Two findings in the code would stop the window at 8.2 ("STOPs before the go", below). Each needs platform-1e's answer before Paul is asked. The barrier, the deadline, the gates and the umask changes are ready. |
| +**Waiting on v0.28.5 (2026-10-19).** platform-1e is fixing S1, S2 and `journal-migrate`'s missing chmod (the `guard/moves` and `guard/seeds` directories in section 4) for v0.28.5. The window waits for that release and for Paul's go. Leo is on v0.28.4 until then. Every code reference in this package is to `f3bf1214` and is re-checked against v0.28.5 before the go. |
| + |
| ### STOPs before the go |
| **S1. `journal-migrate` needs a signed backfill plan that nothing ships to write.** `carlos system journal-migrate` always runs the census path (`RunSystemJournalMigrate` passes `census=true`, `internal/host/claimjournal_migration.go:61`). Its census collector starts by reading `control/hosts-backfill.json` (`hostops.BackfillKey`) and refuses unless that object is root-signed with the release key, at `phase` `collecting`, with a `hosts` map that freezes every instance record in the deployment by key and etag, and `directory_revision` equal to the roster's (`rootMigrationCollector.collect`, `internal/host/rootmigration_census.go:37-75`). On dev the key is 404 (read 20:40Z). At `f3bf1214` nothing outside tests writes a `collecting` plan: the only non-test uses of `"collecting"` are the two readers, the root tests sign the plan with a fixture key (`rootmigration_bootstrap_root_linux_test.go:165-171`), and `claims run` has no such phase (its phases are `initialize`, `configure`, `home`, `invite-box`, `enroll-box`, `invite-console`, `enroll-console`, `bind-key`, `publish-floor`, `sign-policy` and `publish-identity`). The next stage is missing too: `Backfill.Publish`, which moves the plan to `adopting`, has no production caller (its comment names "the #232 coordinator"). The console adopts claims only while the plan reads `adopting` (`ReconcileClaims`, `internal/hostops/backfill_controller.go`), and `control/hosts/` on dev is empty. So as tooled, 8.2's `journal-migrate` fails in its first observation, before it writes `migration.json`. And even with the plan in place, nothing would give `claimtrial` the claim the seed's cold admission needs. The plan would have to freeze all three records in the bucket: `claimtrial`, `rollcheck-0261` and `wakecheck`. platform-1e: is a backfill writer and adopter owed (a `claims run` phase is a signing session, with the release key on a host again), or does the trial reach the census another way? |
| @@ -19,7 +21,7 @@ Prepared 2026-10-19 for platform-1e, who takes it to Paul. Every check below was |
| A refusal there leaves `migration.json` at phase `recovered`, an unfinished migration, with leo held and masked. No witness covers this: the migration root tests (`TestRootRebootPlan14Migration*`, `TestRootRebootPlan14MigrationInstalled*`) use scratch roots, never a box-shaped tree with the window's masks. platform-1e: confirm or correct this reading, ideally with one root-runner run of 8.1 then `journal-migrate` on a copy of leo's tree. Then rule on the fix: census rules, a different mask, and whether the `wakecheck` leftovers on leo (`/data/instances/wakecheck…`, `/data/.instances/wakecheck…`, `/opt/carlos/live/wakecheck…`, `/data/programs/wakecheck…json`) are cleaned up before the window, and by whom. |
| -**Ruling also wanted (R1).** May a window that stops before 8.2's `host.env` edit be undone, rather than held? Until then claims are still off and nothing claim-mode has run: put the twelve unit files back from `$WINDOW/held`, re-enable and start the edge and the triggers, and lift the barrier. The runbook says hold from 8.1 on. Given S1 and S2, an 8.1-only undo is worth deciding before the go. |
| +**Ruled (R1), Paul, 2026-10-19.** A window that stops before 8.2's `host.env` edit is fully undone: the units and the edge are restored, enablement is restored, and the barrier is lifted. Until that edit, claims are still off and nothing claim-mode has run. From 8.2's `host.env` edit onwards, a stop is stop-and-hold, as before. The commands are in section 5, case B1. |
| ### 1. Window ID and deadline |
| @@ -60,7 +62,7 @@ systemctl show carlos-edge.service carlos-instance-control.service -p Id -p Load |
| 1. If the seed override is in place, the watcher captures the invocation ID first, then stops the one-shot and saves its result and journal under `$WINDOW` (`[runbook.cancellation]`). Otherwise it stops nothing. |
| 2. If `journal-migrate` is still running, the primary operator interrupts it (Ctrl-C, its documented cancel: SIGTERM or interrupt cancel the import). If the primary session is gone, the holder sends `kill -TERM <pid>` to the PID the watcher printed. Never `pkill -f`. |
| 3. The primary runs 8.4's first block (remask the edge, remove the override, put the guard drop-in back), or, if 8.3 had not started, nothing. If the primary session is gone, the holder runs that block, with `INVOCATION=$(cat "$WINDOW/deadline-invocation")` and the same `WINDOW`, `SEED` and `GUARD` values. |
| -4. Leo is then in "Stop and hold" with the edge persistently masked: step 9's hold state, which survives a reboot. The barrier stays on (section 5). Evidence is captured and posted on #250. No block starts after the deadline. Anything further needs a new go. |
| +4. Leo is then in "Stop and hold" with the edge persistently masked: step 9's hold state, which survives a reboot. The barrier stays on (section 5). Evidence is captured and posted on #250. No block starts after the deadline. Anything further needs a new go. The exception is a window that never reached 8.2's `host.env` edit: that one is undone (section 5, case B1, ruling R1), not held. |
| ### 2. External traffic barrier |
| @@ -197,14 +199,86 @@ Changes now written into the blocks: |
| `umask 022` loosens nothing private here: every root-only path above sets its mode explicitly. The one exception is `journals.db`, which becomes `0644` inside the `0700` `/var/lib/carlos-claims`, so it stays unreachable. Aside, not umask: `journal-migrate` resets `/data/.hostexec` from `guard-arm`'s `2770` to `0700 carlos:carlos`. Both work for the agent; the two pieces of code disagree. |
| -### 5. Rollback: stop and hold, as commands |
| +### 5. Rollback: undo before 8.2's `host.env` edit, stop and hold from it, as commands |
| -This is step 9's stop-and-hold path ("Stop and hold"), laid out by where the window stopped. Stopping is always allowed. Run it in the primary root session on leo, which keeps `WINDOW`, `SEED`, `GUARD`, `INVOCATION`, `UNITS` and `TRIGGERS` from 8.1 and 8.3. If that session is gone, set them again in the second session from the window ID and 8.1 and 8.3's literal values, with `set -eu`, `umask 077` and `pub() ( umask 022; "$@" )` first. |
| +Paul's ruling R1 (2026-10-19) splits the window in two. A stop before 8.2's `host.env` edit is fully undone (case B1): units and edge restored, enablement restored, barrier lifted. From that edit onwards a stop is step 9's stop-and-hold path ("Stop and hold"), cases B2 to E. Stopping is always allowed. Run it in the primary root session on leo, which keeps `WINDOW`, `SEED`, `GUARD`, `INVOCATION`, `UNITS` and `TRIGGERS` from 8.1 and 8.3. If that session is gone, set them again in the second session from the window ID and 8.1 and 8.3's literal values, with `set -eu`, `umask 077` and `pub() ( umask 022; "$@" )` first. |
| **A. Barrier on, 8.1 not started (abort).** Leo is unchanged. Lift the barrier (section 2, "Reverse"). Done. |
| -**B. From 8.1's first `systemctl stop` until the seed override exists (8.1, 8.2).** |
| -- If the stop came before 8.1's mask loop finished, finish that loop first. The triggers are still enabled, so without their masks a reboot would start them: |
| +**B1. From 8.1's first `systemctl stop` until 8.2 changes `host.env`: undo (R1).** That is all of 8.1, including a mask loop that stopped part-way and 8.1's closing `install-control --dry-run`, which creates nothing, up to the moment 8.2's `host.env` edit begins. 8.2's own dry run comes after that edit, so it falls in B2. Claims are still off and nothing claim-mode has run. |
| + |
| +First prove that 8.2's edit has not happened and that nothing claim-mode has written anything. If either check fails, this is B2, not B1: hold. |
| + |
| +```bash |
| +cmp -- /etc/carlos/host.env "$WINDOW/host.env.before" |
| +test ! -e /var/lib/carlos-claims |
| +``` |
| + |
| +Put the twelve unit files back from `$WINDOW/held` and remove the `/dev/null` links. Each unit is handled in whichever state the stop left it: masked (a `/dev/null` link), moved but not yet linked (absent), or never moved (still the original file). Every original is checked against its saved copy before it moves: |
| + |
| +```bash |
| +for unit in "${UNITS[@]}"; do |
| + path="/etc/systemd/system/$unit" |
| + if [ -L "$path" ]; then |
| + test "$(readlink -- "$path")" = /dev/null |
| + cmp -- "$WINDOW/held/$unit" "$WINDOW/$unit.before" |
| + rm -- "$path" |
| + mv -T -- "$WINDOW/held/$unit" "$path" |
| + elif [ ! -e "$path" ]; then |
| + cmp -- "$WINDOW/held/$unit" "$WINDOW/$unit.before" |
| + mv -T -- "$WINDOW/held/$unit" "$path" |
| + fi |
| + test -f "$path" |
| + test ! -L "$path" |
| + test "$(stat -c '%u %a' -- "$path")" = '0 644' |
| + cmp -- "$path" "$WINDOW/$unit.before" |
| + test ! -e "$WINDOW/held/$unit" |
| +done |
| +systemctl daemon-reload |
| +systemctl show "${UNITS[@]}" -p Id -p LoadState -p UnitFileState |
| +``` |
| + |
| +Expected: every unit `LoadState=loaded`, none `masked`. 8.1 disabled only the edge. The masks hid the other units' wants links but did not remove them, so they are enabled again as soon as their files are back. |
| + |
| +Restore the edge's saved enablement, check every unit's enablement against the baseline, then start the edge and the five triggers. The six services were inactive at baseline, and the triggers start them as before, so the services are not started by hand: |
| + |
| +```bash |
| +if grep -qx 'UnitFileState=enabled' "$WINDOW/carlos-edge.service.state"; then |
| + systemctl enable carlos-edge.service |
| +fi |
| +for unit in "${UNITS[@]}"; do |
| + test "$(systemctl show "$unit" -p UnitFileState)" = "$(grep '^UnitFileState=' "$WINDOW/$unit.state")" |
| +done |
| +systemctl start carlos-edge.service |
| +systemctl is-active carlos-edge.service |
| +systemctl start "${TRIGGERS[@]}" |
| +for unit in "${UNITS[@]}"; do |
| + diff -- <(systemctl show "$unit" -p LoadState -p ActiveState -p UnitFileState) "$WINDOW/$unit.state" || echo "DIFFERS: $unit" |
| +done |
| +ss -ltn '( sport = :80 or sport = :443 )' |
| +``` |
| + |
| +Expected: no `DIFFERS` line, so the baseline from P8 is back: the edge and the five triggers `loaded active enabled`, and the six services `loaded inactive`. A service that a trigger has just fired can show `active` or `activating` for a moment, so re-run the `diff` loop after a minute before reading anything into it. The edge listens on 80 and 443. The edge's guard drop-in (`90-host-floor.conf`) never moved in 8.1, so the edge starts through `guard-exec` as before. |
| + |
| +Then lift the barrier from DOCS-T1 with section 2's "Reverse". The two inbound rules the barrier revoked, `sgr-0d3da6656deec14b0` (TCP 443 from `0.0.0.0/0`) and `sgr-032478c46083d5074` (TCP 80 from `0.0.0.0/0`), cannot be brought back by ID. Re-authorizing ports 80 and 443 makes two new rules with new IDs, in the shape the `region-host` module declares: |
| + |
| +```bash |
| +R="--profile kass --region eu-west-1" |
| +aws $R ec2 authorize-security-group-ingress --group-id sg-054aa165db842b6ba --ip-permissions \ |
| + 'IpProtocol=tcp,FromPort=80,ToPort=80,IpRanges=[{CidrIp=0.0.0.0/0}]' \ |
| + 'IpProtocol=tcp,FromPort=443,ToPort=443,IpRanges=[{CidrIp=0.0.0.0/0}]' |
| +aws $R ec2 describe-security-group-rules --filters Name=group-id,Values=sg-054aa165db842b6ba \ |
| + --query 'SecurityGroupRules[].[IsEgress,IpProtocol,FromPort,ToPort,CidrIpv4]' --output text |
| +``` |
| + |
| +Expected: TCP 80 and TCP 443 inbound from `0.0.0.0/0`, plus the egress rule `True -1 -1 -1 0.0.0.0/0`. Then, outside the sandbox, section 2's bare TCP connect prints `80 OPEN` and `443 OPEN`. Read it the other way round from the barrier check: here open is the expected result. |
| + |
| +Leave `$WINDOW` in place as the window's record. `held/` is empty now, and nothing under `/etc/carlos` is in systemd's search path, so it changes nothing. Capture the evidence below and post it on #250. A later window needs a new window ID (8.1's `mkdir` refuses an existing `$WINDOW`) and a new go. |
| + |
| +If any `cmp` or `test` in B1 fails, stop there. Move nothing more, start nothing, and report on #250 with the evidence. A unit already moved back stays back, and one still held stays masked. |
| + |
| +**B2. From 8.2's `host.env` edit until the seed override exists (8.2): hold.** |
| +- If the mask loop somehow had not finished (it must have, before 8.2 starts), finish that loop first. The triggers are still enabled, so without their masks a reboot would start them: |
| ```bash |
| for unit in "${UNITS[@]}"; do |
| @@ -217,7 +291,7 @@ This is step 9's stop-and-hold path ("Stop and hold"), laid out by where the win |
| systemctl show "${UNITS[@]}" -p Id -p LoadState -p ActiveState |
| ``` |
| -- Then leave every mask, `$WINDOW` and `carlos-instance-control.service` as they are (subject to ruling R1). |
| +- Then leave every mask, `$WINDOW`, `host.env` (edited or not) and `carlos-instance-control.service` as they are. Do not put `host.env.before` back by hand: once 8.2 has touched `host.env`, R1's undo no longer applies. |
| **C. The seed override is in place (8.3).** This is the fragile state (H6): |
| @@ -247,7 +321,7 @@ That is 8.4's first block, with the invocation captured before the stop. Do not |
| **E. Steps 9 and 10.** As "Stop and hold" says: `systemctl stop carlos-edge.service`, stop the five triggers, then `cmp`, move the edge unit to `$WINDOW/held`, link `/dev/null`, `daemon-reload` and check `LoadState=masked`. |
| -**Restoring the edge.** In every case the hold leaves the edge persistently masked: a `/dev/null` link at `/etc/systemd/system/carlos-edge.service`, the original in `$WINDOW/held`, its guard drop-in back at `90-host-floor.conf`, and the seed override gone. That state survives a reboot. Bringing the edge back to serving is step 9's block, and it needs its own go. |
| +**Restoring the edge.** B1 restores the edge and its enablement itself, above. In every hold case (B2 to E) the hold leaves the edge persistently masked: a `/dev/null` link at `/etc/systemd/system/carlos-edge.service`, the original in `$WINDOW/held`, its guard drop-in back at `90-host-floor.conf`, and the seed override gone. That state survives a reboot. Bringing the edge back to serving is step 9's block, and it needs its own go. |
| **Evidence, every case.** On leo, read-only (the "Stop and hold" capture), then post the summary on #250: |
| @@ -259,7 +333,7 @@ cat /var/lib/carlos-claims/authority/migration/migration.json 2>&1 | head -c 400 |
| cat /proc/sys/kernel/random/boot_id |
| ``` |
| -**Lifting the barrier.** The runbook keeps it through any hold, and that stays the default: the barrier comes off only in step 9, after its serving check, with step 9's go. If Paul wants leo's group back to its declared state while leo is held, the precondition, read on leo, is `systemctl show carlos-edge.service -p LoadState --value` printing `masked`, and `ss -ltn '( sport = :80 or sport = :443 )' | sed 1d | wc -l` printing `0`. Then run section 2's "Reverse" and check that the two inbound rules are back. With the edge masked nothing listens on 80 or 443, so lifting it then exposes nothing. |
| +**Lifting the barrier.** Cases A and B1 lift it as their last step. The runbook keeps it through any hold (B2 to E), and that stays the default: the barrier comes off only in step 9, after its serving check, with step 9's go. If Paul wants leo's group back to its declared state while leo is held, the precondition, read on leo, is `systemctl show carlos-edge.service -p LoadState --value` printing `masked`, and `ss -ltn '( sport = :80 or sport = :443 )' | sed 1d | wc -l` printing `0`. Then run section 2's "Reverse" and check that the two inbound rules are back. With the edge masked nothing listens on 80 or 443, so lifting it then exposes nothing. |
| **What must not be touched while leo is held.** Everything in "Stop and hold"'s list: `$WINDOW` and `held/`, `host.env`, `/root/claim-trial`, the mask links, unit files and drop-ins, `/var/lib/carlos-claims`, `host-actor.json`, `host-evidence.key`, `fence-writer/`, the guard, both `carlos` slots, `/data`, sophie's protected console, the identities, the roster, floor and enrollments, and the claim's S3 objects. No `reset-failed` before results are saved, no reboot unless the edge is persistently masked, and no rerun of `journal-migrate`, `install-control` or the seed. Added for the window: |
| - `sg-054aa165db842b6ba` changes only through section 2's two commands; |
| @@ -2281,7 +2355,9 @@ Before the window: |
| - any update or recovery on leo is settled; genuine pending update evidence is kept; |
| - the named-unit preflight still holds: leo has no named-unit or `backing=unit` route (P5). |
| -The window ID, deadline, barrier, gate evidence, umask audit and hold commands are in "Window package (for Paul)" at the top of this file. Its STOPs S1 and S2 must be cleared before the go. |
| +The window ID, deadline, barrier, gate evidence, umask audit and hold commands are in "Window package (for Paul)" at the top of this file. Its STOPs S1 and S2 must be cleared before the go. platform-1e is fixing S1, S2 and `journal-migrate`'s chmod for v0.28.5, and the window waits for that release and Paul's go. |
| + |
| +Rollback (Paul's ruling R1, 2026-10-19): a window that stops before 8.2's `host.env` edit is fully undone. The units and the edge are restored, enablement is restored and the barrier is lifted (window package, section 5, case B1). From 8.2's `host.env` edit onwards a stop is stop-and-hold ("Stop and hold", and section 5's cases B2 to E). |
| The approved description of the step: |
| @@ -2747,13 +2823,15 @@ A 200 acknowledges the request; it does not prove a wake. Check the new retry ep |
| **Rollback slot after session 2 (2026-10-19).** 6.1a's `install-channel --policy --identity` put the v0.28.4 bytes (`736826e4`) into both `/opt/carlos/carlos` and `/opt/carlos/carlos.prev` on leo and sophie. Neither box can roll back to v0.28.3 from its own slot. To recover v0.28.3 on a box, fetch it again from the channel. Promote v0.28.3 on that box's system channel, `stable` for leo or `canary` for sophie, with `carlos system update`. The box's update path then downloads and verifies the v0.28.3 router from the bucket (`bootstrap/router/v0.28.3/carlos-linux-arm64`, sha256 `6154afb4…`) and stages it. Do not copy a binary in by hand. |
| -This replaces a claims-off rollback for the trial. If anything from step 5 on fails, or acceptance fails, stop and leave leo as it is for diagnosis. Nothing on leo is undone, repaired or retried until platform-1e has looked and Paul has decided what comes next. Do not unset the claim flags, unbind the census, delete records or guards, or attempt a data-preserving rollback. Only `claimtrial` is affected, and it holds nothing of value. |
| +This replaces a claims-off rollback for the trial. If anything from step 5 on fails, or acceptance fails, stop and leave leo as it is for diagnosis. The one exception is step 8 stopped before 8.2's `host.env` edit, which is undone (ruling R1, below). Nothing on leo is undone, repaired or retried until platform-1e has looked and Paul has decided what comes next. Do not unset the claim flags, unbind the census, delete records or guards, or attempt a data-preserving rollback. Only `claimtrial` is affected, and it holds nothing of value. |
| Paul's go: not needed to stop. Stopping is always allowed. Anything after it needs a go. |
| **Inside the signing session (5.1 to 7).** Nothing claim-mode runs on leo yet. A stop ends the session: close it at once ("Closing the signing session": shred the key, terminate the signer, delete its role, profile and security group, run the CloudTrail check). Leave leo, sophie, the roster, the floor and the bucket as they are; leo keeps serving claims off. Resuming needs Paul's go for a new signing session, which would put the release key on a host a second time. |
| -**Step 8, before the seed starts (8.1, 8.2).** The six services, five triggers and the edge are already persistently masked. Leave every mask and `$WINDOW` as they are. Leave instance-control as it is (running if 8.2 started it). |
| +**Step 8, before 8.2's `host.env` edit (8.1): undo.** Paul's ruling R1 (2026-10-19). Claims are still off and nothing claim-mode has run, so the window is fully undone, not held. Check that `host.env` still matches `$WINDOW/host.env.before` and that `/var/lib/carlos-claims` does not exist. Then put the twelve unit files back from `$WINDOW/held` and remove the `/dev/null` links, `daemon-reload`, restore the edge's enablement, start the edge and the five triggers, and lift the barrier by re-authorizing ports 80 and 443 on `sg-054aa165db842b6ba`. The exact commands are in the window package, section 5, case B1. If either check fails, it is the next case: hold. |
| + |
| +**Step 8, from 8.2's `host.env` edit until the seed starts (8.2): hold.** The six services, five triggers and the edge are already persistently masked. Leave every mask, `$WINDOW` and `host.env` as they are. Leave instance-control as it is (running if 8.2 started it). |
| **Step 8, while the seed override is in place (8.3).** This is the one fragile state: the edge unit file is back in `/etc`, its guard drop-in is in `$WINDOW/held`, and the seed override lives in `/run`, which a reboot clears. Return to the masked state: |
| 1. At the deadline or on failure, capture `InvocationID` before stopping, as `[runbook.cancellation]` says. |