amadan
docs: cohort-1 trial runbook uses the activation tools; IAM key read resolved
- Steps 3 to 7 now quote platform-1e's activation tools exactly (carlos system claims inspect/run/console-config/console-report/ evidence-key/fence-credentials, the offline-begin/offline-seed/ offline-restore transaction, install-control, console enrollment and the reset API call), from activation-tools-report.md run 2. They are on the unmerged branch feat/claim-activation-tools at 0e81652a, under authority review. The tools replace blocker 1 and blocker 3's runtime masks. The approved seed strings move to an appendix as reference. - P2: box roles can no longer read /carlos/release-key (platform- infrastructure#44). Re-checked read-only by policy read and simulate-principal-policy: explicitDeny for leo and sophie; dev-uk denied for the eu-west-1 key, no key in eu-west-2. Flagship half not checked from here. - Paul's answers: proof* routes removing (2026-10-16), apps/bab// cleanup waits for the e8e961dd release and Paul's approval; step 5.0 approved, applied near the trial after platform-d8 coordination; flagship need not roll before dev's window (question removed). - Stop and hold follows the tools' report: before offline-restore the transaction's persistent mask holds; after it, stop the edge, keep instance-control unless it is failing, and no reboot until a persistent hold procedure is approved.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+381
−229
docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md+381 −229
CI passed — run details
CI log
=== gate ===
cd launchctx && go vet ./... && go test ./... -count=1
ok amadan.net/carlos/platform.git/launchctx 0.011s
? amadan.net/carlos/platform.git/launchctx/controlname [no test files]
? amadan.net/carlos/platform.git/launchctx/wireid [no test files]
go vet ./...
gofmt -l .: clean
go test ./... -count=1 -timeout 30m
ok github.com/carlosframework/platform 304.659s
? github.com/carlosframework/platform/cmd/changelog [no test files]
ok github.com/carlosframework/platform/cmd/console 4.324s
ok github.com/carlosframework/platform/cmd/moveprobe 93.196s
ok github.com/carlosframework/platform/cmd/zddprobe 0.702s
ok github.com/carlosframework/platform/internal/activator 249.406s
ok github.com/carlosframework/platform/internal/agentsock 0.016s
ok github.com/carlosframework/platform/internal/alerting 0.004s
ok github.com/carlosframework/platform/internal/analytics 0.020s
ok github.com/carlosframework/platform/internal/appconfig 2.474s
ok github.com/carlosframework/platform/internal/apperr 0.071s
ok github.com/carlosframework/platform/internal/applog 0.122s
ok github.com/carlosframework/platform/internal/awsinv 0.004s
ok github.com/carlosframework/platform/internal/boxchoice 0.004s
ok github.com/carlosframework/platform/internal/boxid 0.551s
? github.com/carlosframework/platform/internal/boxid/boxidtest [no test files]
? github.com/carlosframework/platform/internal/boxid/internal/signtest [no test files]
ok github.com/carlosframework/platform/internal/boxvolumes 0.008s
ok github.com/carlosframework/platform/internal/caps 0.012s
ok github.com/carlosframework/platform/internal/certwarm 0.002s
ok github.com/carlosframework/platform/internal/cgroup 0.692s
ok github.com/carlosframework/platform/internal/changelog 0.303s
ok github.com/carlosframework/platform/internal/clamd 0.055s
? github.com/carlosframework/platform/internal/cliflag [no test files]
ok github.com/carlosframework/platform/internal/console 393.853s
? github.com/carlosframework/platform/internal/controlname [no test files]
? github.com/carlosframework/platform/internal/controlsock [no test files]
ok github.com/carlosframework/platform/internal/dbdoor 6.102s
ok github.com/carlosframework/platform/internal/dbdoor/dbdoortest 0.009s
ok github.com/carlosframework/platform/internal/dbverify 0.004s
ok github.com/carlosframework/platform/internal/dnsrec 0.002s
ok github.com/carlosframework/platform/internal/docsite 0.005s
ok github.com/carlosframework/platform/internal/domains 2.421s
ok github.com/carlosframework/platform/internal/doorkey 0.006s
ok github.com/carlosframework/platform/internal/economics 0.004s
ok github.com/carlosframework/platform/internal/edge 26.579s
? github.com/carlosframework/platform/internal/edgeproof [no test files]
ok github.com/carlosframework/platform/internal/edgesfile 0.008s
ok github.com/carlosframework/platform/internal/fleetapi 0.132s
ok github.com/carlosframework/platform/internal/gate 1.657s
ok github.com/carlosframework/platform/internal/genselect 0.015s
ok github.com/carlosframework/platform/internal/geo 0.016s
ok github.com/carlosframework/platform/internal/geoip 0.029s
ok github.com/carlosframework/platform/internal/host 166.113s
ok github.com/carlosframework/platform/internal/hostactors 0.425s
ok github.com/carlosframework/platform/internal/hostbox 2.556s
ok github.com/carlosframework/platform/internal/hostclaim 0.512s
ok github.com/carlosframework/platform/internal/hostevidence 0.013s
ok github.com/carlosframework/platform/internal/hostexec 4.768s
ok github.com/carlosframework/platform/internal/hostfence 0.029s
ok github.com/carlosframework/platform/internal/hostfiles 10.394s
ok github.com/carlosframework/platform/internal/hostfloor 0.015s
ok github.com/carlosframework/platform/internal/hostguard 0.970s
ok github.com/carlosframework/platform/internal/hostops 1.651s
? github.com/carlosframework/platform/internal/hostops/fencerelease [no test files]
ok github.com/carlosframework/platform/internal/hostops/internal/fencerelease 0.014s
ok github.com/carlosframework/platform/internal/hostquarantine 0.013s
ok github.com/carlosframework/platform/internal/hostrelocate 0.485s
ok github.com/carlosframework/platform/internal/hostresolve 0.163s
ok github.com/carlosframework/platform/internal/hostwrite 0.013s
ok github.com/carlosframework/platform/internal/infraview 0.014s
ok github.com/carlosframework/platform/internal/instancectl 4.159s
ok github.com/carlosframework/platform/internal/instanceexec 2.720s
ok github.com/carlosframework/platform/internal/instancefs 0.018s
ok github.com/carlosframework/platform/internal/instancekey 0.014s
ok github.com/carlosframework/platform/internal/instanceupdates 0.008s
ok github.com/carlosframework/platform/internal/layout 1.530s
ok github.com/carlosframework/platform/internal/lease 9.793s
ok github.com/carlosframework/platform/internal/lease/leasetest 0.007s
ok github.com/carlosframework/platform/internal/ledger 0.021s
ok github.com/carlosframework/platform/internal/mail 0.174s
ok github.com/carlosframework/platform/internal/maintenance 0.084s
ok github.com/carlosframework/platform/internal/maintenanceexec 5.538s
ok github.com/carlosframework/platform/internal/maintenanceguard 0.083s
ok github.com/carlosframework/platform/internal/metrics 0.006s
ok github.com/carlosframework/platform/internal/moves 20.957s
ok github.com/carlosframework/platform/internal/moves/archive 0.824s
ok github.com/carlosframework/platform/internal/objstore 0.195s
ok github.com/carlosframework/platform/internal/ops 0.171s
ok github.com/carlosframework/platform/internal/park 0.210s
ok github.com/carlosframework/platform/internal/privatefile 0.004s
ok github.com/carlosframework/platform/internal/provision 11.261s
ok github.com/carlosframework/platform/internal/publicrelease 0.007s
? github.com/carlosframework/platform/internal/regionname [no test files]
ok github.com/carlosframework/platform/internal/registry 4.845s
ok github.com/carlosframework/platform/internal/relay 7.216s
ok github.com/carlosframework/platform/internal/release 19.618s
ok github.com/carlosframework/platform/internal/rollout 3.637s
ok github.com/carlosframework/platform/internal/rolloutwords 0.003s
ok github.com/carlosframework/platform/internal/rootaws 0.006s
ok github.com/carlosframework/platform/internal/rootparticipation 0.009s
ok github.com/carlosframework/platform/internal/rootstate 0.026s
ok github.com/carlosframework/platform/internal/runs 0.572s
ok github.com/carlosframework/platform/internal/s3compat 0.005s
ok github.com/carlosframework/platform/internal/safefile 0.004s
ok github.com/carlosframework/platform/internal/schedule 2.484s
ok github.com/carlosframework/platform/internal/seeds 0.011s
ok github.com/carlosframework/platform/internal/sim/hostlife 8.768s
ok github.com/carlosframework/platform/internal/sim/litestreamfacts 4.186s
ok github.com/carlosframework/platform/internal/status 0.038s
ok github.com/carlosframework/platform/internal/steering 0.006s
ok github.com/carlosframework/platform/internal/stopevidence 0.188s
ok github.com/carlosframework/platform/internal/svcrole 0.002s
ok github.com/carlosframework/platform/internal/telemetry 0.004s
ok github.com/carlosframework/platform/internal/termfence 0.005s
ok github.com/carlosframework/platform/internal/tunnelclient 0.006s
ok github.com/carlosframework/platform/internal/unixdial 0.003s
ok github.com/carlosframework/platform/internal/updatewords 0.004s
ok github.com/carlosframework/platform/internal/usage 0.005s
ok github.com/carlosframework/platform/internal/version 0.002s
ok github.com/carlosframework/platform/internal/vet 1.553s
=== cross ===
--- shard 0 of 5 ---
cross-compiling darwin/arm64
--- shard 1 of 5 ---
cross-compiling darwin/amd64
--- shard 2 of 5 ---
cross-compiling linux/arm64
linux/arm64 carlos binary: 47120546 bytes (ceiling 52000000)
--- shard 3 of 5 ---
cross-compiling windows/amd64
--- shard 4 of 5 ---
cross-compiling windows/arm64
=== minio-contract ===
minio contract: arm64, minio RELEASE.2025-09-07T16-13-09Z, mc RELEASE.2025-08-13T08-35-41Z
minio: cached, digest verified
mc: cached, digest verified
minio healthy on 127.0.0.1:20999
Added `local` successfully.
Bucket created successfully `local/contract-bucket`.
Bucket created successfully `local/denied-bucket`.
Created policy `contract-only` successfully.
Added user `scopeduser` successfully.
Attached Policies: [contract-only]
To User: scopeduser
buckets and scoped user ready
=== RUN TestDeliveredContractMinio
=== RUN TestDeliveredContractMinio/put
=== RUN TestDeliveredContractMinio/get_round-trips_the_body
=== RUN TestDeliveredContractMinio/list_shows_the_key
=== RUN TestDeliveredContractMinio/head_and_list_agree_on_the_version
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_its_own_object
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed
=== RUN TestDeliveredContractMinio/delete_removes_it
=== RUN TestDeliveredContractMinio/cross-bucket_denial
--- PASS: TestDeliveredContractMinio (0.02s)
--- PASS: TestDeliveredContractMinio/put (0.00s)
--- PASS: TestDeliveredContractMinio/get_round-trips_the_body (0.00s)
--- PASS: TestDeliveredContractMinio/list_shows_the_key (0.00s)
--- PASS: TestDeliveredContractMinio/head_and_list_agree_on_the_version (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route (0.01s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed (0.00s)
--- PASS: TestDeliveredContractMinio/delete_removes_it (0.00s)
--- PASS: TestDeliveredContractMinio/cross-bucket_denial (0.00s)
PASS
ok github.com/carlosframework/platform/internal/objstore 0.023s
=== RUN TestT4StorageProbeMinio
2026/10/16 15:55:21 moves: T4: STORE: list: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: 18DF0D978A49F06B, HostID: 7c420053e0fbcd7f41653fdd92ffa2802c903d05303b1aa15adc2e8bb65d183f, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
--- PASS: TestT4StorageProbeMinio (0.01s)
PASS
ok github.com/carlosframework/platform/internal/host 0.021s
minio contract: PASS
diff --git a/docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md b/docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md| index 18c6f0429..cea57f158 100644 |
| --- a/docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md |
| +++ b/docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md |
| @@ -16,6 +16,7 @@ Sources: |
| - `docs/host-claim-restore-runbook.md` on main, including "Home-region activation order"; |
| - the files seed, on main as ee35dee2, and its runbook strings, which Paul approved unchanged on 2026-10-15 (copy review `files-seed-copy-review/result.json` in platform-1e's scratch, action `approve`); |
| - Plan 14 Parts 2 and 3, unmerged, on `feat/claim-root-trusted-state-p3` at 305866ad: `claimactivation_preflight.go`, `rootmigration*.go` and `claimjournal_migration.go`; |
| +- platform-1e's activation tools, unmerged, on `feat/claim-activation-tools` at 0e81652a, and their report `activation-tools-report.md` (run 2) in platform-1e's scratch; |
| - amadan #250, #272, #280 and #286. |
| ## Boxes |
| @@ -95,7 +96,10 @@ Today: FAIL, which is expected before step 3 apart from the first two items. |
| - Versioning is `Enabled` on all three buckets. PASS. |
| - `control/host-actors.json` does not exist (404). Dev has no signed roster. |
| - None of the four local files exists on leo. |
| -- Accepted trial gap: leo's role may `ssm:GetParameter` on `parameter/carlos/*` in eu-west-1, which holds `/carlos/release-key`, the deployment root signing key. The same key signs the roster and the floor. Key rotation is deferred for the trial (see "Accepted trial gaps"). |
| +- RESOLVED 2026-10-16 (platform-infrastructure#44, Paul's approval): box roles can no longer read `/carlos/release-key`, the deployment root signing key that signs the roster and the floor. Re-checked read-only the same day by reading the role policies and simulating `ssm:GetParameter` (`aws iam simulate-principal-policy`), with no parameter read and nothing decrypted: |
| + - `carlos-dev-host-b` (leo) and `carlos-dev-host` (sophie): `explicitDeny` on `arn:aws:ssm:eu-west-1:640012953448:parameter/carlos/release-key`, from the new inline policy `deny-release-key-reads` (it also denies `GetParameters`, `GetParameterHistory`, and `GetParametersByPath` on `/carlos`). `/carlos/secrets-key` stays `allowed`. |
| + - `carlos-dev-host-uk` (not in the trial): `explicitDeny` on the eu-west-1 key through its region scope. The simulator says `allowed` for a `/carlos/release-key` in eu-west-2, but no such parameter exists there (`describe-parameters`, metadata only). Do not copy the key into eu-west-2. |
| + - The flagship half (carloku-infrastructure#11) was not checked from here, because this task does not touch flagship. |
| - Accepted trial gap: leo's role may `s3:PutObject` anywhere in the deployment bucket, including `control/hosts/`, so Plan 15's independent claim-writer boundary is not in place on dev. Deferred for the trial (see "Accepted trial gaps"). |
| ### P3. Every roster reader's version |
| @@ -184,21 +188,21 @@ for u in carlos-edge carlos-adopt carlos-stage carlos-config-sync carlos-restart |
| Today: |
| - Kernel 6.12.105, `/data` on ext4, 18G free, 927 MB of memory. PASS. Flag-on instance-control takes `MemoryMax=256M`, a large share of this box; watch it in step 7. |
| - `/`, `/var` and `/var/lib` are `root:root 755`, and no `proof-policy.json` exists. PASS for the Plan 14 Part 1 pre-roll checks. |
| -- systemd 257. Every unit the offline window masks has its file in `/etc/systemd/system`. FAIL, a blocker. A runtime mask is a link in `/run/systemd/system`, and systemd reads `/etc/systemd/system` first, so the `/etc` unit file wins. #272's round-3 dev check saw exactly this: "runtime masks lose to the /etc unit files". Part 3's `verifyOfflineRootMigration` requires `LoadState=masked` on six units, and the seed runbook relies on a runtime mask on the edge. platform-1e is building a persistent-mask installer (blocker 3 below). |
| +- systemd 257. Every unit the offline window masks has its file in `/etc/systemd/system`. FAIL, a blocker. A runtime mask is a link in `/run/systemd/system`, and systemd reads `/etc/systemd/system` first, so the `/etc` unit file wins. #272's round-3 dev check saw exactly this: "runtime masks lose to the /etc unit files". Part 3's `verifyOfflineRootMigration` requires `LoadState=masked` on six units, and the seed runbook relies on a runtime mask on the edge. platform-1e's persistent offline transaction (`system claims offline-begin`, `offline-seed`, `offline-restore`) replaces runtime masks for the trial (blocker 3 below). |
| ## Blockers |
| -1. **Activation runner, pending (platform-1e is building it).** Nothing on main or the Plan 14 branches creates the signed roster (`hostactors.Directory.Initialize`), invites and enrolls a box or the console, installs `/etc/carlos/host-evidence.key`, publishes the floor (`Directory.PublishFloor`) or publishes `home_region`. Steps 3b, 3c and 4b use platform-1e's activation runner once it is reviewed. The same work enrolls the dev console as an actor. |
| -2. **Empty app segment, fix merging.** `apps/bab//` in the deployment bucket (P5) makes the activation preflight and the migration census refuse. The writers are leo's three app-less `proof*` routes. The writer fix merges as `fix-empty-app-key`. After that fix reaches leo, a one-off cleanup removes the objects under `apps/bab//`, with Paul's approval. Whether the `proof*` routes themselves are removed is a pending question for Paul. |
| -3. **Persistent masks, pending (platform-1e is building the installer).** Runtime masks do not mask units installed in `/etc/systemd/system` (P8). Step 5 uses platform-1e's persistent-mask installer once it is reviewed. |
| +1. **Activation runner: built, under authority review.** `carlos system claims ...` on `feat/claim-activation-tools` at 0e81652a (unmerged). It creates and signs the roster, invites and enrolls leo, sophie and the dev console, publishes the floor and `home_region`, and installs the evidence key, the fence-writer credentials and the claim-on instance-control unit. Steps 3 to 7 quote its commands. It must be reviewed, merged and released (step 1); its copy and census rows are still pending review. |
| +2. **Empty app segment.** `apps/bab//` in the deployment bucket (P5) makes the activation preflight and the migration census refuse. Source: leo's three app-less `proof*` routes, removing (2026-10-16), by another agent on Paul's answer. The writer fix is `fix-empty-app-key`. The one-off cleanup of `apps/bab//` waits for the e8e961dd release and Paul's approval. |
| +3. **Masks: built, under authority review.** Runtime masks do not mask units installed in `/etc/systemd/system` (P8). The tools' persistent offline transaction (`system claims offline-begin`, `offline-seed`, `offline-restore`, same branch) replaces them in step 5. |
| 4. **IAM.** Leo's IAM role lacks the version actions, and dev has neither the deployment bucket policy nor a fence-writer role (P2). Step 3a fixes this. |
| -5. **Release.** Plan 14 Parts 2 and 3 are not merged, and no release contains them with the files seed. Step 1 fixes this. |
| +5. **Release.** Plan 14 Parts 2 and 3 and the activation tools are not merged, and no release contains them with the files seed. Step 1 fixes this. |
| ## Accepted trial gaps |
| Paul deferred these for this trial only. They are not blockers here, and they must be closed before any non-trial cohort: |
| - **Plan 15's independent claim writer.** Leo's instance role can write `control/hosts/`, so the box's own credentials could rewrite its claims. |
| -- **Key rotation.** Box roles in eu-west-1 can read `/carlos/release-key` from SSM. No key is rotated or moved for the trial, and no rotation schedule is set for the fence-writer bootstrap keys. |
| +- **Key rotation.** The release signing key is not rotated for the trial, and no rotation schedule is set for the fence-writer bootstrap keys. (The separate exposure, box roles reading `/carlos/release-key`, is resolved: see P2.) |
| ## Step 0. Coordinate |
| @@ -210,11 +214,11 @@ Paul's go: none needed to talk, but nothing below starts until these are done. |
| ## Step 1. Release and roll, claims off |
| -What: cut a release containing Plan 14 Parts 2 and 3 and the files seed, and roll it with every claim switch unset. Dev goes first, then flagship. |
| +What: cut a release containing Plan 14 Parts 2 and 3, the files seed and the activation tools, and roll it with every claim switch unset. Dev goes first. Flagship does not need this release before dev's window (Paul, 2026-10-16); its roll is separate. |
| -Paul's go: to cut the release, then to roll dev, then (separately, not from this runbook) to roll flagship. |
| +Paul's go: to cut the release, then to roll dev. |
| -1. Merge Parts 2 and 3, re-merge main, wait for CI. |
| +1. Merge Parts 2 and 3 and `feat/claim-activation-tools` after their reviews, re-merge main, wait for CI. |
| 2. Cut the release by the README section "Releasing the platform". Check `gh release list -R carlosframework/releases` and `git ls-remote --tags origin` before choosing the number. |
| 3. Roll the dev boxes on the new road only: |
| @@ -227,7 +231,6 @@ Paul's go: to cut the release, then to roll dev, then (separately, not from this |
| `stable` moves leo and dennis together; `canary` moves sophie. Dennis is not in the trial, but it rolls with leo here because both are on `stable`. |
| 4. Roll the dev console direct-bucket with `ship-app.sh` (profile `kass`, bucket `carlos-dev-640012953448`). |
| 5. Check dev end to end: wake `wakecheck` on sophie through DNS and on leo with `curl --resolve wakecheck.dev.carlosframework.com:443:18.200.150.210` (outside the sandbox). |
| -6. Flagship boxes, then the flagship console, by the release recipe, under their own approval. |
| Expected: |
| - `system status` shows all three dev boxes `current` at vX.Y.Z; |
| @@ -255,9 +258,22 @@ Expected: P1 passes and step 1 applied on leo. Today the key is already present |
| Rollback: none; nothing changes. |
| -## Step 3. Actor and IAM certification |
| +## The activation tools |
| + |
| +Steps 3 to 7 use platform-1e's activation tools, `carlos system claims ...`. They are on the unmerged branch `feat/claim-activation-tools` at 0e81652a ("Claim activation tools, run 2 (not for merge)"), under authority review. Their copy is pending Paul's review and their census rows are pending controller review. The binary that runs them must come from a release that contains them (step 1). Every command in steps 3 to 7 is quoted exactly from platform-1e's report, `activation-tools-report.md` ("Exact command sequence for the Leo dev mechanics trial", sections A to F). Where this runbook and the report differ, the report wins. |
| + |
| +The tools replace the earlier draft's pending pieces: |
| +- `system claims inspect`, `run`, `console-config`, `console-report`, `evidence-key` and `fence-credentials` create and sign the roster, enroll leo and the dev console, publish the floor and `home_region`, and install the protected keys (formerly blocker 1); |
| +- `system claims offline-begin`, `offline-seed` and `offline-restore` are a persistent mask transaction (formerly blocker 3's runtime-mask problem). `offline-seed` performs the files seed's delegated one-shot, its override and the edge mask handling; |
| +- `system claims install-control` installs the claim-on instance-control unit. |
| + |
| +Where the commands run: |
| +- **The trusted signing host:** a root shell holding `/root/claim-trial` with `release.pub`, `release.key`, `regions.json`, `leo.json`, `sophie.json`, `console.json` and `floor.json`. The release signing key never goes on either box. |
| +- **Leo and sophie:** root shells through the approved administration transport (the SSM helper above). Inputs are staged at the exact paths shown, root-owned, private files `0600`, no symlinks or extra hard links. Staging is a checkpoint, not permission to make up signatures. |
| -What: put the IAM shape from main's modules on dev, enroll leo as a claim actor, arm its guard and publish the floor. Plan 9 C2 order: arm every enrolled actor, publish only once every actor reports ready. |
| +The controller supplies, and Paul approves, the concrete inputs: the signed release, floor and guard artifacts; namespace, pool and label inputs (leo's pool is `cohort1`, sophie's matches its real placement); the console identity; the fence role and credentials; and the reviewed `/root/claim-trial/leo-host.env`. Revalidate both instance IDs before the trial. |
| + |
| +## Step 3. Actor and IAM certification |
| ### 3a. IAM |
| @@ -272,78 +288,213 @@ Paul's go: to apply the dev root. |
| 3. Apply. Re-run P2's IAM lines. |
| 4. Denied-operation witnesses, through a reviewed runner: leo's instance role is refused a put to `control/box-fences/`, the fence-writer role may create there with `If-None-Match`, and nobody may delete there. |
| +The activation tools do not provision IAM. The fence-writer credentials that step 5 installs come from this step. |
| + |
| Expected: P2's IAM items pass. |
| On failure: revert the dev root to its previous module wiring and apply, but only before any fence object is written. Evidence objects already written stay. |
| -### 3b. Enroll leo and the dev console, and arm leo's guard |
| +### 3b. Roster, enrollment and console enrollment (report section A) |
| -Paul's go: once for the whole of 3b. |
| +Paul's go: once for 3b. |
| -Run with platform-1e's activation runner (pending, blocker 1). The order it follows: |
| +Each box's signed AWS identity record must already be published by the existing box identity mechanism. A missing record is a stop, not permission to make one up. |
| -1. Create the signed roster, `control/host-actors.json`, and a root-signed invitation `control/host-actor-invitations/leo-climbing.json`. Enroll the dev console as an actor in the same roster; what that installs on sophie is part of the runner's review. |
| -2. Install, root-owned and outside any agent-writable path: |
| - - `/etc/carlos/host-evidence.key`, `root`, mode `600`, one link; |
| - - `/etc/carlos/fence-writer/credentials.json`, `root`, mode `400`. |
| -3. Arm the guard: |
| +On the trusted signing host, the helper: |
| - ```bash |
| - /opt/carlos/carlos system guard-arm --policy "$APPROVED_POLICY" --identity "$SIGNED_IDENTITY" --enrollment "$SIGNED_ENROLLMENT" |
| - ``` |
| +```bash |
| +set -euo pipefail |
| +umask 077 |
| +cd /root/claim-trial |
| +CARLOS=/opt/carlos/carlos |
| +B=carlos-dev-640012953448 |
| +R=eu-west-1 |
| +PUB=$(cat release.pub) |
| +KEY=/root/claim-trial/release.key |
| +export AWS_PROFILE=kass |
| +refresh() { |
| + "$CARLOS" system claims inspect --bucket "$B" --aws-region "$R" --public-key "$PUB" > inspect.json |
| + jq '.signed_roster' inspect.json > roster.json |
| + REV=$(jq -er '.roster.revision' inspect.json) |
| +} |
| +apply_plan() { |
| + "$CARLOS" system claims run --plan /root/claim-trial/plan.json --signing-key "$KEY" --public-key "$PUB" --aws-region "$R" --dry-run |
| + "$CARLOS" system claims run --plan /root/claim-trial/plan.json --signing-key "$KEY" --public-key "$PUB" --aws-region "$R" |
| + refresh |
| +} |
| +phase() { |
| + refresh |
| + jq -n --arg phase "$1" --arg bucket "$B" --argjson rev "$REV" \ |
| + --slurpfile input "$2" '{phase:$phase,bucket:$bucket,expected_revision:$rev} + $input[0]' > plan.json |
| + apply_plan |
| +} |
| +``` |
| + |
| +For a new roster only (dev has none today, P2), initialize, configure, then invite and enroll both computes and the console. On resume, run `refresh` and inspect the actual state instead. Never turn a read error into permission to initialize or overwrite. |
| - These are approved, protected files, not guessed strings. |
| -4. Complete leo's enrollment into the roster. |
| +```bash |
| +jq -n --arg b "$B" '{phase:"initialize",bucket:$b,expected_revision:0}' > plan.json |
| +apply_plan |
| +jq -n --slurpfile x regions.json '{regions:$x[0]}' > configure.json |
| +phase configure configure.json |
| +for box in leo sophie; do |
| + jq -n --slurpfile x "$box.json" '{box:$x[0]}' > box-phase.json |
| + phase invite-box box-phase.json |
| + phase enroll-box box-phase.json |
| + done |
| +refresh |
| +jq --arg bucket "$B" --arg e "$(jq -er '.roster.boxes["sophie-knitting"].enrollment' inspect.json)" \ |
| + '.deployment_bucket=$bucket | .compute={box:"sophie-knitting",enrollment:$e}' console.json > console-bound.json |
| +jq -n --slurpfile x console-bound.json '{console:$x[0]}' > console-phase.json |
| +phase invite-console console-phase.json |
| +phase enroll-console console-phase.json |
| +refresh |
| +for box in leo-climbing sophie-knitting; do |
| + jq --arg box "$box" '{box:$box,enrollment:.roster.boxes[$box].enrollment,instance_id:.roster.boxes[$box].cloud.instance_id}' \ |
| + inspect.json > "$box-enrollment.json" |
| +done |
| +``` |
| -Expected: |
| -- `/etc/carlos/host-actor.json` written by `guard-arm`; |
| -- `/opt/carlos/system/guard/policy.json` present; |
| -- the roster's `boxes.leo-climbing` row has leo's enrollment and `cloud.instance_id` `i-046c7fe43620944bc`, and the roster lists the dev console; |
| -- `carlos system status --console https://platform.dev.carlosframework.com` shows leo ready, with the floor digest. |
| +Expected: every dry run, then its apply, succeeds; `inspect.json` lists `leo-climbing` (pool `cohort1`, instance `i-046c7fe43620944bc`), `sophie-knitting` and the console. Sophie is enrolled only as the console's compute host; its host claims stay off. |
| -On failure: hold. Leave every flag unset and change nothing more. Arming is one-way: keep the local armed policy, and never delete the enrollment or the guard. Restore only an image at or above the floor. |
| +Then transfer `roster.json`, `release.pub`, the matching enrollment JSON, the approved armed `policy.json` and the signed `identity.json` to `/root/claim-trial` on each box. No private signing key is transferred. |
| -### 3c. Publish the floor |
| +On failure: hold. A partly applied phase is resumed from the latest authenticated revision, never recreated. |
| -Paul's go: separate from 3b. |
| +### 3c. Arm leo's guard and start the guarded console (report section B) |
| -Run with platform-1e's activation runner (pending, blocker 1). Re-read every report, the roster revision and the candidate digest just before publishing. Publish, then read back the roster. |
| +Paul's go: once for 3c. It includes a `console.service` restart on sophie, which briefly interrupts the dev console. |
| -Expected: the roster has `publication: "published"` and a `floor_sha256` equal to the digest in leo's readiness report. |
| +On leo, after the approved release and protected current and previous binaries are installed: |
| -On failure: hold. A torn publish is resumed with identical bytes. Never erase the floor or substitute another digest. If a check after publishing fails, the claim flag stays off. |
| +```bash |
| +set -euo pipefail |
| +cd /root/claim-trial |
| +set -a |
| +. /etc/carlos/host.env |
| +set +a |
| +/opt/carlos/carlos system guard-arm --policy /root/claim-trial/policy.json \ |
| + --identity /root/claim-trial/identity.json --enrollment /root/claim-trial/leo-climbing-enrollment.json |
| +``` |
| -## Step 4. Plan 13 order: reader, then home, then enable |
| +On sophie, stage `console-path` (the protected absolute console executable path) and `console-id` (the approved ID from `console-bound.json`). `CARLOS_HOST_CLAIM` stays absent or empty in sophie's `/etc/carlos/host.env`; the installer refuses otherwise. |
| -Paul's go: before 4b. The designation becomes write-once as soon as claims are on. |
| +```bash |
| +set -euo pipefail |
| +umask 077 |
| +cd /root/claim-trial |
| +set -a |
| +. /etc/carlos/host.env |
| +set +a |
| +PUB=$(cat release.pub) |
| +CID=$(cat console-id) |
| +CONSOLE_BINARY=$(cat console-path) |
| +/opt/carlos/carlos system claims inspect --bucket carlos-dev-640012953448 --aws-region eu-west-1 --public-key "$PUB" > inspect.json |
| +REV=$(jq -er '.roster.revision' inspect.json) |
| +jq '.signed_roster' inspect.json > roster.json |
| +/opt/carlos/carlos system guard-arm --policy /root/claim-trial/policy.json \ |
| + --identity /root/claim-trial/identity.json --enrollment /root/claim-trial/sophie-knitting-enrollment.json \ |
| + --console "$CONSOLE_BINARY" |
| +/opt/carlos/carlos system claims console-config --roster /root/claim-trial/roster.json --expect-revision "$REV" \ |
| + --bucket carlos-dev-640012953448 --console-id "$CID" --public-key "$PUB" --dry-run |
| +/opt/carlos/carlos system claims console-config --roster /root/claim-trial/roster.json --expect-revision "$REV" \ |
| + --bucket carlos-dev-640012953448 --console-id "$CID" --public-key "$PUB" |
| +systemctl restart console.service |
| +systemctl is-active console.service |
| +/opt/carlos/carlos system claims console-report --bucket carlos-dev-640012953448 --aws-region eu-west-1 \ |
| + --expect-revision "$REV" --console-id "$CID" --public-key "$PUB" --dry-run |
| +/opt/carlos/carlos system claims console-report --bucket carlos-dev-640012953448 --aws-region eu-west-1 \ |
| + --expect-revision "$REV" --console-id "$CID" --public-key "$PUB" |
| +``` |
| + |
| +Expected: `guard-arm` succeeds on both; `console.service` is `active`; the report publishes. If reporting refuses because a just-started console or box has no fresh telemetry yet, hold, read the service journal, wait for its normal report and rerun the reporting pair. Never refresh timestamps by hand. Leo's host claims stay off until the offline window. Get real fresh box evidence from both enrolled computes before step 4. |
| + |
| +On failure: hold. Arming is one-way: keep the local armed policy, and never delete the enrollment or the guard. Restore only an image at or above the floor. |
| + |
| +## Step 4. Plan 13 order: readers, then floor and home, then enable |
| ### 4a. Readers |
| -Re-run P3. Dennis is not in the trial, but its agent reads the roster, so it counts. Expected: leo, sophie, dennis, their `.prev` images, any staged recovery image, the dev console and the operator CLI are all on vX.Y.Z (at minimum, on a Plan 13 build). The signed fleet data does not record every reader's build, so this is the operator's check. |
| +Re-run P3. Dennis is not in the trial, but its agent reads the roster, so it counts. Expected: leo, sophie, dennis, their `.prev` images, any staged recovery image, the dev console and the operator CLI are all on the trial release (at minimum, on a Plan 13 build). The signed fleet data does not record every reader's build, so this is the operator's check. |
| + |
| +### 4b. Publish the floor and home, bind leo's key, enable the console runtime (report section C) |
| + |
| +Paul's go: once for 4b. `home_region` becomes write-once as soon as claims are on. The home name is `ie` (the roster and home designation use `ie`, AWS uses `eu-west-1`, and home boxes leave `CARLOS_REGION` unset). |
| -### 4b. Home designation |
| +On the trusted signing host, immediately after sophie's successful report: |
| -Publish `home_region` as dev's home roster name, with platform-1e's activation runner (pending, blocker 1). The proposed name is `ie` (open question 5). |
| +```bash |
| +jq -n --slurpfile x floor.json '{floor:$x[0]}' > floor-phase.json |
| +phase publish-floor floor-phase.json |
| +printf '%s\n' '{"home_region":"ie"}' > home-phase.json |
| +phase home home-phase.json |
| +refresh |
| +``` |
| + |
| +On leo, read the latest signed roster and generate the protected evidence key. Keep the applied response, not the dry-run one: |
| + |
| +```bash |
| +set -euo pipefail |
| +umask 077 |
| +cd /root/claim-trial |
| +PUB=$(cat release.pub) |
| +/opt/carlos/carlos system claims inspect --bucket carlos-dev-640012953448 --aws-region eu-west-1 --public-key "$PUB" > inspect.json |
| +REV=$(jq -er '.roster.revision' inspect.json) |
| +LE=$(jq -er '.roster.boxes["leo-climbing"].enrollment' inspect.json) |
| +jq '.signed_roster' inspect.json > roster.json |
| +/opt/carlos/carlos system claims evidence-key --roster /root/claim-trial/roster.json --expect-revision "$REV" \ |
| + --bucket carlos-dev-640012953448 --box leo-climbing --enrollment "$LE" --public-key "$PUB" --dry-run |
| +/opt/carlos/carlos system claims evidence-key --roster /root/claim-trial/roster.json --expect-revision "$REV" \ |
| + --bucket carlos-dev-640012953448 --box leo-climbing --enrollment "$LE" --public-key "$PUB" > key-proof.json |
| +``` |
| -Expected: the roster read back has `"home_region": "ie"`, and after step 6 leo's agent does not log "host claim: signed roster has no home region". |
| +Transfer only `key-proof.json` to the trusted signing host. It holds the public binding and proof, never the private key. There: |
| -On failure: before claims are on, a new signed roster revision can correct the name. After claims are on, the name cannot change. |
| +```bash |
| +jq '{binding:.key_proof.binding,proof:.key_proof.proof}' key-proof.json > bind-phase.json |
| +phase bind-key bind-phase.json |
| +refresh |
| +jq -e '.roster.home_region == "ie" and .roster.publication == "published" and .roster.request_limit_ns == 120000000000' inspect.json |
| +``` |
| -### 4c. Enable |
| +On sophie, enable only the console runtime: |
| -Enabling happens in steps 5 and 6. |
| +```bash |
| +set -euo pipefail |
| +umask 077 |
| +cd /root/claim-trial |
| +PUB=$(cat release.pub) |
| +CID=$(cat console-id) |
| +/opt/carlos/carlos system claims inspect --bucket carlos-dev-640012953448 --aws-region eu-west-1 --public-key "$PUB" > inspect.json |
| +REV=$(jq -er '.roster.revision' inspect.json) |
| +jq '.signed_roster' inspect.json > roster.json |
| +/opt/carlos/carlos system claims console-config --roster /root/claim-trial/roster.json --expect-revision "$REV" \ |
| + --bucket carlos-dev-640012953448 --console-id "$CID" --public-key "$PUB" --enable-console-runtime --dry-run |
| +/opt/carlos/carlos system claims console-config --roster /root/claim-trial/roster.json --expect-revision "$REV" \ |
| + --bucket carlos-dev-640012953448 --console-id "$CID" --public-key "$PUB" --enable-console-runtime |
| +systemctl restart console.service |
| +systemctl is-active console.service |
| +/opt/carlos/carlos system claims console-report --bucket carlos-dev-640012953448 --aws-region eu-west-1 \ |
| + --expect-revision "$REV" --console-id "$CID" --public-key "$PUB" --dry-run |
| +/opt/carlos/carlos system claims console-report --bucket carlos-dev-640012953448 --aws-region eu-west-1 \ |
| + --expect-revision "$REV" --console-id "$CID" --public-key "$PUB" |
| +``` |
| + |
| +Expected: the final `jq -e` prints `true`; `/etc/carlos/host-evidence.key` exists on leo (`root`, `600`, one link); `console.service` is `active` and its report publishes. If publication loses a race with the agent's own report, inspect again, report again and retry with the current revision. |
| + |
| +On failure: hold. A torn publish is resumed with identical bytes. Never erase the floor or substitute another digest. The claim flag on leo stays off. |
| ## Step 5. Offline window: migration and files seed |
| -What: Plan 9 milestone 4 on leo, with the files seed inside the same window. Tenants stay stopped from the drain until serving resumes. |
| +What: Plan 9 milestone 4 on leo, with the files seed inside the same window, through the persistent offline transaction. Tenants stay stopped from the drain until step 6. |
| -Paul's go: for the window, with a written deadline. A second go is needed in step 6 before serving resumes. |
| +Paul's go: for the window, with a written deadline. Step 6 needs its own go. |
| -Before the window: blockers 1 to 3 are closed (the `apps/bab//` cleanup done after `fix-empty-app-key` reached leo), 5.0 is done, and platform-d8 has confirmed leo is free. |
| +Before the window: blockers 2 and 5 are closed, the activation tools are merged and released, 5.0 is done, and platform-d8 has confirmed leo is free. |
| ### 5.0 Trial pool and disposable app, claims still off |
| -Paul's go: separate, before the window. This is a host.env change on leo. |
| +Paul's go: APPROVED by Paul on 2026-10-16. Apply it when the trial nears, and only after coordinating with platform-d8, because it stops their test apps landing on leo. This is a host.env change on leo. `leo-host.env` in 5.1 keeps `CARLOS_POOLS=cohort1`. |
| Choice: keep the `CARLOS_POOLS` plan. The disposable app does not make it unnecessary. Dev's two home boxes both mint every unregioned, pool-less record, so a disposable app created without a pool would land on sophie too, which is the #286 problem moved to a new app. The pool does two jobs at once: it takes wakecheck off leo (H1), and it is the only knob that puts the disposable app on leo alone. |
| @@ -376,74 +527,216 @@ Choice: keep the `CARLOS_POOLS` plan. The disposable app does not make it unnece |
| On failure: hold. Leave the pool and the app as they are and report. |
| -### 5.1 Drain and quiesce |
| +### 5.1 Fence credentials, offline transaction, migration, root startup and seed (report section D) |
| + |
| +Before entry: hold the admission and traffic barrier, drain and replicate `claimtrial`, confirm P5 still passes, and verify the independent preflight and retained-state gates from the binding scope. |
| + |
| +The reviewed `/root/claim-trial/leo-host.env` keeps every existing setting and secret, sets `CARLOS_HOST_CLAIM=1`, `CARLOS_HOST_CLAIM_ORIGINS=1` and `CARLOS_POOLS=cohort1`, and leaves `CARLOS_REGION` and `CARLOS_MOVES_PHASE1` absent or empty. Never add a literal `CARLOS_REGION=ie` on a home box. Stage `/root/claim-trial/fence-credentials.json` and `/root/claim-trial/fence-role-arn` from step 3a's provisioning. |
| -Mask the six offline units and the edge with platform-1e's persistent-mask installer (pending, blocker 3). A runtime mask (`systemctl mask --runtime`) does not work on these boxes, because the unit files live in `/etc/systemd/system`. |
| +On leo, as root: |
| ```bash |
| -leo 'systemctl list-units --no-legend --plain "carlos-instance-gen@*" "carlos-instance@*"' |
| -leo 'systemctl stop carlos-adopt.timer carlos-system-update.path carlos-stage.path carlos-restart.path' |
| -# platform-1e's persistent-mask installer masks: carlos-adopt.service carlos-stage.service |
| -# carlos-config-sync.service carlos-restart.service carlos-system-update.service carlos-system-recover.service |
| -leo 'for u in carlos-adopt carlos-stage carlos-config-sync carlos-restart carlos-system-update carlos-system-recover; do echo "$u $(systemctl show $u.service -p LoadState -p ActiveState --value | tr "\n" " ")"; done' |
| -leo 'systemctl stop carlos-edge.service' |
| -# then the persistent-mask installer masks carlos-edge.service |
| -leo 'systemctl show carlos-edge.service -p LoadState -p ActiveState --value; pgrep -u carlos | wc -l' |
| +set -euo pipefail |
| +umask 077 |
| +cd /root/claim-trial |
| +FENCE_ROLE=$(cat fence-role-arn) |
| +/opt/carlos/carlos system claims fence-credentials --source /root/claim-trial/fence-credentials.json \ |
| + --bucket carlos-dev-640012953448 --aws-region eu-west-1 --role "$FENCE_ROLE" --dry-run |
| +/opt/carlos/carlos system claims fence-credentials --source /root/claim-trial/fence-credentials.json \ |
| + --bucket carlos-dev-640012953448 --aws-region eu-west-1 --role "$FENCE_ROLE" |
| +/opt/carlos/carlos system claims offline-begin --dry-run |
| +/opt/carlos/carlos system claims offline-begin |
| +if ! test -e /root/claim-trial/host.env.before; then |
| + cp --preserve=all /etc/carlos/host.env /root/claim-trial/host.env.before |
| +fi |
| +install -o root -g root -m 0600 /root/claim-trial/leo-host.env /etc/carlos/host.env |
| +/opt/carlos/carlos system claims install-control --dry-run |
| +/opt/carlos/carlos system claims install-control |
| +set -a |
| +. /etc/carlos/host.env |
| +set +a |
| +unset CARLOS_REGION CARLOS_MOVES_PHASE1 |
| +/opt/carlos/carlos system journal-migrate |
| +/opt/carlos/carlos system claims install-control --dry-run |
| +/opt/carlos/carlos system claims install-control |
| +systemctl add-wants multi-user.target carlos-instance-control.service |
| +systemctl restart carlos-instance-control.service |
| +systemctl is-active carlos-instance-control.service |
| +/opt/carlos/carlos system claims offline-seed --dry-run |
| +/opt/carlos/carlos system claims offline-seed |
| +cp /etc/carlos/claim-activation/offline/seed-result.txt /root/claim-trial/seed-result.txt |
| +INVOCATION=$(sed -n 's/^InvocationID=//p' /root/claim-trial/seed-result.txt) |
| +test -n "$INVOCATION" |
| +journalctl "_SYSTEMD_INVOCATION_ID=$INVOCATION" --no-pager > /root/claim-trial/seed-journal.txt |
| +cat /root/claim-trial/seed-result.txt |
| +cat /root/claim-trial/seed-journal.txt |
| +systemctl show carlos-edge.service --property=LoadState,ActiveState |
| ``` |
| -Expected: before the stop, no instance unit other than claimtrial's; each of the six units `masked inactive`; the edge `masked inactive`; `0` processes under the carlos user. Stopping the edge drains requests and flushes replication for any awake instance. |
| +`install-control` runs twice on purpose: the second run, after `journal-migrate`, adds the census-dependent root-retry configuration. |
| -The adopt pass must be stopped first. It re-renders the instance-control unit from host.env and restarts it, so with `CARLOS_HOST_CLAIM` set it would start claim mode at an unplanned moment. |
| +Expected: |
| +- `/etc/carlos/fence-writer/credentials.json` installed (`root`, `400`); |
| +- `journal-migrate` exits 0, and `/var/lib/carlos-claims/authority/migration/migration.json` exists; |
| +- `carlos-instance-control.service` is `active`; |
| +- the seed result shows `Host "claimtrial.dev.carlosframework.com": seeded.` and its admitted prefix line, with no retained-state refusal (wakecheck is not on leo); |
| +- the last line shows the edge `LoadState=masked` and `ActiveState=inactive`. |
| -Where the seed strings in 5.6 say "runtime mask", use the installer's mask instead. Their `systemctl unmask --runtime` and `systemctl mask --runtime` commands are replaced by the installer's unmask and mask. |
| +Archive the result and journal even if a command fails. With `set -e`, run the capture commands in a separate root shell after a failure. Before any retry, save that invocation's evidence under a new filename, because the next invocation replaces `seed-result.txt`. Resume an interrupted entry with `offline-begin` and an interrupted seed with `offline-seed`. Neither authorizes serving. A completed offline transaction is not reusable as a new window; keep its record. |
| -### 5.2 Set the flags |
| +The approved files seed runbook strings, quoted verbatim in the appendix, describe what the seed does and how to read its output. Their manual override, mask and unmask commands are not run: `offline-seed` performs them. |
| -```bash |
| -leo 'cp -p /etc/carlos/host.env /etc/carlos/host.env.bak-cohort1-$(date -u +%Y%m%d) |
| -printf "CARLOS_HOST_CLAIM=1\nCARLOS_HOST_CLAIM_ORIGINS=1\n" >> /etc/carlos/host.env |
| -grep -cE "^CARLOS_HOST_CLAIM=1$|^CARLOS_HOST_CLAIM_ORIGINS=1$" /etc/carlos/host.env; grep -c "^CARLOS_MOVES_PHASE1=" /etc/carlos/host.env' |
| -``` |
| +On failure: stop and hold. Leave the transaction's persistent mask and override in place. |
| -Expected: `2`, then `0`. Any non-empty value of `CARLOS_HOST_CLAIM`, including `0`, turns it on. The files seed requires `CARLOS_HOST_CLAIM_ORIGINS` to be exactly `1`. |
| +## Step 6. Serving under the claim (report section E) |
| -### 5.3 Render the claim-on root unit |
| +Paul's go: required, after step 5's migration, seed and retained-state checks pass. Keep the external traffic barrier until then. |
| -The installer that renders this unit while the adopt pass is masked is not named yet (open question 4; it may be part of platform-1e's activation runner). After it runs: |
| +On leo: |
| ```bash |
| -leo 'systemctl cat carlos-instance-control.service | grep -E "^StateDirectory=carlos-claims|^EnvironmentFile=-/etc/carlos/host.env|^MemoryMax=256M"' |
| +/opt/carlos/carlos system claims offline-restore --dry-run |
| +/opt/carlos/carlos system claims offline-restore |
| +systemctl is-active carlos-instance-control.service carlos-edge.service |
| +journalctl -u carlos-edge.service -u carlos-instance-control.service --since '-5min' --no-pager |
| ``` |
| -Expected: all three lines. |
| +`offline-restore` keeps any operator masks that existed before, restores only this transaction's replacements and the previous trigger states, and starts a previously active edge last. |
| -### 5.4 Migrate |
| +Expected: |
| +- both units `active`; |
| +- `REFUSING CARLOS_SWAP=overlap (host-claims-on)` once in the edge journal; |
| +- no "signed roster has no home region" line; |
| +- `carlos system status --console https://platform.dev.carlosframework.com` shows leo current. |
| + |
| +On failure: stop and hold. |
| + |
| +## Step 7. Acceptance |
| + |
| +Paul's go: for the reboot only. |
| + |
| +The acceptance host is the disposable app, `HOST=claimtrial.dev.carlosframework.com`. Never run these checks on wakecheck or a real app. Every curl runs outside the sandbox. |
| + |
| +1. Launch. `curl -s --resolve $HOST:443:18.200.150.210 https://$HOST/`. Expect 200 with `host=ip-172-31-19-180` and `hits=4` (the three seeded rows survived), and on leo a `carlos-instance-gen@$HOST:<n>` unit active. |
| +2. First cold wake. Wait for it to sleep: `carlos logs --app claimtrial --console https://platform.dev.carlosframework.com --since 30m` ends in "instance exited" (idle is 15 minutes). Wake it again. Expect 200, and `hits=` one higher than before: the row written at launch survived the restore. |
| +3. Second cold wake. Repeat. Expect 200 and `hits=` one higher again. |
| +4. Isolation. While it is awake: |
| + |
| + ```bash |
| + leo 'pid=$(systemctl show -p MainPID --value "carlos-instance-gen@'"$HOST"':*"); ls /proc/$pid/root/data/instances/; readlink /proc/$pid/ns/mnt /proc/1/ns/mnt' |
| + ``` |
| + |
| + Expect only the host's own directory, and two different mount namespaces. `carlos status --layout` shows the host `nested`. |
| +5. Reboot. `aws --profile kass --region eu-west-1 ec2 reboot-instances --instance-ids i-046c7fe43620944bc`. After it returns: `carlos-instance-control.service` and `carlos-edge.service` active; `/var/lib/carlos-claims` intact; no recovery or park error in `journalctl -u carlos-instance-control.service -b --no-pager`; a wake answers 200 with `hits=` continuing. |
| +6. Signed repair or update. Apply the approved signed update to leo and confirm the box comes back and `claimtrial` wakes. The exact release input belongs to the approved trial record. |
| + |
| +Pass: all six. Record each response and the journal lines on #250. |
| + |
| +### Resetting an exhausted wake retry (report section F) |
| + |
| +For `claimtrial` only, if its wake-retry budget is exhausted. The authorized operator takes the current holder incarnation and retry episode from the trial's claim evidence. The reset CLI does not expose this yet, so it goes through the console API. Stage a protected curl config with the operator's bearer header at `/root/claim-trial/operator-curl.conf` (never in shell arguments or logs), `reset-input.json` with the exact `incarnation`, `episode` and a unique retained `request_id`, and `acceptance-host` with the disposable hostname. On the trusted operator host: |
| ```bash |
| -leo 'set -a; . /etc/carlos/host.env; set +a; /opt/carlos/carlos system journal-migrate; echo exit=$?' |
| +set -euo pipefail |
| +umask 077 |
| +cd /root/claim-trial |
| +HOST=$(cat acceptance-host) |
| +jq '{action:"reset-wake-retry",incarnation:.incarnation,episode:.episode,request_id:.request_id}' \ |
| + reset-input.json > reset.json |
| +curl --fail-with-body --silent --show-error --config /root/claim-trial/operator-curl.conf \ |
| + --header 'Content-Type: application/json' --data-binary @/root/claim-trial/reset.json \ |
| + "https://platform.dev.carlosframework.com/api/cli/ops/hosts/$HOST/resolve" |
| ``` |
| -On Part 3 this runs the activation preflight, checks the six units are masked and inactive, refuses while any `carlos adopt`, `system apply` or `guard-arm` process runs, then performs the offline root migration and publishes the census report. |
| +A 200 acknowledges the request; it does not prove a wake. Check the new retry episode and zero budget in the trial evidence, with holder, session and stamp unchanged, then check a real wake. A 409 is a stale or refused reset: read the current evidence and investigate. Do not rewrite identifiers or loop. |
| -Expected: `exit=0`; `/var/lib/carlos-claims/authority/migration/migration.json` exists (root-owned). No legacy journals exist on leo, so nothing is imported. |
| +## Stop and hold |
| + |
| +This replaces a claims-off rollback for the trial. If anything from 5.0 on fails, or acceptance fails, stop and leave leo as it is for diagnosis. Nothing on leo is undone, repaired or retried until platform-1e has looked and Paul has decided what comes next. Do not unset the claim flags, unbind the census, delete records or guards, or attempt a data-preserving rollback. Only `claimtrial` is affected, and it holds nothing of value. |
| + |
| +Paul's go: not needed to stop. Stopping is always allowed. Anything after it needs a go. |
| -### 5.5 Root startup |
| +**During the offline transaction (step 5, before `offline-restore`).** The edge is already persistently masked and stopped. Leave the transaction's mask and any seed override in place. Recover only through the owned transaction (`offline-begin` or `offline-seed`), and only with a go. Save `seed-result.txt` and the invocation journal under new filenames. |
| -Plan 9 milestone 4's commands: |
| +**After `offline-restore` (steps 6 and 7).** Keep the traffic and admission barrier, and stop leo's edge, which stops admission and every tenant: |
| ```bash |
| -leo 'systemctl daemon-reload |
| -systemctl add-wants multi-user.target carlos-instance-control.service |
| -systemctl restart carlos-instance-control.service |
| -systemctl is-active carlos-instance-control.service |
| -journalctl -u carlos-instance-control.service --since "-5min" --no-pager | tail -30' |
| +systemctl stop carlos-edge.service |
| +systemctl is-active carlos-edge.service || test "$(systemctl show -p ActiveState --value carlos-edge.service)" = inactive |
| +``` |
| + |
| +Then stop the triggers that could change the box under you, and capture evidence read-only: |
| + |
| +```bash |
| +leo 'systemctl stop carlos-adopt.timer carlos-system-update.path carlos-stage.path carlos-restart.path |
| +systemctl show carlos-edge.service carlos-instance-control.service -p InvocationID -p Result -p ExecMainStatus -p ActiveState -p SubState |
| +journalctl -u carlos-edge.service -u carlos-instance-control.service --since "-2h" --no-pager | tail -400 |
| +ls -la /var/lib/carlos-claims /etc/carlos/claim-activation 2>&1; cat /proc/sys/kernel/random/boot_id' |
| ``` |
| -Expected: `active`, and no migration, recovery or floor error in the journal. A migration or evidence failure leaves the edge stopped. |
| +Post a summary on #250 and tell platform-1e. |
| + |
| +What "hold" leaves running: the operating system (systemd, journald, sshd, the SSM agent), so the box stays reachable for read-only diagnosis, and `carlos-instance-control.service`, the root service that keeps root recovery and retained state consistent. Stop instance-control too (`systemctl stop carlos-instance-control.socket carlos-instance-control.service`) only if it is itself failing or platform-1e asks. Sophie, its console and dennis carry on as normal; sophie keeps serving wakecheck. |
| + |
| +A stopped edge is not a persistent mask once the transaction has completed. Do not reboot or start leo while it is held. A reboot needs a new persistent hold procedure, approved by the controller first. |
| + |
| +What must not be touched while leo is held: |
| +- `/etc/carlos/host.env` (claim flags and `CARLOS_POOLS` stay set) and `/root/claim-trial`, including `host.env.before`; |
| +- `/var/lib/carlos-claims` and `/etc/carlos/claim-activation`, including migration state, journals, parks and the offline transaction record; |
| +- `/etc/carlos/host-actor.json`, `/etc/carlos/host-evidence.key`, `/etc/carlos/fence-writer/` and `/opt/carlos/system/guard`; |
| +- `/opt/carlos/carlos`, `/opt/carlos/carlos.prev` and anything staged: no `carlos system update`, rollback or old-road roll, and no `stable`-channel roll while leo is held, because it is on `stable` with dennis; |
| +- `/data`, including `registry.db`, `/data/instances` and any parked database; |
| +- unit files, drop-ins and masks, as found; |
| +- the roster, floor and enrollments, and the claim's S3 objects: `control/hosts/`, `control/host-actors.json`, `control/host-origins/`, admitted prefixes, and `instances/` in `carlos-dev-ie-640012953448`; |
| +- no `systemctl reset-failed`, no rerun of `journal-migrate`, `install-control` or the seed, no `offline-restore`, and no console `reset-wake-retry`. |
| + |
| +Leaving hold is a separate decision: a fix and resume, or a claims-off teardown under its own reviewed procedure. This runbook does not cover either. Data written under the claim went to the host's claim location, `instances/<host>/<incarnation>`, so a later teardown has to say how it treats that (open question 2). |
| + |
| +## Known hazards |
| + |
| +**H1. wakecheck (#286).** Chosen: wakecheck leaves leo through `CARLOS_POOLS=cohort1` (step 5.0) and stays on sophie, claims off, as today. Acceptance uses the disposable `claimtrial` app in pool `cohort1`. |
| + |
| +Why: wakecheck's record is unregioned, so sophie and leo both mint it. With leo claimed and sophie not, the same host would have two writers on different lineages. Leo's claimed generations restore from the claim location; sophie writes its legacy replica and also starts from its own stale local database (#286). DNS for `*.dev.carlosframework.com` points at sophie, and bots wake wakecheck there, so it cannot be kept to leo. A wake on sophie during the window would also break the drained, replicated stop the seed depends on. The disposable app needs the pool for the same reason: without one, sophie would mint it too. |
| + |
| +A box with `CARLOS_POOLS` set serves only the listed pools, not the shared `""` pool, so leo's reconcile pass removes wakecheck. The activation preflight skips records in other pools. |
| + |
| +**H2. Shared use of leo.** platform-d8 runs zero-downtime dev checks on leo. Notify them before step 5.0 and wait for a clear answer. After step 6, leo declines `CARLOS_SWAP=overlap` with `host-claims-on`, so their checks cannot run on leo for the rest of the trial, including any hold. The pool change also means their test apps would need `--pool cohort1`. |
| + |
| +**H3. #272.** A system pointer whose `result.json` says "applied" is terminal: after an old-road roll the system leg never returns the box to its channel pointer. Roll leo only with `carlos system update`. Separately, #272's stale-artefact refusal is deferred past activation: once a host is retired on leo, installing it again on leo refuses. That fails closed. Never remove `consumed/`, lock or record files by hand. |
| + |
| +**H4. The old-updater instance-control restart.** An updater older than v0.26.0 does not restart `carlos-instance-control`, and the new agent then refuses isolated wakes. Dev's boxes run v0.27.3, whose updater does restart it. Check `ActiveEnterTimestamp` after every roll anyway. Changing host.env never restarts instance-control on its own; step 5.1 restarts it explicitly. |
| + |
| +**H5. Runtime masks lose to `/etc`.** Blocker 3. Never use `systemctl mask --runtime` or `unmask --runtime` on leo. Use only the offline transaction, and check `systemctl show carlos-edge.service --property=LoadState,ActiveState` reads `masked` and `inactive` before migrating. |
| + |
| +**H6. Empty app segment.** Blocker 2. Leo's three app-less `proof*` routes wrote error objects under `apps/bab//errors/leo-climbing/` (newest 06:46Z on 2026-10-16); they are being removed. Clean up only after the e8e961dd release is on leo and Paul approves; before that, new objects may reappear. |
| + |
| +**H7. The dev console as an actor.** Step 3c arms the guard on sophie with `--console` and restarts `console.service` twice (3c and 4b), so the dev console is briefly down each time. Sophie's own host claims stay off. Without the enrolled console, an exhausted wake-retry episode on leo could not be reset. The reset door was proven on platform-1e's root runner (401, 404, 200 and 409 cases). |
| + |
| +**H8. Memory.** Leo has 927 MB. Claim mode raises instance-control's cap to 256M. Watch `systemctl show carlos-instance-control.service -p MemoryCurrent` during step 7. |
| + |
| +## Later option: carlos-dev-uk |
| + |
| +Not part of this trial (Paul, 2026-10-16). Kept here for a later cohort. dennis-stenciling is placement `uk`, serves nothing, and already has its own `CARLOS_SECRETS_KEY` in host.env (fingerprint `3fe29008`, the same as sophie's). Adding it later means running steps 2 to 7 on it as well (SSM region eu-west-2, instance i-009d593f451cfd562), with these differences: |
| + |
| +- IAM: `carlos-dev-host-uk` already has `s3:GetObjectVersion` and `s3:DeleteObjectVersion` on the UK data bucket. It still needs its own fence-writer role, and the deployment bucket policy covers both boxes. |
| +- Roster: its row has region `uk`. The home designation stays the IE name. |
| +- Files seed: it has no hosts, so `carlos system files seed` reports nothing to seed. Acceptance needs an app created with `--region uk`. |
| +- Guard: `/opt/carlos/carlos` on dennis is owned `carlos:carlos`, not root, and it has no `carlos.prev`. Check with platform-1e whether `guard-arm` accepts that before step 3b on dennis. |
| +- Its `carlos-instance-control.service` is idle and socket-activated today. Step 5.1's explicit restart is still required. |
| +- It rolls on the `stable` channel with leo. |
| + |
| +## Open questions |
| + |
| +Answered on 2026-10-16: dev-uk is out of this trial; the dev console is enrolled by the activation tools; the independent writer and key rotation are accepted trial gaps; box roles no longer read `/carlos/release-key`; the activation tools (runner, offline transaction, `install-control`, console enrollment) are built on `feat/claim-activation-tools` at 0e81652a; the home name is `ie`; the `proof*` routes are being removed; step 5.0 is approved; flagship does not need the release before dev's window. |
| + |
| +For platform-1e: |
| +1. Should `hostops.Apps` skip an empty app segment instead of refusing, so one stray object cannot stop activation on a whole deployment? |
| +2. For a later teardown out of hold: how do writes made under the claim reach the claims-off restore path? |
| +3. Before any reboot while held: the persistent hold procedure the tools' report asks the controller to approve. |
| -### 5.6 Files seed |
| +## Appendix: approved files seed runbook strings |
| -Quoted verbatim from the approved copy (2026-10-15). The headings in brackets are the string identifiers. |
| +Quoted verbatim from the files seed copy Paul approved on 2026-10-15. The headings in brackets are the string identifiers. In this trial `system claims offline-seed` (step 5.1) performs the delegation, override, mask and unmask commands below; they are kept here as the approved description, not as steps to run. |
| [runbook.heading] |
| @@ -634,144 +927,3 @@ rm -- /run/systemd/system/carlos-edge.service.d/90-files-seed-activation.conf |
| [runbook.rollback] |
| > Before disabling claims, record outstanding park failures and retained disk space. A box that has enrolled a census still maintains its retained root state during update-time recovery, even when claims are off. The no-census case remains separate. Do not widen unit permissions or delete retained paths by hand; use an approved recovery procedure. |
| - |
| -Expected for leo: `Host "claimtrial.dev.carlosframework.com": seeded.` and its admitted prefix line. wakecheck is no longer on leo. The `proof*` routes, if Paul keeps them, are address-routed and have no app; record whatever is printed for them. Save the full output with the activation record. |
| - |
| -### 5.7 Leave the window, but keep serving off |
| - |
| -Follow `[runbook.finish]` above up to "Keep the traffic barrier until Plan 9 allows serving". Then unmask the six update and adopt units with the persistent-mask installer, and start their timers and paths: |
| - |
| -```bash |
| -leo 'systemctl start carlos-adopt.timer carlos-system-update.path carlos-stage.path carlos-restart.path' |
| -``` |
| - |
| -On failure anywhere in step 5 (after 5.0): stop and hold. Do not restore host.env, re-render the flag-off unit or unmask anything. |
| - |
| -## Step 6. Claims on |
| - |
| -What: lift the traffic barrier so leo serves under the claim. |
| - |
| -Paul's go: required, after 5.5 and 5.6 pass. |
| - |
| -```bash |
| -# first unmask carlos-edge.service with the persistent-mask installer |
| -leo 'systemctl start carlos-edge.service && systemctl is-active carlos-edge.service |
| -journalctl -u carlos-edge.service --since "-5min" --no-pager | grep -iE "host claim|REFUSING CARLOS_SWAP|seed" | tail -20' |
| -``` |
| - |
| -Expected: |
| -- `active`; |
| -- `REFUSING CARLOS_SWAP=overlap (host-claims-on)` once; |
| -- no "signed roster has no home region" line; |
| -- `carlos system status --console https://platform.dev.carlosframework.com` shows leo current. |
| - |
| -On failure: stop and hold. |
| - |
| -## Step 7. Acceptance |
| - |
| -Paul's go: for the reboot only. |
| - |
| -The acceptance host is the disposable app, `HOST=claimtrial.dev.carlosframework.com`. Never run these checks on wakecheck or a real app. Every curl runs outside the sandbox. |
| - |
| -1. Launch. `curl -s --resolve $HOST:443:18.200.150.210 https://$HOST/`. Expect 200 with `host=ip-172-31-19-180` and `hits=4` (the three seeded rows survived), and on leo a `carlos-instance-gen@$HOST:<n>` unit active. |
| -2. First cold wake. Wait for it to sleep: `carlos logs --app claimtrial --console https://platform.dev.carlosframework.com --since 30m` ends in "instance exited" (idle is 15 minutes). Wake it again. Expect 200, and `hits=` one higher than before: the row written at launch survived the restore. |
| -3. Second cold wake. Repeat. Expect 200 and `hits=` one higher again. |
| -4. Isolation. While it is awake: |
| - |
| - ```bash |
| - leo 'pid=$(systemctl show -p MainPID --value "carlos-instance-gen@'"$HOST"':*"); ls /proc/$pid/root/data/instances/; readlink /proc/$pid/ns/mnt /proc/1/ns/mnt' |
| - ``` |
| - |
| - Expect only the host's own directory, and two different mount namespaces. `carlos status --layout` shows the host `nested`. |
| -5. Reboot. `aws --profile kass --region eu-west-1 ec2 reboot-instances --instance-ids i-046c7fe43620944bc`. After it returns: `carlos-instance-control.service` and `carlos-edge.service` active; `/var/lib/carlos-claims` intact; no recovery or park error in `journalctl -u carlos-instance-control.service -b --no-pager`; a wake answers 200 with `hits=` continuing. |
| - |
| -Pass: all five. Record each response and the journal lines on #250. |
| - |
| -## Stop and hold |
| - |
| -This replaces a claims-off rollback for the trial. If any step from 5.0 on fails, or acceptance fails, stop the claim-mode units and leave leo exactly as it is for diagnosis. Nothing on leo is undone, repaired or retried until platform-1e has looked and Paul has decided what comes next. Only claimtrial is affected, and it holds nothing of value. |
| - |
| -Paul's go: not needed to stop. Stopping is always allowed. Anything after it needs a go. |
| - |
| -1. Save the failing step's output, its InvocationID if it was the seed one-shot, and the time. |
| -2. Stop admission and every tenant, then the root service and its socket: |
| - |
| - ```bash |
| - leo 'systemctl stop carlos-edge.service |
| - systemctl stop carlos-adopt.timer carlos-system-update.path carlos-stage.path carlos-restart.path |
| - systemctl stop carlos-instance-control.socket carlos-instance-control.service |
| - systemctl list-units --no-legend --plain "carlos-*" --state=active' |
| - ``` |
| - |
| - Expected: no `carlos-edge`, `carlos-instance-control` or `carlos-instance-gen@` unit active, and no active `carlos-*` timer or path. |
| -3. Capture evidence, read-only: |
| - |
| - ```bash |
| - leo 'systemctl show carlos-edge.service carlos-instance-control.service -p InvocationID -p Result -p ExecMainStatus -p ActiveState -p SubState |
| - journalctl -u carlos-edge.service -u carlos-instance-control.service --since "-2h" --no-pager | tail -400 |
| - ls -la /var/lib/carlos-claims /run/systemd/system/carlos-edge.service.d 2>&1; cat /proc/sys/kernel/random/boot_id' |
| - ``` |
| - |
| - Post a summary on #250 and tell platform-1e. |
| - |
| -What "hold" leaves running: the instance itself and the operating system, including systemd, journald, sshd and the SSM agent, so the box stays reachable for read-only diagnosis. Its disk and every file stay as they were at the stop. Sophie and dennis carry on as normal; sophie keeps serving wakecheck. `system status` shows leo as not reporting. |
| - |
| -What must not be touched while leo is held: |
| -- `/etc/carlos/host.env`: the claim flags and `CARLOS_POOLS` stay set; |
| -- `/var/lib/carlos-claims` and everything under it, including migration state, journals and parks; |
| -- `/etc/carlos/host-actor.json`, `/etc/carlos/host-evidence.key`, `/etc/carlos/fence-writer/` and `/opt/carlos/system/guard`; |
| -- `/opt/carlos/carlos`, `/opt/carlos/carlos.prev` and anything staged: no `carlos system update`, rollback or old-road roll, and no `stable`-channel roll while leo is held, because it is on `stable` with dennis; |
| -- `/data`, including `registry.db`, `/data/instances` and any parked database; |
| -- unit files, drop-ins (including a leftover `90-files-seed-activation.conf`) and masks, as found; |
| -- the claim's S3 objects: `control/hosts/`, `control/host-actors.json`, `control/host-origins/`, admitted prefixes, and `instances/` in `carlos-dev-ie-640012953448`; |
| -- no `systemctl reset-failed`, no reboot (the boot ID and boot-local device numbers are evidence, and boot runs recovery), no rerun of `journal-migrate` or the seed, and no console `reset-wake-retry`. |
| - |
| -Leaving hold is a separate decision: a fix and resume, or a claims-off teardown under its own reviewed procedure. This runbook does not cover either. Data written under the claim went to the host's claim location, `instances/<host>/<incarnation>`, so a later teardown has to say how it treats that (open question 7). |
| - |
| -## Known hazards |
| - |
| -**H1. wakecheck (#286).** Chosen: wakecheck leaves leo through `CARLOS_POOLS=cohort1` (step 5.0) and stays on sophie, claims off, as today. Acceptance uses the disposable `claimtrial` app in pool `cohort1`. |
| - |
| -Why: wakecheck's record is unregioned, so sophie and leo both mint it. With leo claimed and sophie not, the same host would have two writers on different lineages. Leo's claimed generations restore from the claim location; sophie writes its legacy replica and also starts from its own stale local database (#286). DNS for `*.dev.carlosframework.com` points at sophie, and bots wake wakecheck there, so it cannot be kept to leo. A wake on sophie during the window would also break the drained, replicated stop the seed depends on. The disposable app needs the pool for the same reason: without one, sophie would mint it too. |
| - |
| -A box with `CARLOS_POOLS` set serves only the listed pools, not the shared `""` pool, so leo's reconcile pass removes wakecheck. The activation preflight skips records in other pools. |
| - |
| -**H2. Shared use of leo.** platform-d8 runs zero-downtime dev checks on leo. Notify them before step 5.0 and wait for a clear answer. After step 6, leo declines `CARLOS_SWAP=overlap` with `host-claims-on`, so their checks cannot run on leo for the rest of the trial, including any hold. The pool change also means their test apps would need `--pool cohort1`. |
| - |
| -**H3. #272.** A system pointer whose `result.json` says "applied" is terminal: after an old-road roll the system leg never returns the box to its channel pointer. Roll leo only with `carlos system update`. Separately, #272's stale-artefact refusal is deferred past activation: once a host is retired on leo, installing it again on leo refuses. That fails closed. Never remove `consumed/`, lock or record files by hand. |
| - |
| -**H4. The old-updater instance-control restart.** An updater older than v0.26.0 does not restart `carlos-instance-control`, and the new agent then refuses isolated wakes. Dev's boxes run v0.27.3, whose updater does restart it. Check `ActiveEnterTimestamp` after every roll anyway. Changing host.env never restarts instance-control on its own; step 5.5 restarts it explicitly. |
| - |
| -**H5. Runtime masks lose to `/etc`.** Blocker 3. Do not start the window until platform-1e's persistent-mask installer is reviewed and shows `LoadState=masked` on leo. |
| - |
| -**H6. Empty app segment.** Blocker 2. Leo's three app-less `proof*` routes write error objects under `apps/bab//errors/leo-climbing/` (newest 06:46Z on 2026-10-16). Clean up only after `fix-empty-app-key` is on leo; before that, new objects reappear. |
| - |
| -**H7. The dev console as an actor.** platform-1e is building console enrollment, so the dev console can perform an operator `reset-wake-retry` for leo's hosts. Until it is enrolled, an exhausted wake-retry episode on leo stays blocked. What enrollment arms on sophie is part of reviewing platform-1e's runner. |
| - |
| -**H8. Memory.** Leo has 927 MB. Claim mode raises instance-control's cap to 256M. Watch `systemctl show carlos-instance-control.service -p MemoryCurrent` during step 7. |
| - |
| -## Later option: carlos-dev-uk |
| - |
| -Not part of this trial (Paul, 2026-10-16). Kept here for a later cohort. dennis-stenciling is placement `uk`, serves nothing, and already has its own `CARLOS_SECRETS_KEY` in host.env (fingerprint `3fe29008`, the same as sophie's). Adding it later means running steps 2 to 7 on it as well (SSM region eu-west-2, instance i-009d593f451cfd562), with these differences: |
| - |
| -- IAM: `carlos-dev-host-uk` already has `s3:GetObjectVersion` and `s3:DeleteObjectVersion` on the UK data bucket. It still needs its own fence-writer role, and the deployment bucket policy covers both boxes. |
| -- Roster: its row has region `uk`. The home designation stays the IE name. |
| -- Files seed: it has no hosts, so `carlos system files seed` reports nothing to seed. Acceptance needs an app created with `--region uk`. |
| -- Guard: `/opt/carlos/carlos` on dennis is owned `carlos:carlos`, not root, and it has no `carlos.prev`. Check with platform-1e whether `guard-arm` accepts that before step 3b on dennis. |
| -- Its `carlos-instance-control.service` is idle and socket-activated today. Step 5.5's explicit restart is still required. |
| -- It rolls on the `stable` channel with leo. |
| - |
| -## Open questions |
| - |
| -Answered on 2026-10-16: dev-uk is out of this trial; the dev console is enrolled by platform-1e's runner; the independent writer and key rotation are accepted trial gaps; the activation runner and persistent-mask installer are being built by platform-1e; the `apps/bab//` writer fix is `fix-empty-app-key`. |
| - |
| -For Paul: |
| -1. Remove leo's three app-less `proof*` routes (`proof.dev.carlosframework.com`, `proof.dev.oncarlos.com`, `proof2.dev.oncarlos.com`)? They are the source of `apps/bab//`, and nothing has used them since Aug 28. |
| -2. Approve `CARLOS_POOLS=cohort1` on leo (step 5.0), which takes wakecheck off leo and pins `claimtrial` to it. |
| -3. platform-1e's step 1 rolls the release to flagship after dev, claims off. Must flagship be rolled before dev's offline window, or only before a flagship cohort? |
| - |
| -For platform-1e: |
| -4. Does the activation runner also render the claim-on instance-control unit while the adopt pass is masked (5.3), or is that a separate installer? |
| -5. Dev's home roster name for `home_region`: `ie`? |
| -6. Should `hostops.Apps` skip an empty app segment instead of refusing, so one stray object cannot stop activation on a whole deployment? |
| -7. For a later teardown out of hold: how do writes made under the claim reach the claims-off restore path? |