amadan
docs: cohort-1 runbook: 7a complete (KMS teardown applied, #48 merged 5cf9a0ed); swap and staging-file gaps owed before production
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+16
−4
docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md+16 −4
CI runner disappeared — run details
diff --git a/docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md b/docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md| index 28fcba17f..fbf16843d 100644 |
| --- a/docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md |
| +++ b/docs/superpowers/plans/2026-10-16-claim-activation-dev-cohort1.md |
| @@ -312,6 +312,8 @@ Paul deferred these for this trial only. They are not blockers here, and they mu |
| - **Dev module drift.** A whole-root `tofu plan` of `platform-infrastructure/dev` on 2026-10-17 showed 15 adds and 1 change (13 adds after leo's `host_buckets` change and the box-records bucket policy were applied for the trial; whole-root plan from main `a0b3f5a` on 2026-10-18: `13 to add, 0 to change, 0 to destroy`) that come from the platform modules (pinned `?ref=main`) moving ahead of dev: termination SQS queues and EventBridge rules for both IE boxes, the deployment bucket's lifecycle rule and its box-records bucket policy, leo's object-version actions (`host_buckets` gains `PullDoorObjectVersions`, `s3:DeleteObjectVersion` and `PullDoorVersionList`; applied for the trial on 2026-10-18), and an SSM parameter scope policy on both IE roles (a deny on parameter reads outside `/carlos/*`). It is owed: its own plan and Paul's approval later, before production cohorts. It is not part of step 7a, whose plan is targeted at its own three resources. Step 3's IAM overlaps it (see open question 3). |
| - **A trusted build attestation for the console.** `publish-identity` (step 6) accepts a console artefact only if its `version.Build` stamp is the release tag. It reads that stamp from the `-ldflags` the Go build info records (`release.ArtifactBuild`), because release binaries are stripped (`-s -w`) and the linked value itself cannot be read. So the phase checks what the linker was told, not what it linked. Production cohorts need an attestation from a trusted builder, tied to the console artefact's sha256, that says which source and which stamp produced those bytes. The trial accepts the gap because only the operator holds the release key, so only the operator can ship and promote a console build the phase would accept. |
| - **Live denied-operation witnesses.** For the trial, step 3 item 4 ran as IAM policy simulations only (platform-1e's ruling, 2026-10-18). Before production cohorts, the denied writes must be tried live with the boxes' own credentials, through a reviewed runner. Nothing gets installed on the boxes for the trial. |
| +- **Leo has swap.** `swapon --show` on leo lists a swap device (7a.2 pre-checks, 2026-10-19). A tmpfs page can be swapped to disk, so `/run` alone does not keep a secret off disk on that box. Production cohorts should use `fence-credentials --source -` (stdin, #301), or stage only on a box without swap. |
| +- **The fence credentials touched a staging file.** In 7a.2 the credentials sat in `/run/carlos-fence-stage/credentials.json` on tmpfs for about one second before the `trap` removed it. Production cohorts should use `fence-credentials --source -` (stdin, #301), so no staging file exists at all, or a box without swap. |
| ## Step 1. Release and roll, claims off |
| @@ -1679,6 +1681,8 @@ Straight after step 7 passes, or at any stop inside the session. Paul's go is no |
| ## Step 7a. Fence-writer credentials |
| +**Step 7a complete (2026-10-19).** 7a.1 created the key, alias and grant. 7a.2 installed leo's credentials, deleted the parameter and tore the key down (PR carlosframework/platform-infrastructure#48, merged as `5cf9a0ed`). The key `ef26a257` is `PendingDeletion` until 2026-10-26T20:26:18Z. The record is at the end of 7a.2. |
| + |
| Ruled by platform-1e on 2026-10-17: option (a), a one-time SecureString transfer, with two tightenings. This step installs leo's fence-writer credentials (the bootstrap user's access key and the role ARN from step 3) at `/etc/carlos/fence-writer/credentials.json`. |
| Scope: the independent claim writer (Plan 15) is deferred for this trial. This step protects the trial's fence credentials in transit. It does not provide Plan 15's writer boundary. |
| @@ -1941,13 +1945,13 @@ On failure: hold, but always finish parts 3 to 5 first, so no readable copy of t |
| Run by the executor for the controller, on Paul's go confirmed directly with the controller, with the steps as platform-1e specified them. Commands went from DOCS-T1 (profile `kass`, `user/remote-agent`), one SSM `send-command` per leo block. No credential value appeared in a command line, terminal output, an SSM parameter or output, a log, git or a kept file. This record holds only the access key ID's last 4 characters and SHA-256 prefixes. |
| -**7a.2 PASS for parts 1 to 4: credentials installed on leo, parameter deleted after about 18 seconds, CloudTrail clean. Part 5 (the KMS teardown) is planned, not applied: it waits for Paul's go.** |
| +**7a.2 PASS, and 7a complete: credentials installed on leo, parameter deleted after about 18 seconds, CloudTrail clean. Part 5's KMS teardown was applied at 20:26Z under Paul's 7a.2 go, with platform-1e's OK.** |
| Where this run differed from the blocks above: |
| - There was no `fence-credentials.json` from step 3 (the bootstrap user had 0 access keys). The key was created in this sitting: `iam create-access-key` wrote straight to a `0600` file in a private `0700` scratch directory under `$TMPDIR` on DOCS-T1. A Python step built the source file from it (`O_EXCL`, `0600`, compact JSON, 218 bytes) in the exact format `decodeFenceCredentials` takes at `f3bf1214` (`internal/hostfence/writer.go`). That format is five fields, `role`, `region`, `bucket`, `access_key` and `secret_key`, and the decoder refuses unknown fields. Values: role `arn:aws:iam::640012953448:role/carlos-dev-ie-b-fence-writer`, region `eu-west-1` (the bucket's location), bucket `carlos-dev-640012953448`. |
| - The fence role ARN was written into the leo block instead of staged with `box_put`. It is an ARN, not a secret. |
| - The leo block ran inside a shell function fed `</dev/null`, so no command in it could read the rest of the script from bash's stdin. Besides the runbook's fingerprint, it printed a presence flag and `secret_sha8`, the first 8 hex digits of the SHA-256 of `secret_key`. |
| -- Part 5 stopped at the plan. The parameter is gone, so nothing is left to decrypt, but the key, the alias and leo's grant stay live until the apply. |
| +- Part 5 stopped after the plan for a separate OK. The apply followed about five minutes later, from the same saved plan file. |
| Pre-checks, read-only, 20:18Z to 20:20Z, all PASS: |
| - The roster `control/host-actors.json` is 2227 bytes, etag `a69f6d93…`, revision 9, `published`, home `ie`. `leo-climbing` is `active` with its census key bound. Leo's report (20:17:34Z) shows `floor_guard_ready` `True`, floor `1a6a7e50`, binary `736826e4`, `v0.28.4`. |
| @@ -1980,9 +1984,17 @@ CloudTrail since 20:20:00Z, read at 20:24Z in eu-west-1 and us-east-1. No creden |
| - The two `SendCommand`s carry the base64 script only. `DeleteParameter`, `DescribeParameters` and `GetParameter` (`ParameterNotFound`) carry the name only. |
| - A scan of every matching event's request and response for 40-character key-like strings found 0. |
| -Part 5, the KMS teardown, planned and not applied. On DOCS-T1 there is a fresh worktree of the platform-infrastructure GitHub clone, `platform-infrastructure-cohort1-kms-remove`, on branch `claim-cohort1-fence-kms-remove`, made from main `a0b3f5a`. `dev/cohort1-fence-kms.tf` is removed in commit `c31fd38`, as PR carlosframework/platform-infrastructure#48. The targeted plan (the three `-target`s above) is saved as `dev/cohort1-fence-remove.plan`, sha256 `3c8d5f67…5e9e`: `Plan: 0 to add, 0 to change, 3 to destroy.` It deletes `aws_iam_role_policy.cohort1_fence_read`, `aws_kms_alias.cohort1_fence` and `aws_kms_key.cohort1_fence` (7-day window). Nothing else changes, outputs included. After Paul's go, apply that plan file, then run part 5's two checks (`allowed`, then `PendingDeletion`). |
| +Part 5, the KMS teardown, planned and not applied. On DOCS-T1 there is a fresh worktree of the platform-infrastructure GitHub clone, `platform-infrastructure-cohort1-kms-remove`, on branch `claim-cohort1-fence-kms-remove`, made from main `a0b3f5a`. `dev/cohort1-fence-kms.tf` is removed in commit `c31fd38`, as PR carlosframework/platform-infrastructure#48. The targeted plan (the three `-target`s above) is saved as `dev/cohort1-fence-remove.plan`, sha256 `3c8d5f67…5e9e`: `Plan: 0 to add, 0 to change, 3 to destroy.` It deletes `aws_iam_role_policy.cohort1_fence_read`, `aws_kms_alias.cohort1_fence` and `aws_kms_key.cohort1_fence` (7-day window). Nothing else changes, outputs included. |
| + |
| +Part 5 applied, 20:26:15Z to 20:26:19Z, under Paul's 7a.2 go with platform-1e's OK, as `remote-agent` (profile `kass`). The plan file's sha256 was checked first (`3c8d5f67…5e9e`, unchanged). `tofu apply cohort1-fence-remove.plan` printed `Apply complete! Resources: 0 added, 0 changed, 3 destroyed.`, with no stale-plan error. Checks afterwards: |
| +- `get-role-policy` on `carlos-dev-host-b` for `cohort1-fence-read` returns `NoSuchEntity`. The role's inline policies are now `carlos-dev-host-b-buckets`, `carlos-dev-host-b-metrics` and `deny-release-key-reads`. |
| +- `describe-key` on `ef26a257-fc1d-4fe3-ad9e-77a4b7b789f6` shows `PendingDeletion`, with `DeletionDate` 2026-10-26T20:26:18Z (the 7-day window). Until then, `kms cancel-key-deletion` can undo it. |
| +- `list-aliases` has no `alias/cohort1-fence-leo`, and no alias targets the key. |
| +- `simulate-principal-policy` for `carlos-dev-host-b`, `ssm:GetParameter` on the fence parameter, returns `allowed`, not `implicitDeny`. The matched statements are `AmazonSSMManagedInstanceCore` and `carlos-dev-host-b-buckets` (the `/carlos/*` read). That is part 5's own expected result above. It carries no exposure: the parameter does not exist, and the key that encrypted it is pending deletion with no decrypt grant left. |
| + |
| +PR #48 was squash-merged as `5cf9a0eda905eb523aaa3c5e742dec79ed7179e0` (main had not moved from `a0b3f5a`). The saved plan, the worktree's `.terraform` and the worktree itself were then removed, along with the local branch. |
| -State now: leo holds the only copy of the fence-writer credentials, root `600` at `/etc/carlos/fence-writer/credentials.json`. The bootstrap user has one `Active` access key (`…VINM`). No parameter exists. The key, the alias and leo's read grant are live until the apply. Leo's host claims are still off, and no host.env or flag changed. Next: the KMS apply on Paul's go, then step 8 with its own go. |
| +State now: leo holds the only copy of the fence-writer credentials, root `600` at `/etc/carlos/fence-writer/credentials.json`. The bootstrap user has one `Active` access key (`…VINM`). No parameter exists. The trial key is pending deletion, and its alias and leo's read grant are gone. Leo's host claims are still off, and no host.env or flag changed. Next: step 8, with its own go. |
| ## Step 8. Leo's attended offline window |