amadan
docs: the instance-key plan carries the final fix wave: the delivery rule under ruling 5, the Built note, four known gaps and the #273 line (final review I2, I5, minor 6)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+10
−3
docs/superpowers/plans/2026-10-10-instance-key-provision.md+10 −3
diff --git a/docs/superpowers/plans/2026-10-10-instance-key-provision.md b/docs/superpowers/plans/2026-10-10-instance-key-provision.md| index dfaa8d80b..e8cf5bdd9 100644 |
| --- a/docs/superpowers/plans/2026-10-10-instance-key-provision.md |
| +++ b/docs/superpowers/plans/2026-10-10-instance-key-provision.md |
| @@ -7,7 +7,8 @@ |
| - Task 3b: the sidecar worker (`StartWorkerStaged`) and the legacy co-spawned sidecar (`startSidecarChildFrom`) pass the replica gate too, as a seed does (no control socket); a key in a worker's agent-issued `extra` is refused as of unknown origin. `replicaKeyGate` takes the key's layer; the home-keys list is a file the agent reads at every launch, not an environment variable. |
| - Task 4: the claim's guarded write mints the admin token only when the effective env has none, so a configured token is never shadowed by a second one; a missing own file with both values configured proceeds as before; the key's refusal is wrapped with the host once, in the launch source. |
| - Task 6: the activator's start sequence is a named, tested function (`activatorStart.Run`), with the backfill between layout recovery and the pass's publication on a legacy box and nothing on a claim box. |
| -- Known gaps, obligations and follow-ups are at the end, with the rollout's two pre-roll checks. |
| +- Final fix wave (the two final reviews, 2026-10-11; rulings I1 to I5 and P3 in the ledger): nothing is minted until the row's configuration is delivered, by a fresh bucket read before the mint (`ExecBackend.ConfiguredKey`, the field Task 3b called `ReplicaConfigKey`; the delivery rule is under ruling 5; `ErrInstanceKeyDeferred`; the backfill's one-minute budget and its "waiting for their configuration" count); the flat own file and its configuration are opened `O_NONBLOCK` and must be regular files (a FIFO is refused, never waited on), and the flat configuration read is capped at 1 MiB as the nested one is; `SnapshotClaimLaunch` is the gateless read the claim's input policy uses and `SelectClaimLaunch` the launch sources' read with the gate, so an identity check never reads the bucket and keeps `ErrChanged`; the claim path's backend is built by `claimExecBackend` with the same two readers the agent's backend gets (`instanceKeyReaders`); the gate's refusal line is said once an hour per host and reason; the adoption verb refuses a body with anything after its one object; the zero `InstanceKeyOutcome` is `InstanceKeyFailed`, every error's companion; `ensureInstanceKeyFor` and `BackfillInstanceKey` take a context, and the exported ensure wrappers are the unexported `ensureInstanceKeyNoConfig` pair, used by tests. The rollout gained the keyless-replica sidecar and worker check. |
| +- Known gaps, obligations and follow-ups are at the end, with the rollout's pre-roll checks. |
| **Goal:** The platform mints every exec-backed instance's `CARLOS_INSTANCE_KEY` on the box, create-only and durably, into the instance's own `<host>.env` before any process of the instance starts, and backfills it into every existing instance. Relocation accepts a key only with evidence that the app actually seals with it: its own `instance.key`, or an adoption record bound to the writer whose data is pinned, for the exact key the target will deliver, pinned by digest and checked again at install. |
| @@ -194,7 +195,9 @@ Each ruling ends with its cost if wrong. Rulings 1 to 12 are rev 1's, amended wh |
| **4. Create-only, append-only, durable every time (amended for finding 2).** A non-empty `CARLOS_INSTANCE_KEY` in `<host>.env`, of any shape, is left exactly as it is: two lines stay two, and readers take the last. The key is appended on a fresh line, never rewritten. Every ensure that reports success, whether it minted or found the key, has done three things on that call: `fchmod 0600` on a descriptor it opened, fsync of the file, and fsync of its directory. A mint whose sync failed returns the error and leaves its line in place. The next ensure finds the key present, keeps its bytes and redoes all three steps. It succeeds only when they succeed, so no process is started with a key a power cut could take back. An existing file is opened with mode 0 (finding 3). *Cost if wrong:* one open, fchmod and two fsyncs per launch, on a file of a few hundred bytes. |
| -**5. A key from the app's configuration wins, and the platform mints none (amended for finding 4).** If the effective env (ruling 13) takes its key from `<host>.config.env`, the mint writes nothing and logs once an hour. Minting would put a key into `<host>.env`, which overrides the configured one, and change the app's key under its data. *Cost if wrong:* an app that sets the name and later unsets it gets a fresh platform key at its next start, by its own action. Reserving the name instead was open question 2; the provisional default is not to. |
| +**5. A key from the app's configuration wins, and the platform mints none (amended for finding 4; the delivery rule added by the final fix wave).** If the effective env (ruling 13) takes its key from `<host>.config.env`, the mint writes nothing and logs once an hour. Minting would put a key into `<host>.env`, which overrides the configured one, and change the app's key under its data. *Cost if wrong:* an app that sets the name and later unsets it gets a fresh platform key at its next start, by its own action. Reserving the name instead was open question 2; the provisional default is not to. |
| + |
| +*The delivery rule (final review I2, Astra's P2).* The files say what this box has delivered, not what the configuration sets: the fast-converge pass reconciles a row ready and syncs its configuration last, so a first wake can run before `<host>.config.env` is rendered, and a key minted then shadows the configured one for good once it arrives. `SyncExec` removes the file for a route that was never configured, so the file's existence cannot be the delivery signal. The mint therefore asks the bucket, as the replica gate does (`ExecBackend.ConfiguredKey`, `appconfig.ConfiguredKeyDigest`), once, under `adminTokenMu`, only when nothing delivered here sets a key and a mint is next: a key the configuration sets and this box has not delivered defers the launch (`ErrInstanceKeyDeferred`), so does a bucket that cannot be read, and a configuration that sets no key lets the mint go ahead. The launch fails and is retried; the config sync's next pass delivers the file and the next start finds the key configured. A key already delivered, in either file, costs no read. Which boxes have a configuration to read follows the config guard (`configguard.go`): no deployment store under `CARLOS_NO_SEALED_CONFIG` is no configuration source, and the mint proceeds; no store otherwise is an error. *Cost:* a delayed first start while the configuration is in flight (seconds, one pass), and one bucket read per instance, ever. The window the read cannot close is a key configured after the read and before the append, the same in-flight window ruling 27 names for replicas; a key configured after an instance has minted is a rotation, which this plan does not support. |
| **6. Durability before the first start.** |
| - *Legacy boxes:* nothing can publish today, and region HA cannot be on, because it requires host claims. The key's durability is the box's disk, now fsynced, and a move's sealed archive. A dead legacy box loses its keys with its data; that is today's state, unchanged (row 10). |
| @@ -5252,11 +5255,15 @@ Question 5 (restarts inside a claim session) is withdrawn: its premise was wrong |
| - **The residual reclassify window (Task 3's review, #273).** `ensureInstanceKeyFor` refuses a fenced or conflicting row on the paths the launch body classified, but a journal written after `pathsFor` and before the append still mints flat, and `instanceEnv` classifies a third time. Both sit inside the layout pass's fence (`FenceAndStopIfUnfenced`). The admin-token mint has the same window. Closing it means refusing at the top of the launch body: #273's. |
| - **A replica or configured-key row stays at its mode (Task 4's review).** On the claim path a 0644 own file that holds only the token is not tightened: the key's ensure writes nothing on those rows, and the token's mode has never been the claim path's to set. Every launch of such a row enters the guard to do nothing. Not a regression. |
| - **The non-linux `safefile.Write` fsyncs the file, not the directory** (Task 5's review). Boxes are Linux; dev only. |
| +- **The delivery rule's window (final fix wave).** The mint reads the bucket and then appends; a key configured between the two is shadowed, as it would be by a mint one second before the operator's change. The read narrows the gap from "until the next config sync" to the length of one mint; nothing short of reserving the name closes it, and a configured key on an app that has minted is a rotation either way. |
| +- **A box with a deployment store and no `CARLOS_SECRETS_KEY` never mints for an app whose configuration sets a key** (final fix wave). Its config sync loop is off, so the file is never delivered, and the mint defers at every start with a line naming the deferral. The box is misconfigured, as the config guard already says of it for sealed values; the fix is the key. |
| +- **The backfill's cost at agent start (final fix wave).** Each keyless row with an own file costs one bucket read before its key is minted, serial under `adminTokenMu`, before the activator starts. The pass has a one-minute budget, so a box whose bucket is unreachable at start is held at most a minute and the rows past the budget mint at their next start. With the bucket up it is a few hundred milliseconds a row, once. |
| +- **The second read asymmetry on the flat path (final review minor 6, #273).** `instanceEnvParts` reads a flat own file through `readEnvFile`, a plain `os.ReadFile`, right after the mint refused a link or a FIFO at it through `openOwnEnvNoFollow`; a link planted between the two is followed by the read. The pre-branch shape, as the token's. Listed on #273 so the token fix closes both with one shared open; the first asymmetry (the unbounded flat configuration read) is closed. |
| ## Follow-ups this plan does not build |
| - **`RoleLaunch.Start` refuses by name until claim Plan 9 Group D.** `TestStartAdmittedRetriesReuseTheirInputsAndMaterializeFirst` (claimretry_instancekey_linux_test.go) pins the key's survival across a refused start by relying on that refusal and asserting `ErrClaimIsolationUnavailable`. When Group D makes `Start` launch, the test's `want` list and that assertion change: a tripwire to expect, not a regression. |
| -- **#273, the admin token's own holes.** Three things this plan fixed for the key and left for the token, filed together: the flat mint path follows a symlink at `<host>.env` (Task 2 opens with `O_NOFOLLOW`; the token's `Lstat` guard is check-then-open on the present path and absent on the mint path); the token is minted before the fenced/conflict refusal on a row reclassified mid-wake (Task 3 refuses inside `ensureInstanceKeyFor`; the claim closure mints the token first); and the residual reclassify window above. One fix: refuse at the top of each launch body and share the key's open. |
| +- **#273, the admin token's own holes.** Three things this plan fixed for the key and left for the token, filed together: the flat mint path follows a symlink at `<host>.env` (Task 2 opens with `O_NOFOLLOW`; the token's `Lstat` guard is check-then-open on the present path and absent on the mint path); the token is minted before the fenced/conflict refusal on a row reclassified mid-wake (Task 3 refuses inside `ensureInstanceKeyFor`; the claim closure mints the token first); the residual reclassify window above; and the flat own file's second read in `instanceEnvParts`, by `readEnvFile`, which follows a link or waits on a FIFO the mint's open just refused (final review minor 6). One fix: refuse at the top of each launch body and share the key's open, `openOwnEnv`, for every read of the file. |
| - **O8, the home's platform key for replicas** (replica placements, #162; platform keys only, ruling 28). Until it lands, replicas run only with a verified configured key, or keyless with a control socket. |
| - **O7, then O6.** Both are other plans' work. O7 (moves' artifact lifecycle) ships first; O6 (the writer, on claim launches) ships only after it (ruling 24), so no record is ever written that moves cannot retire. |
| - **Low-entropy configured keys.** A `config` record publishes the SHA-256 of a configured key in the data bucket (ruling 9). For a random key that reveals nothing. For a short key an owner typed by hand, a data-bucket reader could test guesses against it. Rastrillo's docs could say that a configured `CARLOS_INSTANCE_KEY` should be random. |