instance-key-build
Instance key provision: the platform mints each instance's key (region HA L21).
Why. Region HA restores a dead box's instance from its replicated database and its files backup. Today an app creates its own instance key when it first starts. If that key is never backed up, the data sealed with it can't be read after a restore. Paul decided on 2026-10-09 that the platform creates the key, before the app first starts and backed up with the instance's files. Rastrillo's side shipped as carlos.InstanceKey (rastrillo e3ea08d, rastrillo#66). The app's own key wins. Otherwise the app takes $CARLOS_INSTANCE_KEY. On CARLOS with no key at all, the app refuses to start.
What changes:
internal/instancekey: one rule for the env parse, the effective layer, the digest and the canonical adoption record.- The mint.
CARLOS_INSTANCE_KEYis written into<host>.env, the file that already holdsCARLOS_ADMIN_TOKEN: - create-only: an existing key is never rewritten; - durable before success: chmod 0600, file fsync and directory fsync; - flat files are openedO_NOFOLLOW|O_NONBLOCK, regular files only; - a failed write is truncated back; - fenced and conflicting rows are refused; - nothing is minted until the instance's configuration has been delivered. The bucket is asked, in two phases, outsideadminTokenMu. A configured key always wins. - Every launch path mints: seed, spawn, nested prepare, and the claim launch source, which mints under the guard and names the key it delivers. The startup backfill does the same on legacy boxes only (
activatorStart.Run). - Replica gate (ruling 26). A replica runs only with its home's verified key, or with no key and a control socket the helper refuses on. The verified keys are in
/etc/carlos/replica-home-keys, which is re-read at every launch. Every replica launch path is gated: seed, spawn, worker, flat and nested sidecar, every nested role, andSelectClaimLaunch. - Adoption verb (
PUT /instance-key/adoption). It writes a record bound to the writer, and publishes it through a session relay. It is mounted nowhere until O6 and O7. - Re-render and moves. The key survives a re-render and travels byte for byte through a move.
Dormant in effect. An env var no app reads yet is harmless. Apps adopt it by moving to rastrillo's helper. The behaviour changes on current fleets are listed in the plan's rollout section, and each one has a pre-roll check:
- a symlinked or FIFO flat
<host>.envnow fails launch; - keyless replicas' legacy sidecars and workers are refused.
Review:
- Plan (
docs/superpowers/plans/2026-10-10-instance-key-provision.md): Astra, seven rounds, ready to build at rev 5.1. - Tasks: each task had its own review and fix rounds; reports are in the branch's SDD ledger.
- Final review: Fable and Astra, with one fix wave, then two scoped re-reviews. The last one judged it merge-ready at 114ef741.
- Main merged in at f6a2aa5f (the zdd surfaces). Main added no new launch paths.
Tests:
- Every guard has a test that fails when the guard is flipped.
- The full suite passes, sandbox off: 94 packages.
Follow-ups, not built here:
- #273: the admin token's own symlink, reclassify and second-read gaps.
- O6: a control socket and the adoption verb on claim launches.
- O7: moves retire the adoption record.
- O8: replicas fetch the home's key.
The plan's "Known gaps" section lists the rest.
Merging is Paul's call.
Discussions
- No discussions on this branch yet.
Tasks
- No tasks yet.
Activity
- claude agent, operated by paul@keymail.dev pushed 258dd81ee docs: the instance-key rollout orders the backfill before any move of a keyless instance (merge review residual)
- claude agent, operated by paul@keymail.dev pushed f6a2aa5f2 Merge origin/main into instance-key-build (zdd surfaces)
- claude agent, operated by paul@keymail.dev pushed 114ef7412 activator: the two-phase ensure's unasked-mint deferral is pinned by a test, and the slow-read test waits for host A to be inside its read before host B starts.
- claude agent, operated by paul@keymail.dev pushed 5fe080b68 activator: the mint's bucket read runs outside adminTokenMu, and the decision is the locked read's alone (final re-review)
- claude agent, operated by paul@keymail.dev pushed a0c906cc1 docs: the instance-key plan carries the final fix wave: the delivery rule under ruling 5, the Built note, four known gaps and the #273 line (final review I2, I5, minor 6)
- claude agent, operated by paul@keymail.dev pushed b628e4eea activator: one guard for the two ensure entry points, one open and one sync for both layouts, the test seams in one struct, the dropped key fields named (final review minors 3, 4, 7)
- claude agent, operated by paul@keymail.dev pushed 1ea86df4c activator: an adoption body is one object and nothing after it (Astra P3, final review minor 1)
- claude agent, operated by paul@keymail.dev pushed 0ee78be84 activator: the replica gate's refusal is logged once an hour per host and reason, and the rollout lists keyless replicas' sidecars and workers (final review I5, minor 5)
- claude agent, operated by paul@keymail.dev pushed 3c213fda5 host: the claim path's backend gets the instance-key readers (final review I1)
- claude agent, operated by paul@keymail.dev pushed 92e6d5c3c activator: the replica gate leaves the claim's identity check and runs in the launch sources only (final review I4)
- claude agent, operated by paul@keymail.dev pushed 96a5ac092 activator: nothing is minted until the row's configuration is delivered (final review I2, Astra P2; minor 2)
- claude agent, operated by paul@keymail.dev pushed c74d0cda4 activator: flat env and config reads open O_NONBLOCK and accept only a regular file (final review I3, minor 6)
- claude agent, operated by paul@keymail.dev pushed 365422843 Instance key provision: Astra final build review, unedited
- claude agent, operated by paul@keymail.dev pushed 3e32533ea Instance key provision: Astra final build review brief
- claude agent, operated by paul@keymail.dev pushed 8ff16f4ee docs: the instance-key plan matches what was built, with a Built note, the rollout's two pre-roll checks, the known gaps, O6's threading and loop obligations, and the follow-ups (end sweep)
- claude agent, operated by paul@keymail.dev pushed c283e1b2a test: the re-render test proves SyncExec rendered and checks its errors, the moves tests print lengths, and a flat row's Start is pinned end to end (end sweep, tasks 3 and 7)
- claude agent, operated by paul@keymail.dev pushed 69fd647e4 hostfiles: the platform relay logs a stage's failure and recovery once each, as the publisher's loop logs a scan, not every retry (end sweep, task 5)
- claude agent, operated by paul@keymail.dev pushed 42f0cd8bd activator: the adoption verb answers 500 for a launch state whose record the pinner could not read, the 500 guards are tested, the swapped sync is restored by t.Cleanup, and no test message prints a record (end sweep, task 5)
- claude agent, operated by paul@keymail.dev pushed b0a6a3889 activator: the claim path's mode comment says what the key's ensure does, the claim snapshot test pins the delivered configured key, and a failure message prints the own file's length (end sweep, task 4)
- claude agent, operated by paul@keymail.dev pushed b42dedcfb test: the instance key survives re-render and travels byte for byte through a move (L21)
- claude agent, operated by paul@keymail.dev pushed 0e766ee2a activator: the claim launch source names the host once around a key refusal, and the tests pin host-once on every path (task 6 fix round 1)
- claude agent, operated by paul@keymail.dev pushed f9372fd5f agent: the activator's start sequence is a named, tested function, so a claim box never backfills and a legacy box backfills between recovery and the pass's publication (task 6 fix round 1)
- claude agent, operated by paul@keymail.dev pushed 2cd8d79a7 activator: the instance-key refusals name the host once, through their callers (task 3 review minor)
- claude agent, operated by paul@keymail.dev pushed 4c9256bb3 agent: backfill the instance key into existing <host>.env files at startup (L21)
- claude agent, operated by paul@keymail.dev pushed 1fbc04cf8 activator: the instance key's adoption verb, bound to the launch's writer and published on the session (L21)
- claude agent, operated by paul@keymail.dev pushed 4a7ee0e60 claim: the launch source mints the instance key under the guard, syncs it before Establish, gates a replica on its home's key, and names the key it delivers (L21)
- claude agent, operated by paul@keymail.dev pushed 3ab3cbe19 activator: the legacy co-spawned sidecar on a replica host is gated as the worker is, and refusals name /etc/carlos/replica-home-keys (L21 fix round 1)
- claude agent, operated by paul@keymail.dev pushed 37d652df6 activator: a replica row's worker is gated as its seed is, and the nested test isolates the home-key digest check (L21)
- claude agent, operated by paul@keymail.dev pushed 366067d54 activator: a replica launches only with its home's key, verified, or none and a socket its helper refuses on (L21)
- claude agent, operated by paul@keymail.dev pushed f896570c3 activator: seed, start and nested prepare mint the instance key durably before the env is read, and the mint refuses a row fenced or conflicting on the launch body's own paths (L21)