carlos / platform Public

Clone
git clone https://amadan.net/carlos/platform

Plain git — no account needed to clone.

Download

Download this file

File too large to show in full — download it above (showing a truncated preview below).

Instance Key Provision (Region HA L21) Implementation Plan, rev 5.1

> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Built (branch instance-key-build, tasks 1 to 7 and 3b, 2026-10-11). The code blocks below are the branch's. Where the build diverged from rev 5.1, each by a ledger ruling (.superpowers/sdd/2026-10-10-instance-key-provision/progress.md):

  • Task 2: the flat own file is opened with O_NOFOLLOW (openOwnEnvNoFollow, with an Lstat stand-in for windows), a failed append is truncated back to the descriptor's size before the error returns, and ensureInstanceKeyFor refuses a fenced or conflicting row itself (the launch bodies classify again after the dispatcher's check). A symlinked flat <host>.env now fails the launch; the rollout checks for one first.
  • Task 3b: the sidecar worker (StartWorkerStaged) and the legacy co-spawned sidecar (startSidecarChildFrom) pass the replica gate too, as a seed does (no control socket); a key in a worker's agent-issued extra is refused as of unknown origin. replicaKeyGate takes the key's layer; the home-keys list is a file the agent reads at every launch, not an environment variable.
  • Task 4: the claim's guarded write mints the admin token only when the effective env has none, so a configured token is never shadowed by a second one; a missing own file with both values configured proceeds as before; the key's refusal is wrapped with the host once, in the launch source.
  • Task 6: the activator's start sequence is a named, tested function (activatorStart.Run), with the backfill between layout recovery and the pass's publication on a legacy box and nothing on a claim box.
  • Final fix wave (the two final reviews, 2026-10-11; rulings I1 to I5 and P3 in the ledger): nothing is minted until the row's configuration is delivered, by a fresh bucket read before the mint (ExecBackend.ConfiguredKey, the field Task 3b called ReplicaConfigKey; the delivery rule is under ruling 5; ErrInstanceKeyDeferred; the backfill's one-minute budget and its "waiting for their configuration" count); the flat own file and its configuration are opened O_NONBLOCK and must be regular files (a FIFO is refused, never waited on), and the flat configuration read is capped at 1 MiB as the nested one is; SnapshotClaimLaunch is the gateless read the claim's input policy uses and SelectClaimLaunch the launch sources' read with the gate, so an identity check never reads the bucket and keeps ErrChanged; the claim path's backend is built by claimExecBackend with the same two readers the agent's backend gets (instanceKeyReaders); the gate's refusal line is said once an hour per host and reason; the adoption verb refuses a body with anything after its one object; the zero InstanceKeyOutcome is InstanceKeyFailed, every error's companion; ensureInstanceKeyFor and BackfillInstanceKey take a context, and the exported ensure wrappers are the unexported ensureInstanceKeyNoConfig pair, used by tests. The rollout gained the keyless-replica sidecar and worker check.
  • Known gaps, obligations and follow-ups are at the end, with the rollout's pre-roll checks.

Goal: The platform mints every exec-backed instance's CARLOS_INSTANCE_KEY on the box, create-only and durably, into the instance's own <host>.env before any process of the instance starts, and backfills it into every existing instance. Relocation accepts a key only with evidence that the app actually seals with it: its own instance.key, or an adoption record bound to the writer whose data is pinned, for the exact key the target will deliver, pinned by digest and checked again at install.

Architecture: A small leaf package, internal/instancekey, holds the one rule for the key: the env parser, which file's key the app receives, and the adoption record. The activator mints by it (instancekey.go), right after the existing CARLOS_ADMIN_TOKEN mint on every launch path and under the same lock, and makes the file durable on every call. A control-socket verb, PUT /instance-key/adoption, records a process's adoption beside <host>.env, bound to the writer the agent admitted it as, and publishes it on the running session through hostfiles.PlatformRelay; claim launches mount it once they bind a control socket (O6). A startup backfill in internal/host appends a key to every existing <host>.env on boxes where host claims are off. On claim boxes the launch source mints under the claim's guard before Materialize. Stage B's pinner (O2'), the C8 provider (O1'), the relocation install (O3') and moves (O7) carry the evidence across boxes and retire it. No flag.

Tech Stack: Go, crypto/rand, encoding/base64, crypto/sha256, internal/instancefs (openat2), internal/safefile, the registry, the claim launch source, the control socket.

Spec:

  • Decision: the L21 row of docs/superpowers/plans/2026-10-07-region-ha-1b-next-relocation.md on branch region-ha-1b-next-build (line 357; Paul, 2026-10-09).
  • Region HA spec docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md: H6 (line 131) and §5.6 (line 539): instance.key, report.token, <host>.env are always continuous.
  • Framework contract: rastrillo/rastrillo discussion #66 (agreed; landed on rastrillo main as e3ea08d). carlos.InstanceKey(own string) (string, error) resolves, in order: (1) own, the app's own key; (2) $CARLOS_INSTANCE_KEY, verbatim; (3) on CARLOS (carlos.Running(), which checks $CARLOS_CONTROL_SOCKET) with no key, ErrNoInstanceKey; (4) off CARLOS, ErrNotOnCarlos, today's behaviour. Read at e3ea08de in ~/amadan.net/rastrillo/rastrillo (carlos/instancekey.go, carlos/schedule.go:28).
  • Stage B plan docs/superpowers/plans/2026-10-10-region-ha-1b-next-stage-b.md on region-ha-1b-next-build (919662d2): rulings 25 and 55, Task 8b step 0 (its O2' variant with a KeyEvidence reader) and Task 8c (O3). This plan's O2' replaces that step.
  • Review: Astra rounds 1 to 6, docs/superpowers/plans/CODEX-PLAN-INSTANCE-KEY-R1.md to -R6.md (verdict each time: not ready; rounds 3 to 6 found no P1). Their findings are answered in the six tables below, newest first.
  • Task 1 is built and reviewed on this branch (c568c13b, afa89057, e8487c3a); its code below is the branch's.

Rev 5.1: round 6 dispositions

Astra's round 6 (CODEX-PLAN-INSTANCE-KEY-R6.md, reviewed at 2c39819c) closed R5-2 and R5-3, and judged R5-1's approach sound. It found two P2s. Task 1 is now built and reviewed on this branch (c568c13b, afa89057, e8487c3a). Rev 5.1 changes none of Task 1's code; it copies the reviewed code into Task 1 so the plan and the branch match.

| # | Finding | Disposition | Change | Test | |---|---|---|---|---| | R6-1 | P2. A concurrent config removal bypasses the fresh check. The launch captures configured key A, but the gate re-read the files to find the source: a sync that removed the key in between makes Effective say LayerNone, and the gate admitted the launch without asking the bucket, while the child still got A. | Accepted (ruling 29). Verified: rev 5's gate called instanceEnvParts again. | One read decides both the env and the key's source: instanceEnvLayer returns mergeEnv's env and Effective's layer from the same two maps. Every gate call (Seed, Start, the nested prepare, the claim snapshot) passes that layer. The gate reads no env file, and refuses a key whose layer is unknown. | TestReplicaGateDecidesOnTheLaunchesOwnRead: a deterministic interleaving through Start, where a hook rewrites <host>.config.env without the key between the launch's read and the gate. It is refused; with rev 5's second read restored, the app ran with A. Also TestReplicaGateReadsNoEnvFile | | R6-2 | P2. The procedure never applied list changes to the running agent: replicaHomeKeys parsed the environment once at startup. | Accepted (ruling 30): read on each launch. | The list moves from the agent's environment to a file, /etc/carlos/replica-home-keys, written whole by the pre-roll check (temporary file, then rename). The agent reads it again at every replica launch and keeps no copy. An invalidation takes effect at the next launch, with no agent restart. The procedure checks the file before the key changes, with the replicas kept stopped throughout. | TestReplicaHomeKeysReadsTheListAtEveryLaunch (a removed entry stops verifying at the next call; a replaced one verifies the new digest); TestReplicaHomeKeysLogsAMalformedListWithoutItsText (also an unreadable list) |

Counts: 2 accepted, 0 declined.

Also in rev 5.1:

  • Task 1's code is the reviewed branch's, from c568c13b to e8487c3a. ParseAdoption accepts only the exact bytes AdoptionRecord writes; ParseEnv's behaviour is pinned by a test; and the two activator doc comments name the empty-key exception. Task 1 also gains TestEmptyOwnInstanceKeyShadowsNothing.
  • A rollout note from Task 1's review: a mixed-version roll and the moves readiness check (rollout step 3).

Effect on stage B (IK-1): none in this revision. The reviewed ParseAdoption is stricter: it accepts only the canonical bytes, which is all AdoptionRecord ever writes. O2' and its test were re-run against stage B's current tip (b9a753f0) with the reviewed instancekey, and pass.

Rev 5: round 5 dispositions

Astra's round 5 (CODEX-PLAN-INSTANCE-KEY-R5.md, reviewed at 37c48a40) accepted the rest of the plan, including the keyless gate on every launch path, and childEnv stripping the list from inherited tenant environments. It found two P2s and a P3, all about replicas. Paul's rulings 27 and 28 answer the P2s. The replica scope stays narrow.

| # | Finding | Disposition | Change | Test | |---|---|---|---|---| | R5-1 | P2. A stale digest admits the wrong key after an ordinary configuration change. SyncExec reconciles each box separately and hourly, so the home can hold B while the replica still holds A and still matches A's entry. | Accepted (ruling 27). Verified: configSyncLoop runs hourly per box (agent.go:1078). | A configured key is never changed in place; that would be a rotation, which this plan does not support. The rollout gives the procedure for any configured-key change on an app with replicas. There is also a cheap fresh-evidence check that needs no O8: a replica whose key comes from its configuration is started only if the bucket's configuration renders that same key now (appconfig.ConfiguredKeyDigest, through ExecBackend.ConfiguredKey). That catches a shared configuration change the replica's box has not synced. The list itself is written only by the pre-roll check, never refreshed from a replica's configuration. | TestReplicaLaunchWithoutAControlSocket: "the home updated, this replica's configuration delayed: refused" (it fails with the fresh check removed); "the bucket cannot be read: refused"; "no fresh reader: refused"; "residual: a change only on the home is not seen". Also TestConfiguredKeyDigestReadsTheBucketNow and TestReplicaConfigKeyReadsTheBucketNow | | R5-2 | P2. O8 has no source for a configured key's value: the published <host>.env leaves it out (ruling 5), and the adoption record holds only its digest. | Accepted (ruling 28). | O8 delivers only a key the platform minted into the home's <host>.env. A home whose key exists only in configuration is explicitly deferred: its keyless replica keeps refusing, and its configured-key replica follows ruling 27. | TestReplicaLaunchWithoutAControlSocket: "a config-only home's keyless replica is refused" | | R5-3 | P3. A malformed list entry can reach the log unredacted: a bare digest with no replica-host= lands in the error. | Accepted. | ParseHomeKeys reports the entry's position and the reason it was rejected, never the entry's text. The agent logs that error and nothing else. | TestParseHomeKeys (no error echoes its input); TestReplicaHomeKeysLogsAMalformedListWithoutItsText (the logging path) |

Counts: 3 accepted, 0 declined.

Effect on stage B (IK-1): none. IK-1's symbols (ParseEnv, Effective, Digest, Binding, Adoption, AdoptionRecord, ParseAdoption, Binds, VerifyDelivered, AdoptionItem, AdoptionSuffix) are unchanged in name, signature and behaviour. The only instancekey change is the error text of ParseHomeKeys, which stage B does not use. O2' and its test are unchanged.

Rev 4: round 4 dispositions

Astra's round 4 (CODEX-PLAN-INSTANCE-KEY-R4.md, reviewed at cba8ae4e) found two P2s in the rev 3.1 replica fix. Paul decided the follow-up instead of deferring it: a replica gets its home's key (ruling 26, which replaces rulings 17 and 25).

| # | Finding | Disposition | Change | Test | |---|---|---|---|---| | R4-1 | P2. The unconditional strip breaks existing replicas that share a configured key matching the home's. "Runs everywhere" is wrong. | Accepted (ruling 26). | The strip is reverted: Task 3b's fileEnvFor is gone, and mergeEnv composes a replica's env as any other. A replica whose key is exactly its home's keeps it. One whose key differs is refused with a log line. A pre-roll check (rollout step 1) lists every replica's key against its home's before any box is upgraded, and records the matches in CARLOS_REPLICA_HOME_KEYS. The wording at rollout step 6 is corrected. | TestReplicaLaunchWithoutAControlSocket (the home's configured key runs; a pre-existing env key that is the home's runs; a differing key is refused; an unverifiable key is refused); TestClaimReplicaLaunchIsGated; TestNestedReplicaPrepareIsGated; TestParseHomeKeys, TestReplicaHomeKeysFromTheEnvironment | | R4-2 | P2. Removing the variable does not refuse on a claim launch: with no control socket, InstanceKey("") returns ErrNotOnCarlos and the app mints its own key over the home's data. | Accepted (ruling 26). Verified: rastrillo e3ea08de's step 4, and SnapshotClaimLaunch's env has no socket (claim_snapshot.go:116). | An agent-side refusal, replicaKeyGate, runs on every replica launch before anything starts. A replica with no key of its home's starts only with a control socket, where the helper refuses at step 3. Without one (a seed, a claim launch before O6, a failed bind) the agent refuses the launch itself. | TestReplicaLaunchWithoutAControlSocket and TestReplicaSeedIsGated run the app as rastrillo's resolution. A keyless replica is refused and no app runs; with the gate disabled the app reports "minted". TestKeylessReplicaWithAControlSocketLeavesTheHelperToRefuse (the helper reports ErrNoInstanceKey) |

Counts: 2 accepted, 0 declined.

New obligation O8 (replica placements, #162). The platform has no channel today for a replica to read its home's key. A replica record names no home: provision.Replica copies the app, account, pool and channel, not the home's host. The one-way mirror between home and replica is the app's own (titogo's), not the platform's. A home's <host>.env is published only on claim boxes, once O1' is in, into the home region's data bucket. So obtaining the home key's value for a replica is O8, with this plan's fallback: a replica that cannot be handed its home's key gets none, and refuses to start reliably (ruling 26). Until O8, verification uses the digests the pre-roll check records.

Effect on stage B and O2':

  • O2' and its test are unchanged.
  • SnapshotClaimLaunch can now return activator.ErrReplicaKey for a replica row. A relocated replica's launch passes the same gate, so it needs its home's key, verified, or O6's socket.
  • LaunchEnvironment is as in rev 3 again, so the moves env report lists a replica's key like any other.

Rev 3.1: round 3 dispositions

Astra's round 3 (CODEX-PLAN-INSTANCE-KEY-R3.md, reviewed at f649d972) found no P1. It closed round 2's five findings at the contract level and judged the source check sound. Two fixes remain.

| # | Finding | Disposition | Change | Test | |---|---|---|---|---| | R3-1 | P2. The replica exclusion stops minting but not delivery. A replica whose configuration supplies B, or whose own env already holds a key, is handed it, and the helper's step 2 returns it over the home's replicated data. The adoption 409 cannot stop that, because the helper ignores failures. | Accepted (ruling 25). Verified: InstanceKeyReplica only skipped the mint; mergeEnv still delivered the configured or existing key. | No launch of a replica row is handed CARLOS_INSTANCE_KEY: fileEnvFor strips it from the composed env on every launch path (Start, Seed, the nested prepare, LaunchEnvironment, the claim snapshot). With the variable absent, the helper's step 2 has nothing to return. No rastrillo change. | TestReplicaLaunchIsHandedNoInstanceKey (a real replica Start and Seed: a configured key, a pre-existing env key, both; the home row delivers the key as the control case); TestNestedReplicaPrepareIsHandedNoInstanceKey (run and seed roles); TestClaimReplicaLaunchIsHandedNoInstanceKey (the claim launch env and the snapshot) | | R3-2 | P3. The follow-ups said "O6 and O7, in that order", contradicting ruling 24. | Accepted. | The follow-up now says O7 ships before O6, and why. | none |

Counts: 2 accepted, 0 declined.

Effect on stage B and O2': none. O2' and its test are unchanged. Stage B consumes instancekey, ClaimLaunchSnapshot and PinObject as before. One behaviour visible elsewhere: the moves env report for a replica (LaunchEnvironment) no longer lists CARLOS_INSTANCE_KEY, on the source and the target alike, so the two still compare equal.

Rev 3: round 2 dispositions

Astra's round 2 (CODEX-PLAN-INSTANCE-KEY-R2.md, reviewed at a21e8225) found three P1s and two P2s. It also withdrew round 1's finding 6, and accepted the missing claim socket as an explicit refusal. Each finding was checked against the code, and rulings 19 to 24 (Paul's R-A to R-F) answer them. Source (c) and appconfig.ConfiguredInstanceKey are gone. The rev 2 table below stays as history; where it and this table disagree, this one stands.

| # | Finding | Disposition | Change | Test | |---|---|---|---|---| | R2-1 | P1. A configured-key source still bypasses the refusal: configuration supplies A, the app's own unpublished B wins (step 1), and (c) passes. | Accepted (ruling 20). Verified: rev 2's O2' consulted ConfigKey with no word from the app. | Source (c) and ConfiguredInstanceKey are dropped. A configured key counts only through an adoption record, reported by the helper with source: config, exactly as a platform key. An app on its own key B sends no report, so there is no record and the pin refuses. | O2' TestStorePinnerKeyEvidence, row "configured A, the app's own unpublished B: no report, no record" | | R2-2 | P1. Configuration evidence neither identifies nor pins the key: a configuration changed from A to B before the box dies, or between pin and install, delivers B over data sealed with A. | Accepted (ruling 20). Verified: ConfiguredInstanceKey returned a boolean from the current configuration. | The record names the key by digest. The pinner pins it (PinObject.InstanceKey). The install checks that the env files it is about to launch with deliver exactly that key (instancekey.VerifyDelivered), and refuses otherwise (O3'). | TestVerifyDeliveredRefusesAnyOtherKey ("the configuration changed since the pin"); O2' config rows; O3''s install test | | R2-3 | P1. An adoption record outlives the writer it describes: a later release on its own key, or a rollback, inherits an earlier session's record. | Accepted (rulings 20, 21). Verified: rev 2's record held only a version, a name and a digest. | The record binds host, incarnation, writer session and generation, all taken from the agent's own launch state for that process. Every new writer admission starts with no record (O1'). The pinner accepts only a record that binds the pinned writer. | TestAdoptionRecordBindsOneWriter; O2' rows "a stale record from a previous release", "a rollback", "a record from another installation"; TestInstanceKeyAdoptionRecordsTheLaunchesWriter (a session in the request is a 400) | | R2-4 | P2. A first adoption has no publication path during the running session, and O6 is not gated in the HA checklist. | Accepted (rulings 22, 23). | The verb hands the durable record to the session's publisher through hostfiles.PlatformRelay, and the publisher's continuous loop commits it. 204 means durable on disk; publication follows the publisher's bound. O6 is a gate in O4's checklist. | TestAdoptionSurvivesBoxLossWithoutAnotherAdmissionOrRelease, TestPlatformRelayStagesTheLatestBodyWithoutBlockingItsCaller, TestInstanceKeyAdoptionIsDurableBeforeItAnswers | | R2-5 | P2. Deferring the record's retirement breaks a later install on the source box: the census refuses the leftover. | Accepted (ruling 24). Verified: installArtifactPaths (movesinstall.go:751) does not name it, and EmptyInstallationCensus.CheckLocal refuses an unclassified host artifact. | O7 puts the record in moves' journalled artifact lifecycle, with the exact lists. Its writer (O6) does not ship before O7. | O7's TestMoveAwayRetiresTheAdoptionRecordSoTheHostCanReturn (in the moves change) | | note | The record is tenant-forgeable. | Answered (ruling 19). | The record is a trusted-helper assertion against accidental loss, not independent evidence. A tenant who forges it can lose only its own data. | none | | R1-6 | Round 1's merge option. | Closed: Astra withdrew it. Reconciling AGENTS.md's local-squash text with the hub rule is Paul's call. | none | none | | gap | Claim launches have no control socket. | Accepted as an explicit refusal (ruling 23). | O4's checklist gates relocation for apps on a platform or configured key on O6. | none |

Counts: findings R2-1 to R2-5: 5 accepted, 0 declined. The forgeability note is answered by ruling 19, and R1-6 is closed.

Found while revising:

  • The helper cannot tell the source itself. A platform key and a configured key both arrive as $CARLOS_INSTANCE_KEY (ruling 13). So O6 delivers the source beside the key, as an agent-owned CARLOS_INSTANCE_KEY_SOURCE, and the helper echoes it. The agent still checks the echo against its own launch state, which is authoritative.
  • Stage B's rev 3 (919662d2) already changed its O2. It moved to a StorePinner.KeyEvidence reader (its ruling 55) that still accepts a configured key on presence, and still parses export lines. This plan's O2' replaces that reader. The evidence is in the pinned manifest itself, so the pinner needs no outside reader.

Rev 2: dispositions of Astra round 1

Each finding was checked against the code on this branch (cut from origin/main 0901e835) before it was answered. "Provisional default" rows are Paul's open questions 1 to 4: they are the defaults this plan builds on, and Paul has not yet confirmed them.

| # | Finding | Disposition | Change | Test | |---|---|---|---|---| | 1 | P1. O2 accepts a platform key the app may never have used: the app's own instance.key wins in rastrillo, and a missing instance.key beside a backed-up platform key would pass. | Accepted. Verified: rastrillo e3ea08de InstanceKey returns own first, and rev 1's mint ran whatever the app used. | Ruling 15: relocation accepts a key only with positive evidence, (a) instance.key, (b) the platform key plus an adoption record for that exact key, or (c) a key from the configuration. The control socket has no verb for (b) today, so Task 5 adds PUT /instance-key/adoption and the record beside <host>.env. O1' publishes the record, O2' requires it, and the rastrillo ask below asks the helper to call it. | O2' TestStorePinnerKeyEvidence (compiled and run on a stage B scratch), row "a backed-up platform key, the app's own key missing" refuses; TestInstanceKeyAdoptionRecordsOnlyTheOwnFilesKey; TestNestedControlSocketRecordsAdoptionDurably | | 2 | P1. A failed durability step becomes success on retry: after the append, a failed chmod or sync leaves a key the next ensure returns early on; the claim early return skips it too. | Accepted. Verified at rev 1's lines 585-647 and claimsnapshot.go:56-58. | Ruling 4: every ensure ends with fchmod 0600, file fsync and directory fsync, also when the key is already there; the bytes are never rewritten. The claim early return fsyncs the file and its directory without writing (ruling 18), and takes the guarded path when the mode needs setting. | TestEnsureInstanceKeyRedoesAFailedSyncBeforeReportingSuccess, TestExecBackendStartStartsNoChildUntilTheKeyIsSynced, TestEnsureClaimAdminTokenSyncsAPresentKeyBeforeItAnswers | | 3 | P2. Every nested backfill that appends fails with EINVAL: create=false drops `OCREAT but still passes mode 0600 to openat2. | **Accepted.** Verified: instancefs.OpenBeneath puts perm in OpenHow.Mode (instancefs_linux.go:93-97), and openat2 refuses a non-zero mode without OCREAT. | openInstanceEnv passes mode 0 unless it creates. The fchmod on the descriptor sets 0600. | TestBackfillInstanceKeyAppendsToANestedEnvFile | | 4 | P2. An empty CARLOSINSTANCEKEY= in <host>.env defeats a configured key: the mint defers to config, but mergeEnv hands the child the empty override. | **Accepted.** Verified: mergeEnv` lets the overlay win every conflict, empty values included (backendexec.go:1980-1990). | Ruling 13: the mint decides on the effective env (instancekey.Effective), and mergeEnv applies the same rule, so an empty own key never shadows a non-empty configured one. The child never gets an empty key while a non-empty one exists. LaunchEnvironment stops reporting that name as shadowed. | TestExecBackendStartNeverHandsTheChildAnEmptyKeyWhileOneExists (both files, through Start); TestEffectiveNeverHandsOutAnEmptyKeyWhileOneExists; TestEnsureInstanceKeyDefersToTheRenderedConfiguration | | 5 | P2. O2 refuses a key that comes from the configuration, which ruling 5 supports. | Accepted. | Ruling 15 (c): the configuration in the bucket is the evidence, and the target renders the same configuration. appconfig.ConfiguredInstanceKey renders it with SyncExec's own loader and renderer; O2' calls it through StorePinner.ConfigKey. | TestConfiguredInstanceKeyReadsTheRenderedConfiguration; O2' rows "(c) ..." | | 6 | P3. The rollout's amadan branch merge -squash contradicts AGENTS.md's local-squash workflow. | Declined. The hub has -squash, and Paul's standing rule (2026-09-22) is squash-merge on the hub. AGENTS.md's local-squash text predates that rule. | Rollout step 1 now says: merging is Paul's call, as a squash on the hub, after CI is green on the tip. | none | | 7 | Open question 5: claim-session restarts can mint outside the C8 publish point. Astra: the premise is not established. | Accepted: the premise was wrong. Verified: startAdmitted calls Prepare once (claimwake.go:355) and loops over attempts, calling Materialize before each (:405-412); the assembly's inputs are captured once (claimwakeassembly.go:353, :377); production claim swaps refuse (claimsSwapRefusal, claimgen.go:30). | Ruling 16 states the corrected premise. Question 5 is removed. | TestStartAdmittedRetriesReuseTheirInputsAndMaterializeFirst | | 8 | Coverage gaps: a first flat seed, the nested seed role, a real hibernation restore (a second Start does not exercise Restore). | Accepted. | Tests added. | TestExecBackendSeedMintsTheKeyForAFreshFlatInstance; TestPrepareMintsTheInstanceKeyIntoTheNestedEnvFile/seed; TestHibernationRestoreKeepsTheInstanceKey | | 9 | O1's capturing fake proves ordering only, not committed publication or refusal on a failed publication. | Accepted, moved. Those need the provider. | O1' names the provider's integration tests, in the C8 plan. Task 4's test keeps proving the ordering. | in the C8 provider's plan (O1') | | 10 | Legacy durability is disk plus move archives: no off-box recovery after box loss. | Acknowledged, no change. | Ruling 6 already says so. Relocation cannot run on a legacy box. | none | | Q1 | Replicas get their own key; should a replica get its home's key? | Provisional default, Paul not yet confirmed. Deferred to a follow-up. | Ruling 17: until then replicas are excluded from platform-key adoption. No path mints a key on a replica row, the backfill skips replicas, and the adoption verb refuses on one. An adopting app on a replica refuses to start unless it passes its own key. | TestReplicaRowsGetNoPlatformKey; the backfill test's replica row; the adoption test's replica case | | Q2 | A key set in configuration: defer to it, or reserve the name? | Provisional default, Paul not yet confirmed. | Configuration wins (ruling 5). The name is not reserved. | TestEnsureInstanceKeyDefersToTheRenderedConfiguration | | Q3 | Should the platform provision report.token? | Provisional default, Paul not yet confirmed. | No report.token. | none | | Q4 | Workers and sidecars: should they get their instance's key? | Provisional default, Paul not yet confirmed. | A follow-up; out of scope (ruling 12). | none |

Counts: findings 1 to 10: 8 accepted, 1 declined (6), 1 acknowledged with no change (10). Open questions 1 to 4: 4 provisional defaults.

Found while revising (not in Astra's list):

  • Stage B's O2 parses the env item with its own rule, accepting export CARLOS_INSTANCE_KEY="k1". The activator's parser reads that line as the variable export CARLOS_INSTANCE_KEY, so the app never receives it. O2' parses with instancekey.ParseEnv, the activator's own parser, and its test pins the exported line as no key.
  • Claim launches carry no control socket. SnapshotClaimLaunch composes the env with agentOwnedVars(rt, "", vars, false) (claimsnapshot.go:116), which strips `CARLOSCONTROL_SOCKET`. An app first adopting the platform key on a claim box cannot record adoption until claim launches bind one (obligation O6).

Global Constraints

  • Format (rastrillo #66): 32 CSPRNG bytes, standard padded base64 (44 characters). The key is the exact bytes the env file yields: no quotes, no trailing newline in the value, no trimming. Changing the encoding later is a key rotation.
  • Create-only: a CARLOS_INSTANCE_KEY already in <host>.env is never rewritten, by any path, whatever its shape.
  • Durable before success: an ensure that reports success has set 0600 on the file, fsynced it and fsynced its directory, in that call.
  • Variable name: CARLOS_INSTANCE_KEY, exactly.
  • File mode: <host>.env and the adoption record are 0600.
  • Never logged, never in an error, never in the console. Errors and log lines name paths, never values. The adoption record holds a SHA-256 digest, never the key.
  • Rollout order (rastrillo #66): the key is on every box, existing instances included, before any app adopts the helper.
  • Neutral examples: shop, example.com, acct. No tenant or app names in code, tests or comments.
  • Cross-compile: internal/host, internal/activator, internal/appconfig and internal/instancekey build for darwin, windows and linux/arm64 (20-cross). Tests that need openat2, mkfifo or a unix socket go in *_linux_test.go.
  • Gate (AGENTS.md "Build & test"): go build ./..., go vet ./..., go test ./... -count=1, gofmt -l . clean.
  • No user-facing copy. Log lines are operator-facing, in the surrounding style. The adoption verb answers with http.StatusText bodies only.

What the code does today (branch instance-key-build, cut from origin/main 0901e835)

Two things are called "overlay". The decision's "overlay env file, alongside $CARLOS_ADMIN_TOKEN" is <host>.env, the instance's own env file:

  • ExecBackend.pathsFor names it <dataDir>/<host>.env beside <host>.config.env (internal/activator/paths.go:50-52); nested, dataDir is the instance directory (paths.go:36-37).
  • mergeEnv calls it the overlay: "base (the sync-owned <host>.config.env, bucket-derived) under overlay (the instance's own <host>.env). The overlay wins every conflict" (internal/activator/backendexec.go:1974-1990). It wins with an empty value too: `CARLOSINSTANCE_KEY= in <host>.env` hands the child an empty key over a configured one (finding 4).
  • The moves archive calls it "the <host>.env overlay" (internal/moves/archive/archive_test.go:36-37), and region HA's H6 names <host>.env as always continuous.

The other overlay is appconfig's instance config layer, apps/<acct>/<app>/instances-config/<host>.json, written by appconfig.ApplyInstance (internal/appconfig/instances.go:1-24, 371-376). It is app configuration, sealed with the deployment's secrets key, rendered into <host>.config.env by SyncExec (internal/appconfig/syncexec.go:1-9). LoadMergedHostVersions (instances.go:222) and RenderEnv (materialize.go:37) are the one loader and renderer that SyncExec, the held render and CheckRenderedIdentity (claimidentity.go:13) share. Sync never reads, writes or reaps <host>.env (internal/appconfig/sync.go:270-275, pinned by TestSyncExecNeverTouchesInstanceEnvOrOutOfScopeRoutes, syncexectest.go:683).

The admin token is the precedent (internal/activator/admintoken.go):

  • minted by ensureAdminToken (:142-238): one read of the raw bytes, parseEnvBytes (:245-257), 32 crypto/rand bytes hex-encoded, appended with O_APPEND|O_CREATE at 0600 on a fresh line (:205-229), then chmod 0600 (:234). It never fsyncs;
  • create-only, malformed tokens logged once an hour per file (logPerFile, :70-77) and left in place (:189-203);
  • serialized by the process-wide adminTokenMu (:43);
  • nested files go through instancefs.ReadBeneathLimit / OpenBeneath (:150-216). OpenBeneath passes its perm straight into openat2's OpenHow.Mode (internal/instancefs/instancefslinux.go:93-97); the kernel refuses a non-zero mode without `OCREAT with EINVAL` (finding 3).

Its callers are every launch path, each before the env is read: seedChild (backendexec.go:831), spawnChildFrom, which flat Start and StartGen share (:1369), the nested prepareAt for the run and seed roles (internal/activator/prepare.go:60), the claim's EnsureClaimAdminToken (internal/activator/claimsnapshot.go:40-67), called by the production launch source claimServingLaunchSource under claimAdminTokens.With (internal/host/claimproduction.go:356-428), and the scheduler tick's scheduleToken (internal/host/schedule_wiring.go:150-180).

Creation against first start. The console writes the instance record. The box's reconcile mints the registry row with Status: "ready" and starts nothing (internal/provision/reconcile.go:425-435). The first process is started by the first wake, doWake: Restore (internal/activator/activator.go:1797), then Seed when the database is fresh (:1817), then Start. Restore restores the database only (backend_exec.go:663-757). Both seed and start mint before they read the env.

The control socket. edge.ListenControl (internal/edge/purge.go:126) binds <socket>.control before every exec spawn and serves POST /purge (when there is a cache) plus whatever the activator mounts: today schedule.MountGated, which serves GET /schedule-capabilities, PUT and DELETE /schedule/{name} and GET /schedule, each bearer-gated on the host's own CARLOS_ADMIN_TOKEN (internal/schedule/control.go:97-225). It is mounted in spawnChildFrom (backend_exec.go:1406-1414) and in the nested listenControl (prepare.go:135-155), and only when b.Timers != nil. Rastrillo e3ea08de's carlos package calls only PUT and DELETE /schedule/{name} (carlos/schedule.go). Nothing on the socket lets an app tell the agent which key it uses, so source (b') needs a new verb here and a call in rastrillo (the ask below).

Claim launches have no control socket. SnapshotClaimLaunch composes the launch env with agentOwnedVars(rt, "", vars, false) (claimsnapshot.go:116), which strips `CARLOSCONTROLSOCKET. A claim-launched app is not carlos.Running(), but rastrillo's step 2 still returns $CARLOSINSTANCE_KEY`. It cannot call the adoption verb (O6).

Claim retries. startAdmitted reads the claim, calls Prepare once (internal/activator/claimwake.go:355), restores, then loops over attempts: Reserve, then Materialize, then the start (:405-412). The assembly captures the launch inputs once, from the selection (_, in := a.snapshot(), internal/host/claimwakeassembly.go:353), and Materialize is bound to them (:377). ClaimStart documents that "process retries reuse it". Production claim swaps refuse before they raise anything (claimsSwapRefusal, internal/activator/claimgen.go:30). So no claim attempt reaches the launch source again, and none runs before its Materialize.

Backup of <host>.env today.

  • Legacy boxes (host claims off; every production box today): nothing publishes <host>.env. internal/hostfiles is imported by no production package (internal/hostfiles/contract.go:13-15). hostfiles.EnvItem exists only on region-ha-1b-next-build (internal/hostfiles/platform.go:13 there). The file's only off-box copy is a move's sealed archive (archive.Build, internal/moves/archive/archive.go:395).
  • Claim boxes: the serving owners call the launch source in Select (internal/host/claimowners.go:375-416) and then Materialize, which calls ClaimFilesManifest.Establish "before the app executes" (claimowners.go:125-138, 425-438). No C8 provider exists on main: Files == nil refuses every serving wake (claimFilesRefusal, :140-143).

Moves (L16's shared install routine on main). installFiles extracts the archive and placeStaged moves the env into place only when <host>.env is absent or is this install's own (internal/host/moveslandinstall.go:130-280). The move's env report HMACs every value and classes any name it does not know as equal (internal/moves/envreport.go:24-38, 90-103, 117-150). The archive carries the data directory and <host>.env only. The source's finalization parks a fixed, journalled list of flat paths (installArtifactPaths, internal/host/movesinstall.go:751; sourceArtifactPaths, movesfinal.go:586).

The layout pass moves every flat entry the flat naming rules give a host (<host> plus a dot or a dash and more) into the instance directory at the same name, and back on a revert (internal/layout/extras.go:14-60, belongsTo). An adoption record named <host>.instance-key-adoption.json travels with it.

Relocation's pinner (stage B branch). StorePinner.Pin runs on the region runtime, which holds the control store and the box's secrets key (RelocationAssemblyConfig.SecretsKey, from CARLOS_SECRETS_KEY). With a nonempty database written by an admitted writer it requires instance.key in the pinned manifest (needKey, pin.go:250 there). The target renders its held config from the same bucket configuration (appconfig.RenderHeldUnder in Task 8c), over the route relocationRecord mints.

Design and rulings

Each ruling ends with its cost if wrong. Rulings 1 to 12 are rev 1's, amended where marked. Rulings 13 to 18 are new in rev 2.

1. The key lives in <host>.env, and is minted on the box. <host>.env is already the platform-managed per-instance secret file, H6 makes it always continuous, moves carry it, and stage B restores it (EnvItem). Minting in the console, into appconfig's instance config layer, is rejected: that layer is the app's own editable configuration (carlos env unset would delete the key), and it renders into <host>.config.env, which sits under <host>.env in mergeEnv. Cost if wrong: if Paul wanted the key in the bucket before the box first sees the instance, O1' would be unnecessary. Moving the key later is a copy of the same bytes, not a rotation.

2. Format: base64.StdEncoding of 32 bytes from crypto/rand, 44 characters, one line CARLOS_INSTANCE_KEY=<44 chars>\n. Every reader of <host>.env yields these bytes exactly: the parser cuts at the first = and trims whitespace, which base64 never contains, and the nested launch file passes KEY=value strings to execve unchanged (prepare.go:204). Cost if wrong: the encoding is fixed by the first mint. Changing it later is a rotation for every adopted app.

3. Every launch path mints, right after the admin token, under the same lock. These are seedChild, spawnChildFrom, prepareAt (run and seed roles) and EnsureClaimAdminToken, through one helper, ensureInstanceKeyFor(rt, paths). The scheduler tick does not mint: any tick that wakes the instance goes through Start. A failed mint fails the wake, as a failed token mint does. Cost if wrong: a missed launch path makes an adopting app refuse to start there (ErrNoInstanceKey), loudly.

4. Create-only, append-only, durable every time (amended for finding 2). A non-empty CARLOS_INSTANCE_KEY in <host>.env, of any shape, is left exactly as it is: two lines stay two, and readers take the last. The key is appended on a fresh line, never rewritten. Every ensure that reports success, whether it minted or found the key, has done three things on that call: fchmod 0600 on a descriptor it opened, fsync of the file, and fsync of its directory. A mint whose sync failed returns the error and leaves its line in place. The next ensure finds the key present, keeps its bytes and redoes all three steps. It succeeds only when they succeed, so no process is started with a key a power cut could take back. An existing file is opened with mode 0 (finding 3). Cost if wrong: one open, fchmod and two fsyncs per launch, on a file of a few hundred bytes.

5. A key from the app's configuration wins, and the platform mints none (amended for finding 4; the delivery rule added by the final fix wave). If the effective env (ruling 13) takes its key from <host>.config.env, the mint writes nothing and logs once an hour. Minting would put a key into <host>.env, which overrides the configured one, and change the app's key under its data. Cost if wrong: an app that sets the name and later unsets it gets a fresh platform key at its next start, by its own action. Reserving the name instead was open question 2; the provisional default is not to.

The delivery rule (final review I2, Astra's P2). The files say what this box has delivered, not what the configuration sets: the fast-converge pass reconciles a row ready and syncs its configuration last, so a first wake can run before <host>.config.env is rendered, and a key minted then shadows the configured one for good once it arrives. SyncExec removes the file for a route that was never configured, so the file's existence cannot be the delivery signal. The mint therefore asks the bucket, as the replica gate does (ExecBackend.ConfiguredKey, appconfig.ConfiguredKeyDigest), once, only when nothing delivered here sets a key and a mint is next: a key the configuration sets and this box has not delivered defers the launch (ErrInstanceKeyDeferred), so does a bucket that cannot be read, and a configuration that sets no key lets the mint go ahead. The read is two-phase (the re-review's finding): outside adminTokenMu, the files are read and, if a mint looks next, the bucket is asked; under the lock the files are read again and the decision is that read's alone. A key that appeared in either file meanwhile is present or configured and nothing is minted; a mint goes ahead only when the locked read still finds no key and the bucket was asked and said none; a mint that looks next under the lock without a verdict defers, and the next attempt asks. So a bucket that is slow or down for one row never holds the lock every other row's mint waits on. The launch fails and is retried; the config sync's next pass delivers the file and the next start finds the key configured. A key already delivered, in either file, costs no read. Which boxes have a configuration to read follows the config guard (configguard.go): no deployment store under CARLOS_NO_SEALED_CONFIG is no configuration source, and the mint proceeds; no store otherwise is an error. Cost: a delayed first start while the configuration is in flight (seconds, one pass), and one bucket read per instance, ever. The window the read cannot close is a key configured after the read and before the append, the same in-flight window ruling 27 names for replicas; a key configured after an instance has minted is a rotation, which this plan does not support.

6. Durability before the first start.

  • Legacy boxes: nothing can publish today, and region HA cannot be on, because it requires host claims. The key's durability is the box's disk, now fsynced, and a move's sealed archive. A dead legacy box loses its keys with its data; that is today's state, unchanged (row 10).
  • Claim boxes are region HA's only path. The launch source mints in Select. Materialize then calls ClaimFilesManifest.Establish, and the app executes only after that returns. O1' binds the provider to publish the env file, committed, before it returns, and to start the new generation with no adoption record.
  • Bucket unreachable at first start: Establish fails, so the wake fails and the app does not start. The key stays on disk unchanged. The next wake publishes the same bytes.

Cost if wrong: a C8 provider that ignores O1' reopens the window. O2' is the backstop: without evidence, the host goes stuck and is never relocated without its key.

7. The backfill: the agent at startup, legacy boxes only, existing files only.

  • Who and when: runAgent, inside the startup closure that runs after layout recovery has settled every journal, and before the layout pass is published and before svc.Start wakes anything (internal/host/agent.go:476-484).
  • Which rows: ready, activator-spawned instances (Kind == "instance", a socket, not Platform, not unit-backed, no tunnel, not a replica, Status empty or ready). Held rows (moving, landing, restoring, releasing), provisioning rows, sidecars, replicas and TCP-addr routes are skipped.
  • Existing files only: a missing <host>.env means the host was never started here, and its first start mints. Creating one could occupy the path a landing's placeStaged needs empty.
  • Idempotent: a present key writes nothing (it is made durable), so every agent start re-runs it harmlessly.
  • Visible: it logs one count line, which the rollout reads.
  • Claim boxes are not backfilled. The claim's launch source mints under the claim's guard before every cold admission.

Cost if wrong: a box whose registry cannot be listed at startup logs it, and its instances get keys at their next start instead.

8. Never overwrite, on every path:

| Path | What guarantees it | Test | |---|---|---| | create (first seed or start) | create-only mint | Task 2 TestEnsureInstanceKeyMintsOnceInTheAgreedFormat; Task 3 TestExecBackendStartHandsTheChildTheMintedInstanceKey, TestExecBackendSeedMintsTheKeyForAFreshFlatInstance, TestPrepareMintsTheInstanceKeyIntoTheNestedEnvFile (run and seed) | | a retry after a failed sync | the present path keeps the bytes and redoes the durability | Task 2 TestEnsureInstanceKeyRedoesAFailedSyncBeforeReportingSuccess; Task 3 TestExecBackendStartStartsNoChildUntilTheKeyIsSynced | | backfill | same body, existing files only | Task 2 TestBackfillInstanceKeyAppendsOnlyToAnExistingEnvFile; Task 3 TestBackfillInstanceKeyAppendsToANestedEnvFile; Task 6 TestBackfillInstanceKeysWritesOnlyReadyActivatorInstances | | any existing key | create-only, any shape | Task 2 TestEnsureInstanceKeyNeverRewritesAnExistingKey, ...LeavesAFileWithTwoKeysAlone | | claim launch, retried | the launch source mints create-only; a session's attempts reuse their inputs | Task 4 TestClaimLaunchMintsAKeyBesideAnExistingToken, TestStartAdmittedRetriesReuseTheirInputsAndMaterializeFirst | | re-render (config change) | sync never touches <host>.env | Task 7 TestSyncExecLeavesTheInstanceKeyAlone | | restore (hibernation wake) | Restore restores the database only; Start mints create-only after it | Task 3 TestHibernationRestoreKeepsTheInstanceKey | | move | landing installs env only on an empty path; archive carries bytes exactly; env report fails a missing or different key | Task 7 TestArchiveCarriesTheInstanceKeyByteForByte, TestEnvReportComparesTheInstanceKey; existing TestLandingNeverDeletesFilesItDidNotCreate | | relocation install | stage B writes EnvItem's bytes; the next launch finds the key present | O3' (stage B branch) | | layout move (flat to nested) | <host>.env is one of the pass's named artifacts; the adoption record is one of its extras | existing layout pass tests | | adoption record | written only for the launch's own writer and delivered key; identical bytes are not rewritten; a new admission starts with none (O1') | Task 5 TestInstanceKeyAdoptionRecordsTheLaunchesWriter |

9. Secrecy.

  • No log line or error takes the value. The shape warning names the path only, and a test checks the value is not in the log.
  • <host>.env is 0600 (ruling 4), as is the adoption record and the nested launch file (prepare.go:204).
  • The adoption record holds sha256(key), never the key. For a platform key it is published beside the env item, which holds the key itself, so it gives a bucket reader nothing new. For a configured key (rev 3, ruling 20) it publishes the digest of a configuration secret: nothing for a random key, a guessing target for a weak one (follow-ups).
  • The console never reads a box's <host>.env. carlos env list reads the bucket configuration through the console API (appconfig_cli.go:462-507), never <host>.env. The moves env report carries HMACs only.
  • Once region HA publishes the env item, the key sits in the data bucket beside the database it seals. That is inherent in making the key recoverable with the data.

Cost if wrong: a leak of one instance's key exposes only what that app sealed with it.

10. Moves and relocation carry the key inside <host>.env.

  • A move archives the file, sealed (archive.go:395), extracts it byte for byte (extract.go:35), and places it only on an empty path (moveslandinstall.go:212-280).
  • If the source never had a key, the target's first start mints one. That is correct: no source process ever held a key.
  • The move's env report classes CARLOS_INSTANCE_KEY as equal. A target that would start with a different key, or none, is a differs or missing difference, never a note.
  • A move's probe launches with the env its readiness pin checks against the source's report (internal/host/movesprobeowners.go:106-120). So the probe path never mints.
  • A move does not carry the adoption record (the archive holds the data directory and <host>.env only). The target's sessions are new writers and start with none (ruling 20); the app re-reports at its first process start there. The source's record is retired with the move's artifacts (O7).
  • Relocation's install (stage B Task 8c) writes the pinned manifest's EnvItem verbatim and checks the pinned key before anything launches (O3'). The next launch's mint finds the key present.

Cost if wrong: a key lost in transit is the L21 failure itself. The env report makes it visible on moves.

11. No switch. Nothing reads CARLOS_INSTANCE_KEY until an app adopts the helper, so writing it is harmless. The new verb answers only a caller that holds the host's admin token. The claim's launch identity, a digest over the env, changes once per host. Cost if wrong: the mint is on every launch path, so a bug in it fails wakes across the fleet. The tests, and an error shape identical to the token mint's, are the mitigation. Rollback is reverting the binary. Keys and records already written stay, and are harmless.

12. Scope: exec-backed activator instances (amended for replicas). These get no platform key:

  • unit-backed tenants: their env is /etc/carlos/apps/<host>.env, root-only and unreadable to the agent (backend_unit.go:174-178);
  • TCP-addr routes: an app supervisor, not the agent, starts them;
  • sidecars and workers: they have their own rows and env files (open question 4, provisional default: a follow-up);
  • replica rows: ruling 26 (only the home's key, never a minted one).

The agent sets CARLOS_CONTROL_SOCKET only for instance children (agentOwnedVars), so carlos.Running() is false for unit tenants, TCP-addr routes, sidecars and workers, and the helper keeps today's behaviour. Cost if wrong: an excluded process that runs with CARLOS_CONTROL_SOCKET set and adopts the helper refuses to start. That is loud, not silent.

13. One rule for the effective key (new; finding 4). instancekey.Effective(config, own) is the key a launched app receives: the own file's when non-empty, else the configuration's when non-empty, else none. An empty own value never shadows a non-empty configured one. mergeEnv applies it on top of its merge, so the child's env follows the rule; the mint decides by it; the adoption verb checks by it; and the pinner reads the env item with the same parser (instancekey.ParseEnv, which parseEnvBytes now calls). Cost if wrong: a deliberately empty CARLOS_INSTANCE_KEY= in <host>.env can no longer hide a configured key. Nobody can rely on that today: an empty key makes an adopting app refuse to start.

14. The adoption record (rev 2; superseded by rulings 20 to 22). Rev 2's record named only a key, was written for any launch with a control socket, and was not bound to a writer. Rev 3's record, its writer and its publication are rulings 20 to 22.

15. Relocation's key evidence (rev 2; superseded by ruling 20). Rev 2 accepted three sources: (a) instance.key, (b) an adopted platform key, and (c) a configured key on presence. Rev 3 keeps (a), replaces (b) with the writer-bound (b'), and drops (c).

16. Claim retries reuse their inputs (new; question 5's premise corrected). On a claim box every serving start is a claim wake. The launch source, which mints, runs in Select. A session's retries reuse the captured inputs and call Materialize, and so Establish, before every attempt. The legacy spawn path does not serve a claim host, and production claim swaps refuse. No claim attempt can run with a key minted after its files manifest was established. Any future path that mints before an asynchronous publish needs a synchronous publication gate. Cost if wrong: TestStartAdmittedRetriesReuseTheirInputsAndMaterializeFirst fails on the change that breaks it.

17. Replicas are excluded from platform-key adoption (rev 2; superseded by ruling 26). Rev 2 deferred a replica's key and minted none; ruling 26 decides it.

18. The claim early return syncs without writing (new; finding 2). When the token and the key are both present, the claim launch source writes nothing and needs no authority, as before. It still opens the env file read-only, checks the mode, fsyncs the file and fsyncs its directory before it answers. fsync changes no byte, mode or name, so it is not a host write and runs outside the claim's guard. A file whose mode is not 0600 needs a write, so it takes the guarded path, whose ensure sets it. Cost if wrong: if fsync were judged a host write, this step would move under authorize, which re-reads the claim on every selection.

19. Threat model: the adoption record is a trusted-helper assertion (rev 3; Paul's R-A). The record exists to stop accidental loss, such as a relocation that restores a database without the key that sealed it. It is not independent platform evidence. The tenant holds the bearer and the key, the nested record sits in tenant-writable storage, and the helper is the tenant's code. A tenant who forges a record, or reports a key it does not seal with, can lose only its own data. Authentication pins the request to the listener's host. Writer binding (ruling 20) stops a record outliving its writer. Neither proves which key seals the database, and nothing here claims to. Cost if wrong: if the record were ever treated as a security boundary between tenants, it would be the wrong tool. It is not used as one.

20. One evidence rule, two sources (rev 3; Paul's R-B; findings R2-1 to R2-3). For a nonempty database written by an admitted writer, the pinner (O2') accepts the pinned manifest only with:

  • (a) a live instance.key entry, unchanged; or
  • (b') an adoption record, the platform item instance-key-adoption.json, bound to the writer whose data is pinned. That writer is the session admitted as the pinned database's generation: the holder's own session when that generation is the holder's, else the matching admission (pinnedWriter). All three of these must hold: - the record binds exactly that writer's host, incarnation, session and generation; - its digest is the digest of the key the target will deliver. A platform record must match the pinned env item's own key, which the install restores byte for byte. A config record needs an env item with no own key, because one would override the configuration at launch; - the key is pinned by digest (PinObject.InstanceKey) and verified again at install. The install refuses if the env files it would launch with deliver any other key, for example a configuration changed since the pin (O3', instancekey.VerifyDelivered).

Source (c) and ConfiguredInstanceKey are gone: a configured key is covered by (b'). Invalidation: every new writer admission starts with no record (O1'). A later release that picks its own key (step 1) sends no report, so it has no record and never inherits an earlier session's. A rollback is a new session and is the same case. Without (a) or (b'), relocation refuses (stuck), as before L21. Cost if wrong: an app that adopts but whose report never landed for the pinned session goes stuck rather than relocating. That is safe and visible, and the next process start re-reports.

21. The agent binds the record from its own launch state (rev 3; Paul's R-B). The helper reports on every process start, at most once per process: the digest of the key it resolved at step 2, and its source, platform or config. The agent checks both against the launch state it holds for that process (AdoptionLaunch). That state is the writer (holder incarnation, session, generation at admission), plus the digest and source of the key the launch env delivered, which SnapshotClaimLaunch now records (InstanceKeySHA256, InstanceKeySource, computed with instancekey.Effective from the same two files as the env). It then writes the record: {"v":2,"var":"CARLOS_INSTANCE_KEY","host","incarnation","session","generation","source","sha256"}. Nothing in the record comes from the request; a body naming a session is a 400. The verb is mounted only on launches that have a writer session, which means claim launches once they bind a control socket (O6). A legacy launch has no session to bind, so nothing mounts the verb there, and the helper's call is the 404 it ignores. Rev 2 mounted it on legacy launches; that is withdrawn. Cost if wrong: if a launch's state were taken from anything but its admission, a record could bind the wrong writer. AdoptionLaunch is built once per launch, by O6, from the admission.

22. Adoption is published on the running session (rev 3; Paul's R-C; finding R2-4).

  • The verb writes the record beside <host>.env: atomically, 0600, file and directory fsynced.
  • It then hands the record to the session's files publisher through hostfiles.PlatformRelay and answers 204. 204 means the record is durable on disk; it does not wait on the bucket.
  • The relay stages the record as the platform item (StagePlatform), retrying while the store refuses. The publisher's continuous loop (hostfiles.Run) commits it within its deadline, as it commits every staged change. The item is always continuous, whatever the app's files durability.
  • An adoption followed by box loss, with no further admission or release, is therefore in the bucket for the pinner.

Cost if wrong: a box lost inside the publisher's bound (one deadline, a second by default) after the first adoption loses that report, and the pin refuses until a later session re-reports. That is safe.

23. O6 is a gate in the HA rollout (rev 3; Paul's R-D). Claim launches bind no control socket today (claimsnapshot.go:116 strips `CARLOSCONTROL_SOCKET`). Until O6, an app on the platform key or a configured key cannot report adoption on a claim box. Its relocation refuses: an explicit refusal, not platform-key HA support. O4's checklist says so. Cost if wrong: none beyond the refusal, which is the point.

24. The move lifecycle includes the record (rev 3; Paul's R-E; finding R2-5). The record is an installation artifact. Nested, it lives in the instance directory and is retired with it. Flat, it must be in moves' journalled artifact lists, so a move away retires it and a later empty-installation census on that box finds nothing of the host's. That is O7, with the exact lists. The writer (O6) does not ship before O7. The census classification does not change: the record stays an unclassified host artifact, so a leftover still refuses, which is the check that keeps O7 honest. Cost if wrong: a host that moved away could not be installed on its old box again until an operator removed the file.

25. A replica is handed no key at all (rev 3.1; superseded by ruling 26). Rev 3.1 stripped CARLOS_INSTANCE_KEY from every replica launch. Astra's round 4 showed that broke replicas that already share their home's configured key, and did not refuse on a launch without a control socket. The strip is reverted.

26. A replica gets exactly its home's key (rev 4; Paul's decision, 2026-10-11; replaces rulings 17 and 25; Astra round 4). A replica (#162) mirrors its home's data, which is sealed under the home's effective key. A replica is handed exactly that key or none, and it never mints.

  • Never minted. No path mints on a replica row (InstanceKeyReplica), and the backfill skips replicas.
  • The gate, replicaKeyGate, runs on every replica launch before anything starts: Start, Seed, the nested prepare's run and seed roles, and the claim launch source. - If the replica's env files deliver a key, the replica starts only when that key's digest is its home's (ExecBackend.ReplicaHomeKeys). A key that differs is refused, with the log line activator: replica <host> not started: CARLOS_INSTANCE_KEY is not its home's key. A key that cannot be checked is refused too. - If the env files deliver no key, the replica starts only with a bound control socket. There an app on rastrillo's InstanceKey with no key of its own gets ErrNoInstanceKey and refuses (step 3), and an app that never asks for a key runs as before. Without a socket (every seed, every claim launch before O6, a flat launch whose bind failed) the helper would fall to step 4 and mint over the home's data. So the agent refuses the launch itself: it does not leave the refusal to the helper.
  • Where the home's key comes from. The home's key reaches a replica through O8: a replica record that names its home, and a read of the home's published env item, verified by digest against the home's adoption record or env item. Until O8, nothing can hand a keyless replica its home's key, so it gets none and is gated as above. Verification uses the digests the pre-roll check records in the box's list, /etc/carlos/replica-home-keys (replica-host=<sha256> entries, read by replicaHomeKeys at every launch; ruling 30). A list that does not parse verifies nothing.
  • No regression on upgrade. A replica that shares a configured key that is its home's effective key keeps it: the pre-roll check lists it, and the gate passes it. A replica whose configured key differs is refused, and the pre-roll check shows it before any box is upgraded (rollout step 1).
  • The adoption verb still answers 409 on a replica: a replica's record would bind a writer whose data is the home's.

Cost if wrong:

  • A keyless replica of an app with a seed command cannot seed until O8 or a verified configured key, because seeds have no control socket.
  • A keyless replica on a claim box does not start before O6. Claim launches start nothing before Group D anyway.
  • A flat replica whose control bind fails does not start.
  • A replica whose key the operator never listed is refused after the roll. That is loud and fixed by the list.
  • The list's freshness, and any change to a configured key, are ruling 27 (rev 5). O8's scope is ruling 28.

27. A configured key is never changed in place (rev 5; Paul's ruling; Astra round 5 finding 1). Changing a configured CARLOS_INSTANCE_KEY on an app that has data is a key rotation, and this plan does not support rotation. SyncExec reconciles each box on its own schedule, hourly, so during any configured-key change the home and a replica can briefly hold different keys.

  • The procedure (rollout step 1 and the operator note): stop the app's replicas; remove their entries from the box's list and check that the file no longer names them (the agent keeps no copy, ruling 30); change the key; let every box sync; re-run the pre-roll check, which writes the new entries; then start the replicas.
  • The safety net. On every replica launch whose key comes from its configuration, the agent checks two things. The key's digest must equal the replica's list entry, which only the pre-roll check writes and nothing refreshes from the replica's configuration. And the bucket's configuration must render that same key now (ConfiguredKey, a fresh read on every such launch). Any mismatch, or a read that fails, refuses the launch with a log line naming the host and the reason.
  • The fresh read is the cheap fresh evidence that works without O8. A shared configuration change reaches the bucket before any box syncs it, so a replica whose box lags the home is refused.

Residual (procedure not followed): a change the bucket does not hold is invisible to the replica's box. Examples are the home's own <host>.env edited by hand, or a per-host key configured on the home alone. The replica's list entry and the bucket still agree, so it runs with the old key. A test documents this case; only the procedure, or O8's fresh evidence for platform keys, closes it.

Cost if wrong: a bucket read on every launch of a replica whose key is configured. A bucket that cannot be reached refuses those launches, which is safe but costs availability while the bucket is down.

28. O8 delivers platform keys only (rev 5; Paul's ruling; Astra round 5 finding 2). O8 hands a keyless replica only a key the platform minted into the home's <host>.env, read from the home's published env item and verified against it. A home whose key exists only in its configuration is explicitly deferred: its value is not in any published item, and the adoption record holds only its digest. Its keyless replica keeps refusing (ruling 26), and a replica configured with the same key follows ruling 27. Cost if wrong: an app that keys its home by configuration must also configure its replicas with that key (the shared app-level key, listed by the pre-roll check) for them to run.

29. The replica gate decides on the launch's own read (rev 5.1; Astra round 6 finding 1). Configuration sync shares no lock with a launch, so the env files can change between the read that composes a child's env and any later read. The source of the key the child is handed is therefore taken from the same read: instanceEnvLayer returns mergeEnv(cfg, own) and Effective(cfg, own)'s layer together. Every gate call passes that layer, and the gate never calls Effective on a second read. A non-empty key with no known layer refuses. Cost if wrong: none at run time, since it is one read where there were two. If a future caller composed the env some other way and passed a stale layer, the gate would still compare the delivered key's digest with the list and the bucket, and an unknown layer still refuses.

30. The verified list is read at every replica launch (rev 5.1; Astra round 6 finding 2).

  • Where it lives. The list (still called CARLOS_REPLICA_HOME_KEYS in errors and docs) moves from the agent's environment, which a running agent never re-reads, to a file: /etc/carlos/replica-home-keys. It holds replica-host=<sha256> entries separated by newlines or commas, root-owned and readable by the agent (0640 root:carlos).
  • How it is written. The pre-roll check writes it whole, to a temporary file renamed into place, so a reader sees the old list or the new one, never part of one. The agent writes nothing to it.
  • How it is read. replicaHomeKeys reads the file at every call and keeps no copy. A removed entry stops verifying at the very next replica launch, and a replacement verifies from the next launch on, with no agent restart (which would restart every instance on the box). An absent file verifies nothing. An unreadable or malformed one verifies nothing and is logged without its text.

Cost if wrong: one read of a small file per replica launch. A list written by hand without the rename can be read mid-write. A cut inside an entry fails to parse and refuses every keyed replica until the file is whole. A cut after a complete entry still parses, so the entries before the cut keep verifying and those after it are refused until the file is whole. Both outcomes refuse rather than admit a wrong key, and the prescribed write-then-rename avoids either (Astra round 7, P3).

Obligations on other plans

Rev 3 rewrites O1', O2', O3', O4, O5 and O6, and adds O7. Rev 4 adds O8.

  • O1' (the C8 files-manifest provider, ClaimFilesManifest): Establish leaves a committed manifest whose hostfiles.EnvItem holds exactly the bytes of <host>.env when it runs. A new admission starts with no adoption record (ruling 20): the new generation's manifest carries no instancekey.AdoptionItem (StagePlatformDelete), and Establish removes the local <host>.instance-key-adoption.json under the start's captured work. It never writes <host>.env. Any error (store unreachable, If-Match lost) is returned, and the wake fails before the app executes. Tests in the provider's plan, run against the provider itself rather than a fake: - TestEstablishPublishesTheEnvFileWithItsInstanceKey: a dir store, a predecessor manifest whose env item lacks the key and which carries an adoption item, and a local <host>.env with the key plus a local record. After Establish, ReadPlatform(EnvItem) equals the local bytes, the new manifest has no adoption item, the local record is gone, and the manifest is the committed revision. - TestEstablishFailsTheWakeWhenThePublishFails: a store that refuses the PUT. Establish errors, and through claimServingOwners.Materialize and startAdmitted no start is reserved.
  • O2' (stage B, hostrelocate/pin.go on region-ha-1b-next-build): the pinner's check for a nonempty database is (a) or (b') (ruling 20). It replaces rev 2's O2' and stage B's own rev 3 variant: ruling 55, its StorePinner.KeyEvidence reader and its KeyQuestion, KeySource and envHasInstanceKey. The evidence is in the pinned manifest, so the pinner needs no outside reader and no wiring. The diff and the test are below ("O2' for stage B"). Stage B's ruling 55 becomes: "The instance key is evidenced by (a) instance.key or (b') an adoption record bound to the pinned writer for the key the target delivers, pinned by digest (instance-key plan rev 3, ruling 20). The env item is parsed with instancekey.ParseEnv."
  • O3' (stage B Task 8c, relocation install): - The install writes the pinned manifest's EnvItem byte for byte with writeFileSynced, or removes <host>.env when the item is absent. It never mints. - It does not restore the adoption item: the target's sessions are new writers, which start with no record (ruling 20). - When the pin carries InstanceKey, then after the env file and the held render (RenderHeldUnder) are on disk and before anything launches, it reads both files and calls instancekey.VerifyDelivered(config, env, pin.InstanceKey.SHA256). On an error it refuses (the operation goes stuck) and starts nothing. - Tests: TestRelocationInstallKeepsThePublishedEnvByteForByte gains "after Install, activator.EnsureInstanceKeyBeneath(dir, host+".env") returns InstanceKeyPresent and leaves the bytes unchanged". New TestRelocationInstallDeliversThePinnedKey has three cases: a pinned platform key installs; a pinned configured key whose configuration is unchanged installs; a configuration changed between the pin and the install refuses with nothing launched.
  • O4 (stage B Task 10 checklist item 9) becomes: "L21. The instance-key plan is rolled to the region's boxes, and O1', O2', O3', O5 and O7 are in. O6 gates platform-key and configured-key relocation: until claim launches bind a control socket and mount the adoption verb, an app whose key is not its own instance.key cannot report adoption on a claim box, and its relocation refuses (stuck). That is an explicit refusal, not HA support for those apps. Apps must also run a rastrillo release with the adoption call (#66 ask and addendum)."
  • O5 (stage B's release flush, internal/host/relocationflush.go on that branch): stages the local adoption record as AdoptionItem when it parses and binds the releasing writer (the holder's incarnation, session and generation); otherwise it stages StagePlatformDelete. Both go in the same commit as the env item. Test: a flush with the releasing writer's record publishes it; with a stale record, or none, publishes none.
  • O6 (claim Plan 9 Group D, the isolated-root launcher), the gate of ruling 23: - When claim launches bind a control socket, mount ExecBackend.MountInstanceKeyAdoption on it. - Thread ClaimLaunchSnapshot.InstanceKeySHA256 and InstanceKeySource through ClaimServingLaunch and ClaimLaunchInputs. Today claimServingLaunchSource (claimproduction.go) drops both; the snapshot computes them and nothing carries them further. - Its launch returns the process's AdoptionLaunch: Writer from the admission (holder incarnation, session and generation as admitted); KeySHA256 and Source from the threaded fields; and the instance directory. It must reflect the current admission: the verb binds whatever launch() returns at the moment of the request, so once the session has ended or been superseded launch returns false (a 404), never the old session's writer. A late helper report after a re-admission then writes nothing. The verb refuses a writer binding the pinner could not read (500), but it cannot tell a stale complete one from a current one; that is the callback's lifetime, which O6's review checks against the session's. - Publish is a hostfiles.PlatformRelay over the session's files publisher. O6 runs both hostfiles.Run and relay.Run on the session's publisher: the relay only stages, and a staged item nobody commits never reaches the bucket (TestAdoptionSurvivesBoxLossWithoutAnotherAdmissionOrRelease is the shape). - The record's write holds adminTokenMu across its read and two fsyncs (once per helper report, once per process start); on a slow disk that stalls every token mint and key ensure on the box for that long. Known and accepted; O6's review should know the cost. - The record's write runs under the claim's guard (a hostwrite selection for the host, as the token's mint does). - The launch env gains the agent-owned CARLOS_INSTANCE_KEY_SOURCE (platform or config, from InstanceKeySource), set only when the socket is bound. It is stripped from tenant files like CARLOS_CONTROL_SOCKET (agentOwnedVars) and added to appconfig's agentOwned list. - SnapshotClaimLaunch's replica gate (ruling 26) then takes the bound socket as haveControl, so a keyless replica runs and its helper refuses at step 3. - O7 ships before O6; O6 does not ship before O7 (ruling 24). Until O6, ruling 23's refusal stands.
  • O7 (moves, the journalled artifact lifecycle; ruling 24). These lists change: - installArtifactPaths (internal/host/movesinstall.go:751-754): append host + instancekey.AdoptionSuffix after host + ".env". It flows into the dirty return's pre-move park (movesinstall.go:852), into sourceArtifactPaths (movesfinal.go:586-587) for the source finalization's park (movesfinal.go:527), and into the rollback's candidates (movesrollback.go:1314). - The landing's flat inventory (movesinstall.go:664): paths gains host + instancekey.AdoptionSuffix. The installation's inventory then claims the path its app's sessions will write. - rollbackOwnership.owns (movesrollback.go:1449-1470): a case for the record. It is the target's when it parses (instancekey.ParseAdoption) and binds the move's incarnation. Anything else is not ours. - The census classification does not change (ruling 24). A nested record lives in the instance directory and is retired with it. - Journals: the lists are journalled in order, and a journal written before O7 keeps its recorded list. No record exists anywhere before O6 ships, so an older journal cannot miss one. - Tests (in the moves change): TestMoveAwayRetiresTheAdoptionRecordSoTheHostCanReturn. A flat source holds a record; the move completes; the source finalization parks the record; EmptyInstallationCensus.CheckLocal for a reinstall of the host on the source box passes; a move back lands. Also the dirty-return variant: a return over a source that still holds a stale record parks it before placing anything.
  • O8 (replica placements, #162; rulings 26 and 28). It hands a keyless replica its home's platform-minted key, and only that. Today's fallback: a replica that cannot be handed its home's key gets none, and refuses to start reliably (replicaKeyGate). - The replica record names its home. provision.Replica and the console's create door record the home's host (and so its incarnation, through the home's claim) in an immutable field. Today a replica record names only the app, account, region and role. - The replica's box reads the home's published EnvItem from the home's files manifest, in the home region's data bucket (O1' publishes it). That needs read access for the replica's region to the home region's bucket, which no box has today. - It takes the env item's own key, the one the platform minted, and verifies it by digest against the home's adoption record (instancekey.Binds to the home's current writer) or the env item itself. - It writes the key, create-only, into the replica's own <host>.env, and supplies ExecBackend.ReplicaHomeKeys from the same read, replacing CARLOS_REPLICA_HOME_KEYS for that replica. - A home whose key exists only in configuration is out of O8's scope (ruling 28). Its value is in no published item. Its keyless replica keeps refusing, and a configured-key replica follows ruling 27. - Test, in O8's plan: a keyless replica of an adopting app whose home holds a platform key starts with exactly that key, and its helper resolves it. A home whose env item is unreadable, unverifiable or keyless leaves the replica refused.

O2' for stage B

Apply to internal/hostrelocate/pin.go on region-ha-1b-next-build after merging main, which brings internal/instancekey. It goes in place of Task 8b step 0's diff, and that step's KeyEvidence reader goes with it. It was verified in a scratch copy of origin/region-ha-1b-next-build (919662d2) with internal/instancekey added: go build ./internal/... and go vet clean, and go test ./internal/hostrelocate/ -count=1 passes, every existing pinner test included.

--- a/internal/hostrelocate/pin.go
+++ b/internal/hostrelocate/pin.go
@@ -15,6 +15,7 @@
 	"github.com/carlosframework/platform/internal/genselect"
 	"github.com/carlosframework/platform/internal/hostclaim"
 	"github.com/carlosframework/platform/internal/hostfiles"
+	"github.com/carlosframework/platform/internal/instancekey"
 	"github.com/carlosframework/platform/internal/release"
 )
 
@@ -37,6 +38,18 @@
 	// Release is the app release version the relocation installs, read
 	// at pinning from StorePinner.Release (R9); a reuse carries it.
 	Release string `json:"release"`
+	// InstanceKey is the key a nonempty database is sealed with when the
+	// pinned manifest evidences it by an adoption record rather than by
+	// instance.key (O2', instance-key plan rev 3): its digest and source.
+	// The install delivers exactly this key or refuses (O3').
+	InstanceKey *KeyPin `json:"instance_key,omitempty"`
+}
+
+// KeyPin is a pinned instance key: the SHA-256 of the key and the file
+// that supplies it at launch (instancekey.SourcePlatform or SourceConfig).
+type KeyPin struct {
+	Source string `json:"source"`
+	SHA256 string `json:"sha256"`
 }
 
 // FilesPin names one generation's files manifest by prefix, sequence and
@@ -142,11 +155,15 @@
 	// that writer's instance.key: the pinned manifest must carry it (rev
 	// 18, round 17 finding 1).
 	needKey := c.Holder.Session != 0 && obj.DB != nil && obj.DB.MaxTXID > 0
-	files, retained, err := p.filesPin(ctx, c.Holder, released && rel.ManifestSeq == 0, needKey)
+	// The writer whose data is pinned: the session admitted as the pinned
+	// database's generation (O2', instance-key plan ruling 20). Asked only
+	// when instance.key is missing.
+	writer := func() (instancekey.Binding, error) { return pinnedWriter(c, obj.DB.Generation) }
+	files, keyPin, retained, err := p.filesPin(ctx, writer, c.Holder, released && rel.ManifestSeq == 0, needKey)
 	if err != nil {
 		return hostclaim.RecoveryPin{}, err
 	}
-	obj.Files = files
+	obj.Files, obj.InstanceKey = files, keyPin
 	if released && obj.Files.Seq < rel.ManifestSeq {
 		return hostclaim.RecoveryPin{}, fmt.Errorf("%w: the files manifest is at %d, the release at %d", ErrNotYet, obj.Files.Seq, rel.ManifestSeq)
 	}
@@ -221,7 +238,7 @@
 // stays restorable at g10. Every referenced object must lie under this
 // holder's location (its incarnation) in this region's data bucket, at
 // files/<generation>/objects/; any other reference refuses the pin.
-func (p StorePinner) filesPin(ctx context.Context, h *hostclaim.Holder, releasedEmpty, needKey bool) (FilesPin, []byte, error) {
+func (p StorePinner) filesPin(ctx context.Context, writer func() (instancekey.Binding, error), h *hostclaim.Holder, releasedEmpty, needKey bool) (FilesPin, *KeyPin, []byte, error) {
 	chain := []string{h.Generation}
 	for i := len(h.Admissions) - 1; i >= 0; i-- {
 		chain = append(chain, h.Admissions[i].Generation, h.Admissions[i].Predecessor)
@@ -241,24 +258,33 @@
 			continue
 		}
 		if err != nil {
-			return FilesPin{}, nil, err
+			return FilesPin{}, nil, nil, err
 		}
 		var m hostfiles.Manifest // decoded from the bytes the digest names
 		if err := json.Unmarshal(raw, &m); err != nil || m.V != hostfiles.ManifestVersion {
-			return FilesPin{}, nil, errors.Join(fmt.Errorf("%w: files manifest %s", ErrRefused, prefix), err)
-		}
-		if needKey && !liveEntry(m, InstanceKeyPath) {
-			// The manifest was copied before the app started and the
-			// key it minted afterwards never published (§5.6's
-			// continuous-sync window, L21): installing this set would mint
-			// a new key over data sealed with the old one.
-			return FilesPin{}, nil, fmt.Errorf("%w: %s's pinned manifest has no %s for a nonempty database", ErrFilesUnevidenced, prefix, InstanceKeyPath)
+			return FilesPin{}, nil, nil, errors.Join(fmt.Errorf("%w: files manifest %s", ErrRefused, prefix), err)
 		}
 		if origins, err = referencedOrigins(h.Location, origins, m); err != nil {
-			return FilesPin{}, nil, err
+			return FilesPin{}, nil, nil, err
+		}
+		var keyPin *KeyPin
+		if needKey && !liveEntry(m, InstanceKeyPath) {
+			// (a) failed: no instance.key. (b'): an adoption record bound
+			// to the pinned writer, for the key the target will deliver
+			// (O2', instance-key plan rev 3). Without it the manifest may
+			// have been copied before the app minted a key of its own
+			// (§5.6's continuous-sync window), and installing this set
+			// would mint a new key over data sealed with the old one.
+			w, err := writer()
+			if err != nil {
+				return FilesPin{}, nil, nil, err
+			}
+			if keyPin, err = p.adoptedKey(ctx, w, prefix, m, origins); err != nil {
+				return FilesPin{}, nil, nil, err
+			}
 		}
 		sum := sha256.Sum256(raw)
-		return FilesPin{Prefix: prefix, Seq: m.Seq, SHA256: hex.EncodeToString(sum[:]), Origins: origins}, raw, nil
+		return FilesPin{Prefix: prefix, Seq: m.Seq, SHA256: hex.EncodeToString(sum[:]), Origins: origins}, keyPin, raw, nil
 	}
 	// No manifest anywhere in the chain. An empty files pin needs the
 	// same kind of evidence an empty database pin does (ruling 15): no
@@ -272,14 +298,14 @@
 	// 17, finding F2), never an install that mints a fresh key over data
 	// sealed with the old one.
 	if h.Session != 0 && !releasedEmpty {
-		return FilesPin{}, nil, fmt.Errorf("%w: %s has admitted a writer but no files manifest exists in its chain", ErrFilesUnevidenced, h.Location)
+		return FilesPin{}, nil, nil, fmt.Errorf("%w: %s has admitted a writer but no files manifest exists in its chain", ErrFilesUnevidenced, h.Location)
 	}
 	gen := h.Generation
 	if gen == "" {
 		gen = hostclaim.RootGeneration
 	}
 	prefix := h.Location + "/files/" + gen + "/"
-	return FilesPin{Prefix: prefix, Origins: []string{prefix}}, nil, nil
+	return FilesPin{Prefix: prefix, Origins: []string{prefix}}, nil, nil, nil
 }
 
 // referencedOrigins adds to origins the files prefix of every generation
@@ -395,3 +421,72 @@
 	}
 	return pinned, nil
 }
+
+// pinnedWriter is the writer whose data a pin of generation gen holds: the
+// holder's own session when gen is the holder's generation, else the
+// admitted session recorded for gen. A generation no admission names has
+// no writer to bind evidence to, and refuses.
+func pinnedWriter(c *hostclaim.Claim, gen string) (instancekey.Binding, error) {
+	h := c.Holder
+	w := instancekey.Binding{Host: c.Host, Incarnation: h.Incarnation, Generation: gen}
+	if gen == h.Generation {
+		w.Session = h.Session
+	}
+	for i := len(h.Admissions) - 1; w.Session == 0 && i >= 0; i-- {
+		if h.Admissions[i].Generation == gen {
+			w.Session = h.Admissions[i].Session
+		}
+	}
+	if w.Session == 0 || w.Incarnation == "" || gen == "" {
+		return instancekey.Binding{}, fmt.Errorf("%w: no admitted writer for generation %q", ErrFilesUnevidenced, gen)
+	}
+	return w, nil
+}
+
+// adoptedKey is O2' source (b') (instance-key plan rev 3, ruling 20): the
+// pinned manifest's adoption record must bind exactly the pinned writer,
+// and name the key the target will deliver. A platform-sourced record must
+// match the env item's own key, which the install restores byte for byte.
+// A config-sourced record needs an env item with no own key (one would
+// override the configuration at launch); its digest is pinned, and the
+// install verifies the configuration it renders still delivers it (O3').
+// The env item is parsed with instancekey.ParseEnv, the activator's own
+// rule. A named item that cannot be read is an error, never "no evidence".
+func (p StorePinner) adoptedKey(ctx context.Context, writer instancekey.Binding, prefix string, m hostfiles.Manifest, origins []string) (*KeyPin, error) {
+	read := func(name string) ([]byte, bool, error) {
+		if e, ok := m.Platform[name]; !ok || e.Deleted {
+			return nil, false, nil
+		}
+		raw, err := hostfiles.ReadPlatform(ctx, p.Data, prefix, m, name, origins)
+		return raw, err == nil, err
+	}
+	rec, ok, err := read(instancekey.AdoptionItem)
+	if err != nil {
+		return nil, err
+	}
+	refuse := func(why string) (*KeyPin, error) {
+		return nil, fmt.Errorf("%w: %s's pinned manifest has no %s and %s, for a nonempty database", ErrFilesUnevidenced, prefix, InstanceKeyPath, why)
+	}
+	if !ok {
+		return refuse("no adoption record")
+	}
+	a, err := instancekey.ParseAdoption(rec)
+	if err != nil {
+		return refuse("its adoption record is malformed")
+	}
+	if !a.Binds(writer) {
+		return refuse("its adoption record is not the pinned writer's")
+	}
+	env, _, err := read(hostfiles.EnvItem)
+	if err != nil {
+		return nil, err
+	}
+	own, _ := instancekey.Effective(nil, instancekey.ParseEnv(env))
+	switch {
+	case a.Source == instancekey.SourcePlatform && own != "" && instancekey.Digest(own) == a.SHA256:
+	case a.Source == instancekey.SourceConfig && own == "":
+	default:
+		return refuse("its adoption record does not name the key the env delivers")
+	}
+	return &KeyPin{Source: a.Source, SHA256: a.SHA256}, nil
+}

internal/hostrelocate/pin_envkey_test.go (in place of stage B's TestStorePinnerNeedsPositiveEvidenceForAnEnvKey):

package hostrelocate

import (
	"context"
	"crypto/sha256"
	"encoding/hex"
	"encoding/json"
	"errors"
	"fmt"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/hostclaim"
	"github.com/carlosframework/platform/internal/hostfiles"
	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/release"
)

// O2' (instance-key plan rev 3, replacing stage B's O2 and ruling 55's
// KeyEvidence reader): a nonempty database's pinned manifest evidences its
// key by (a) instance.key, or (b') an adoption record bound to the pinned
// writer for the key the target will deliver. Presence of a key, in the env
// or in the configuration, is never evidence (Astra rounds 1 and 2).
func TestStorePinnerKeyEvidence(t *testing.T) {
	ctx := context.Background()
	at := time.Date(2026, 10, 7, 9, 0, 0, 0, time.UTC)
	const key = "q+/Zk3vJ0aW1Yx9sR7tUe2oPl4mN8bC6dF5gH0iJ1kA="
	writer := instancekey.Binding{Host: host, Incarnation: "inc-1", Session: 3, Generation: "g3"}
	earlier := instancekey.Binding{Host: host, Incarnation: "inc-1", Session: 2, Generation: "g2"}
	envWithKey := "CARLOS_ADMIN_TOKEN=t\nCARLOS_INSTANCE_KEY=" + key + "\n"
	rec := func(b instancekey.Binding, source, k string) string {
		return string(instancekey.AdoptionRecord(b, source, instancekey.Digest(k)))
	}
	for _, tc := range []struct {
		name          string
		appKey        bool   // (a) instance.key in the manifest
		env, adoption string // "" is no item
		unreadable    string // a named item whose object is missing
		holder        func(*hostclaim.Holder)
		want          error
		pinned        *KeyPin
	}{
		{name: "(a) the app's own instance.key", appKey: true, env: envWithKey, want: nil},
		{name: "(b') the platform key, adopted by the pinned writer", env: envWithKey, adoption: rec(writer, "platform", key), pinned: &KeyPin{Source: "platform", SHA256: instancekey.Digest(key)}},
		{name: "(b') a configured key, adopted by the pinned writer", env: "CARLOS_ADMIN_TOKEN=t\n", adoption: rec(writer, "config", "configured-A"), pinned: &KeyPin{Source: "config", SHA256: instancekey.Digest("configured-A")}},
		{name: "(b') a configured key under an empty override", env: "CARLOS_ADMIN_TOKEN=t\nCARLOS_INSTANCE_KEY=\n", adoption: rec(writer, "config", "configured-A"), pinned: &KeyPin{Source: "config", SHA256: instancekey.Digest("configured-A")}},
		{name: "a backed-up platform key, the app's own key missing", env: envWithKey, want: ErrFilesUnevidenced},
		{name: "configured A, the app's own unpublished B: no report, no record", env: "CARLOS_ADMIN_TOKEN=t\n", want: ErrFilesUnevidenced},
		{name: "a stale record from a previous release", env: envWithKey, adoption: rec(earlier, "platform", key), want: ErrFilesUnevidenced},
		{name: "a rollback: a new session, the old session's record", env: envWithKey, adoption: rec(writer, "platform", key),
			holder: func(h *hostclaim.Holder) {
				h.Session, h.Generation = 4, "g4"
				h.Admissions = append(h.Admissions, hostclaim.AdmissionRecord{Generation: "g4", Predecessor: "g3", Session: 4})
			}, want: ErrFilesUnevidenced},
		{name: "a record from another installation", env: envWithKey, adoption: rec(instancekey.Binding{Host: host, Incarnation: "inc-0", Session: 3, Generation: "g3"}, "platform", key), want: ErrFilesUnevidenced},
		{name: "a platform record for another key", env: envWithKey, adoption: rec(writer, "platform", "another-key"), want: ErrFilesUnevidenced},
		{name: "a config record while the env's own key overrides it", env: envWithKey, adoption: rec(writer, "config", key), want: ErrFilesUnevidenced},
		{name: "a platform record without the platform key", env: "CARLOS_ADMIN_TOKEN=t\n", adoption: rec(writer, "platform", key), want: ErrFilesUnevidenced},
		{name: "an exported key line is not what the app receives", env: "export CARLOS_INSTANCE_KEY=" + key + "\n", adoption: rec(writer, "platform", key), want: ErrFilesUnevidenced},
		{name: "a malformed record", env: envWithKey, adoption: `{"v":1}`, want: ErrFilesUnevidenced},
		{name: "no admitted writer for the pinned generation", env: envWithKey, adoption: rec(writer, "platform", key), holder: func(h *hostclaim.Holder) { h.Incarnation = "" }, want: ErrFilesUnevidenced},
		{name: "a named adoption item that cannot be read", env: envWithKey, unreadable: instancekey.AdoptionItem, want: errUnreadable},
		{name: "a named env item that cannot be read", adoption: rec(writer, "config", "configured-A"), unreadable: hostfiles.EnvItem, want: errUnreadable},
	} {
		t.Run(tc.name, func(t *testing.T) {
			claims, data := release.NewDirStore(t.TempDir()), release.NewDirStore(t.TempDir())
			obs := pinClaim()
			obs.Claim.Holder.Incarnation = "inc-1"
			gen := "g3"
			if tc.holder != nil {
				tc.holder(obs.Claim.Holder)
				gen = obs.Claim.Holder.Generation
			}
			m := hostfiles.Manifest{V: hostfiles.ManifestVersion, Seq: 4, Entries: map[string]hostfiles.Entry{}, Platform: map[string]hostfiles.Entry{}}
			if tc.appKey {
				m = keyed(m)
			}
			item := func(name, body string, put bool) {
				sum := sha256.Sum256([]byte(body))
				sha := hex.EncodeToString(sum[:])
				obj := pinLocation + "/files/" + gen + "/objects/" + fmt.Sprintf("%016d-%s", 4, sha)
				if put {
					if _, err := data.Put(ctx, obj, []byte(body), ""); err != nil {
						t.Fatal(err)
					}
				}
				m.Platform[name] = hostfiles.Entry{Object: obj, SHA256: sha, Size: int64(len(body)), Mode: 0o600, Seq: 4}
			}
			if tc.env != "" {
				item(hostfiles.EnvItem, tc.env, true)
			}
			if tc.adoption != "" {
				item(instancekey.AdoptionItem, tc.adoption, true)
			}
			if tc.unreadable != "" {
				item(tc.unreadable, "never stored\n", false)
			}
			putManifest(t, data, gen, m)
			p := StorePinner{Claims: claims, Data: data, Release: routeRelease, Bucket: "data", Region: "ie", SelectDB: selectAt(41)}
			ref, err := p.Pin(ctx, obs, at)
			switch {
			case tc.want == errUnreadable:
				if err == nil || errors.Is(err, ErrFilesUnevidenced) {
					t.Fatalf("got %v, want a read error, never a refusal for no evidence", err)
				}
				return
			case !errors.Is(err, tc.want):
				t.Fatalf("got %v, want %v", err, tc.want)
			}
			if err != nil {
				return
			}
			raw, _, err := claims.Get(ctx, ref.Key)
			if err != nil {
				t.Fatal(err)
			}
			var obj PinObject
			if err := json.Unmarshal(raw, &obj); err != nil {
				t.Fatal(err)
			}
			if (obj.InstanceKey == nil) != (tc.pinned == nil) || (tc.pinned != nil && *obj.InstanceKey != *tc.pinned) {
				t.Fatalf("pinned key %+v, want %+v", obj.InstanceKey, tc.pinned)
			}
		})
	}
}

var errUnreadable = errors.New("sentinel: any read error")

Ask for rastrillo #66

For Paul to post on rastrillo/rastrillo discussion #66. Rev 2's ask is below, updated; the rev 3 addendum follows it.

The ask. When carlos.InstanceKey(own) returns $CARLOS_INSTANCE_KEY (step 2), have it tell the agent so, on every process start, so the platform can record that this app's current writer seals with that key. Failure is non-fatal to the app.

Why. Region HA relocates an instance's data to another box. It may hand the app the delivered key there only if that key is the one the data is sealed with. A key being present proves nothing: an app with its own key keeps it (step 1). So the platform needs positive word, and only the app's helper knows which step it took.

The call.

  • PUT http://carlos/instance-key/adoption over $CARLOS_CONTROL_SOCKET, the same transport and placeholder host ScheduleAt uses.
  • Header Authorization: Bearer $CARLOS_ADMIN_TOKEN; Content-Type: application/json.
  • Body {"sha256":"<hex>","source":"<platform|config>"}. sha256 is the lowercase hex SHA-256 of the key string exactly as resolved; never send the key. source is $CARLOS_INSTANCE_KEY_SOURCE verbatim, which the platform sets beside the key.
  • When: inside InstanceKey, only on the step 2 return, only when Running() and the token are set, at most once per process (a sync.Once), on every process start.
  • How long: synchronous, before InstanceKey returns, so the record is on disk before the app seals anything. It is bounded by a short timeout of its own (2 seconds suggested), since InstanceKey takes no context.
  • Answers: 204 means recorded. 404 or 405 means an agent or a launch without the verb: ignore it silently. 409 means the agent's launch delivered another key or source, or the host is a replica: worth one log line. 400, 401, 5xx, a timeout or a dial error: one log line at most. Never retry in a loop, and never log the key.
  • Whatever the answer, InstanceKey returns the key and a nil error.

Not called at step 1 (the app's own key), step 3 or step 4, or with no $CARLOS_ADMIN_TOKEN.

What changes for apps. Nothing visible. An app on a rastrillo release without the call still runs. Its data does not relocate (it is held as stuck) until it runs a release with the call.

Addendum (rev 3), for Paul to post

Two changes to the ask above, and only the second is new:

  1. Every process start. The helper reports once per process, on every process start, not once per agent or per deployment. The platform binds each report to the writer session that started the process, and a new session starts with no record.
  2. New: source. The body adds "source": "platform" or "source": "config", copied verbatim from $CARLOS_INSTANCE_KEY_SOURCE, which the platform sets beside $CARLOS_INSTANCE_KEY. Both keys arrive as the same variable, so the helper cannot tell them apart without it. If $CARLOS_INSTANCE_KEY_SOURCE is unset, skip the call: that agent cannot record adoption.

Review Focus

  1. O2''s evidence table. Astra's configured-A, own-B case refuses. So do a stale record from a previous release, a rollback, another installation, and a record naming a key the env does not deliver. The pinned key is in the pin object.
  2. The install's last check (O3'). A configuration changed between the pin and the install refuses (TestVerifyDeliveredRefusesAnyOtherKey).
  3. The verb takes nothing from the request but the digest and the source. A session in the body is a 400. The record binds the launch's writer. 204 means durable on disk (Task 5).
  4. Adoption, then box loss. The record is in the bucket with no admission or release after it (TestAdoptionSurvivesBoxLossWithoutAnotherAdmissionOrRelease).
  5. A failed sync, then a retry. The retry keeps the bytes, redoes the fchmod and both syncs, and only then succeeds; no child starts in between (Tasks 2 to 4).
  6. An empty CARLOS_INSTANCE_KEY= in <host>.env with a configured key. The child gets the configured key and nothing is minted (Task 3).
  7. A replica without a control socket. A keyless replica is refused by the agent, before its app could mint; a replica whose key is not its home's is refused; its home's key runs (Task 3b, through the helper's real resolution).
  8. A configured key in flight (ruling 27). The bucket says B while the replica's box still holds A, and the replica is refused. The documented residual, a change only on the home, runs with A (Task 3b).
  9. A sync between the launch's read and the gate (ruling 29), and a list change while the agent runs (ruling 30). The gate decides on the launch's own read, and the list is read at every launch (Task 3b).

File structure

  • Create internal/instancekey/instancekey.go and instancekey_test.go. They hold the parser, Effective, Digest, the sources, the writer-bound adoption record (Binding, Adoption, AdoptionRecord, ParseAdoption, Binds) and VerifyDelivered. Standard library only.
  • Modify internal/activator/admintoken.go (parseEnvBytes calls instancekey.ParseEnv) and internal/activator/backend_exec.go (mergeEnv, LaunchEnvironment's shadow list).
  • Create internal/activator/instancekey.go: the mint (EnsureInstanceKey, EnsureInstanceKeyBeneath, ensureInstanceKeyFor, BackfillInstanceKey, settleClaimInstanceEnv, InstanceKeyVar, InstanceKeyOutcome).
  • Create internal/activator/instancekey_test.go, instancekey_linux_test.go, replica_instancekey_test.go, replica_instancekey_linux_test.go, claim_instancekey_test.go and claimretry_instancekey_linux_test.go.
  • Create internal/activator/replicakey.go (replicaKeyGate, ErrReplicaKey), internal/host/replicahomekeys.go (replicaHomeKeys, configuredKey) and internal/appconfig/instancekey.go (ConfiguredKeyDigest), with internal/host/replicahomekeys_test.go, internal/appconfig/instancekey_test.go and internal/activator/replicakey_race_test.go. Modify internal/activator/backend_exec.go (ExecBackend.ReplicaHomeKeys and ConfiguredKey; the gate in seedChild and spawnChildFrom), internal/activator/prepare.go and internal/host/agent.go (Task 3b).
  • Modify internal/activator/backend_exec.go (seedChild :831, spawnChildFrom :1369), internal/activator/prepare.go (:60) and internal/activator/claim_snapshot.go (the snapshot's fields, EnsureClaimAdminToken :40-67, SnapshotClaimLaunch's env read).
  • Create internal/activator/keyadoption.go, keyadoption_test.go and keyadoption_linux_test.go: the adoption verb, mounted by O6.
  • Create internal/hostfiles/relay.go and relay_test.go: PlatformRelay.
  • Modify internal/host/claimowners.go (:125-138), for O1' on ClaimFilesManifest. Create internal/host/claim_instancekey_test.go, claim_replica_instancekey_test.go and instancekey_adoption_test.go.
  • Create internal/host/instancekeys.go and instancekeys_test.go: the backfill. Modify internal/host/agent.go (:476-484).
  • Create internal/appconfig/syncexec_instancekey_test.go, internal/moves/envreport_instancekey_test.go and internal/moves/archive/instancekey_test.go.

Sandbox note for every command below:

  • If the toolchain reports "read-only file system" for the Go cache, prefix commands with GOCACHE=$TMPDIR/gocache. Re-export it per command, and never delete that shared directory. In a worktree, also set GOFLAGS=-buildvcs=false.
  • On a shared machine, run commands under nice -n 19.
  • The sandbox refuses unix sockets ("socket: operation not permitted"). Tests that start a child or prepare a nested role must run with the sandbox off.
  • go test ./internal/host/ -run Claim takes longer than go test's default 10-minute timeout; pass -timeout 30m.

---

Task 1: The shared rule

Files:

  • Create: internal/instancekey/instancekey.go, internal/instancekey/instancekey_test.go
  • Modify: internal/activator/admintoken.go:245-257 (parseEnvBytes), internal/activator/backend_exec.go:1879-1905 (LaunchEnvironment), :1980-1990 (mergeEnv)

Interfaces:

  • Produces: instancekey.Var, AdoptionSuffix, AdoptionItem, ParseEnv, Layer (LayerNone, LayerOwn, LayerConfig), Effective, Digest, SourcePlatform, SourceConfig, SourceOf, Binding, Adoption, AdoptionRecord, ParseAdoption, (Adoption).Binds, VerifyDelivered, ValidDigest, ParseHomeKeys (ruling 26, for Task 3b).
  • [ ] Step 1: Write the failing tests

Create internal/instancekey/instancekey_test.go:

package instancekey

import (
	"strings"
	"testing"
)

// Finding 4: an empty own value never shadows a configured key; a
// non-empty own value always wins.
func TestEffectiveNeverHandsOutAnEmptyKeyWhileOneExists(t *testing.T) {
	for _, tc := range []struct {
		name        string
		config, own string
		want        string
		layer       Layer
	}{
		{"own wins", Var + "=cfg\n", Var + "=own\n", "own", LayerOwn},
		{"empty own, configured", Var + "=cfg\n", Var + "=\n", "cfg", LayerConfig},
		{"no own, configured", Var + "=cfg\n", "A=1\n", "cfg", LayerConfig},
		{"own only", "", Var + "=own\n", "own", LayerOwn},
		{"both empty", Var + "=\n", Var + "=\n", "", LayerNone},
		{"exported is not the key", "", "export " + Var + "=own\n", "", LayerNone},
	} {
		t.Run(tc.name, func(t *testing.T) {
			got, layer := Effective(ParseEnv([]byte(tc.config)), ParseEnv([]byte(tc.own)))
			if got != tc.want || layer != tc.layer {
				t.Fatalf("Effective = %q, %v; want %q, %v", got, layer, tc.want, tc.layer)
			}
		})
	}
}

// Ruling 20 (R-B): a record binds one writer. Another session, another
// generation, another installation or another host never matches, so a
// later release that picks its own key, a rollback and a reinstall never
// inherit an earlier writer's adoption.
func TestAdoptionRecordBindsOneWriter(t *testing.T) {
	key := strings.Repeat("A", 43) + "="
	w := Binding{Host: "shop.example.com", Incarnation: "inc-1", Session: 3, Generation: "g3"}
	rec := AdoptionRecord(w, SourcePlatform, Digest(key))
	if strings.Contains(string(rec), key) {
		t.Fatal("the record holds the key itself")
	}
	a, err := ParseAdoption(rec)
	if err != nil || !a.Binds(w) || a.Source != SourcePlatform || a.SHA256 != Digest(key) {
		t.Fatalf("ParseAdoption = %+v, %v", a, err)
	}
	for name, other := range map[string]Binding{
		"an earlier session (an earlier release)": {Host: w.Host, Incarnation: w.Incarnation, Session: 2, Generation: "g2"},
		"a later session (a rollback)":            {Host: w.Host, Incarnation: w.Incarnation, Session: 4, Generation: "g4"},
		"the same session, another generation":    {Host: w.Host, Incarnation: w.Incarnation, Session: 3, Generation: "g3b"},
		"another installation (a reinstall)":      {Host: w.Host, Incarnation: "inc-2", Session: 3, Generation: "g3"},
		"another host":                            {Host: "other.example.com", Incarnation: w.Incarnation, Session: 3, Generation: "g3"},
		"no writer":                               {},
	} {
		if a.Binds(other) {
			t.Errorf("%s: the record matched", name)
		}
	}
	for name, raw := range map[string]string{
		"version 1":        `{"v":1,"var":"CARLOS_INSTANCE_KEY","sha256":"` + Digest(key) + `"}`,
		"no binding":       `{"v":2,"var":"CARLOS_INSTANCE_KEY","source":"platform","sha256":"` + Digest(key) + `"}`,
		"unknown source":   `{"v":2,"var":"CARLOS_INSTANCE_KEY","host":"h","incarnation":"i","session":1,"generation":"g","source":"own","sha256":"` + Digest(key) + `"}`,
		"an unknown field": `{"v":2,"var":"CARLOS_INSTANCE_KEY","host":"h","incarnation":"i","session":1,"generation":"g","source":"config","sha256":"` + Digest(key) + `","key":"x"}`,
		"trailing data":    string(rec) + `{}`,
		"garbage":          "not json",
		// The canonical bytes are the only record (review minor 1):
		// encoding/json alone accepts each of these.
		"a trailing }":         string(rec) + `}`,
		"a trailing ]":         string(rec) + `]`,
		"an upper-case key":    strings.Replace(string(rec), `"sha256"`, `"SHA256"`, 1),
		"a key given twice":    strings.Replace(string(rec), `"source":"platform"`, `"source":"config","source":"platform"`, 1),
		"reordered fields":     strings.Replace(string(rec), `{"v":2,"var":"CARLOS_INSTANCE_KEY",`, `{"var":"CARLOS_INSTANCE_KEY","v":2,`, 1),
		"an upper-case digest": strings.Replace(string(rec), Digest(key), strings.ToUpper(Digest(key)), 1),
		"a short digest":       strings.Replace(string(rec), Digest(key), Digest(key)[:63], 1),
		"another variable":     strings.Replace(string(rec), `"var":"CARLOS_INSTANCE_KEY"`, `"var":"OTHER"`, 1),
	} {
		if raw == string(rec) {
			t.Fatalf("%s: the case did not change the record", name)
		}
		if _, err := ParseAdoption([]byte(raw)); err == nil {
			t.Errorf("%s: parsed", name)
		}
	}
}

// ParseEnv is exactly the activator's old parseEnvBytes, so what a reader
// parses is what a child received: quotes kept, whole-line comments only,
// the last duplicate winning, CRLF and surrounding whitespace trimmed.
func TestParseEnvMatchesWhatAChildReceives(t *testing.T) {
	got := ParseEnv([]byte("A=\"quoted\"\nB='single'\n# C=1\n   # D=1\nE=1 # kept\nF=1\nF=2\n  G = spaced  \nH=a=b\nI=1\r\nnoequals\n\n"))
	want := map[string]string{"A": `"quoted"`, "B": "'single'", "E": "1 # kept", "F": "2", "G": "spaced", "H": "a=b", "I": "1"}
	if len(got) != len(want) {
		t.Fatalf("ParseEnv = %q, want %q", got, want)
	}
	for k, v := range want {
		if got[k] != v {
			t.Fatalf("ParseEnv = %q, want %q", got, want)
		}
	}
}

// O3': the install refuses env files that would deliver any key but the
// pinned one, including a configuration changed since the pin.
func TestVerifyDeliveredRefusesAnyOtherKey(t *testing.T) {
	a, b := strings.Repeat("A", 43)+"=", strings.Repeat("B", 43)+"="
	for _, tc := range []struct {
		name, config, own string
		ok                bool
	}{
		{"the configured key, unchanged", Var + "=" + a + "\n", "T=1\n", true},
		{"the configuration changed since the pin", Var + "=" + b + "\n", "T=1\n", false},
		{"the configured key removed", "", "T=1\n", false},
		{"an empty override over the configured key", Var + "=" + a + "\n", Var + "=\n", true},
		{"the platform key, restored", "", Var + "=" + a + "\n", true},
		{"a platform key over the pinned configured one", Var + "=" + a + "\n", Var + "=" + b + "\n", false},
	} {
		if err := VerifyDelivered([]byte(tc.config), []byte(tc.own), Digest(a)); (err == nil) != tc.ok {
			t.Errorf("%s: err = %v", tc.name, err)
		}
	}
}

// Ruling 26: the pre-roll check's verified home keys, as the box reads
// them. A list that may be wrong verifies nothing, and its errors never
// echo an entry, which may hold a digest (Astra round 5 finding 3).
func TestParseHomeKeys(t *testing.T) {
	a, b := Digest("home-key-a"), Digest("home-key-b")
	got, err := ParseHomeKeys(" shop-au.example.com=" + a + ", shop-uk.example.com=" + b + ",")
	if err != nil || len(got) != 2 || got["shop-au.example.com"] != a || got["shop-uk.example.com"] != b {
		t.Fatalf("ParseHomeKeys = %v, %v", got, err)
	}
	if got, err := ParseHomeKeys(""); err != nil || len(got) != 0 {
		t.Fatalf("empty = %v, %v", got, err)
	}
	for _, tc := range []struct{ value, want string }{
		{a, "entry 1: no \"replica-host=\""},
		{"shop-au.example.com=" + a + "," + b, "entry 2: no \"replica-host=\""},
		{"shop-au.example.com=" + a[:40], "entry 1: the digest is not"},
		{"=" + a, "entry 1: the host is empty"},
		{a[:20] + "=" + a, "entry 1: the host is empty or not a host name"},
		{"shop-au.example.com=" + a + ",shop-au.example.com=" + b, "entry 2: its host is listed"},
	} {
		_, err := ParseHomeKeys(tc.value)
		if err == nil || !strings.Contains(err.Error(), tc.want) {
			t.Errorf("%q: err = %v, want %q", tc.value, err, tc.want)
			continue
		}
		if strings.Contains(err.Error(), a[:16]) || strings.Contains(err.Error(), b[:16]) || strings.Contains(err.Error(), "shop-") {
			t.Errorf("the error echoes its input: %v", err)
		}
	}
}
  • [ ] Step 2: Run them to verify they fail

Run: nice -n 19 go test ./internal/instancekey/ -count=1 Expected: build failure, no non-test Go files.

  • [ ] Step 3: Write the package

Create internal/instancekey/instancekey.go:

// Package instancekey is the one rule for the platform's per-instance key
// (region HA L21, rastrillo #66): how an env file is parsed, which file's
// CARLOS_INSTANCE_KEY a launched app receives, and the adoption record, a
// writer-bound assertion from the app's helper that it seals with that
// key. The activator mints and
// delivers the key by these rules, the control socket records adoption by
// them, and the relocation pinner checks its evidence by them, so the three
// can never disagree about what the app was given.
//
// It imports only the standard library, so the pinner can use it without
// importing the activator.
package instancekey

import (
	"bytes"
	"crypto/sha256"
	"encoding/hex"
	"encoding/json"
	"errors"
	"fmt"
	"strings"
)

// Var is the variable the key is delivered as.
const Var = "CARLOS_INSTANCE_KEY"

// AdoptionSuffix names the adoption record beside an instance's own env
// file: <host>.env's sibling <host>.instance-key-adoption.json. It is an
// installation artifact: moves retire it with <host>.env (O7), and nested
// it lives in the instance directory, retired with it.
const AdoptionSuffix = ".instance-key-adoption.json"

// AdoptionItem is the adoption record's name among a files manifest's
// platform items, beside hostfiles' EnvItem ("host.env").
const AdoptionItem = "instance-key-adoption.json"

// ParseEnv parses an EnvironmentFile: KEY=VALUE lines, blank lines and #
// comments skipped, the key and the value trimmed of surrounding
// whitespace, the last of two lines for one key winning. It is exactly
// what the activator hands a child, so a reader that wants to know what an
// app received must parse with it. "export KEY=..." is the key "export
// KEY", and a quoted value keeps its quotes.
func ParseEnv(raw []byte) map[string]string {
	m := map[string]string{}
	for _, line := range strings.Split(string(raw), "\n") {
		line = strings.TrimSpace(line)
		if line == "" || strings.HasPrefix(line, "#") {
			continue
		}
		if k, v, ok := strings.Cut(line, "="); ok {
			m[strings.TrimSpace(k)] = strings.TrimSpace(v)
		}
	}
	return m
}

// Layer is which env file supplied the effective key.
type Layer int

const (
	// LayerNone: neither file supplies a non-empty key.
	LayerNone Layer = iota
	// LayerOwn: the instance's own <host>.env, where the platform mints.
	LayerOwn
	// LayerConfig: the rendered <host>.config.env, the app's configuration.
	LayerConfig
)

// Effective is the key a launched app receives from its two env files, and
// which file supplied it. The instance's own file wins, as in every other
// variable (the activator's mergeEnv), but an empty value supplies no key:
// an empty CARLOS_INSTANCE_KEY= in the own file never shadows a non-empty
// configured key, so an app is never handed an empty key while a non-empty
// one exists.
func Effective(config, own map[string]string) (string, Layer) {
	if k := own[Var]; k != "" {
		return k, LayerOwn
	}
	if k := config[Var]; k != "" {
		return k, LayerConfig
	}
	return "", LayerNone
}

// Digest is the lowercase hex SHA-256 of key: what an app's helper sends
// to say which key it resolved, and what the adoption record holds. The
// record is published beside the env file that holds a platform key
// itself, so the digest gives that file's reader nothing new.
func Digest(key string) string {
	sum := sha256.Sum256([]byte(key))
	return hex.EncodeToString(sum[:])
}

// The two sources rastrillo's step 2 can have resolved. Both arrive as
// $CARLOS_INSTANCE_KEY once the env files are merged.
const (
	SourcePlatform = "platform" // the instance's own <host>.env, where the platform mints
	SourceConfig   = "config"   // the rendered <host>.config.env, the app's configuration
)

// SourceOf names the source of a layer, "" for LayerNone.
func SourceOf(l Layer) string {
	switch l {
	case LayerOwn:
		return SourcePlatform
	case LayerConfig:
		return SourceConfig
	}
	return ""
}

// Binding is the writer an adoption belongs to: the host, its installation
// (the claim holder's incarnation), and the writer session and generation
// the process was admitted as. The agent takes all four from its own
// launch state for the process, never from the request.
type Binding struct {
	Host        string `json:"host"`
	Incarnation string `json:"incarnation"`
	Session     uint64 `json:"session"`
	Generation  string `json:"generation"`
}

func (b Binding) valid() bool {
	return b.Host != "" && b.Incarnation != "" && b.Session != 0 && b.Generation != ""
}

// Adoption is the record the agent writes when a process reports that its
// helper resolved the key at step 2: which writer, which source, which
// key (by digest). It is a trusted-helper assertion, not independent
// evidence: it stops accidental loss, and a tenant who forges one can lose
// only its own data (instance-key plan ruling 19).
type Adoption struct {
	V   int    `json:"v"`
	Var string `json:"var"`
	Binding
	Source string `json:"source"`
	SHA256 string `json:"sha256"`
}

// AdoptionRecord is the canonical bytes of the record for writer b, which
// was handed the key with digest sha from source.
func AdoptionRecord(b Binding, source, sha string) []byte {
	raw, _ := json.Marshal(Adoption{V: 2, Var: Var, Binding: b, Source: source, SHA256: sha})
	return append(raw, '\n')
}

// ParseAdoption reads a record strictly: version 2, this variable, a
// complete binding, a known source and a digest, in exactly the bytes
// AdoptionRecord writes for them. Anything else is an error, never a
// record: trailing data, a key in another case, a key given twice, or any
// other spelling of the same fields (encoding/json alone accepts the
// last three).
func ParseAdoption(record []byte) (Adoption, error) {
	var a Adoption
	dec := json.NewDecoder(bytes.NewReader(record))
	dec.DisallowUnknownFields()
	if err := dec.Decode(&a); err != nil {
		return Adoption{}, fmt.Errorf("instancekey: adoption record: %w", err)
	}
	if dec.More() || a.V != 2 || a.Var != Var || !a.Binding.valid() || (a.Source != SourcePlatform && a.Source != SourceConfig) || !ValidDigest(a.SHA256) {
		return Adoption{}, errors.New("instancekey: adoption record: not a version 2 record for " + Var)
	}
	if !bytes.Equal(record, AdoptionRecord(a.Binding, a.Source, a.SHA256)) {
		return Adoption{}, errors.New("instancekey: adoption record: not in the canonical form AdoptionRecord writes")
	}
	return a, nil
}

// Binds reports whether the record belongs to exactly writer w. A record
// from another installation, another session or another generation (an
// earlier release, a rollback, a reinstall) never vouches for w.
func (a Adoption) Binds(w Binding) bool {
	return w.valid() && a.Binding == w
}

// VerifyDelivered checks that the env files a relocation target is about
// to launch with deliver the key pinned by digest (O3'): the effective key
// of the rendered configuration and the restored own env file, by
// Effective's rule. A configuration changed since the pin, a key removed,
// or a platform key where a configured one was pinned, is an error; the
// install refuses rather than hand the app a different key.
func VerifyDelivered(configEnv, ownEnv []byte, pinnedSHA string) error {
	key, _ := Effective(ParseEnv(configEnv), ParseEnv(ownEnv))
	if key == "" || Digest(key) != pinnedSHA {
		return errors.New("instancekey: the env files would not deliver the pinned " + Var)
	}
	return nil
}

// ValidDigest reports whether s is a lowercase hex SHA-256.
func ValidDigest(s string) bool {
	if len(s) != 64 {
		return false
	}
	for _, r := range s {
		if (r < '0' || r > '9') && (r < 'a' || r > 'f') {
			return false
		}
	}
	return true
}

// ParseHomeKeys reads CARLOS_REPLICA_HOME_KEYS, the box's list of verified
// home keys for its replica rows (instance-key plan rulings 26 and 27):
// "replica-host=sha256,replica-host=sha256", each digest the home's
// effective key as the pre-roll check computed it. Only the pre-roll check
// writes it; nothing refreshes it from a replica's own configuration.
// Whitespace around entries is ignored and "" is an empty list. A
// malformed entry, or a host named twice, fails the whole value: a list
// that may be wrong is not a list to verify keys against. Its errors name
// the entry by position and the reason, never the entry's text, which may
// hold a digest.
func ParseHomeKeys(s string) (map[string]string, error) {
	out := map[string]string{}
	n := 0
	for _, e := range strings.Split(s, ",") {
		e = strings.TrimSpace(e)
		if e == "" {
			continue
		}
		n++
		host, sha, ok := strings.Cut(e, "=")
		host, sha = strings.TrimSpace(host), strings.TrimSpace(sha)
		switch {
		case !ok:
			return nil, fmt.Errorf("instancekey: CARLOS_REPLICA_HOME_KEYS entry %d: no \"replica-host=\" before the digest", n)
		case host == "" || strings.ContainsAny(host, " /=") || !strings.Contains(host, "."):
			return nil, fmt.Errorf("instancekey: CARLOS_REPLICA_HOME_KEYS entry %d: the host is empty or not a host name", n)
		case !ValidDigest(sha):
			return nil, fmt.Errorf("instancekey: CARLOS_REPLICA_HOME_KEYS entry %d: the digest is not 64 lowercase hex characters", n)
		}
		if _, dup := out[host]; dup {
			return nil, fmt.Errorf("instancekey: CARLOS_REPLICA_HOME_KEYS entry %d: its host is listed by an earlier entry", n)
		}
		out[host] = sha
	}
	return out, nil
}
  • [ ] Step 4: One parser, and the merge follows the rule

In internal/activator/admintoken.go, replace parseEnvBytes's body (keep its doc comment), drop the now-unused "strings" import and add "github.com/carlosframework/platform/internal/instancekey":

func parseEnvBytes(raw []byte) map[string]string {
	return instancekey.ParseEnv(raw)
}

In internal/activator/backend_exec.go, add the same import, and end mergeEnv with the rule:

	for k, v := range overlay {
		out[k] = v
	}
	// An empty CARLOS_INSTANCE_KEY in the instance's own file supplies no
	// key, so it never shadows a configured one: the child is never handed
	// an empty key while a non-empty one exists (instance-key plan ruling
	// 13). instancekey.Effective is the rule the key's mint decides by.
	if key, _ := instancekey.Effective(base, overlay); key != "" {
		out[instancekey.Var] = key
	}
	return out
}

In LaunchEnvironment, an empty own key is not a shadow:

	for k := range ownEnv {
		if _, rendered := cfgEnv[k]; rendered {
			if k == instancekey.Var && ownEnv[k] == "" {
				continue // an empty own key shadows nothing (mergeEnv)
			}
			shadowed = append(shadowed, k)
		}
	}

Name the exception in both doc comments (task 1 review):

// mergeEnv composes the child's env-file layers: base (the sync-owned
// <host>.config.env, bucket-derived) under overlay (the instance's own
// <host>.env). The overlay wins every conflict but one — CARLOS_ADMIN_TOKEN
// lives only in the instance file and must never be clobberable from
// the bucket, and a box-local override deliberately beats central
// config. The exception: an empty CARLOS_INSTANCE_KEY in the overlay
// supplies no key, so a non-empty one in base stands (instancekey.Effective).
// Returns a fresh map; inputs are never mutated.
func mergeEnv(base, overlay map[string]string) map[string]string {
// shadowed is the sorted names present in both the rendered configuration
// file and the instance's own <host>.env: the override mergeEnv applies
// (moves spec §7.3 T3 reports it), collected here because this is the only
// layer that sees both files. The one exception is mergeEnv's: an empty
// CARLOS_INSTANCE_KEY in the own file overrides nothing, so it is not
// listed. The move's launch report (§7.3, codex R2-8)
// is computed over exactly this env, so a target compares what would run,
// not a reconstruction of it. The values are raw: the caller HMACs them
// and must never log or return them.
func (b *ExecBackend) LaunchEnvironment(rt registry.Route) (map[string]string, []string, error) {

Add to internal/activator/backend_exec_test.go, after TestLaunchEnvironmentComposesTheSpawnEnvironment:

// An empty CARLOS_INSTANCE_KEY in the instance's own file supplies no key
// (instance-key plan ruling 13): the merge keeps the configured key, and
// the launch report does not list the empty one as shadowing it. A
// non-empty own key still wins and is still reported.
func TestEmptyOwnInstanceKeyShadowsNothing(t *testing.T) {
	cfg := map[string]string{"CARLOS_INSTANCE_KEY": "cfg"}
	if got := mergeEnv(cfg, map[string]string{"CARLOS_INSTANCE_KEY": ""}); got["CARLOS_INSTANCE_KEY"] != "cfg" {
		t.Fatalf("mergeEnv with an empty own key: CARLOS_INSTANCE_KEY = %q, want the configured key", got["CARLOS_INSTANCE_KEY"])
	}
	b := &ExecBackend{DataDir: t.TempDir(), Region: "ie"}
	rt := registry.Route{Host: "k.go.example.com", Socket: filepath.Join(b.DataDir, "k.sock")}
	write := func(name, body string) {
		t.Helper()
		if err := os.WriteFile(filepath.Join(b.DataDir, rt.Host+name), []byte(body), 0o600); err != nil {
			t.Fatal(err)
		}
	}
	write(".config.env", "CARLOS_INSTANCE_KEY=cfg\nMINE=0\n")
	write(".env", "CARLOS_INSTANCE_KEY=\nMINE=1\n")
	env, shadowed, err := b.LaunchEnvironment(rt)
	if err != nil {
		t.Fatal(err)
	}
	if env["CARLOS_INSTANCE_KEY"] != "cfg" || len(shadowed) != 1 || shadowed[0] != "MINE" {
		t.Fatalf("empty own key: CARLOS_INSTANCE_KEY is the configured key: %v; shadowed %v, want [MINE]", env["CARLOS_INSTANCE_KEY"] == "cfg", shadowed)
	}
	write(".env", "CARLOS_INSTANCE_KEY=own\nMINE=1\n")
	if env, shadowed, err = b.LaunchEnvironment(rt); err != nil || env["CARLOS_INSTANCE_KEY"] != "own" || strings.Join(shadowed, ",") != "CARLOS_INSTANCE_KEY,MINE" {
		t.Fatalf("non-empty own key: CARLOS_INSTANCE_KEY is the own key: %v; shadowed %v; err %v", env["CARLOS_INSTANCE_KEY"] == "own", shadowed, err)
	}
}
  • [ ] Step 5: Run the tests

Run: nice -n 19 go test ./internal/instancekey/ -count=1 && nice -n 19 go test ./internal/activator/ -run 'TestExecBackendStartMergesConfigEnvUnderOwnEnv|LaunchEnvironment|TestEmptyOwnInstanceKeyShadowsNothing|TestEnsureAdminToken' -count=1 (sandbox off for the second: it starts children) Expected: PASS.

  • [ ] Step 6: Commit
gofmt -l internal/instancekey internal/activator
git add internal/instancekey internal/activator/admintoken.go internal/activator/backend_exec.go internal/activator/backend_exec_test.go
git commit -m "instancekey: one rule for the instance key's parse, effective layer and adoption record (L21)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"

---

Task 2: The mint, durable every time

Files:

  • Create: internal/activator/instancekey.go, internal/activator/ownenv_unix.go and ownenv_other.go (openOwnEnvNoFollow: O_NOFOLLOW on unix, an Lstat check-then-open on windows, which the agent never runs on)
  • Test: internal/activator/instancekey_test.go, internal/activator/instancekey_linux_test.go

Built: the code block in Step 3 is the branch's, with three things rev 5.1 did not have, each by ruling. The flat own file is opened with O_NOFOLLOW in the open itself (not check-then-open), mirroring EnsureAdminToken's symlink refusal; a symlinked flat <host>.env fails the launch (the admin-token mint path has the same hole on main: #273). A failed or short append is undone: the file is truncated back to the size the open descriptor reported before the write (undoInstanceKeyAppend) and fsynced, so a short key line never survives to be read as present; under adminTokenMu that size is len(raw). And ensureInstanceKeyFor refuses a fenced or conflicting row itself, deciding on the paths the mint would use, because the launch bodies classify again in pathsFor after the dispatcher's own check and a migration starting mid-wake would otherwise mint into the flat directory of a half-migrated host.

Interfaces:

  • Consumes: Task 1's instancekey; adminTokenMu (admintoken.go:43), logPerFile (:70), parseEnvBytes, instancefs.ReadBeneathLimit, instancefs.OpenBeneath, (*ExecBackend).pathsFor (paths.go:33), Layout* constants (layout.go:21-26).
  • Produces: - const InstanceKeyVar = instancekey.Var - type InstanceKeyOutcome int with InstanceKeyPresent, InstanceKeyMinted, InstanceKeyFromConfig, InstanceKeyNoEnvFile, InstanceKeyReplica - func EnsureInstanceKey(path string) (InstanceKeyOutcome, error) - func EnsureInstanceKeyBeneath(dir, name string) (InstanceKeyOutcome, error) - func (b *ExecBackend) ensureInstanceKeyFor(rt registry.Route, p instancePaths) (InstanceKeyOutcome, error) - func (b *ExecBackend) BackfillInstanceKey(rt registry.Route) (InstanceKeyOutcome, error) - func (b *ExecBackend) settleClaimInstanceEnv(rt registry.Route) error, errInstanceEnvNeedsWrite - the test seam instanceKeySync (file and directory sync), and syncDir
  • [ ] Step 1: Write the failing tests

Create internal/activator/instancekey_test.go:

package activator

import (
	"bytes"
	"encoding/base64"
	"errors"
	"log"
	"os"
	"path/filepath"
	"strings"
	"sync"
	"testing"

	"github.com/carlosframework/platform/internal/registry"
)

// keyLines returns every CARLOS_INSTANCE_KEY value in raw, in order.
func keyLines(raw []byte) []string {
	var out []string
	for _, line := range strings.Split(string(raw), "\n") {
		if v, ok := strings.CutPrefix(line, InstanceKeyVar+"="); ok {
			out = append(out, v)
		}
	}
	return out
}

// syncProbe counts the key's syncs and fails them on demand. Set the
// error fields only between calls.
type syncProbe struct {
	file, dir       int
	fileErr, dirErr error
}

func probeInstanceKeySync(t *testing.T) *syncProbe {
	t.Helper()
	p := &syncProbe{}
	adminTokenMu.Lock()
	old := instanceKeySync
	instanceKeySync.file = func(f *os.File) error {
		p.file++
		if p.fileErr != nil {
			return p.fileErr
		}
		return f.Sync()
	}
	instanceKeySync.dir = func(d string) error {
		p.dir++
		if p.dirErr != nil {
			return p.dirErr
		}
		return syncDir(d)
	}
	adminTokenMu.Unlock()
	t.Cleanup(func() {
		adminTokenMu.Lock()
		instanceKeySync = old
		adminTokenMu.Unlock()
	})
	return p
}

func TestEnsureInstanceKeyMintsOnceInTheAgreedFormat(t *testing.T) {
	p := filepath.Join(t.TempDir(), "shop.example.com.env")
	got, err := EnsureInstanceKey(p)
	if err != nil || got != InstanceKeyMinted {
		t.Fatalf("first ensure = %v, %v; want minted", got, err)
	}
	raw, _ := os.ReadFile(p)
	keys := keyLines(raw)
	if len(keys) != 1 {
		t.Fatalf("file holds %d keys, want 1", len(keys))
	}
	dec, err := base64.StdEncoding.Strict().DecodeString(keys[0])
	if err != nil || len(dec) != 32 || len(keys[0]) != 44 {
		t.Fatalf("key is not 32 bytes in padded standard base64 (len %d, decode err %v)", len(keys[0]), err)
	}
	if !bytes.HasSuffix(raw, []byte("\n")) {
		t.Fatal("the key line has no newline: the next append would join it")
	}
	if st, _ := os.Stat(p); st.Mode().Perm() != 0o600 {
		t.Fatalf("mode = %o, want 600", st.Mode().Perm())
	}
	// The parser every launch path uses yields the exact characters minted.
	env, _ := readEnvFile(p)
	if env[InstanceKeyVar] != keys[0] {
		t.Fatal("readEnvFile does not yield the minted key verbatim")
	}
	again, err := EnsureInstanceKey(p)
	if err != nil || again != InstanceKeyPresent {
		t.Fatalf("second ensure = %v, %v; want present", again, err)
	}
	raw2, _ := os.ReadFile(p)
	if !bytes.Equal(raw, raw2) {
		t.Fatal("second ensure changed the file")
	}
}

// Never overwrite: a key already there, of any shape, is kept byte for byte
// and only reported.
func TestEnsureInstanceKeyNeverRewritesAnExistingKey(t *testing.T) {
	for _, existing := range []string{
		"CARLOS_INSTANCE_KEY=hand-written-not-base64\n",
		"A=1\nCARLOS_INSTANCE_KEY=" + strings.Repeat("A", 43) + "=", // well formed, no trailing newline
	} {
		p := filepath.Join(t.TempDir(), "shop.example.com.env")
		if err := os.WriteFile(p, []byte(existing), 0o600); err != nil {
			t.Fatal(err)
		}
		var buf bytes.Buffer
		log.SetOutput(&buf)
		got, err := EnsureInstanceKey(p)
		log.SetOutput(os.Stderr)
		if err != nil || got != InstanceKeyPresent {
			t.Fatalf("ensure over %q = %v, %v; want present", existing, got, err)
		}
		if raw, _ := os.ReadFile(p); string(raw) != existing {
			t.Fatalf("file changed: %q -> %q", existing, raw)
		}
		if strings.Contains(buf.String(), "hand-written-not-base64") {
			t.Fatal("the shape warning logged the key's value")
		}
	}
}

func TestEnsureInstanceKeyAppendsOnAFreshLineAndKeepsEveryOtherLine(t *testing.T) {
	p := filepath.Join(t.TempDir(), "shop.example.com.env")
	before := "CARLOS_ADMIN_TOKEN=" + strings.Repeat("a", 64) + "\n# comment\nDATABASE_URL=sqlite" // no trailing newline
	if err := os.WriteFile(p, []byte(before), 0o644); err != nil {
		t.Fatal(err)
	}
	if got, err := EnsureInstanceKey(p); err != nil || got != InstanceKeyMinted {
		t.Fatalf("ensure = %v, %v", got, err)
	}
	raw, _ := os.ReadFile(p)
	if !strings.HasPrefix(string(raw), before+"\n"+InstanceKeyVar+"=") {
		t.Fatalf("file = %q: the key must land on its own line after the untouched original", raw)
	}
	env, _ := readEnvFile(p)
	if env["DATABASE_URL"] != "sqlite" || len(env["CARLOS_ADMIN_TOKEN"]) != 64 {
		t.Fatal("an existing line was lost")
	}
	if st, _ := os.Stat(p); st.Mode().Perm() != 0o600 {
		t.Fatalf("mode = %o, want the 0644 file tightened to 600", st.Mode().Perm())
	}
}

// A key the app's configuration supplies is the app's key: minting one into
// <host>.env would override it (mergeEnv) and change the key under its data.
// The decision is the effective env's, so an empty CARLOS_INSTANCE_KEY= in
// the own file does not hide the configured key (finding 4).
func TestEnsureInstanceKeyDefersToTheRenderedConfiguration(t *testing.T) {
	for _, own := range []string{"A=1\n", "A=1\n" + InstanceKeyVar + "=\n"} {
		dir := t.TempDir()
		p := filepath.Join(dir, "shop.example.com.env")
		if err := os.WriteFile(filepath.Join(dir, "shop.example.com.config.env"), []byte(InstanceKeyVar+"=from-config\n"), 0o600); err != nil {
			t.Fatal(err)
		}
		if err := os.WriteFile(p, []byte(own), 0o600); err != nil {
			t.Fatal(err)
		}
		got, err := EnsureInstanceKey(p)
		if err != nil || got != InstanceKeyFromConfig {
			t.Fatalf("own %q: ensure = %v, %v; want from-config", own, got, err)
		}
		if raw, _ := os.ReadFile(p); string(raw) != own {
			t.Fatalf("own file changed: %q", raw)
		}
	}
}

func TestBackfillInstanceKeyAppendsOnlyToAnExistingEnvFile(t *testing.T) {
	b := &ExecBackend{DataDir: t.TempDir()}
	fresh := registry.Route{Host: "new.example.com", Socket: "/run/carlos/new.sock"}
	got, err := b.BackfillInstanceKey(fresh)
	if err != nil || got != InstanceKeyNoEnvFile {
		t.Fatalf("backfill with no env file = %v, %v; want no-env-file", got, err)
	}
	if _, err := os.Lstat(filepath.Join(b.DataDir, fresh.Host+".env")); !os.IsNotExist(err) {
		t.Fatalf("backfill created an env file: %v", err)
	}
	old := registry.Route{Host: "shop.example.com", Socket: "/run/carlos/shop.sock"}
	envPath := filepath.Join(b.DataDir, old.Host+".env")
	tok := "CARLOS_ADMIN_TOKEN=" + strings.Repeat("b", 64) + "\n"
	if err := os.WriteFile(envPath, []byte(tok), 0o600); err != nil {
		t.Fatal(err)
	}
	if got, err := b.BackfillInstanceKey(old); err != nil || got != InstanceKeyMinted {
		t.Fatalf("backfill = %v, %v; want minted", got, err)
	}
	raw, _ := os.ReadFile(envPath)
	if !strings.HasPrefix(string(raw), tok) || len(keyLines(raw)) != 1 {
		t.Fatalf("file = %q", raw)
	}
	// Idempotent: a second backfill (the next agent start) writes nothing.
	if got, err := b.BackfillInstanceKey(old); err != nil || got != InstanceKeyPresent {
		t.Fatalf("second backfill = %v, %v; want present", got, err)
	}
	if raw2, _ := os.ReadFile(envPath); !bytes.Equal(raw, raw2) {
		t.Fatal("second backfill changed the file")
	}
}

// Review focus 1: a hand-edited file with two keys. Readers take the last
// (parseEnvBytes), and the ensure writes nothing.
func TestEnsureInstanceKeyLeavesAFileWithTwoKeysAlone(t *testing.T) {
	p := filepath.Join(t.TempDir(), "shop.example.com.env")
	body := InstanceKeyVar + "=first\n" + InstanceKeyVar + "=second\n"
	if err := os.WriteFile(p, []byte(body), 0o600); err != nil {
		t.Fatal(err)
	}
	if got, err := EnsureInstanceKey(p); err != nil || got != InstanceKeyPresent {
		t.Fatalf("ensure = %v, %v; want present", got, err)
	}
	if raw, _ := os.ReadFile(p); string(raw) != body {
		t.Fatalf("file changed: %q", raw)
	}
	if env, _ := readEnvFile(p); env[InstanceKeyVar] != "second" {
		t.Fatalf("readers see %q, want the last line's value", env[InstanceKeyVar])
	}
}

// Review focus 2: an empty value is no key. One is appended after it, and
// readers take the appended one.
func TestEnsureInstanceKeyTreatsAnEmptyValueAsAbsent(t *testing.T) {
	p := filepath.Join(t.TempDir(), "shop.example.com.env")
	if err := os.WriteFile(p, []byte(InstanceKeyVar+"=\n"), 0o600); err != nil {
		t.Fatal(err)
	}
	if got, err := EnsureInstanceKey(p); err != nil || got != InstanceKeyMinted {
		t.Fatalf("ensure = %v, %v; want minted", got, err)
	}
	env, _ := readEnvFile(p)
	if !wellFormedInstanceKey(env[InstanceKeyVar]) {
		t.Fatal("readers do not see the appended key")
	}
}

// Review focus 3: a seed, a visitor's wake and the backfill can ask at
// once. Exactly one key is written.
func TestEnsureInstanceKeyMintsOneKeyUnderConcurrentCallers(t *testing.T) {
	p := filepath.Join(t.TempDir(), "shop.example.com.env")
	var wg sync.WaitGroup
	for i := 0; i < 8; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			if _, err := EnsureInstanceKey(p); err != nil {
				t.Error(err)
			}
		}()
	}
	wg.Wait()
	if raw, _ := os.ReadFile(p); len(keyLines(raw)) != 1 {
		t.Fatalf("%d keys written", len(keyLines(raw)))
	}
}

// Review focus 5: an empty CARLOS_INSTANCE_KEY in the rendered
// configuration supplies nothing, so the platform mints one.
func TestEnsureInstanceKeyMintsWhenTheConfigurationValueIsEmpty(t *testing.T) {
	dir := t.TempDir()
	if err := os.WriteFile(filepath.Join(dir, "shop.example.com.config.env"), []byte(InstanceKeyVar+"=\n"), 0o600); err != nil {
		t.Fatal(err)
	}
	if got, err := EnsureInstanceKey(filepath.Join(dir, "shop.example.com.env")); err != nil || got != InstanceKeyMinted {
		t.Fatalf("ensure = %v, %v; want minted", got, err)
	}
}

// Finding 2: a mint whose file or directory sync fails reports failure,
// and so does every retry until a sync succeeds. The retry finds the key
// present, keeps its bytes, and still completes the chmod and both syncs
// before it reports success.
func TestEnsureInstanceKeyRedoesAFailedSyncBeforeReportingSuccess(t *testing.T) {
	for _, tc := range []struct {
		name string
		fail func(*syncProbe, error)
	}{
		{"file sync", func(p *syncProbe, err error) { p.fileErr = err }},
		{"directory sync", func(p *syncProbe, err error) { p.dirErr = err }},
	} {
		t.Run(tc.name, func(t *testing.T) {
			probe := probeInstanceKeySync(t)
			p := filepath.Join(t.TempDir(), "shop.example.com.env")
			tc.fail(probe, errors.New("injected: EIO"))
			if _, err := EnsureInstanceKey(p); err == nil {
				t.Fatal("a mint whose sync failed reported success")
			}
			raw, _ := os.ReadFile(p)
			if len(keyLines(raw)) != 1 {
				t.Fatalf("the failed mint left %d keys, want its 1", len(keyLines(raw)))
			}
			if _, err := EnsureInstanceKey(p); err == nil {
				t.Fatal("a retry reported success while the sync still fails")
			}
			if err := os.Chmod(p, 0o644); err != nil {
				t.Fatal(err)
			}
			tc.fail(probe, nil)
			before := *probe
			got, err := EnsureInstanceKey(p)
			if err != nil || got != InstanceKeyPresent {
				t.Fatalf("retry = %v, %v; want present", got, err)
			}
			if probe.file != before.file+1 || probe.dir != before.dir+1 {
				t.Fatalf("the retry synced the file %d and the directory %d times, want once each", probe.file-before.file, probe.dir-before.dir)
			}
			if raw2, _ := os.ReadFile(p); !bytes.Equal(raw, raw2) {
				t.Fatal("a retry rewrote the key")
			}
			if st, _ := os.Stat(p); st.Mode().Perm() != 0o600 {
				t.Fatalf("mode = %o, want the present path to set 600", st.Mode().Perm())
			}
		})
	}
}

// Ruling 26: a replica row is never minted a key, on any path; the only
// key it may hold is its home's (replicaKeyGate).
func TestReplicaRowsGetNoPlatformKey(t *testing.T) {
	b := &ExecBackend{DataDir: t.TempDir()}
	rt := registry.Route{Host: "shop-au.example.com", Kind: "instance", Socket: "/run/carlos/shop-au.sock", Replica: true}
	envPath := filepath.Join(b.DataDir, rt.Host+".env")
	if err := os.WriteFile(envPath, []byte("CARLOS_ADMIN_TOKEN=t\n"), 0o600); err != nil {
		t.Fatal(err)
	}
	if got, err := b.ensureInstanceKeyFor(rt, b.pathsFor(rt)); err != nil || got != InstanceKeyReplica {
		t.Fatalf("launch ensure = %v, %v; want replica", got, err)
	}
	if got, err := b.BackfillInstanceKey(rt); err != nil || got != InstanceKeyReplica {
		t.Fatalf("backfill = %v, %v; want replica", got, err)
	}
	if raw, _ := os.ReadFile(envPath); len(keyLines(raw)) != 0 {
		t.Fatal("a replica row was given a platform key")
	}
}
  • [ ] Step 2: Run them to verify they fail

Run: nice -n 19 go test ./internal/activator/ -run 'TestEnsureInstanceKey|TestBackfillInstanceKey|TestReplicaRows' -count=1 Expected: build failure, undefined: EnsureInstanceKey (and InstanceKeyVar, instanceKeySync, ...).

  • [ ] Step 3: Write the implementation

Create internal/activator/instancekey.go:

package activator

// The instance key (region HA request L21; rastrillo/rastrillo #66). An app
// on CARLOS seals its data with a key. Before this file the app minted that
// key itself after it started, so a box that died before the key reached a
// backup left a database nothing could unseal. The platform now mints the
// key on the box, into the instance's own <host>.env as
// CARLOS_INSTANCE_KEY, before the app's first start, and never rewrites it.
// rastrillo's carlos.InstanceKey reads it verbatim when the app has no key
// of its own.
//
// The rules, each a ruling in
// docs/superpowers/plans/2026-10-10-instance-key-provision.md:
//   - create-only: a key already in the file is never rewritten, whatever
//     its shape. A malformed one is reported and left in place, as
//     EnsureAdminToken does with a malformed token;
//   - the decision is made on the effective env (instancekey.Effective,
//     the rule mergeEnv applies): a key the rendered configuration
//     supplies is the app's key and nothing is minted; an empty own value
//     is no key;
//   - durable every time: before an ensure reports success the file is
//     0600 (fchmod on its descriptor), the file is fsynced and so is its
//     directory, when the key was already there too. A sync that failed
//     once is redone by the next ensure, never skipped because the bytes
//     are already there. The known limit: once fsync has returned EIO,
//     Linux may mark the pages clean, so a later fsync's success does not
//     prove the bytes reached the disk. The retry narrows the gap; it
//     cannot close it, and no create-only design can;
//   - a mint that fails part way through its append is undone: the file
//     is truncated back to its length before the append and fsynced, so a
//     short key line never survives to be read as present. Those bytes
//     were never reported, so removing them rewrites no key;
//   - the value is never logged, never put in an error, and never leaves
//     the file except in the launch environment;
//   - a flat row's own file is opened O_NOFOLLOW on every path, read and
//     append alike (openOwnEnvNoFollow): a symlink there is refused, never
//     followed into a file a same-uid sibling chose. A nested row's file
//     gets the same from instancefs.

import (
	"crypto/rand"
	"encoding/base64"
	"errors"
	"fmt"
	"io"
	"os"
	"path/filepath"
	"strings"

	"github.com/carlosframework/platform/internal/instancefs"
	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/registry"
)

// InstanceKeyVar is the variable the key is delivered as.
const InstanceKeyVar = instancekey.Var

// instanceKeyBytes is the key's length before encoding: 32 CSPRNG bytes,
// standard base64 with padding, 44 characters. rastrillo uses the exact
// bytes the env file yields, so this encoding is fixed for the life of
// every key minted with it; changing it is a rotation.
const instanceKeyBytes = 32

// InstanceKeyOutcome is what one ensure did.
type InstanceKeyOutcome int

const (
	// InstanceKeyPresent: the own env file already carried a key. No byte
	// was written; the file was made durable.
	InstanceKeyPresent InstanceKeyOutcome = iota
	// InstanceKeyMinted: a key was appended and made durable.
	InstanceKeyMinted
	// InstanceKeyFromConfig: the rendered configuration supplies the key.
	// Nothing was written.
	InstanceKeyFromConfig
	// InstanceKeyNoEnvFile: the backfill found no own env file, so the host
	// has never been started here. Nothing was written; its first start
	// mints the key.
	InstanceKeyNoEnvFile
	// InstanceKeyReplica: a replica row (#162). A replica is never minted
	// a key: it may hold only its home's (ruling 26, replicaKeyGate).
	InstanceKeyReplica
)

func (o InstanceKeyOutcome) String() string {
	switch o {
	case InstanceKeyPresent:
		return "present"
	case InstanceKeyMinted:
		return "minted"
	case InstanceKeyFromConfig:
		return "from-config"
	case InstanceKeyNoEnvFile:
		return "no-env-file"
	case InstanceKeyReplica:
		return "replica"
	}
	return fmt.Sprintf("outcome(%d)", int(o))
}

// instanceKeySync is the durability step's two syncs, a seam so a test can
// fail each one. Read and replaced only under adminTokenMu.
var instanceKeySync = struct {
	file func(*os.File) error
	dir  func(string) error
}{file: (*os.File).Sync, dir: syncDir}

// syncDir fsyncs a directory, so an entry created in it survives a power
// cut.
func syncDir(dir string) error {
	d, err := os.Open(dir)
	if err != nil {
		return err
	}
	err = d.Sync()
	if cerr := d.Close(); err == nil {
		err = cerr
	}
	return err
}

// instanceKeyWrite is the mint's append, a seam so a test can make it fail
// part way (a short write, ENOSPC). Read and replaced only under
// adminTokenMu.
var instanceKeyWrite = func(f *os.File, b []byte) (int, error) { return f.Write(b) }

// instanceKeyBeforeOpen, when set, runs under adminTokenMu after the read
// and the decision and before the ensure opens the file to append or
// settle: a test hook to change the file between the two, so the open's
// own symlink refusal is pinned and not only the read's. nil outside
// tests.
var instanceKeyBeforeOpen func(path string)

func mintInstanceKey() (string, error) {
	raw := make([]byte, instanceKeyBytes)
	if _, err := rand.Read(raw); err != nil {
		return "", err
	}
	return base64.StdEncoding.EncodeToString(raw), nil
}

// wellFormedInstanceKey reports whether key has the shape this file mints.
// A key that fails it is not necessarily broken (an operator may have
// written one by hand), which is why a failure is a warning, never a
// rotation.
func wellFormedInstanceKey(key string) bool {
	if len(key) != base64.StdEncoding.EncodedLen(instanceKeyBytes) {
		return false
	}
	raw, err := base64.StdEncoding.Strict().DecodeString(key)
	return err == nil && len(raw) == instanceKeyBytes
}

// configEnvName is the rendered configuration's file name beside an own env
// file: <host>.env's sibling <host>.config.env, the pair pathsFor names.
func configEnvName(envName string) string {
	return strings.TrimSuffix(envName, ".env") + ".config.env"
}

// EnsureInstanceKey makes sure path (an instance's own <host>.env on the
// flat layout) carries CARLOS_INSTANCE_KEY, creating the file if needed.
// Launch paths call it right after EnsureAdminToken.
func EnsureInstanceKey(path string) (InstanceKeyOutcome, error) {
	return ensureInstanceKey(filepath.Dir(path), filepath.Base(path), false, true)
}

// EnsureInstanceKeyBeneath is EnsureInstanceKey for a nested row's own env
// file, read and written through instancefs because the directory is
// tenant-writable (isolated-exec design §3.2).
func EnsureInstanceKeyBeneath(dir, name string) (InstanceKeyOutcome, error) {
	return ensureInstanceKey(dir, name, true, true)
}

// ensureInstanceKeyFor is the launch paths' ensure for rt: none for a
// replica row, the nested body for a nested row, the flat one otherwise.
//
// A fenced or conflicting row is refused and nothing is written, as
// BackfillInstanceKey refuses it. Every launch dispatcher (Start,
// StartGen, StartStaged, Seed) refuses those rows already, but on its own
// classification: the launch body classifies again in pathsFor, and p is
// that second answer. A layout journal written between the two (a
// migration starting mid-wake) makes p fenced, with the flat directory as
// its dataDir, and minting there would write to a half-migrated host
// (isolated-exec design §8.2). The refusal is decided on p, the paths the
// mint would use. Every caller names the host around the error ("instance
// key for <host>: ..."), so the refusal does not.
func (b *ExecBackend) ensureInstanceKeyFor(rt registry.Route, p instancePaths) (InstanceKeyOutcome, error) {
	if rt.Replica {
		return InstanceKeyReplica, nil
	}
	switch p.layout {
	case LayoutFenced, LayoutConflict:
		return InstanceKeyPresent, fmt.Errorf("not minted: %s", p.reason)
	}
	if p.layout == LayoutNested {
		return EnsureInstanceKeyBeneath(p.dataDir, rt.Host+".env")
	}
	return EnsureInstanceKey(p.envFile)
}

// BackfillInstanceKey is the startup backfill's step for one row: a key is
// appended only to an own env file that already exists. A host with no file
// has never been started on this box, and its first start mints the key;
// creating a file here could also occupy the path a move's install expects
// to be empty (moveslandinstall.go placeStaged). A row this backend would
// refuse to start (fenced, conflict) is an error and nothing is written.
// The agent's backfill names the host when it logs the error, so the
// refusal does not.
func (b *ExecBackend) BackfillInstanceKey(rt registry.Route) (InstanceKeyOutcome, error) {
	if rt.Replica {
		return InstanceKeyReplica, nil
	}
	p := b.pathsFor(rt)
	switch p.layout {
	case LayoutNested:
		return ensureInstanceKey(p.dataDir, rt.Host+".env", true, false)
	case LayoutFenced, LayoutConflict:
		return InstanceKeyPresent, fmt.Errorf("not backfilled: %s", p.reason)
	}
	return ensureInstanceKey(p.dataDir, rt.Host+".env", false, false)
}

// openInstanceEnv opens an own env file for append. Only a create passes
// a mode: openat2 refuses a non-zero mode without O_CREAT (EINVAL), and an
// existing file's mode is set by the fchmod that follows anyway.
func openInstanceEnv(dir, name string, beneath, create bool) (*os.File, error) {
	flags, perm := os.O_APPEND|os.O_WRONLY, os.FileMode(0)
	if create {
		flags, perm = flags|os.O_CREATE, 0o600
	}
	if beneath {
		return instancefs.OpenBeneath(dir, name, flags, perm)
	}
	return openOwnEnvNoFollow(filepath.Join(dir, name), flags, perm)
}

// errOwnEnvSymlink is a flat row's own env file found to be a symlink. The
// ensure refuses it on every path (mint, existing key, backfill, the claim
// settle): following it would append to, chmod and fsync a file a same-uid
// sibling chose (spec §7.2).
var errOwnEnvSymlink = errors.New("activator: the instance env file is a symlink, refused")

// readOwnEnv reads a flat row's own env file through openOwnEnvNoFollow,
// so the read refuses a link exactly as the append does.
func readOwnEnv(path string) ([]byte, error) {
	f, err := openOwnEnvNoFollow(path, os.O_RDONLY, 0)
	if err != nil {
		return nil, err
	}
	defer f.Close()
	return io.ReadAll(f)
}

// settleInstanceEnv is the durability step every successful ensure ends
// with: 0600 on the descriptor, then the file, then its directory. Caller
// holds adminTokenMu.
func settleInstanceEnv(f *os.File, dir, path string) error {
	if err := f.Chmod(0o600); err != nil {
		return fmt.Errorf("chmod %s: %w", path, err)
	}
	if err := instanceKeySync.file(f); err != nil {
		return fmt.Errorf("sync %s: %w", path, err)
	}
	if err := instanceKeySync.dir(dir); err != nil {
		return fmt.Errorf("sync %s: %w", dir, err)
	}
	return nil
}

// ensureInstanceKey is the shared body. It holds adminTokenMu, the lock
// every mint into <host>.env takes, so a key append and a token append can
// never interleave. create false is the backfill: an absent file is
// InstanceKeyNoEnvFile and nothing is created.
func ensureInstanceKey(dir, name string, beneath, create bool) (InstanceKeyOutcome, error) {
	path := filepath.Join(dir, name)
	adminTokenMu.Lock()
	defer adminTokenMu.Unlock()
	read := func(n string) ([]byte, error) {
		if beneath {
			return instancefs.ReadBeneathLimit(dir, n, 1<<20)
		}
		return os.ReadFile(filepath.Join(dir, n))
	}
	var raw []byte
	var err error
	if beneath {
		raw, err = read(name)
	} else {
		raw, err = readOwnEnv(path)
	}
	switch {
	case errors.Is(err, os.ErrNotExist):
		if !create {
			return InstanceKeyNoEnvFile, nil
		}
	case err != nil:
		return InstanceKeyPresent, fmt.Errorf("read %s: %w", path, err)
	}
	cfg, err := read(configEnvName(name))
	if err != nil && !errors.Is(err, os.ErrNotExist) {
		return InstanceKeyPresent, fmt.Errorf("read %s: %w", filepath.Join(dir, configEnvName(name)), err)
	}
	// The decision is the effective env's: the key the launch will hand
	// the app, by the same rule mergeEnv applies.
	key, layer := instancekey.Effective(parseEnvBytes(cfg), parseEnvBytes(raw))
	switch layer {
	case instancekey.LayerConfig:
		logPerFile("key-config", path, fmt.Sprintf("activator: %s is set by the app's configuration for %s: the platform mints none while it is", InstanceKeyVar, path))
		return InstanceKeyFromConfig, nil
	case instancekey.LayerOwn:
		if !wellFormedInstanceKey(key) {
			logPerFile("key-shape", path, fmt.Sprintf("activator: %s in %s is not the shape the platform mints (44 base64 characters, 32 bytes): left in place, because the app may already have sealed data with it", InstanceKeyVar, path))
		}
		// Present is not a no-op: a mint whose sync failed left these
		// bytes behind, and this call is the retry that makes them
		// durable before anything is started with them.
		if instanceKeyBeforeOpen != nil {
			instanceKeyBeforeOpen(path)
		}
		f, err := openInstanceEnv(dir, name, beneath, false)
		if err != nil {
			return InstanceKeyPresent, fmt.Errorf("open %s: %w", path, err)
		}
		defer f.Close()
		if err := settleInstanceEnv(f, dir, path); err != nil {
			return InstanceKeyPresent, err
		}
		return InstanceKeyPresent, nil
	}
	minted, err := mintInstanceKey()
	if err != nil {
		return InstanceKeyPresent, fmt.Errorf("mint instance key for %s: %w", path, err)
	}
	if instanceKeyBeforeOpen != nil {
		instanceKeyBeforeOpen(path)
	}
	f, err := openInstanceEnv(dir, name, beneath, create)
	if errors.Is(err, os.ErrNotExist) && !create {
		return InstanceKeyNoEnvFile, nil // removed since the read: nothing to backfill
	}
	if err != nil {
		return InstanceKeyPresent, fmt.Errorf("open %s: %w", path, err)
	}
	defer f.Close()
	// The length the append starts from, taken from the open descriptor:
	// what a failed append is undone to. Under adminTokenMu it is
	// len(raw); reading it here also keeps any byte another process
	// appended since the read.
	st, err := f.Stat()
	if err != nil {
		return InstanceKeyPresent, fmt.Errorf("stat %s: %w", path, err)
	}
	prefix := ""
	if len(raw) > 0 && raw[len(raw)-1] != '\n' {
		prefix = "\n"
	}
	line := []byte(prefix + InstanceKeyVar + "=" + minted + "\n")
	if n, err := instanceKeyWrite(f, line); err != nil || n != len(line) {
		if err == nil {
			err = io.ErrShortWrite
		}
		return InstanceKeyPresent, undoInstanceKeyAppend(f, st.Size(), path, err)
	}
	if err := settleInstanceEnv(f, dir, path); err != nil {
		return InstanceKeyPresent, err
	}
	return InstanceKeyMinted, nil
}

// undoInstanceKeyAppend removes a failed append's bytes: it truncates f
// back to size, the length before the append, and fsyncs, so a short key
// line can never survive to be read as present. Nothing past size was
// ever reported to anyone. Returns the write error, joined with any
// failure to undo it. Caller holds adminTokenMu.
func undoInstanceKeyAppend(f *os.File, size int64, path string, werr error) error {
	err := fmt.Errorf("write %s: %w", path, werr)
	if terr := f.Truncate(size); terr != nil {
		return errors.Join(err, fmt.Errorf("undo the partial write to %s: %w", path, terr))
	}
	if serr := instanceKeySync.file(f); serr != nil {
		return errors.Join(err, fmt.Errorf("sync %s after undoing the partial write: %w", path, serr))
	}
	return err
}

// errInstanceEnvNeedsWrite is settleClaimInstanceEnv's answer for a file
// whose mode is not 0600: making it so is a write, which the claim path
// does only under its guard.
var errInstanceEnvNeedsWrite = errors.New("activator: the instance env file needs its mode set")

// settleClaimInstanceEnv is the claim launch source's durability step for
// an env file that already carries what the launch needs: it makes the
// file and its directory durable and writes nothing. fsync changes no
// byte, mode or name, so it is not a host write and runs without the
// claim's guard. A file that is not 0600 is errInstanceEnvNeedsWrite, and
// the caller takes the guarded path, whose ensure sets the mode when the
// own file holds the key (and leaves it when the key is configured or the
// row is a replica: EnsureClaimAdminToken).
func (b *ExecBackend) settleClaimInstanceEnv(rt registry.Route) error {
	p := b.pathsFor(rt)
	name := rt.Host + ".env"
	adminTokenMu.Lock()
	defer adminTokenMu.Unlock()
	var f *os.File
	var err error
	if p.layout == LayoutNested {
		f, err = instancefs.OpenBeneath(p.dataDir, name, os.O_RDONLY, 0)
	} else {
		f, err = openOwnEnvNoFollow(p.envFile, os.O_RDONLY, 0)
	}
	if err != nil {
		return fmt.Errorf("open %s: %w", p.envFile, err)
	}
	defer f.Close()
	st, err := f.Stat()
	if err != nil {
		return fmt.Errorf("stat %s: %w", p.envFile, err)
	}
	if st.Mode().Perm() != 0o600 {
		return errInstanceEnvNeedsWrite
	}
	if err := instanceKeySync.file(f); err != nil {
		return fmt.Errorf("sync %s: %w", p.envFile, err)
	}
	if err := instanceKeySync.dir(p.dataDir); err != nil {
		return fmt.Errorf("sync %s: %w", p.dataDir, err)
	}
	return nil
}

Create internal/activator/ownenv_unix.go (the open that refuses a link in the open itself) and ownenv_other.go (the windows stand-in, so the cross-compiles build; the agent never runs there):

//go:build unix

package activator

import (
	"errors"
	"os"

	"golang.org/x/sys/unix"
)

// openOwnEnvNoFollow opens a flat row's own <host>.env and refuses a
// symlink at it in the open itself (O_NOFOLLOW), so there is no window
// between a check and the open for a same-uid sibling (spec §7.2) to plant
// a link in. A refused link is the bare errOwnEnvSymlink: the caller names
// the path, once. A nested row's file needs none of this: instancefs.OpenBeneath already refuses links.
func openOwnEnvNoFollow(path string, flag int, perm os.FileMode) (*os.File, error) {
	f, err := os.OpenFile(path, flag|unix.O_NOFOLLOW, perm)
	if errors.Is(err, unix.ELOOP) {
		return nil, errOwnEnvSymlink
	}
	return f, err
}
//go:build !unix

package activator

import (
	"os"
)

// openOwnEnvNoFollow is the unix helper's stand-in where there is no
// O_NOFOLLOW (windows). The agent only runs on Linux; this keeps the
// cross-compiled builds working and still refuses a link it can see, by
// the same Lstat check EnsureAdminToken makes. It is check-then-open, so
// it is weaker than the unix open, which is the one that matters.
func openOwnEnvNoFollow(path string, flag int, perm os.FileMode) (*os.File, error) {
	if fi, err := os.Lstat(path); err == nil && fi.Mode()&os.ModeSymlink != 0 {
		return nil, errOwnEnvSymlink
	}
	return os.OpenFile(path, flag, perm)
}
  • [ ] Step 4: Run the tests to verify they pass

Run: nice -n 19 go test ./internal/activator/ -run 'TestEnsureInstanceKey|TestBackfillInstanceKey|TestReplicaRows|TestEnsureAdminToken' -count=1 Expected: PASS. The admin token tests are in the run because the key mint shares their lock and log throttle.

  • [ ] Step 5: Commit
gofmt -l internal/activator
git add internal/activator/instancekey.go internal/activator/instancekey_test.go
git commit -m "activator: mint CARLOS_INSTANCE_KEY create-only into <host>.env, durable on every call (L21)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"

---

Task 3: Every activator launch path mints the key

Files:

  • Modify: internal/activator/backend_exec.go:831-833 (seedChild), :1369-1371 (spawnChildFrom)
  • Modify: internal/activator/prepare.go:60-62 (prepareAt)
  • Test: internal/activator/instancekey_test.go (append), internal/activator/instancekey_linux_test.go (create)

Interfaces:

  • Consumes: Task 2's ensureInstanceKeyFor, BackfillInstanceKey, probeInstanceKeySync, wellFormedInstanceKey; test helpers testExecBackend (activatortest.go:4262), stageRecordingLiveFullEnv (:4712), waitForFile (:4741), tmpSock (:954), writeMinimalSQLite (backendexectest.go:343), the restoreReplica seam (backendexec.go:757), nestedBackend, fakeControl, fakeUnit (unitproc_test.go:24-160).
  • Produces: nothing new; the launch env of every exec start and seed carries CARLOS_INSTANCE_KEY, and no child starts before it is durable.
  • [ ] Step 1: Write the failing tests

Append to internal/activator/instancekey_test.go, and add "context" and "time" to its imports:

// startRecorded starts host on b with the recording binary and returns the
// environment the child saw, or nil when no child ran within a second.
func startRecorded(t *testing.T, b *ExecBackend, host string) (map[string]string, error) {
	t.Helper()
	_, envPath := stageRecordingLiveFullEnv(t, b, host, "v1")
	rt := registry.Route{Host: host, Kind: "instance", Socket: tmpSock(t)}
	ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
	defer cancel()
	proc, _, err := b.Start(ctx, rt) // the fake binary never binds: the health wait fails, as in the sibling tests
	if proc != nil {
		defer proc.Stop(context.Background())
	}
	deadline := time.Now().Add(time.Second)
	for time.Now().Before(deadline) {
		if raw, rerr := os.ReadFile(envPath); rerr == nil {
			child := map[string]string{}
			for _, line := range strings.Split(string(raw), "\n") {
				if k, v, ok := strings.Cut(line, "="); ok {
					child[k] = v
				}
			}
			return child, err
		}
		time.Sleep(10 * time.Millisecond)
	}
	return nil, err
}

// Start hands the child the key it minted, and a second Start (a restart, a
// redeploy) hands it the same one.
func TestExecBackendStartHandsTheChildTheMintedInstanceKey(t *testing.T) {
	b := testExecBackend(t)
	host := "shop.example.com"
	first, _ := startRecorded(t, b, host)
	minted, err := readEnvFile(filepath.Join(b.DataDir, host+".env"))
	if err != nil {
		t.Fatal(err)
	}
	if first == nil || !wellFormedInstanceKey(minted[InstanceKeyVar]) || first[InstanceKeyVar] != minted[InstanceKeyVar] {
		t.Fatal("the child was not handed the key minted into <host>.env")
	}
	if second, _ := startRecorded(t, b, host); second == nil || second[InstanceKeyVar] != minted[InstanceKeyVar] {
		t.Fatal("a second start handed the child a different key")
	}
}

// Finding 2 through a launch: while the key's sync fails, Start fails and
// no child runs. Once the sync succeeds, the child is handed the key the
// failed attempt wrote, never a second one.
func TestExecBackendStartStartsNoChildUntilTheKeyIsSynced(t *testing.T) {
	for _, tc := range []struct {
		name string
		fail func(*syncProbe, error)
	}{
		{"file sync", func(p *syncProbe, err error) { p.fileErr = err }},
		{"directory sync", func(p *syncProbe, err error) { p.dirErr = err }},
	} {
		t.Run(tc.name, func(t *testing.T) {
			probe := probeInstanceKeySync(t)
			b := testExecBackend(t)
			host := "shop.example.com"
			tc.fail(probe, errors.New("injected: EIO"))
			for i := 0; i < 2; i++ {
				child, err := startRecorded(t, b, host)
				if err == nil || !strings.Contains(err.Error(), "instance key for "+host) {
					t.Fatalf("start %d with a failing sync: err = %v, want the instance key's error", i, err)
				}
				if child != nil {
					t.Fatalf("start %d ran a child before the key was durable", i)
				}
			}
			written, _ := readEnvFile(filepath.Join(b.DataDir, host+".env"))
			tc.fail(probe, nil)
			child, _ := startRecorded(t, b, host)
			if child == nil || child[InstanceKeyVar] == "" || child[InstanceKeyVar] != written[InstanceKeyVar] {
				t.Fatal("after the sync succeeded the child was not handed the key the failed attempt wrote")
			}
		})
	}
}

// Finding 4 through a launch: <host>.env and <host>.config.env together.
// An empty override never hands the child an empty key while the
// configuration supplies one, and no key is minted over the configured one.
// With neither supplying one, the child gets the minted key.
func TestExecBackendStartNeverHandsTheChildAnEmptyKeyWhileOneExists(t *testing.T) {
	tok := "CARLOS_ADMIN_TOKEN=" + strings.Repeat("c", 64) + "\n"
	for _, tc := range []struct {
		name, own, config string
		configured        bool
	}{
		{"empty override, configured key", tok + InstanceKeyVar + "=\n", InstanceKeyVar + "=from-config\n", true},
		{"no override, configured key", tok, InstanceKeyVar + "=from-config\n", true},
		{"empty override, empty configuration", tok + InstanceKeyVar + "=\n", InstanceKeyVar + "=\n", false},
	} {
		t.Run(tc.name, func(t *testing.T) {
			b := testExecBackend(t)
			host := "shop.example.com"
			own := filepath.Join(b.DataDir, host+".env")
			if err := os.WriteFile(own, []byte(tc.own), 0o600); err != nil {
				t.Fatal(err)
			}
			if err := os.WriteFile(filepath.Join(b.DataDir, host+".config.env"), []byte(tc.config), 0o600); err != nil {
				t.Fatal(err)
			}
			child, _ := startRecorded(t, b, host)
			if child == nil {
				t.Fatal("no child ran")
			}
			raw, _ := os.ReadFile(own)
			if tc.configured {
				if child[InstanceKeyVar] != "from-config" {
					t.Fatalf("the child was handed %q, want the configured key", child[InstanceKeyVar])
				}
				if string(raw) != tc.own {
					t.Fatal("a key was minted over the configured one")
				}
				return
			}
			env, _ := readEnvFile(own)
			if !wellFormedInstanceKey(child[InstanceKeyVar]) || child[InstanceKeyVar] != env[InstanceKeyVar] {
				t.Fatal("the child was not handed the minted key")
			}
		})
	}
}

// Coverage (Astra round 1): the first process of a fresh flat instance is
// often its seed. The seed mints the key, sees it, and the first start
// hands the instance the same one.
func TestExecBackendSeedMintsTheKeyForAFreshFlatInstance(t *testing.T) {
	b := testExecBackend(t)
	host := "seeded.example.com"
	_, envPath := stageRecordingLiveFullEnv(t, b, host, "v1")
	if err := b.Seed(context.Background(), registry.Route{Host: host, Kind: "instance", Seed: `["seed"]`}); err != nil {
		t.Fatalf("Seed: %v", err)
	}
	seed := map[string]string{}
	for _, line := range strings.Split(waitForFile(t, envPath, time.Second), "\n") {
		if k, v, ok := strings.Cut(line, "="); ok {
			seed[k] = v
		}
	}
	own, _ := readEnvFile(filepath.Join(b.DataDir, host+".env"))
	if !wellFormedInstanceKey(own[InstanceKeyVar]) || seed[InstanceKeyVar] != own[InstanceKeyVar] {
		t.Fatal("the seed did not mint the key and see it")
	}
	if child, _ := startRecorded(t, b, host); child == nil || child[InstanceKeyVar] != own[InstanceKeyVar] {
		t.Fatal("the first start handed the instance a different key from its seed's")
	}
}

// Coverage (Astra round 1): a hibernation wake restores the database and
// leaves <host>.env alone, so the start after the restore hands the
// instance the key it had.
func TestHibernationRestoreKeepsTheInstanceKey(t *testing.T) {
	b := testExecBackend(t)
	host := "sleepy.example.com"
	if child, _ := startRecorded(t, b, host); child == nil {
		t.Fatal("no child ran")
	}
	envPath := filepath.Join(b.DataDir, host+".env")
	before, _ := os.ReadFile(envPath)
	key := parseEnvBytes(before)[InstanceKeyVar]
	if !wellFormedInstanceKey(key) {
		t.Fatal("the first start minted no key")
	}
	old := restoreReplica
	restoreReplica = func(_ context.Context, dbPath, bucket, prefix, region string) (bool, error) {
		return true, writeMinimalSQLite(dbPath)
	}
	t.Cleanup(func() { restoreReplica = old })
	_ = os.Remove(filepath.Join(b.DataDir, host+".db")) // hibernated: the database is gone
	rt := registry.Route{Host: host, Kind: "instance", Socket: tmpSock(t)}
	if fresh, err := b.Restore(context.Background(), rt); err != nil || fresh {
		t.Fatalf("Restore = fresh %v, %v; want a restored database", fresh, err)
	}
	if after, _ := os.ReadFile(envPath); !bytes.Equal(before, after) {
		t.Fatal("the restore changed <host>.env")
	}
	if child, _ := startRecorded(t, b, host); child == nil || child[InstanceKeyVar] != key {
		t.Fatal("the start after a restore handed the instance a different key")
	}
}

Create internal/activator/instancekey_linux_test.go:

package activator

import (
	"os"
	"path/filepath"
	"strings"
	"testing"
	"time"

	"golang.org/x/sys/unix"

	"github.com/carlosframework/platform/internal/instancectl"
	"github.com/carlosframework/platform/internal/provision"
)

// The nested prepare mints the key into the instance directory's own env
// file and hands it to the role in the same env it returns: the run role
// and the seed role (coverage, Astra round 1).
func TestPrepareMintsTheInstanceKeyIntoTheNestedEnvFile(t *testing.T) {
	for _, role := range []instancectl.Role{instancectl.RoleRun, instancectl.RoleSeed} {
		t.Run(string(role), func(t *testing.T) {
			b, rt := nestedBackend(t, &fakeControl{inv: "inv1"}, &fakeUnit{})
			_, ctl, logs, env, err := b.prepare(rt, role)
			if err != nil {
				t.Fatalf("prepare: %v", err)
			}
			logs.Close()
			if ctl != nil {
				ctl.Close()
			}
			own, err := readEnvFileBeneath(provision.InstanceDir(b.DataDir, rt.Host), rt.Host+".env")
			if err != nil {
				t.Fatal(err)
			}
			if !wellFormedInstanceKey(own[InstanceKeyVar]) || env[InstanceKeyVar] != own[InstanceKeyVar] {
				t.Fatal("prepare's env does not carry the key minted into the nested env file")
			}
		})
	}
}

// Finding 3: the nested backfill opens an existing file with mode 0
// (openat2 refuses a mode without O_CREAT), appends, and sets 0600 on the
// descriptor.
func TestBackfillInstanceKeyAppendsToANestedEnvFile(t *testing.T) {
	b, rt := nestedBackend(t, &fakeControl{inv: "inv1"}, &fakeUnit{})
	dir := provision.InstanceDir(b.DataDir, rt.Host)
	if err := os.MkdirAll(dir, 0o700); err != nil {
		t.Fatal(err)
	}
	envPath := filepath.Join(dir, rt.Host+".env")
	tok := "CARLOS_ADMIN_TOKEN=" + strings.Repeat("e", 64) + "\n"
	if err := os.WriteFile(envPath, []byte(tok), 0o644); err != nil {
		t.Fatal(err)
	}
	if got, err := b.BackfillInstanceKey(rt); err != nil || got != InstanceKeyMinted {
		t.Fatalf("nested backfill = %v, %v; want minted", got, err)
	}
	raw, _ := os.ReadFile(envPath)
	if !strings.HasPrefix(string(raw), tok) || len(keyLines(raw)) != 1 || !wellFormedInstanceKey(keyLines(raw)[0]) {
		t.Fatalf("nested env file after the backfill: %d bytes, %d keys", len(raw), len(keyLines(raw)))
	}
	if st, _ := os.Stat(envPath); st.Mode().Perm() != 0o600 {
		t.Fatalf("mode = %o, want 600", st.Mode().Perm())
	}
	if got, err := b.BackfillInstanceKey(rt); err != nil || got != InstanceKeyPresent {
		t.Fatalf("second nested backfill = %v, %v; want present", got, err)
	}
	if raw2, _ := os.ReadFile(envPath); string(raw2) != string(raw) {
		t.Fatal("the second backfill changed the file")
	}
}

// A nested own env file is on tenant-writable ground: a symlink planted at
// its name is refused, and the file it points at is not touched.
func TestEnsureInstanceKeyBeneathRefusesASymlink(t *testing.T) {
	dir := t.TempDir()
	target := filepath.Join(t.TempDir(), "elsewhere.env")
	if err := os.WriteFile(target, []byte("X=1\n"), 0o644); err != nil {
		t.Fatal(err)
	}
	if err := os.Symlink(target, filepath.Join(dir, "shop.example.com.env")); err != nil {
		t.Fatal(err)
	}
	if _, err := EnsureInstanceKeyBeneath(dir, "shop.example.com.env"); err == nil {
		t.Fatal("ensure followed a symlink planted at the env file's name")
	}
	if raw, _ := os.ReadFile(target); string(raw) != "X=1\n" {
		t.Fatalf("the symlink's target changed: %q", raw)
	}
}

// Review focus 4: a FIFO planted at a nested env file's name fails the
// backfill fast; it never blocks agent start.
func TestBackfillInstanceKeyRefusesAFIFOWithoutBlocking(t *testing.T) {
	b, rt := nestedBackend(t, &fakeControl{inv: "inv1"}, &fakeUnit{})
	dir := provision.InstanceDir(b.DataDir, rt.Host)
	if err := os.MkdirAll(dir, 0o700); err != nil {
		t.Fatal(err)
	}
	if err := unix.Mkfifo(filepath.Join(dir, rt.Host+".env"), 0o600); err != nil {
		t.Fatal(err)
	}
	done := make(chan error, 1)
	go func() {
		_, err := b.BackfillInstanceKey(rt)
		done <- err
	}()
	select {
	case err := <-done:
		if err == nil {
			t.Fatal("backfill over a FIFO succeeded, want an error")
		}
	case <-time.After(2 * time.Second):
		t.Fatal("backfill blocked on a FIFO")
	}
}
  • [ ] Step 2: Run them to verify they fail

Run (sandbox off: these start children and bind sockets): nice -n 19 go test ./internal/activator/ -run 'TestExecBackendStartHandsTheChild|TestExecBackendStartStartsNoChild|TestExecBackendStartNeverHands|TestExecBackendSeedMintsTheKey|TestHibernationRestoreKeeps|TestPrepareMintsTheInstanceKey|TestBackfillInstanceKeyAppendsToANested|TestEnsureInstanceKeyBeneathRefuses|TestBackfillInstanceKeyRefusesAFIFO' -count=1 -v Expected (checked in the scratch build): FAIL for TestExecBackendStartHandsTheChildTheMintedInstanceKey, both TestExecBackendStartStartsNoChildUntilTheKeyIsSynced cases, the "empty override, empty configuration" case of TestExecBackendStartNeverHandsTheChildAnEmptyKeyWhileOneExists (its two configured cases already pass, from Task 1's mergeEnv), TestExecBackendSeedMintsTheKeyForAFreshFlatInstance, TestHibernationRestoreKeepsTheInstanceKey and both roles of TestPrepareMintsTheInstanceKeyIntoTheNestedEnvFile. The nested backfill, symlink and FIFO tests already pass, from Task 2's body; they pin the nested path.

  • [ ] Step 3: Call the mint on each launch path

In internal/activator/backend_exec.go, seedChild, after the admin token mint:

	if err := EnsureAdminToken(paths.envFile); err != nil {
		return fmt.Errorf("admin token for %s: %w", host, err)
	}
	// The instance key, before the env is read for the same reason: a seed
	// is usually the first process of a fresh instance (L21).
	if _, err := b.ensureInstanceKeyFor(rt, paths); err != nil {
		return fmt.Errorf("instance key for %s: %w", host, err)
	}
	fileEnv, err := b.instanceEnv(rt)

In spawnChildFrom, after the admin token mint:

	if err := EnsureAdminToken(paths.envFile); err != nil {
		return nil, "", fmt.Errorf("admin token for %s: %w", host, err)
	}
	// The instance key is minted here for the same reason and fails the
	// wake the same way: an app that takes its key from the platform
	// refuses to start without one (L21). It returns only once the key is
	// durable, so no child starts with a key a power cut could take back.
	if _, err := b.ensureInstanceKeyFor(rt, paths); err != nil {
		return nil, "", fmt.Errorf("instance key for %s: %w", host, err)
	}

In internal/activator/prepare.go, prepareAt, after EnsureAdminTokenBeneath:

		if err := EnsureAdminTokenBeneath(p.dataDir, rt.Host+".env"); err != nil {
			return p, nil, nil, nil, fmt.Errorf("admin token for %s: %w", rt.Host, err)
		}
		if _, err := b.ensureInstanceKeyFor(rt, p); err != nil {
			return p, nil, nil, nil, fmt.Errorf("instance key for %s: %w", rt.Host, err)
		}
  • [ ] Step 4: Run the tests, then the package's launch tests

Run (sandbox off): the Step 2 command. Expected: PASS.

Run (sandbox off): nice -n 19 go test ./internal/activator/ -run 'Seed|Start|Prepare|LaunchEnvironment|Control|InstanceKey|AdminToken' -count=1 Expected: PASS. If an existing test compares a child's whole environment or an env file's exact bytes after a start or seed, it now sees one more line, CARLOS_INSTANCE_KEY=.... Update that assertion to allow the key line (assert it is well formed). Do not weaken any other check, and list each test you changed in the commit message. (None needed changing in the scratch build.)

  • [ ] Step 5: Commit
gofmt -l internal/activator
git add internal/activator/backend_exec.go internal/activator/prepare.go internal/activator/instancekey_test.go internal/activator/instancekey_linux_test.go
git commit -m "activator: seed, start and nested prepare mint the instance key durably before the env is read (L21)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"

---

Task 3b: A replica gets exactly its home's key, or none and a refusal (ruling 26)

Files:

  • Create: internal/activator/replicakey.go, internal/host/replicahomekeys.go, internal/appconfig/instancekey.go
  • Modify: internal/activator/backend_exec.go (ExecBackend.ReplicaHomeKeys and ConfiguredKey; the gate in seedChild, spawnChildFrom, StartWorkerStaged and startSidecarChildFrom), internal/activator/prepare.go (prepareAt), internal/host/agent.go (the wiring)
  • Test: internal/activator/replica_instancekey_test.go, internal/activator/replica_instancekey_linux_test.go, internal/activator/replicakey_race_test.go, internal/host/replicahomekeys_test.go, internal/appconfig/instancekey_test.go

Interfaces:

  • Consumes: Task 1's instancekey.Digest, Effective and ParseHomeKeys; appconfig's LoadMergedHostVersions, RenderEnv, routeBinding, and test kit execKit, SetEnv, ApplyInstance, Seal; Task 2's InstanceKeyReplica (never minted); test helpers testExecBackend, tmpSock, nestedBackend, readEnvFile.
  • Produces: ExecBackend.ReplicaHomeKeys func(host string) (sha256 string, ok bool) and ConfiguredKey func(ctx, rt) (sha256 string, err error); (*ExecBackend).replicaKeyGate(rt, env, layer, haveControl) error, with layer the one the env's key came from, decided by the same read that composed the env (instanceEnvLayer, ruling 29); activator.ErrReplicaKey; appconfig.ConfiguredKeyDigest(ctx, s, priv, rt) (string, error); host.replicaHomeKeys(read), which parses the list the reader returns (/etc/carlos/replica-home-keys, read again at every replica launch; ruling 30), and host.configuredKey(getenv, open), the fresh read (ruling 27). Task 4's snapshot calls the gate.

Built: two launch paths rev 5.1 left ungated pass the gate as a seed does (no control socket: the home's verified key or none, refused otherwise), by the ledger's rulings on the 3b build and its review. The sidecar worker (StartWorkerStaged) reads the env and its key's layer in one read, and when the agent-issued extra carries CARLOS_INSTANCE_KEY the layer is set to none, so the gate refuses it as a key of unknown origin: a worker never mints and extra has no business carrying a key. The legacy co-spawned sidecar (startSidecarChildFrom) gates its own env file's key the same way (sidecarKeyLayer). Refusal and log text name the file path, not the variable the plan calls the list by. The bucket read in the gate runs on its own bounded context (prepareAt has none to thread).

  • [ ] Step 1: Write the failing tests

The app these tests launch is the test binary itself, running TestInstanceKeyHelperProcess: rastrillo e3ea08de's resolution and its documented caller. So each case records what an adopting app actually did ("key <sha256>", "refused" or "minted"), not only which variables it was handed. "No control socket" is a socket path too long to bind, so the launch withholds CARLOS_CONTROL_SOCKET, exactly as a claim launch before O6 does.

Create internal/activator/replica_instancekey_test.go:

package activator

import (
	"context"
	"errors"
	"fmt"
	"os"
	"path/filepath"
	"strings"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/registry"
)

// The app the replica tests launch is this test binary, running
// TestInstanceKeyHelperProcess: rastrillo e3ea08de's carlos.InstanceKey
// and its documented caller, so a test sees what an adopting app does, not
// only which variables it was handed.
const helperOutEnv = "GO_TEST_INSTANCE_KEY_HELPER_OUT"

var (
	errHelperNoInstanceKey = errors.New("ErrNoInstanceKey")
	errHelperNotOnCarlos   = errors.New("ErrNotOnCarlos")
)

// helperInstanceKey is rastrillo e3ea08de's InstanceKey(own) with Running():
// own, then $CARLOS_INSTANCE_KEY verbatim, then ErrNoInstanceKey when
// $CARLOS_CONTROL_SOCKET is set, else ErrNotOnCarlos.
func helperInstanceKey(own string) (string, error) {
	if own != "" {
		return own, nil
	}
	if k := os.Getenv(InstanceKeyVar); k != "" {
		return k, nil
	}
	if os.Getenv("CARLOS_CONTROL_SOCKET") != "" {
		return "", errHelperNoInstanceKey
	}
	return "", errHelperNotOnCarlos
}

// TestInstanceKeyHelperProcess is the app, not a test: with no output path
// set it does nothing. It resolves its key as an adopting app with no key
// of its own does, records the outcome, and exits:
//   - "key <sha256>": it sealed with the key it was handed;
//   - "refused": ErrNoInstanceKey, it did not start;
//   - "minted": ErrNotOnCarlos, and the documented caller minted its own.
func TestInstanceKeyHelperProcess(t *testing.T) {
	out := os.Getenv(helperOutEnv)
	if out == "" {
		return
	}
	var outcome string
	switch key, err := helperInstanceKey(""); {
	case err == nil:
		outcome = "key " + instancekey.Digest(key)
	case errors.Is(err, errHelperNoInstanceKey):
		outcome = "refused"
	default:
		outcome = "minted"
	}
	_ = os.WriteFile(out+".tmp", []byte(outcome), 0o600)
	_ = os.Rename(out+".tmp", out)
	os.Exit(0)
}

// stageHelperApp makes this test binary host's live build, running only
// TestInstanceKeyHelperProcess, and returns where its outcome lands.
func stageHelperApp(t *testing.T, b *ExecBackend, host, ver string) string {
	t.Helper()
	self, err := os.Executable()
	if err != nil {
		t.Fatal(err)
	}
	out := filepath.Join(t.TempDir(), "outcome")
	t.Setenv(helperOutEnv, out)
	vdir := filepath.Join(b.OptDir, "versions", ver)
	if err := os.MkdirAll(vdir, 0o755); err != nil {
		t.Fatal(err)
	}
	bin := filepath.Join(vdir, "app")
	script := fmt.Sprintf("#!/bin/sh\nexec '%s' -test.run='^TestInstanceKeyHelperProcess$'\n", self)
	if err := os.WriteFile(bin, []byte(script), 0o755); err != nil {
		t.Fatal(err)
	}
	if err := os.MkdirAll(filepath.Join(b.OptDir, "live"), 0o755); err != nil {
		t.Fatal(err)
	}
	os.Remove(b.liveLink(host))
	if err := os.Symlink(bin, b.liveLink(host)); err != nil {
		t.Fatal(err)
	}
	return out
}

// outcomeOf waits up to two seconds for the helper's outcome; "" is no app
// ran at all.
func outcomeOf(path string) string {
	deadline := time.Now().Add(2 * time.Second)
	for time.Now().Before(deadline) {
		if raw, err := os.ReadFile(path); err == nil {
			return string(raw)
		}
		time.Sleep(10 * time.Millisecond)
	}
	return ""
}

// noControlSocket is a socket path too long to bind, so the launch's
// control listener fails and CARLOS_CONTROL_SOCKET is withheld: the same
// launch a claim box runs before O6, and a flat launch whose bind failed.
func noControlSocket(t *testing.T) string {
	return filepath.Join(t.TempDir(), strings.Repeat("s", 120)+".sock")
}

type replicaCase struct {
	name             string
	replica          bool
	own, config      string // env file bodies ("" = no config file)
	homeKey          string // the home's effective key the box lists ("" = none listed)
	start            string // "refused" (no app runs) or the helper outcome wanted
	wantKeyFromOwnOK bool   // a home row: the outcome is the key minted into <host>.env
	// bucket is the key the replica's configuration in the bucket renders
	// now (ConfiguredKey; "" renders none). bucketDown makes that read
	// fail; noBucket leaves ConfiguredKey nil.
	bucket               string
	bucketDown, noBucket bool
}

// freshConfig is ExecBackend.ConfiguredKey for one case.
func freshConfig(tc replicaCase) func(context.Context, registry.Route) (string, error) {
	if tc.noBucket {
		return nil
	}
	return func(context.Context, registry.Route) (string, error) {
		switch {
		case tc.bucketDown:
			return "", errors.New("bucket unreachable")
		case tc.bucket == "":
			return "", nil
		}
		return instancekey.Digest(tc.bucket), nil
	}
}

// Rulings 26 to 28 through a real launch and the helper (Astra rounds 4 and
// 5): a replica runs only with exactly its home's key, listed by the
// pre-roll check and, when configured, still the bucket's configured key;
// or with none and a control socket its helper refuses on. Every launch
// here has no control socket, which is where the helper would otherwise
// mint.
func TestReplicaLaunchWithoutAControlSocket(t *testing.T) {
	tok := "CARLOS_ADMIN_TOKEN=" + strings.Repeat("c", 64) + "\n"
	homeKey := strings.Repeat("H", 43) + "="
	for _, tc := range []replicaCase{
		{name: "a keyless replica is refused by the agent, not left to mint", replica: true, own: tok, start: "refused"},
		{name: "a configured key that is the home's runs and seals with it", replica: true, own: tok, config: InstanceKeyVar + "=" + homeKey + "\n", homeKey: homeKey, bucket: homeKey, start: "key " + instancekey.Digest(homeKey)},
		{name: "a pre-existing env key that is the home's runs", replica: true, own: tok + InstanceKeyVar + "=" + homeKey + "\n", homeKey: homeKey, start: "key " + instancekey.Digest(homeKey)},
		{name: "a configured key that differs is refused", replica: true, own: tok, config: InstanceKeyVar + "=configured-B\n", homeKey: homeKey, bucket: "configured-B", start: "refused"},
		{name: "a configured key with no verified home key is refused", replica: true, own: tok, config: InstanceKeyVar + "=" + homeKey + "\n", bucket: homeKey, start: "refused"},
		// Ruling 27, Astra round 5 finding 1: the key changed in the bucket
		// to B and the home already holds it, while this box's sync is
		// behind and the replica's configuration and list entry still say A.
		{name: "the home updated, this replica's configuration delayed: refused", replica: true, own: tok, config: InstanceKeyVar + "=" + homeKey + "\n", homeKey: homeKey, bucket: "configured-B", start: "refused"},
		{name: "the bucket cannot be read: refused", replica: true, own: tok, config: InstanceKeyVar + "=" + homeKey + "\n", homeKey: homeKey, bucketDown: true, start: "refused"},
		{name: "no fresh reader: refused", replica: true, own: tok, config: InstanceKeyVar + "=" + homeKey + "\n", homeKey: homeKey, noBucket: true, start: "refused"},
		// Ruling 27's residual, documented: a change the bucket does not
		// hold (the home's own <host>.env edited by hand, or a per-host key
		// configured on the home only) is invisible here. The replica's
		// list entry and the bucket both still say A, so it runs with A.
		// Only the procedure, or O8's fresh evidence, closes this.
		{name: "residual: a change only on the home is not seen, and the replica runs with the old key", replica: true, own: tok, config: InstanceKeyVar + "=" + homeKey + "\n", homeKey: homeKey, bucket: homeKey, start: "key " + instancekey.Digest(homeKey)},
		// Ruling 28: a home whose key exists only in configuration, with a
		// replica that is handed none (its own overlay blanks it). O8 never
		// delivers a configured key, so the replica stays keyless and is
		// refused here, where it has no control socket.
		{name: "a config-only home's keyless replica is refused", replica: true, own: tok, config: InstanceKeyVar + "=\n", homeKey: homeKey, bucket: "", start: "refused"},
		{name: "a home row mints and its app seals with the minted key", own: tok, wantKeyFromOwnOK: true},
	} {
		t.Run(tc.name, func(t *testing.T) {
			b := testExecBackend(t)
			rt := registry.Route{Host: "shop-au.example.com", Kind: "instance", Replica: tc.replica, Socket: noControlSocket(t)}
			own := filepath.Join(b.DataDir, rt.Host+".env")
			if err := os.WriteFile(own, []byte(tc.own), 0o600); err != nil {
				t.Fatal(err)
			}
			if tc.config != "" {
				if err := os.WriteFile(filepath.Join(b.DataDir, rt.Host+".config.env"), []byte(tc.config), 0o600); err != nil {
					t.Fatal(err)
				}
			}
			if tc.homeKey != "" {
				b.ReplicaHomeKeys = func(h string) (string, bool) {
					return instancekey.Digest(tc.homeKey), h == rt.Host
				}
			}
			b.ConfiguredKey = freshConfig(tc)
			out := stageHelperApp(t, b, rt.Host, "v1")
			ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
			defer cancel()
			proc, _, err := b.Start(ctx, rt)
			if proc != nil {
				defer proc.Stop(context.Background())
			}
			got := outcomeOf(out)
			if tc.wantKeyFromOwnOK {
				env, _ := readEnvFile(own)
				if got != "key "+instancekey.Digest(env[InstanceKeyVar]) {
					t.Fatalf("home row: outcome %q, want the minted key", got)
				}
				return
			}
			if tc.start == "refused" {
				if !errors.Is(err, ErrReplicaKey) || got != "" {
					t.Fatalf("Start = %v, app outcome %q; want ErrReplicaKey and no app", err, got)
				}
			} else if got != tc.start {
				t.Fatalf("app outcome %q, want %q (Start: %v)", got, tc.start, err)
			}
			if raw, _ := os.ReadFile(own); string(raw) != tc.own {
				t.Fatal("a replica launch changed its own env file: a replica never mints")
			}
		})
	}
}

// A replica's seed has no control socket: a keyless replica's seed is
// refused before it runs; one handed its home's key runs.
func TestReplicaSeedIsGated(t *testing.T) {
	tok := "CARLOS_ADMIN_TOKEN=" + strings.Repeat("c", 64) + "\n"
	homeKey := strings.Repeat("H", 43) + "="
	for _, tc := range []struct {
		name, config string
		want         string
	}{
		{"keyless", "", ""},
		{"the home's key", InstanceKeyVar + "=" + homeKey + "\n", "key " + instancekey.Digest(homeKey)},
	} {
		t.Run(tc.name, func(t *testing.T) {
			b := testExecBackend(t)
			rt := registry.Route{Host: "shop-au.example.com", Kind: "instance", Replica: true, Seed: `["seed"]`}
			if err := os.WriteFile(filepath.Join(b.DataDir, rt.Host+".env"), []byte(tok), 0o600); err != nil {
				t.Fatal(err)
			}
			if tc.config != "" {
				if err := os.WriteFile(filepath.Join(b.DataDir, rt.Host+".config.env"), []byte(tc.config), 0o600); err != nil {
					t.Fatal(err)
				}
			}
			b.ReplicaHomeKeys = func(h string) (string, bool) { return instancekey.Digest(homeKey), true }
			b.ConfiguredKey = freshConfig(replicaCase{bucket: homeKey})
			out := stageHelperApp(t, b, rt.Host, "v1")
			err := b.Seed(context.Background(), rt)
			got := outcomeOf(out)
			if tc.want == "" {
				if !errors.Is(err, ErrReplicaKey) || got != "" {
					t.Fatalf("Seed = %v, outcome %q; want ErrReplicaKey and no seed process", err, got)
				}
				return
			}
			if err != nil || got != tc.want {
				t.Fatalf("Seed = %v, outcome %q; want %q", err, got, tc.want)
			}
		})
	}
}

Create internal/activator/replica_instancekey_linux_test.go:

package activator

import (
	"context"
	"errors"
	"net"
	"os"
	"path/filepath"
	"strings"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/instancectl"
	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/provision"
	"github.com/carlosframework/platform/internal/registry"
)

// socketsAllowed reports whether this process may bind a unix socket (the
// sandbox refuses it).
func socketsAllowed(t *testing.T) bool {
	dir, err := os.MkdirTemp("", "ik") // short: a test's own temp dir can exceed a socket path's limit
	if err != nil {
		t.Fatal(err)
	}
	defer os.RemoveAll(dir)
	l, err := net.Listen("unix", filepath.Join(dir, "p.sock"))
	if err != nil {
		return false
	}
	l.Close()
	return true
}

// Ruling 26 with a control socket: a keyless replica runs, and the app's
// helper refuses at step 3 (ErrNoInstanceKey) instead of minting.
func TestKeylessReplicaWithAControlSocketLeavesTheHelperToRefuse(t *testing.T) {
	if !socketsAllowed(t) {
		t.Skip("needs unix sockets (run with the sandbox off)")
	}
	b := testExecBackend(t)
	rt := registry.Route{Host: "shop-au.example.com", Kind: "instance", Replica: true, Socket: tmpSock(t)}
	if err := os.WriteFile(filepath.Join(b.DataDir, rt.Host+".env"), []byte("CARLOS_ADMIN_TOKEN="+strings.Repeat("c", 64)+"\n"), 0o600); err != nil {
		t.Fatal(err)
	}
	out := stageHelperApp(t, b, rt.Host, "v1")
	ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
	defer cancel()
	proc, _, _ := b.Start(ctx, rt)
	if proc != nil {
		defer proc.Stop(context.Background())
	}
	if got := outcomeOf(out); got != "refused" {
		t.Fatalf("app outcome %q, want the helper's refusal", got)
	}
}

// Ruling 26 on the nested layout: the seed role has no control socket, so
// a keyless replica's seed is refused at prepare; a run role handed a key
// that is not the home's is refused too; one handed the home's key is
// prepared with it.
func TestNestedReplicaPrepareIsGated(t *testing.T) {
	homeKey := strings.Repeat("H", 43) + "="
	for _, tc := range []struct {
		name, config string
		role         instancectl.Role
		refused      bool
	}{
		{"a keyless seed", "", instancectl.RoleSeed, true},
		{"a run role with another key", InstanceKeyVar + "=configured-B\n", instancectl.RoleRun, true},
		{"a run role with the home's key", InstanceKeyVar + "=" + homeKey + "\n", instancectl.RoleRun, false},
	} {
		t.Run(tc.name, func(t *testing.T) {
			b, rt := nestedBackend(t, &fakeControl{inv: "inv1"}, &fakeUnit{})
			rt.Replica = true
			b.ReplicaHomeKeys = func(string) (string, bool) { return instancekey.Digest(homeKey), true }
			b.ConfiguredKey = func(context.Context, registry.Route) (string, error) { return instancekey.Digest(homeKey), nil }
			dir := provision.InstanceDir(b.DataDir, rt.Host)
			if err := os.MkdirAll(dir, 0o700); err != nil {
				t.Fatal(err)
			}
			if tc.config != "" {
				if err := os.WriteFile(filepath.Join(dir, rt.Host+".config.env"), []byte(tc.config), 0o600); err != nil {
					t.Fatal(err)
				}
			}
			_, ctl, logs, env, err := b.prepare(rt, tc.role)
			if tc.refused {
				if !errors.Is(err, ErrReplicaKey) {
					t.Fatalf("prepare = %v, want ErrReplicaKey", err)
				}
				return
			}
			if err != nil {
				t.Fatalf("prepare: %v", err)
			}
			logs.Close()
			if ctl != nil {
				ctl.Close()
			}
			if env[InstanceKeyVar] != homeKey {
				t.Fatal("the prepared env does not carry the home's key")
			}
		})
	}
}

Create internal/appconfig/instancekey_test.go:

package appconfig

import (
	"testing"

	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/registry"
)

// Ruling 27's fresh check: the digest of the key the bucket's
// configuration renders now, from the app's bundle or the host's overlay,
// plain or sealed; "" when it renders none.
func TestConfiguredKeyDigestReadsTheBucketNow(t *testing.T) {
	ctx, s, _, _, priv, pub := execKit(t)
	pk, _ := ParsePrivateKey(priv, "test")
	pubKey, _ := ParsePublicKey(pub, "test")
	rt := registry.Route{Host: "shop-au.example.com", Kind: "instance", App: "shop", Account: "acct", Replica: true}
	check := func(want string) {
		t.Helper()
		got, err := ConfiguredKeyDigest(ctx, s, pk, rt)
		if err != nil || got != want {
			t.Fatalf("ConfiguredKeyDigest = %q, %v; want %q", got, err, want)
		}
	}
	check("")
	if err := SetEnv(ctx, s, "acct", "shop", map[string]string{"CARLOS_INSTANCE_KEY": "key-A"}); err != nil {
		t.Fatal(err)
	}
	check(instancekey.Digest("key-A"))
	if err := SetEnv(ctx, s, "acct", "shop", map[string]string{"CARLOS_INSTANCE_KEY": "key-B"}); err != nil {
		t.Fatal(err)
	}
	check(instancekey.Digest("key-B"))
	sealed, err := Seal(pubKey, Binding{Account: "acct", App: "shop"}, []byte("key-C"))
	if err != nil {
		t.Fatal(err)
	}
	if _, _, err := ApplyInstance(ctx, s, "acct", "shop", rt.Host, map[string]Var{"CARLOS_INSTANCE_KEY": {Kind: "secret", Envelope: sealed}}, nil); err != nil {
		t.Fatal(err)
	}
	check(instancekey.Digest("key-C"))
}

Create internal/activator/replicakey_race_test.go, ruling 29's deterministic interleaving:

package activator

import (
	"context"
	"errors"
	"os"
	"path/filepath"
	"strings"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/registry"
)

// Ruling 29 (Astra round 6 finding 1), deterministic: the launch reads a
// configured key A, and then a config sync removes the key from this box's
// <host>.config.env (the bucket no longer holds it) before the replica
// gate runs. The gate decides on the layer the launch's own read found, so
// it still consults the bucket, which now renders no key, and refuses. A
// second read of the files would have found no key, skipped the bucket
// and admitted a child that is handed A.
func TestReplicaGateDecidesOnTheLaunchesOwnRead(t *testing.T) {
	tok := "CARLOS_ADMIN_TOKEN=" + strings.Repeat("c", 64) + "\n"
	keyA := strings.Repeat("A", 43) + "="
	b := testExecBackend(t)
	rt := registry.Route{Host: "shop-au.example.com", Kind: "instance", Replica: true, Socket: noControlSocket(t)}
	cfgPath := filepath.Join(b.DataDir, rt.Host+".config.env")
	if err := os.WriteFile(filepath.Join(b.DataDir, rt.Host+".env"), []byte(tok), 0o600); err != nil {
		t.Fatal(err)
	}
	if err := os.WriteFile(cfgPath, []byte(InstanceKeyVar+"="+keyA+"\n"), 0o600); err != nil {
		t.Fatal(err)
	}
	b.ReplicaHomeKeys = func(string) (string, bool) { return instancekey.Digest(keyA), true }
	b.ConfiguredKey = func(context.Context, registry.Route) (string, error) { return "", nil } // the bucket: key removed
	syncs := 0
	replicaGateHook = func() {
		syncs++
		if err := os.WriteFile(cfgPath, []byte("OTHER=1\n"), 0o600); err != nil {
			t.Error(err)
		}
	}
	t.Cleanup(func() { replicaGateHook = nil })
	out := stageHelperApp(t, b, rt.Host, "v1")
	ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
	defer cancel()
	proc, _, err := b.Start(ctx, rt)
	if proc != nil {
		defer proc.Stop(context.Background())
	}
	if syncs != 1 {
		t.Fatalf("the sync ran %d times between the read and the gate, want once", syncs)
	}
	if got := outcomeOf(out); !errors.Is(err, ErrReplicaKey) || got != "" {
		t.Fatalf("Start = %v, app outcome %q; want ErrReplicaKey and no app", err, got)
	}
}

// The gate reads no env file: given the env and layer the launch's read
// captured, it decides the same with the files gone.
func TestReplicaGateReadsNoEnvFile(t *testing.T) {
	keyA := strings.Repeat("A", 43) + "="
	b := &ExecBackend{DataDir: t.TempDir()}
	rt := registry.Route{Host: "shop-au.example.com", Kind: "instance", Replica: true}
	if err := os.WriteFile(filepath.Join(b.DataDir, rt.Host+".config.env"), []byte(InstanceKeyVar+"="+keyA+"\n"), 0o600); err != nil {
		t.Fatal(err)
	}
	env, layer, err := b.instanceEnvLayer(rt)
	if err != nil || layer != instancekey.LayerConfig || env[InstanceKeyVar] != keyA {
		t.Fatalf("instanceEnvLayer = %v, %v", layer, err)
	}
	if err := os.RemoveAll(b.DataDir); err != nil {
		t.Fatal(err)
	}
	b.ReplicaHomeKeys = func(string) (string, bool) { return instancekey.Digest(keyA), true }
	for _, tc := range []struct {
		bucket string
		ok     bool
	}{{keyA, true}, {"", false}} {
		b.ConfiguredKey = func(context.Context, registry.Route) (string, error) {
			if tc.bucket == "" {
				return "", nil
			}
			return instancekey.Digest(tc.bucket), nil
		}
		if err := b.replicaKeyGate(rt, env, layer, false); (err == nil) != tc.ok {
			t.Fatalf("bucket %q: gate = %v, want ok %v", tc.bucket, err, tc.ok)
		}
	}
	if err := b.replicaKeyGate(rt, env, instancekey.LayerNone, false); !errors.Is(err, ErrReplicaKey) {
		t.Fatalf("a key with no known layer = %v, want a refusal", err)
	}
}

Create internal/host/replicahomekeys_test.go:

package host

import (
	"bytes"
	"context"
	"errors"
	"log"
	"os"
	"path/filepath"
	"strings"
	"testing"

	"github.com/carlosframework/platform/internal/appconfig"
	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/registry"
	"github.com/carlosframework/platform/internal/release"
)

// Ruling 30 (Astra round 6 finding 2): the list is read again at every
// call, so the procedure's invalidation and its new entries take effect at
// the next replica launch, with no restart. An absent list verifies
// nothing.
func TestReplicaHomeKeysReadsTheListAtEveryLaunch(t *testing.T) {
	a, b := instancekey.Digest("home-key-a"), instancekey.Digest("home-key-b")
	path := filepath.Join(t.TempDir(), "replica-home-keys")
	keys := replicaHomeKeys(func() ([]byte, error) { return os.ReadFile(path) })
	if _, ok := keys("shop-au.example.com"); ok {
		t.Fatal("an absent list verified a key")
	}
	write := func(body string) {
		t.Helper()
		tmp := path + ".tmp"
		if err := os.WriteFile(tmp, []byte(body), 0o640); err != nil {
			t.Fatal(err)
		}
		if err := os.Rename(tmp, path); err != nil {
			t.Fatal(err)
		}
	}
	write("shop-au.example.com=" + a + "\nshop-uk.example.com=" + a + "\n")
	if got, ok := keys("shop-au.example.com"); !ok || got != a {
		t.Fatalf("listed replica = %q, %v", got, ok)
	}
	// The procedure's invalidation: the entry is gone at the next call.
	write("shop-uk.example.com=" + a + "\n")
	if _, ok := keys("shop-au.example.com"); ok {
		t.Fatal("a removed entry still verified: the list was cached")
	}
	// And its replacement, after the key change and the re-run check.
	write("shop-au.example.com=" + b + ",shop-uk.example.com=" + a)
	if got, ok := keys("shop-au.example.com"); !ok || got != b {
		t.Fatalf("the replaced entry = %q, %v; want the new digest", got, ok)
	}
	if _, ok := keys("shop-nz.example.com"); ok {
		t.Fatal("an unlisted replica verified")
	}
}

// Astra round 5 finding 3: a malformed list is logged by entry position and
// reason, never by its text (a bare digest, with no host before it, would
// otherwise reach the log), and verifies nothing.
func TestReplicaHomeKeysLogsAMalformedListWithoutItsText(t *testing.T) {
	sha := instancekey.Digest("home-key")
	var buf bytes.Buffer
	log.SetOutput(&buf)
	defer log.SetOutput(os.Stderr)
	keys := replicaHomeKeys(func() ([]byte, error) { return []byte("shop-au.example.com=" + sha + "\n" + sha + "\n"), nil })
	if _, ok := keys("shop-au.example.com"); ok {
		t.Fatal("a malformed list verified a key")
	}
	line := buf.String()
	if !strings.Contains(line, "CARLOS_REPLICA_HOME_KEYS entry 2") || strings.Contains(line, sha[:16]) || strings.Contains(line, "shop-au") {
		t.Fatalf("log line %q: want the entry's position and reason, and none of its text", line)
	}
	buf.Reset()
	if _, ok := replicaHomeKeys(func() ([]byte, error) { return nil, errors.New("permission denied") })("shop-au.example.com"); ok {
		t.Fatal("an unreadable list verified a key")
	}
	if !strings.Contains(buf.String(), "could not be read") {
		t.Fatalf("log line %q", buf.String())
	}
}

// Ruling 27's fresh read, as the agent wires it: the bucket's current
// configured key, through the deploy store and the box's secrets key. With
// no secrets key it is an error, so the replica is refused.
func TestReplicaConfigKeyReadsTheBucketNow(t *testing.T) {
	ctx := context.Background()
	s := release.NewDirStore(t.TempDir())
	priv, _, err := appconfig.GenerateKey()
	if err != nil {
		t.Fatal(err)
	}
	if err := appconfig.SetEnv(ctx, s, "acct", "shop", map[string]string{instancekey.Var: "key-A"}); err != nil {
		t.Fatal(err)
	}
	rt := registry.Route{Host: "shop-au.example.com", Kind: "instance", App: "shop", Account: "acct", Replica: true}
	opened := 0
	open := func(context.Context) (release.Store, error) { opened++; return s, nil }
	read := configuredKey(func(k string) string {
		if k == "CARLOS_SECRETS_KEY" {
			return priv
		}
		return ""
	}, open)
	for i := 0; i < 2; i++ {
		if got, err := read(ctx, rt); err != nil || got != instancekey.Digest("key-A") {
			t.Fatalf("read %d = %q, %v", i, got, err)
		}
	}
	if opened != 1 {
		t.Fatalf("the store was opened %d times, want once", opened)
	}
	failing := configuredKey(func(string) string { return priv }, func(context.Context) (release.Store, error) { return nil, errors.New("bucket unreachable") })
	if _, err := failing(ctx, rt); err == nil {
		t.Fatal("an unreachable bucket was not an error")
	}
	if _, err := configuredKey(func(string) string { return "" }, open)(ctx, rt); err == nil {
		t.Fatal("no secrets key was not an error")
	}
}
  • [ ] Step 2: Run them to verify they fail

Run: nice -n 19 go test ./internal/activator/ -run 'TestReplicaLaunchWithoutAControlSocket|TestReplicaSeedIsGated|TestReplicaGate|TestKeylessReplica|TestNestedReplicaPrepareIsGated' -count=1 -v; nice -n 19 go test ./internal/host/ -run 'TestReplicaHomeKeys|TestReplicaConfigKey' -count=1; nice -n 19 go test ./internal/appconfig/ -run TestConfiguredKeyDigest -count=1 Expected: build failures, undefined: ErrReplicaKey (and ReplicaHomeKeys, ConfiguredKey, replicaHomeKeys, configuredKey, ConfiguredKeyDigest). In the scratch build, with the gate present but disabled, the keyless and differing-key cases failed: the app reported "minted", or ran with the wrong key. That is Astra's round 4 failure. With only the fresh read disabled, "the home updated, this replica's configuration delayed" failed: the app ran with the old key. That is Astra's round 5 failure. With rev 5's second read of the files restored in the gate, TestReplicaGateDecidesOnTheLaunchesOwnRead failed: the app ran with A. That is Astra's round 6 failure.

  • [ ] Step 3: Write the gate

Create internal/activator/replicakey.go:

package activator

// A replica gets its home's key (instance-key plan rulings 26 to 28). A
// replica (#162) mirrors its home's data, sealed with the home's key, so
// the only key it may be handed is exactly the home's effective key. The
// platform never mints one on a replica row (ensureInstanceKeyFor), and
// every replica launch passes replicaKeyGate before anything runs:
//
//   - a replica whose env files deliver a key starts only when that key's
//     digest is its entry in the box's verified list (ReplicaHomeKeys, which
//     only the pre-roll check writes, read again at every launch) and, when
//     the key comes from the replica's configuration, the configuration in
//     the bucket renders that same key now (ConfiguredKey). Where the key
//     came from is decided by the same read that composed the env the child
//     gets (instanceEnvLayer, ruling 29), never by reading the files again. A key that differs from either, or
//     that cannot be checked, refuses the launch with a log line. The fresh
//     read catches a configuration change that this box has not synced yet
//     while the home already holds the new key (ruling 27);
//   - a replica whose env files deliver no key starts only with a control
//     socket. There an app on rastrillo's InstanceKey with no key of its
//     own gets ErrNoInstanceKey and refuses (step 3), and an app that never
//     asks for a key runs as before. Without a socket (a seed, a claim
//     launch before O6, a control bind that failed) the helper would fall
//     to step 4 and mint its own key over the home's data, so the agent
//     refuses the launch itself rather than leave that to the helper.
//
// Delivering the home's key to a replica that has none is O8's, for a
// platform-minted home key only (ruling 28): until then, and always for a
// home whose key exists only in configuration, a keyless replica gets
// nothing and is gated as above.

import (
	"context"
	"errors"
	"fmt"
	"log"
	"time"

	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/registry"
)

// ErrReplicaKey is a replica launch refused by replicaKeyGate.
var ErrReplicaKey = errors.New("activator: replica launch refused: its instance key is not its home's")

// configReadTimeout bounds replicaKeyGate's fresh read of the bucket.
const configReadTimeout = 10 * time.Second

// instanceEnvLayer is instanceEnv returning, from the same read, the layer
// its CARLOS_INSTANCE_KEY came from (ruling 29). mergeEnv delivers exactly
// the key instancekey.Effective names, so env[Var] and the layer always
// describe the same bytes, however the files change afterwards.
func (b *ExecBackend) instanceEnvLayer(rt registry.Route) (map[string]string, instancekey.Layer, error) {
	cfgEnv, ownEnv, err := b.instanceEnvParts(rt)
	if err != nil {
		return nil, instancekey.LayerNone, err
	}
	_, layer := instancekey.Effective(cfgEnv, ownEnv)
	return mergeEnv(cfgEnv, ownEnv), layer, nil
}

// replicaGateHook runs at the start of every replica gate, after the env
// was read. A test seam: it lets a test change the env files between the
// read and the gate, as a concurrent config sync can. Nil in production.
var replicaGateHook func()

// replicaKeyGate is rulings 26, 27 and 29's check for a launch of rt with
// env (the composed env files), layer (where env's key came from, decided
// by the same read) and, when haveControl, a bound control socket. It reads
// no env file. It is nil for every row that is not a replica. Its errors and
// log lines name the host and the reason, never a key or a digest.
func (b *ExecBackend) replicaKeyGate(rt registry.Route, env map[string]string, layer instancekey.Layer, haveControl bool) error {
	if !rt.Replica {
		return nil
	}
	if replicaGateHook != nil {
		replicaGateHook()
	}
	refuse := func(why string) error {
		log.Printf("activator: replica %s not started: %s", rt.Host, why)
		return fmt.Errorf("%w: %s: %s", ErrReplicaKey, rt.Host, why)
	}
	key := env[instancekey.Var]
	if key == "" {
		if haveControl {
			return nil
		}
		return refuse("it has no key of its home's and no control socket, so an app on the instance-key helper would mint its own")
	}
	var home string
	var ok bool
	if b.ReplicaHomeKeys != nil {
		home, ok = b.ReplicaHomeKeys(rt.Host)
	}
	switch {
	case !ok:
		return refuse(instancekey.Var + " cannot be checked against its home's key (no entry in CARLOS_REPLICA_HOME_KEYS; O8)")
	case instancekey.Digest(key) != home:
		return refuse(instancekey.Var + " is not its home's key")
	}
	switch layer {
	case instancekey.LayerOwn:
		return nil // the replica's own file: it changes only by hand
	case instancekey.LayerConfig:
	default:
		return refuse("where its " + instancekey.Var + " came from is unknown")
	}
	if b.ConfiguredKey == nil {
		return refuse("its configured " + instancekey.Var + " cannot be checked against the bucket's configuration")
	}
	ctx, cancel := context.WithTimeout(context.Background(), configReadTimeout)
	defer cancel()
	fresh, err := b.ConfiguredKey(ctx, rt)
	switch {
	case err != nil:
		return refuse("the bucket's configuration could not be read: " + err.Error())
	case fresh != instancekey.Digest(key):
		return refuse("its configured " + instancekey.Var + " is not the one the bucket's configuration now holds: a key change is in flight (ruling 27)")
	}
	return nil
}

In internal/activator/backend_exec.go, add the two fields to ExecBackend, after Region:

	// ReplicaHomeKeys answers, for a replica row's host, the SHA-256 of its
	// home's effective CARLOS_INSTANCE_KEY (instance-key plan rulings 26 and
	// 30). O8 will supply it from the home's published env item; until then
	// the agent reads the box's list (/etc/carlos/replica-home-keys), which
	// only the pre-roll check writes, again at every call. Nil, or false for
	// a host, means the home key cannot be verified, and a replica launch
	// that would be handed a key is refused (replicaKeyGate).
	ReplicaHomeKeys func(host string) (sha256 string, ok bool)
	// ConfiguredKey reads rt's configuration from the bucket now and
	// returns the SHA-256 of the CARLOS_INSTANCE_KEY it renders, "" for
	// none (appconfig.ConfiguredKeyDigest). replicaKeyGate refuses a
	// replica whose configured key is not that digest: a configuration
	// change is in flight (ruling 27). Nil refuses every configured key.
	ConfiguredKey func(ctx context.Context, rt registry.Route) (sha256 string, err error)

In seedChild, read the env and its key's layer together (instanceEnvLayer, in replicakey.go), then gate:

	fileEnv, keyLayer, err := b.instanceEnvLayer(rt)
	if err != nil {
		return err
	}
	// A seed has no control socket, so a replica's seed is gated as a
	// launch without one (ruling 26), on the layer this same read found.
	if err := b.replicaKeyGate(rt, fileEnv, keyLayer, false); err != nil {
		return err
	}
	dbPath := paths.dbPath

In spawnChildFrom, replace the instanceEnv read after the key mint with one read of the env and its key's layer:

	// One read gives the env and the layer its CARLOS_INSTANCE_KEY came
	// from, for the replica gate below (ruling 29).
	fileEnv, keyLayer, err := b.instanceEnvLayer(rt)
	if err != nil {
		return nil, "", err
	}

In spawnChildFrom, once the control listener is bound or has failed, before the child's env is composed:

	// A replica is started only with its home's key, verified, or with
	// none and a control socket its helper can refuse on (ruling 26).
	if err := b.replicaKeyGate(rt, fileEnv, keyLayer, ctl != nil); err != nil {
		if ctl != nil {
			ctl.Close()
		}
		return nil, "", err
	}
	childVars := b.agentOwnedVars(rt, controlSocket, fileEnv, ctl != nil)

In internal/activator/prepare.go, add the instancekey import; in prepareAt, declare the layer beside env and read them together (the lines between are unchanged):

	var env map[string]string
	var keyLayer instancekey.Layer // the layer env's key came from, read with it (ruling 29)
	if role == instancectl.RoleRun || role == instancectl.RoleSeed {
		// p.dataDir is the instance directory here, tenant-writable ground
		// (design §3.2), so the token file is minted and read through
		// instancefs rather than a plain path-based open.
		if err := EnsureAdminTokenBeneath(p.dataDir, rt.Host+".env"); err != nil {
			return p, nil, nil, nil, fmt.Errorf("admin token for %s: %w", rt.Host, err)
		}
		if _, err := b.ensureInstanceKeyFor(rt, p); err != nil {
			return p, nil, nil, nil, fmt.Errorf("instance key for %s: %w", rt.Host, err)
		}
		var err error
		env, keyLayer, err = b.instanceEnvLayer(rt)

In internal/activator/prepare.go, prepareAt, before the log listener:

	// A replica's run and seed roles are gated like a flat launch: the
	// seed has no control socket, the run role has the one just bound or
	// none (ruling 26). The sidecar role reads no instance env.
	if role == instancectl.RoleRun || role == instancectl.RoleSeed {
		if err := b.replicaKeyGate(rt, env, keyLayer, ctl != nil); err != nil {
			if ctl != nil {
				ctl.Close()
			}
			return p, nil, nil, nil, err
		}
	}
	logs, err := listenLogs(b.logSocketFor(rt.Host, role, gen), b.logEmitter(rt))

Create internal/appconfig/instancekey.go, the fresh read:

package appconfig

import (
	"context"
	"crypto/ecdh"

	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/registry"
	"github.com/carlosframework/platform/internal/release"
)

// ConfiguredKeyDigest is the SHA-256 of the CARLOS_INSTANCE_KEY rt's
// configuration in the bucket renders now, "" when it renders none: the
// same merge and renderer SyncExec writes <host>.config.env with
// (LoadMergedHostVersions, RenderEnv), read fresh rather than from the
// box's last sync. A replica launch whose configured key is not this
// digest is refused (instance-key plan ruling 27): a configuration change
// is in flight, and the home may already hold the new key. The value
// never leaves this function.
func ConfiguredKeyDigest(ctx context.Context, s release.Store, priv *ecdh.PrivateKey, rt registry.Route) (string, error) {
	m, _, err := LoadMergedHostVersions(ctx, s, release.RouteAccount(rt), rt.App, rt.Environment, rt.Host)
	if err != nil || m == nil {
		return "", err
	}
	rendered, err := RenderEnv(m, priv, routeBinding(rt))
	if err != nil {
		return "", err
	}
	key, _ := instancekey.Effective(instancekey.ParseEnv(rendered), nil)
	if key == "" {
		return "", nil
	}
	return instancekey.Digest(key), nil
}

Create internal/host/replicahomekeys.go:

package host

import (
	"context"
	"crypto/ecdh"
	"errors"
	"log"
	"os"
	"strings"
	"sync"

	"github.com/carlosframework/platform/internal/appconfig"
	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/registry"
	"github.com/carlosframework/platform/internal/release"
)

// replicaHomeKeysPath is the box's list of verified home keys for its
// replica rows (the list the plan calls CARLOS_REPLICA_HOME_KEYS):
// "replica-host=sha256" entries, separated by commas or newlines. Only the
// pre-roll check writes it, as root, replacing it whole (write a temporary
// file, then rename it into place); the agent only reads it.
const replicaHomeKeysPath = "/etc/carlos/replica-home-keys"

// replicaHomeKeys is ExecBackend.ReplicaHomeKeys (instance-key plan rulings
// 26, 27 and 30): it reads the list again at every call, so an entry the
// procedure removes or replaces takes effect at the next replica launch,
// with no agent restart and no cached copy to go stale. Nothing refreshes
// the list from a replica's own configuration. An absent list verifies
// nothing. A list that cannot be read or parsed is logged by the error's
// position and reason, never its text, and verifies nothing, so every
// replica launch that would be handed a key is refused until it is fixed.
func replicaHomeKeys(read func() ([]byte, error)) func(host string) (string, bool) {
	return func(host string) (string, bool) {
		raw, err := read()
		if errors.Is(err, os.ErrNotExist) {
			return "", false
		}
		if err != nil {
			log.Printf("agent: replica home keys: the list could not be read (no replica home key verifies): %v", err)
			return "", false
		}
		keys, err := instancekey.ParseHomeKeys(strings.ReplaceAll(string(raw), "\n", ","))
		if err != nil {
			log.Printf("agent: replica home keys: %v (no replica home key verifies until it is fixed)", err)
			return "", false
		}
		sha, ok := keys[host]
		return sha, ok
	}
}

// configuredKey is ExecBackend.ConfiguredKey (ruling 27): the digest
// of the key rt's configuration in the bucket renders now, read with the
// box's secrets key through the deploy store. The store and key are opened
// on first use and kept once both open; a failure is returned, so the
// replica is refused, and is tried again at its next launch.
func configuredKey(getenv func(string) string, open func(context.Context) (release.Store, error)) func(context.Context, registry.Route) (string, error) {
	var (
		mu    sync.Mutex
		store release.Store
		priv  *ecdh.PrivateKey
	)
	return func(ctx context.Context, rt registry.Route) (string, error) {
		mu.Lock()
		if store == nil {
			raw := getenv("CARLOS_SECRETS_KEY")
			if raw == "" {
				mu.Unlock()
				return "", errors.New("CARLOS_SECRETS_KEY is not set")
			}
			k, err := appconfig.ParsePrivateKey(raw, "CARLOS_SECRETS_KEY")
			if err != nil {
				mu.Unlock()
				return "", err
			}
			st, err := open(ctx)
			if err != nil {
				mu.Unlock()
				return "", err
			}
			store, priv = st, k
		}
		st, k := store, priv
		mu.Unlock()
		return appconfig.ConfiguredKeyDigest(ctx, st, k, rt)
	}
}

In internal/host/agent.go, after the ExecBackend literal (beside eb.Logs):

		// The verified home keys of this box's replica rows (L21 rulings 26
		// and 30), read again at every replica launch, until O8 reads them
		// from each home's published env item.
		eb.ReplicaHomeKeys = replicaHomeKeys(func() ([]byte, error) { return os.ReadFile(replicaHomeKeysPath) })
		// The bucket's configuration, read fresh at each replica launch
		// whose key is configured (ruling 27).
		eb.ConfiguredKey = configuredKey(os.Getenv, openDeployStore)
  • [ ] Step 4: Run the tests, then the launch tests around them

Run (sandbox off: the control-socket case and the nested prepare bind sockets): the Step 2 commands, then nice -n 19 go test ./internal/activator/ -run 'Replica|InstanceKey|Seed|Start|Prepare|LaunchEnvironment|Control' -count=1 Expected: PASS. (With the sandbox on, the control-socket case skips and the nested run-role case cannot bind its log socket.)

  • [ ] Step 5: Commit
gofmt -l internal/activator internal/host internal/appconfig
git add internal/activator/replicakey.go internal/activator/backend_exec.go internal/activator/prepare.go internal/activator/replica_instancekey_test.go internal/activator/replica_instancekey_linux_test.go internal/host/replicahomekeys.go internal/host/replicahomekeys_test.go internal/host/agent.go internal/appconfig/instancekey.go internal/appconfig/instancekey_test.go internal/activator/replicakey_race_test.go
git commit -m "activator: a replica launches only with its home's key, verified, or none and a socket its helper refuses on (L21)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"

---

Task 4: The claim's launch source mints under the guard, syncs before it answers, and names the key it delivers

Files:

  • Modify: internal/activator/claim_snapshot.go:17-25 (ClaimLaunchSnapshot), :40-67 (EnsureClaimAdminToken), :69-85 (SnapshotClaimLaunch's env read)
  • Modify: internal/host/claimowners.go:125-138 (doc on ClaimFilesManifest, O1')
  • Test: internal/activator/claim_instancekey_test.go, internal/activator/claimretry_instancekey_linux_test.go, internal/host/claim_instancekey_test.go, internal/host/claim_replica_instancekey_test.go (create all four)

Interfaces:

  • Consumes: Task 1's instancekey.Effective, Digest, SourceOf; Task 2's ensureInstanceKeyFor, settleClaimInstanceEnv, errInstanceEnvNeedsWrite, probeInstanceKeySync; instanceEnvParts (backendexec.go:1836); claimServingLaunchSource (claimproduction.go:421), claimServingOwners (claimowners.go:293) with Select (:375) and Materialize (:425); test fixture newAdminTokenFixture, adminTokenHost, f.envPath(), f.token() (claimproductionadmintokentest.go:20-118); leasetest.NewVersionedBucket; Service.startAdmitted (claimwake.go:330) as `claimlooplinux_test.go` drives it.
  • Produces: EnsureClaimAdminToken initializes the token and the key inside the one authorized write, and syncs a present key before it answers. ClaimLaunchSnapshot.InstanceKeySHA256 and InstanceKeySource name the key the snapshot's env delivers, for O6 (ruling 21).

Built: the guarded write mints the admin token only when the effective env has none (rev 5.1 called EnsureAdminToken unconditionally, which would mint a second token into the own file over a configured one and change a working launch); the key is ensured after it either way. The early return has a third arm: no own env file with the token and the key both from the rendered configuration proceeds as it did before the key existed (pre-flight ruling A; a configured key is not the platform's to make durable, ruling 5, and an absent file is not created, ruling 7). The claim launch source is the one launch path that returns the key's refusal without a wrapper of its own, so it wraps it with the host, once (Task 6's ruling). claimServingLaunchSource does not yet carry InstanceKeySHA256 and InstanceKeySource into ClaimServingLaunch: that is O6's.

  • [ ] Step 1: Write the failing tests

Create internal/activator/claim_instancekey_test.go:

package activator

import (
	"context"
	"errors"
	"os"
	"path/filepath"
	"strings"
	"testing"

	"github.com/carlosframework/platform/internal/registry"
)

// Finding 2 on the claim path: when the token and the key are already
// present, EnsureClaimAdminToken writes nothing and calls no authorizer,
// but it does not answer until the env file and its directory are synced.
// A file that is not 0600 goes through the authorized write, which sets it.
func TestEnsureClaimAdminTokenSyncsAPresentKeyBeforeItAnswers(t *testing.T) {
	probe := probeInstanceKeySync(t)
	const host = "claim-key.example"
	b := &ExecBackend{DataDir: t.TempDir()}
	rt := registry.Route{Host: host, Socket: "/run/carlos/claim-key.sock", DBPath: "/data/claim-key.db"}
	envPath := filepath.Join(b.DataDir, host+".env")
	body := "CARLOS_ADMIN_TOKEN=" + strings.Repeat("d", 64) + "\n" + InstanceKeyVar + "=" + strings.Repeat("A", 43) + "=\n"
	if err := os.WriteFile(envPath, []byte(body), 0o600); err != nil {
		t.Fatal(err)
	}
	authorized := 0
	authorize := func(_ context.Context, write func() error) error { authorized++; return write() }
	for _, fail := range []func(error){func(e error) { probe.fileErr = e }, func(e error) { probe.dirErr = e }} {
		fail(errors.New("injected: EIO"))
		if err := b.EnsureClaimAdminToken(context.Background(), rt, authorize); err == nil {
			t.Fatal("the launch source answered while the env file's sync fails")
		}
		fail(nil)
	}
	before := *probe
	if err := b.EnsureClaimAdminToken(context.Background(), rt, authorize); err != nil {
		t.Fatal(err)
	}
	if authorized != 0 || probe.file != before.file+1 || probe.dir != before.dir+1 {
		t.Fatalf("authorized %d, synced file %d dir %d; want 0, 1, 1", authorized, probe.file-before.file, probe.dir-before.dir)
	}
	if err := os.Chmod(envPath, 0o644); err != nil {
		t.Fatal(err)
	}
	if err := b.EnsureClaimAdminToken(context.Background(), rt, authorize); err != nil {
		t.Fatal(err)
	}
	if st, _ := os.Stat(envPath); authorized != 1 || st.Mode().Perm() != 0o600 {
		t.Fatalf("a 0644 file: authorized %d, mode %o; want the guarded path to set 600", authorized, st.Mode().Perm())
	}
	if raw, _ := os.ReadFile(envPath); string(raw) != body {
		t.Fatal("the claim path rewrote the env file")
	}
}

Create internal/activator/claimretry_instancekey_linux_test.go:

//go:build linux

package activator

import (
	"context"
	"errors"
	"os/exec"
	"slices"
	"sync"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/hostclaim"
	"github.com/carlosframework/platform/internal/hostwrite"
	"github.com/carlosframework/platform/internal/registry"
)

// Open question 5's premise, corrected (instance-key plan rev 2): on a
// claim box a session's process retries do not go back to the launch
// source, which is where the key is minted. Prepare runs once per
// admitted session and captures the launch inputs, and every attempt
// calls Materialize (the C8 provider's Establish) before it reserves a
// start. So no attempt runs with a key minted after the files manifest
// was established. This pins that shape of startAdmitted: one Prepare,
// and Materialize before each attempt's start.
func TestStartAdmittedRetriesReuseTheirInputsAndMaterializeFirst(t *testing.T) {
	var (
		mu       sync.Mutex
		events   []string
		prepares int
	)
	note := func(e string) { mu.Lock(); events = append(events, e); mu.Unlock() }
	obs := hostclaim.Observed{Host: "x.example", ETag: "one", Claim: &hostclaim.Claim{Holder: &hostclaim.Holder{Location: "instances/x.example/inc", Session: 1, Generation: "one"}, Op: hostclaim.Op{Kind: "landing", ID: "op", Phase: "probing", Exec: "box"}}}
	work, _ := (hostwrite.Config{}).Select(t.Context(), obs.Host, "spawn")
	a := &Service{cfg: Config{HostWrites: hostwrite.Config{Enabled: true}, Claims: &ClaimRuntime{Box: "box", Read: func(context.Context, string) (hostclaim.Observed, error) { return obs, nil }, Prepare: func(context.Context, registry.Route, hostclaim.Observed, startPolicy) (ClaimStart, error) {
		mu.Lock()
		prepares++
		mu.Unlock()
		return ClaimStart{RoleLauncher: &RoleLaunch{}, CurrentAttempt: func() hostclaim.WakeRetryAttempt { return hostclaim.WakeRetryAttempt{} }, Host: obs.Host, Work: work, Identity: hostwrite.Identity{Config: "c", Release: "r"},
			OpenProbe: func(context.Context, time.Time) (time.Time, error) {
				return time.Time{}, errors.New("unreachable while Start refuses")
			},
			Restore: func(context.Context) error { note("restore"); return nil },
			Materialize: func(context.Context) error {
				note("materialize")
				return nil
			},
			Build:     func() *exec.Cmd { return nil },
			Ready:     func(context.Context, *exec.Cmd) error { return nil },
			Reserve:   func(context.Context, int) error { note("reserve"); return nil },
			Failed:    func(context.Context, int) error { note("failed"); return nil },
			Healthy:   func(context.Context, int, *exec.Cmd) error { return nil },
			Replicate: func(context.Context, string) (Rep, error) { return nil, nil },
			Publish:   func(context.Context, hostclaim.Observed) (hostclaim.Observed, error) { return obs, nil },
			Retain:    func(Proc, Rep, error) {},
		}, nil
	}}}}
	// Two starts: the first refused (RoleLaunch.Start refuses until claim
	// Plan 9 Group D), a 1 s backoff, then the second.
	err := a.startAdmitted(t.Context(), &actEntry{host: obs.Host}, obs, startPolicy{MaxStarts: 2, Window: 30 * time.Second, Deadline: time.Now().Add(time.Minute), Probe: true})
	if !errors.Is(err, ErrClaimIsolationUnavailable) {
		t.Fatalf("the loop should end on the last start's refusal: %v", err)
	}
	want := []string{"restore", "reserve", "materialize", "failed", "reserve", "materialize", "failed"}
	mu.Lock()
	defer mu.Unlock()
	if prepares != 1 || !slices.Equal(events, want) {
		t.Fatalf("prepares %d, events %v; want 1 and %v", prepares, events, want)
	}
}

Create internal/host/claim_instancekey_test.go:

package host

import (
	"context"
	"os"
	"path/filepath"
	"strings"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/activator"
	"github.com/carlosframework/platform/internal/hostclaim"
	"github.com/carlosframework/platform/internal/hostwrite"
	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/lease/leasetest"
)

// keyCapturingFiles is a C8 provider that records the instance's own env
// file as it stands when Establish runs: the moment the provider must
// publish it (ruling 6).
type keyCapturingFiles struct {
	path string
	seen []byte
}

func (k *keyCapturingFiles) Establish(ctx context.Context, o hostclaim.Observed, in ClaimLaunchInputs, work hostwrite.Work) error {
	raw, err := os.ReadFile(k.path)
	k.seen = raw
	return err
}

type keyTestMarkers struct{}

func (keyTestMarkers) PutMarker(context.Context, string, string, []byte, string, func(context.Context) error) (string, error) {
	return "v1", nil
}

type passWork struct{}

func (passWork) With(_ context.Context, _ hostwrite.Identity, fn func() error) error { return fn() }

// The claim's first wake mints the key in Select (the launch source), so
// the env file the C8 provider publishes in Materialize already carries it,
// and the launch env the app will run with carries the same value.
func TestClaimFirstWakeKeyIsOnDiskBeforeTheFilesManifest(t *testing.T) {
	f := newAdminTokenFixture(t, "box-1")
	files := &keyCapturingFiles{path: f.envPath()}
	owners := &claimServingOwners{Data: leasetest.NewVersionedBucket(), Bucket: "data", Region: "ie", Box: "box-1",
		NewID: func() string { return "g1" }, Now: func() time.Time { return time.Date(2026, 10, 10, 9, 0, 0, 0, time.UTC) },
		Sources: ClaimServingSources{Launch: claimServingLaunchSource(f.backend, f.tokens), Markers: keyTestMarkers{}, Files: files}}
	o, err := hostclaim.Read(t.Context(), f.claims, adminTokenHost)
	if err != nil {
		t.Fatal(err)
	}
	in, err := owners.Select(t.Context(), f.rt, o, activator.StartPolicy{})
	if err != nil {
		t.Fatalf("select: %v", err)
	}
	if err := owners.Materialize(t.Context(), o, in, passWork{}); err != nil {
		t.Fatalf("materialize: %v", err)
	}
	var key string
	for _, line := range strings.Split(string(files.seen), "\n") {
		if v, ok := strings.CutPrefix(line, activator.InstanceKeyVar+"="); ok {
			key = v
		}
	}
	if len(key) != 44 {
		t.Fatalf("the env file at Establish carries no minted key: %d bytes", len(files.seen))
	}
	found := false
	for _, kv := range in.Env {
		found = found || kv == activator.InstanceKeyVar+"="+key
	}
	if !found {
		t.Fatal("the launch env does not carry the key the provider published")
	}
}

// An instance from before this build has a token and no key. Its next
// claim launch mints the key under the same guarded write, leaves the
// token alone, and a retry keeps both.
func TestClaimLaunchMintsAKeyBesideAnExistingToken(t *testing.T) {
	f := newAdminTokenFixture(t, "box-1")
	tok := strings.Repeat("c", 64)
	if err := os.WriteFile(f.envPath(), []byte("CARLOS_ADMIN_TOKEN="+tok+"\n"), 0o600); err != nil {
		t.Fatal(err)
	}
	launch := claimServingLaunchSource(f.backend, f.tokens)
	first, err := launch(t.Context(), f.rt)
	if err != nil {
		t.Fatal(err)
	}
	raw, _ := os.ReadFile(f.envPath())
	if f.token() != tok || !strings.HasPrefix(string(raw), "CARLOS_ADMIN_TOKEN="+tok+"\n"+activator.InstanceKeyVar+"=") {
		t.Fatalf("env file after the launch: %d bytes, token kept %v", len(raw), f.token() == tok)
	}
	again, err := launch(t.Context(), f.rt)
	if err != nil {
		t.Fatal(err)
	}
	raw2, _ := os.ReadFile(f.envPath())
	if string(raw2) != string(raw) || again.Identity != first.Identity {
		t.Fatal("a retried launch changed the env file or the launch identity")
	}
}

// Ruling 21: the claim snapshot names the key its env delivers, by digest,
// and the file that supplied it, for O6 to check a process's report
// against. The identity is the env's alone, so it does not change.
func TestClaimSnapshotNamesTheDeliveredKeyAndItsSource(t *testing.T) {
	f := newAdminTokenFixture(t, "box-1")
	first, err := claimServingLaunchSource(f.backend, f.tokens)(t.Context(), f.rt)
	if err != nil {
		t.Fatal(err)
	}
	snap, err := f.backend.SnapshotClaimLaunch(t.Context(), f.rt)
	if err != nil {
		t.Fatal(err)
	}
	own, _ := os.ReadFile(f.envPath())
	var key string
	for _, line := range strings.Split(string(own), "\n") {
		if v, ok := strings.CutPrefix(line, activator.InstanceKeyVar+"="); ok {
			key = v
		}
	}
	if snap.InstanceKeySource != instancekey.SourcePlatform || snap.InstanceKeySHA256 != instancekey.Digest(key) || snap.Identity != first.Identity {
		t.Fatalf("platform key: source %q, digest match %v, identity kept %v", snap.InstanceKeySource, snap.InstanceKeySHA256 == instancekey.Digest(key), snap.Identity == first.Identity)
	}
	tok := "CARLOS_ADMIN_TOKEN=" + f.token() + "\n"
	if err := os.WriteFile(f.envPath(), []byte(tok+activator.InstanceKeyVar+"=\n"), 0o600); err != nil {
		t.Fatal(err)
	}
	if err := os.WriteFile(filepath.Join(f.data, adminTokenHost+".config.env"), []byte(activator.InstanceKeyVar+"=from-config\n"), 0o600); err != nil {
		t.Fatal(err)
	}
	snap, err = f.backend.SnapshotClaimLaunch(t.Context(), f.rt)
	if err != nil || snap.InstanceKeySource != instancekey.SourceConfig || snap.InstanceKeySHA256 != instancekey.Digest("from-config") {
		t.Fatalf("configured key: source %q, %v", snap.InstanceKeySource, err)
	}
}

Create internal/host/claim_replica_instancekey_test.go, ruling 26 on the claim path:

package host

import (
	"context"
	"errors"
	"os"
	"path/filepath"
	"strings"
	"testing"

	"github.com/carlosframework/platform/internal/activator"
	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/registry"
)

// Ruling 26 on the claim path, which has no control socket before O6: a
// keyless replica is never selected (its helper would mint), a key that is
// not the home's is refused, and the home's key is selected and named.
func TestClaimReplicaLaunchIsGated(t *testing.T) {
	homeKey := strings.Repeat("H", 43) + "="
	for _, tc := range []struct {
		name, config string
		refused      bool
	}{
		{"keyless", "", true},
		{"another key", activator.InstanceKeyVar + "=configured-B\n", true},
		{"the home's key", activator.InstanceKeyVar + "=" + homeKey + "\n", false},
	} {
		t.Run(tc.name, func(t *testing.T) {
			f := newAdminTokenFixture(t, "box-1")
			f.rt.Replica = true
			f.backend.ReplicaHomeKeys = func(string) (string, bool) { return instancekey.Digest(homeKey), true }
			f.backend.ConfiguredKey = func(context.Context, registry.Route) (string, error) { return instancekey.Digest(homeKey), nil }
			own := "CARLOS_ADMIN_TOKEN=" + strings.Repeat("c", 64) + "\n"
			if err := os.WriteFile(f.envPath(), []byte(own), 0o600); err != nil {
				t.Fatal(err)
			}
			if tc.config != "" {
				if err := os.WriteFile(filepath.Join(f.data, adminTokenHost+".config.env"), []byte(tc.config), 0o600); err != nil {
					t.Fatal(err)
				}
			}
			launch, err := claimServingLaunchSource(f.backend, f.tokens)(t.Context(), f.rt)
			if tc.refused {
				if !errors.Is(err, activator.ErrReplicaKey) {
					t.Fatalf("launch source = %v, want ErrReplicaKey", err)
				}
			} else {
				found := false
				for _, kv := range launch.Env {
					found = found || kv == activator.InstanceKeyVar+"="+homeKey
				}
				if err != nil || !found {
					t.Fatalf("launch source = %v, key delivered %v", err, found)
				}
			}
			if raw, _ := os.ReadFile(f.envPath()); string(raw) != own {
				t.Fatal("a replica's claim launch changed its own env file: a replica never mints")
			}
		})
	}
}
  • [ ] Step 2: Run them to verify they fail

Run: nice -n 19 go test ./internal/activator/ -run 'TestEnsureClaimAdminTokenSyncs|TestStartAdmittedRetriesReuse' -count=1 -v && nice -n 19 go test ./internal/host/ -run 'TestClaimFirstWakeKey|TestClaimLaunchMintsAKey|TestClaimSnapshotNames|TestClaimReplicaLaunch' -count=1 -v Expected, as checked in the scratch build:

  • TestEnsureClaimAdminTokenSyncsAPresentKeyBeforeItAnswers fails: the early return answers without a sync.
  • TestStartAdmittedRetriesReuseTheirInputsAndMaterializeFirst passes: it pins existing behaviour (ruling 16).
  • The host package fails to build: snap.InstanceKeySHA256 undefined.
  • [ ] Step 3: Mint, sync, and name the key in the launch source

In internal/activator/claim_snapshot.go, add "errors" and "github.com/carlosframework/platform/internal/instancekey" to the imports, and add the two fields to ClaimLaunchSnapshot:

	Roles                       []string
	Limit                       cgroup.Limit
	// InstanceKeySHA256 and InstanceKeySource name the CARLOS_INSTANCE_KEY
	// Env delivers (its digest) and the file that supplied it
	// (instancekey.SourcePlatform or SourceConfig); both empty when Env
	// delivers none. O6 hands them to the process's adoption verb.
	InstanceKeySHA256, InstanceKeySource string
}

Replace the body of EnsureClaimAdminToken from the early return to its end:

	// The instance key is initialized the same way and in the same
	// authorized write (L21). vars is the effective env, so a key the
	// rendered configuration supplies counts as present, and a replica
	// gets none (ruling 26). When both are present nothing is written and
	// authorize is not called, but the file is still made durable before
	// the launch source answers: a key whose mint failed its sync is never
	// launched with until a sync succeeds (settleClaimInstanceEnv, ruling
	// 18). A file that is not 0600 takes the guarded path. The key's
	// ensure sets the mode when the own file holds the key; when the key
	// is the configuration's, or the row is a replica, the ensure writes
	// nothing, the mode stays as it is, and the next launch of that row
	// asks the guard again. The token's mode has never been the claim
	// path's to tighten.
	if vars["CARLOS_ADMIN_TOKEN"] != "" && (rt.Replica || vars[InstanceKeyVar] != "") {
		err := b.settleClaimInstanceEnv(rt)
		switch {
		case err == nil:
			return nil
		case errors.Is(err, os.ErrNotExist):
			// No own env file: the token and the key both come from the
			// rendered configuration. A configured key is the app's and
			// is not the platform's to make durable (ruling 5), and an
			// absent file is not created here (ruling 7), so the launch
			// proceeds as it did before the key existed.
			return nil
		case !errors.Is(err, errInstanceEnvNeedsWrite):
			return err
		}
	}
	if authorize == nil {
		return hostwrite.ErrUnavailable
	}
	paths := b.pathsFor(rt)
	return authorize(ctx, func() error {
		// Only a missing token is minted: one the rendered configuration
		// supplies is the one the launch delivers, and a second one in the
		// own file would shadow it.
		if vars["CARLOS_ADMIN_TOKEN"] == "" {
			if paths.layout == LayoutNested {
				if err := EnsureAdminTokenBeneath(paths.dataDir, rt.Host+".env"); err != nil {
					return err
				}
			} else if err := EnsureAdminToken(paths.envFile); err != nil {
				return err
			}
		}
		// The one place the claim launch source names the host around the
		// key's error: the refusal itself does not, and every other launch
		// path wraps it the same way.
		if _, err := b.ensureInstanceKeyFor(rt, paths); err != nil {
			return fmt.Errorf("instance key for %s: %w", rt.Host, err)
		}
		return nil
	})
}

("fmt" and "os" join the imports.)

In SnapshotClaimLaunch, read the two layers, keep instanceEnv's merge, pass a replica through Task 3b's gate with the layer from this same read (ruling 29; a claim launch has no control socket before O6), and record the delivered key. The identity is computed from Env as before, so it does not change:

	cfgEnv, ownEnv, err := b.instanceEnvParts(rt)
	if err != nil {
		return out, err
	}
	vars := mergeEnv(cfgEnv, ownEnv) // instanceEnv, with its two layers kept
	if vars["CARLOS_ADMIN_TOKEN"] == "" {
		return out, hostwrite.ErrUnavailable
	}
	// The key this launch delivers and the file that supplied it, from
	// the same read as vars: what the replica gate decides on (ruling 29)
	// and what the process's adoption report is checked against (ruling
	// 21, O6). Not in the identity: the env already is.
	key, layer := instancekey.Effective(cfgEnv, ownEnv)
	// A claim launch has no control socket until O6, so a replica is
	// selected only with its home's key, verified (ruling 26).
	if err := b.replicaKeyGate(rt, vars, layer, false); err != nil {
		return out, err
	}
	if key != "" {
		out.InstanceKeySHA256, out.InstanceKeySource = instancekey.Digest(key), instancekey.SourceOf(layer)
	}

In internal/host/claimowners.go, extend the ClaimFilesManifest doc comment (O1'):

// bootstrap; only an empty bootstrap (in.Empty, after EmptyAuthority) starts
// from no manifest.
//
// The instance key (L21, plan 2026-10-10-instance-key-provision.md
// obligation O1'): the launch source has already minted
// CARLOS_INSTANCE_KEY into the host's own <host>.env, durably, when
// Establish runs, and the app executes only after it returns. So the
// manifest Establish leaves must carry the env file (region HA §5.6's
// always-continuous platform item) with exactly the bytes on disk when it
// runs, staged and committed before it returns. A new admission starts
// with no adoption record: the manifest carries no instancekey.AdoptionItem
// (StagePlatformDelete), and the local <host>.instance-key-adoption.json is
// removed under work. An error, a bucket that cannot be reached included,
// fails the wake and the app does not start. Establish never writes the env
// file on disk.
type ClaimFilesManifest interface {
  • [ ] Step 4: Run the tests, then the claim tests around them

Run: nice -n 19 go test ./internal/host/ -run 'InstanceKey|ClaimSnapshot|AdminToken|Rev2Shape|SnapshotClaimLaunch' -count=1 Expected: PASS, including TestClaimProductionAdminTokenRefusesWithoutAuthority: without authority no <host>.env is written, so no key either.

Run: nice -n 19 go test ./internal/activator/ -run 'Claim|StartAdmitted' -count=1 Expected: PASS, including TestEnsureClaimAdminTokenInspectsSidecarBeforeAuthorize.

  • [ ] Step 5: Commit
gofmt -l internal/activator internal/host
git add internal/activator/claim_snapshot.go internal/activator/claim_instancekey_test.go internal/activator/claimretry_instancekey_linux_test.go internal/host/claimowners.go internal/host/claim_instancekey_test.go internal/host/claim_replica_instancekey_test.go
git commit -m "claim: the launch source mints the instance key under the guard, syncs it before Establish, and names it (L21)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"

---

Task 5: The adoption verb and its publication

The verb is built and tested here and mounted nowhere on main (ruling 21). O6 mounts it on claim launches, after O7 (ruling 24).

Files:

  • Create: internal/hostfiles/relay.go, internal/activator/keyadoption.go
  • Test: internal/hostfiles/relay_test.go, internal/activator/keyadoption_test.go, internal/activator/keyadoption_linux_test.go, internal/host/instancekey_adoption_test.go

Interfaces:

  • Consumes: Task 1's instancekey (Binding, AdoptionRecord, ParseAdoption, ValidDigest, the sources, AdoptionSuffix, AdoptionItem); Task 2's syncDir; adminTokenMu; instancefs.WriteAtomic, instancefs.ReadBeneathLimit, safefile.WriteFile (which fsyncs the file and its directory); hostfiles.Open, Run, Options, ReadPlatform, ManifestKey, (*Publisher).StagePlatform.
  • Produces: activator.AdoptionLaunch, (*ExecBackend).MountInstanceKeyAdoption; hostfiles.PlatformStager, hostfiles.PlatformRelay (Publish, Run); the test seam adoptionRecordSync.
  • [ ] Step 1: Write the failing tests

Create internal/hostfiles/relay_test.go:

package hostfiles

import (
	"context"
	"errors"
	"sync"
	"testing"
	"time"
)

type flakyStager struct {
	mu     sync.Mutex
	fail   int
	staged [][]byte
	block  chan struct{}
}

func (f *flakyStager) StagePlatform(_ context.Context, name string, body []byte) error {
	if f.block != nil {
		<-f.block
	}
	f.mu.Lock()
	defer f.mu.Unlock()
	if f.fail > 0 {
		f.fail--
		return errors.New("store unreachable")
	}
	f.staged = append(f.staged, append([]byte(nil), body...))
	return nil
}

func (f *flakyStager) last() []byte {
	f.mu.Lock()
	defer f.mu.Unlock()
	if len(f.staged) == 0 {
		return nil
	}
	return f.staged[len(f.staged)-1]
}

// Ruling 22: Publish never waits on the store; Run stages the latest body,
// retrying a failed stage, and stages nothing again once it has.
func TestPlatformRelayStagesTheLatestBodyWithoutBlockingItsCaller(t *testing.T) {
	st := &flakyStager{fail: 2, block: make(chan struct{})}
	r := &PlatformRelay{Name: "instance-key-adoption.json", Stage: st, Retry: 10 * time.Millisecond}
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()
	done := make(chan error, 1)
	go func() { done <- r.Run(ctx) }()
	published := make(chan struct{})
	go func() { r.Publish([]byte("one")); r.Publish([]byte("two")); close(published) }()
	select {
	case <-published:
	case <-time.After(time.Second):
		t.Fatal("Publish waited on a store that does not answer")
	}
	close(st.block)
	deadline := time.Now().Add(2 * time.Second)
	for string(st.last()) != "two" {
		if time.Now().After(deadline) {
			t.Fatalf("staged %q, want the latest body after two failed stages", st.last())
		}
		time.Sleep(5 * time.Millisecond)
	}
	time.Sleep(50 * time.Millisecond)
	st.mu.Lock()
	n := len(st.staged)
	st.mu.Unlock()
	if n != 1 {
		t.Fatalf("staged %d times, want once", n)
	}
	cancel()
	if err := <-done; !errors.Is(err, context.Canceled) {
		t.Fatalf("Run = %v", err)
	}
}

Create internal/activator/keyadoption_test.go:

package activator

import (
	"net/http"
	"net/http/httptest"
	"os"
	"path/filepath"
	"strings"
	"testing"

	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/registry"
)

// adoptionFixture is one claim-launched process's control socket, served
// in memory: its launch state, and every record handed to the publisher.
type adoptionFixture struct {
	mux       *http.ServeMux
	launch    AdoptionLaunch
	hasLaunch bool
	published [][]byte
}

func newAdoptionFixture(t *testing.T, rt registry.Route, tok, key, source string) *adoptionFixture {
	t.Helper()
	f := &adoptionFixture{hasLaunch: true}
	f.launch = AdoptionLaunch{
		Writer:    instancekey.Binding{Host: rt.Host, Incarnation: "inc-1", Session: 3, Generation: "g3"},
		KeySHA256: instancekey.Digest(key), Source: source, Dir: t.TempDir(),
		Publish: func(rec []byte) { f.published = append(f.published, rec) },
	}
	f.mux = http.NewServeMux()
	(&ExecBackend{}).MountInstanceKeyAdoption(f.mux, rt, func() (string, error) { return tok, nil },
		func() (AdoptionLaunch, bool) { return f.launch, f.hasLaunch })
	return f
}

func (f *adoptionFixture) put(token, body string) int {
	req := httptest.NewRequest(http.MethodPut, instanceKeyAdoptionPath, strings.NewReader(body))
	if token != "" {
		req.Header.Set("Authorization", "Bearer "+token)
	}
	w := httptest.NewRecorder()
	f.mux.ServeHTTP(w, req)
	return w.Code
}

func adoptionBody(key, source string) string {
	return `{"sha256":"` + instancekey.Digest(key) + `","source":"` + source + `"}`
}

// Rulings 20 and 21: the agent records the launch's own writer and the key
// the launch delivered, from its own launch state. A request cannot name a
// writer, a digest or a source the launch did not have.
func TestInstanceKeyAdoptionRecordsTheLaunchesWriter(t *testing.T) {
	tok := strings.Repeat("f", 64)
	key := strings.Repeat("K", 43) + "="
	rt := registry.Route{Host: "shop.example.com", Kind: "instance"}
	f := newAdoptionFixture(t, rt, tok, key, instancekey.SourcePlatform)
	recPath := filepath.Join(f.launch.Dir, rt.Host+instancekey.AdoptionSuffix)
	for _, tc := range []struct {
		name, token, body string
		want              int
	}{
		{"no bearer", "", adoptionBody(key, "platform"), http.StatusUnauthorized},
		{"another host's bearer", strings.Repeat("0", 64), adoptionBody(key, "platform"), http.StatusUnauthorized},
		{"not a digest", tok, `{"sha256":"abc","source":"platform"}`, http.StatusBadRequest},
		{"an unknown source", tok, adoptionBody(key, "own"), http.StatusBadRequest},
		{"a session in the request", tok, `{"sha256":"` + instancekey.Digest(key) + `","source":"platform","session":9}`, http.StatusBadRequest},
		{"another key", tok, adoptionBody("some-other-key", "platform"), http.StatusConflict},
		{"another source", tok, adoptionBody(key, "config"), http.StatusConflict},
	} {
		if got := f.put(tc.token, tc.body); got != tc.want {
			t.Fatalf("%s: %d, want %d", tc.name, got, tc.want)
		}
		if _, err := os.Stat(recPath); !os.IsNotExist(err) || len(f.published) != 0 {
			t.Fatalf("%s: a record was written or published", tc.name)
		}
	}
	want := instancekey.AdoptionRecord(f.launch.Writer, instancekey.SourcePlatform, instancekey.Digest(key))
	for i := 0; i < 2; i++ { // the helper reports on every process start
		if got := f.put(tok, adoptionBody(key, "platform")); got != http.StatusNoContent {
			t.Fatalf("report %d: %d, want 204", i, got)
		}
		rec, err := os.ReadFile(recPath)
		if err != nil || string(rec) != string(want) {
			t.Fatalf("report %d: record = %q, %v", i, rec, err)
		}
		if st, _ := os.Stat(recPath); st.Mode().Perm() != 0o600 {
			t.Fatalf("record mode = %o, want 600", st.Mode().Perm())
		}
		if len(f.published) != i+1 || string(f.published[i]) != string(want) {
			t.Fatalf("report %d: published %d records", i, len(f.published))
		}
	}

	// A configured key is recorded the same way, with its source (ruling 20).
	cfg := newAdoptionFixture(t, rt, tok, "from-config", instancekey.SourceConfig)
	if got := cfg.put(tok, adoptionBody("from-config", "config")); got != http.StatusNoContent {
		t.Fatalf("configured key: %d, want 204", got)
	}
	if a, err := instancekey.ParseAdoption(cfg.published[0]); err != nil || a.Source != instancekey.SourceConfig || !a.Binds(cfg.launch.Writer) {
		t.Fatalf("configured key's record = %+v, %v", a, err)
	}

	// No launch state (a legacy launch) is the 404 an agent without the
	// verb answers; a replica refuses (ruling 26).
	none := newAdoptionFixture(t, rt, tok, key, instancekey.SourcePlatform)
	none.hasLaunch = false
	replicaRT := rt
	replicaRT.Replica = true
	replica := newAdoptionFixture(t, replicaRT, tok, key, instancekey.SourcePlatform)
	if got := none.put(tok, adoptionBody(key, "platform")); got != http.StatusNotFound {
		t.Fatalf("no launch state: %d, want 404", got)
	}
	if got := replica.put(tok, adoptionBody(key, "platform")); got != http.StatusConflict {
		t.Fatalf("replica: %d, want 409", got)
	}
	if len(none.published)+len(replica.published) != 0 {
		t.Fatal("a record was published without a writer")
	}
}

Create internal/activator/keyadoption_linux_test.go:

package activator

import (
	"errors"
	"net/http"
	"os"
	"path/filepath"
	"strings"
	"testing"

	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/registry"
)

// Ruling 22 (R-C): 204 means the record is durable on disk. A nested
// record whose directory sync fails answers 500 and publishes nothing; the
// next report writes it and publishes it.
func TestInstanceKeyAdoptionIsDurableBeforeItAnswers(t *testing.T) {
	tok := strings.Repeat("f", 64)
	key := strings.Repeat("K", 43) + "="
	rt := registry.Route{Host: "shop.example.com", Kind: "instance"}
	f := newAdoptionFixture(t, rt, tok, key, instancekey.SourcePlatform)
	f.launch.Beneath = true
	adminTokenMu.Lock()
	old := adoptionRecordSync
	adoptionRecordSync = func(string) error { return errors.New("injected: EIO") }
	adminTokenMu.Unlock()
	if got := f.put(tok, adoptionBody(key, "platform")); got != http.StatusInternalServerError {
		t.Fatalf("a failing directory sync: %d, want 500", got)
	}
	if len(f.published) != 0 {
		t.Fatal("a record that is not durable was published")
	}
	adminTokenMu.Lock()
	adoptionRecordSync = old
	adminTokenMu.Unlock()
	if got := f.put(tok, adoptionBody(key, "platform")); got != http.StatusNoContent {
		t.Fatalf("retry: %d, want 204", got)
	}
	rec, err := os.ReadFile(filepath.Join(f.launch.Dir, rt.Host+instancekey.AdoptionSuffix))
	if err != nil || len(f.published) != 1 || string(rec) != string(f.published[0]) {
		t.Fatalf("after the retry: record %q (%v), %d published", rec, err, len(f.published))
	}
}

Create internal/host/instancekey_adoption_test.go, the box-loss test of ruling 22:

package host

import (
	"context"
	"encoding/json"
	"net/http"
	"net/http/httptest"
	"os"
	"path/filepath"
	"strings"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/activator"
	"github.com/carlosframework/platform/internal/hostfiles"
	"github.com/carlosframework/platform/internal/instancekey"
	"github.com/carlosframework/platform/internal/registry"
	"github.com/carlosframework/platform/internal/release"
)

// Ruling 22 (R-C): an adoption is published by the running session's
// publisher, with no admission or release after it. The box is then lost:
// every process and every local file is gone. The data bucket's manifest
// for the session's generation still holds the record, bound to the
// session's writer, which is what O2' pins.
func TestAdoptionSurvivesBoxLossWithoutAnotherAdmissionOrRelease(t *testing.T) {
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()
	data := release.NewDirStore(t.TempDir())
	writer := instancekey.Binding{Host: "shop.example.com", Incarnation: "inc-1", Session: 3, Generation: "g3"}
	prefix := "instances/" + writer.Host + "/" + writer.Incarnation + "/files/" + writer.Generation + "/"
	pub, err := hostfiles.Open(ctx, data, prefix)
	if err != nil {
		t.Fatal(err)
	}
	box := t.TempDir()
	filesDir, envDir := filepath.Join(box, "files"), filepath.Join(box, "env")
	for _, d := range []string{filesDir, envDir} {
		if err := os.MkdirAll(d, 0o700); err != nil {
			t.Fatal(err)
		}
	}
	// The session's continuous publisher and the adoption relay.
	running := make(chan error, 2)
	go func() {
		running <- hostfiles.Run(ctx, pub, filesDir, hostfiles.Options{Deadline: 40 * time.Millisecond, Scan: time.Minute, Poll: 40 * time.Millisecond})
	}()
	relay := &hostfiles.PlatformRelay{Name: instancekey.AdoptionItem, Stage: pub, Retry: 10 * time.Millisecond}
	go func() { running <- relay.Run(ctx) }()

	key := strings.Repeat("K", 43) + "="
	tok := strings.Repeat("f", 64)
	launch := activator.AdoptionLaunch{Writer: writer, KeySHA256: instancekey.Digest(key), Source: instancekey.SourcePlatform, Dir: envDir, Publish: relay.Publish}
	mux := http.NewServeMux()
	(&activator.ExecBackend{}).MountInstanceKeyAdoption(mux, registry.Route{Host: writer.Host, Kind: "instance"},
		func() (string, error) { return tok, nil }, func() (activator.AdoptionLaunch, bool) { return launch, true })
	req := httptest.NewRequest(http.MethodPut, "/instance-key/adoption", strings.NewReader(`{"sha256":"`+instancekey.Digest(key)+`","source":"platform"}`))
	req.Header.Set("Authorization", "Bearer "+tok)
	w := httptest.NewRecorder()
	mux.ServeHTTP(w, req)
	if w.Code != http.StatusNoContent {
		t.Fatalf("adoption = %d, want 204", w.Code)
	}

	// Within the publisher's continuous bound the record is committed.
	readRecord := func() ([]byte, error) {
		raw, _, err := data.Get(context.Background(), hostfiles.ManifestKey(prefix))
		if err != nil {
			return nil, err
		}
		var m hostfiles.Manifest
		if err := json.Unmarshal(raw, &m); err != nil {
			return nil, err
		}
		return hostfiles.ReadPlatform(context.Background(), data, prefix, m, instancekey.AdoptionItem, nil)
	}
	deadline := time.Now().Add(2 * time.Second)
	for {
		if _, err := readRecord(); err == nil {
			break
		}
		if time.Now().After(deadline) {
			t.Fatal("the adoption was not published within the continuous bound")
		}
		time.Sleep(10 * time.Millisecond)
	}

	// Box loss: the processes stop and the disk is gone.
	cancel()
	<-running
	<-running
	if err := os.RemoveAll(box); err != nil {
		t.Fatal(err)
	}
	rec, err := readRecord()
	if err != nil {
		t.Fatalf("after box loss: %v", err)
	}
	a, err := instancekey.ParseAdoption(rec)
	if err != nil || !a.Binds(writer) || a.SHA256 != instancekey.Digest(key) || a.Source != instancekey.SourcePlatform {
		t.Fatalf("after box loss the bucket holds %+v (%v), want the session's record", a, err)
	}
}
  • [ ] Step 2: Run them to verify they fail

Run: nice -n 19 go test ./internal/hostfiles/ -run TestPlatformRelay -count=1; nice -n 19 go test ./internal/activator/ -run Adoption -count=1; nice -n 19 go test ./internal/host/ -run TestAdoptionSurvivesBoxLoss -count=1 Expected: build failures: undefined: PlatformRelay, undefined: MountInstanceKeyAdoption (and AdoptionLaunch, instanceKeyAdoptionPath, adoptionRecordSync).

  • [ ] Step 3: Write the relay

Create internal/hostfiles/relay.go:

package hostfiles

import (
	"context"
	"log"
	"sync"
	"time"
)

// PlatformStager is the part of a session's Publisher a PlatformRelay
// needs. *Publisher implements it.
type PlatformStager interface {
	StagePlatform(ctx context.Context, name string, body []byte) error
}

// PlatformRelay hands one platform item to a running session's publisher
// without making its caller wait on the store (instance-key plan ruling
// 22). Publish records the latest body and returns at once; Run stages it
// with StagePlatform, retrying until it is staged, and the publisher's own
// continuous loop (Run in run.go) commits it within its deadline, as it
// commits every staged change. A body staged once is not staged again
// unless Publish is called with it again.
type PlatformRelay struct {
	Name  string
	Stage PlatformStager
	Retry time.Duration // between failed stages; 0 is one second

	mu      sync.Mutex
	pending []byte
	wake    chan struct{}
}

func (r *PlatformRelay) signal() chan struct{} {
	r.mu.Lock()
	defer r.mu.Unlock()
	if r.wake == nil {
		r.wake = make(chan struct{}, 1)
	}
	return r.wake
}

// Publish queues body as the item's next value. It never blocks.
func (r *PlatformRelay) Publish(body []byte) {
	w := r.signal()
	r.mu.Lock()
	r.pending = append([]byte(nil), body...)
	r.mu.Unlock()
	select {
	case w <- struct{}{}:
	default:
	}
}

// Run stages queued bodies until ctx ends, and returns ctx.Err().
func (r *PlatformRelay) Run(ctx context.Context) error {
	retry := r.Retry
	if retry <= 0 {
		retry = time.Second
	}
	w := r.signal()
	for {
		r.mu.Lock()
		body := r.pending
		r.mu.Unlock()
		if body != nil {
			if err := r.Stage.StagePlatform(ctx, r.Name, body); err != nil {
				log.Printf("hostfiles: stage platform item %s: %v (retrying)", r.Name, err)
				select {
				case <-ctx.Done():
					return ctx.Err()
				case <-time.After(retry):
				}
				continue
			}
			r.mu.Lock()
			if string(r.pending) == string(body) {
				r.pending = nil
			}
			r.mu.Unlock()
			continue
		}
		select {
		case <-ctx.Done():
			return ctx.Err()
		case <-w:
		}
	}
}
  • [ ] Step 4: Write the verb

Create internal/activator/keyadoption.go:

package activator

// The instance key's adoption record (instance-key plan rev 3, rulings 19
// to 22). A key in <host>.env or <host>.config.env proves only that one
// was delivered: an app with a key of its own keeps it (rastrillo's step
// 1), and its data is sealed under that key. So relocation accepts a
// delivered key only when the app's helper, having resolved
// $CARLOS_INSTANCE_KEY (step 2), told the agent so, and the agent recorded
// it for the writer the process was admitted as. The record is a
// trusted-helper assertion: it stops accidental loss, and a tenant that
// forges one can lose only its own data (ruling 19).
//
// The verb is mounted only where a launch has a writer session: claim
// launches, once they bind a control socket (obligation O6). A legacy
// launch has no session to bind a record to, so nothing mounts it there and
// the helper's call is a 404 it ignores.

import (
	"crypto/sha256"
	"cry