amadan
spec: instance update settings rev 6 — answers Astra S4 (no automatic backfill, monotonic completion watermark, completion at execution)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 files changed,
+95
−19
docs/superpowers/specs/2026-10-07-instance-update-settings-design.md+65 −19docs/superpowers/specs/CODEX-FINDINGS-SETTINGS-S4.md+30 −0
CI passed — run details
diff --git a/docs/superpowers/specs/2026-10-07-instance-update-settings-design.md b/docs/superpowers/specs/2026-10-07-instance-update-settings-design.md| index b213cb52c..8c457c3df 100644 |
| --- a/docs/superpowers/specs/2026-10-07-instance-update-settings-design.md |
| +++ b/docs/superpowers/specs/2026-10-07-instance-update-settings-design.md |
| @@ -1,6 +1,6 @@ |
| # Instance update settings: auto-update, channel, major hold, pin |
| -Status: draft rev 5, 2026-10-07. Split out of the zero-downtime deploys |
| +Status: draft rev 6, 2026-10-07. Split out of the zero-downtime deploys |
| spec (`2026-10-07-zero-downtime-deploys-design.md`) at its rev 6. Rev 1 was |
| a design with console-signed per-instance directives. Astra's round 6 |
| (`CODEX-FINDINGS-ZERO-DOWNTIME-R6.md`) rejected it with 6 findings. |
| @@ -17,7 +17,8 @@ adding a durable signed record of every promoted build (§4.1). Astra's S2 |
| review of rev 3 (`CODEX-FINDINGS-SETTINGS-S2.md`) closed two of those five |
| and left three partial. Rev 4 answered those three and one new finding |
| (§10). S3 (`CODEX-FINDINGS-SETTINGS-S3.md`) closed one and left three |
| -partial. Rev 5 answers them (§11). |
| +partial. Rev 5 answered them (§11). S4 (`CODEX-FINDINGS-SETTINGS-S4.md`) |
| +closed two and left two partial. Rev 6 answers them (§12). |
| ## 1. Why |
| @@ -131,9 +132,23 @@ plus a reused version name could authorize bytes nobody promoted. That |
| weakness exists in the release chain today, independent of this spec. It |
| is filed as #253, and this spec never widens it. |
| -**Backfill.** When this feature rolls out, the console writes promotion |
| -records for every version still in each channel's signed history. Their |
| -digests are taken from the canonical manifests at that time. |
| +**Builds promoted before this feature have no promotion record, and |
| +nothing is backfilled automatically.** Signing whatever is in the bucket |
| +now would vouch for bytes that could have been swapped since the build was |
| +promoted. |
| + |
| +For such a build: |
| +- **On a box that already has it staged**, it is recovered from that |
| + staged, already-verified copy. That uses the zero-downtime spec's |
| + retained identity, and needs no record. |
| +- **Anywhere else**, a pin or recovery install is refused with *no |
| + promotion record: authorize it or pick a newer build*. An app member can |
| + run `carlos releases authorize <app> <version>`. It shows the manifest |
| + and artifact digests, and on confirmation signs a promotion record for |
| + them. That is an explicit act by the publisher, not an automatic one. |
| + |
| +From the rollout on, every promotion writes its record at promotion |
| +time. |
| Promotion is the act that authorizes a build, so the record keeps the |
| authorization valid after V falls out of the pointer's ten-entry history. |
| @@ -160,12 +175,15 @@ is the zero-downtime spec's retained identity. |
| The **build state** object holds two things: |
| - **`last_executed`**: the build whose process last opened the instance's |
| database; |
| -- **the install request root last resolved**, as {R, T}, and whether it |
| - has completed. |
| +- **the install request root last resolved**, as {R, T}; |
| +- **the highest request id completed**. |
| **When it is written.** Before any generation runs a build X against the |
| instance's database, the agent writes the build state with |
| -`last_executed = X` and the current request state. That covers a wake, a |
| +`last_executed = X` and the current request state. **That write is also |
| +what completes a request.** If X is the target T of the resolved request R, |
| +the same write records R as completed. A request completes when its build |
| +first runs against the data, not when root installs it. That covers a wake, a |
| swap's successor, a fallback, a worker or sidecar spawn, and a recycle. |
| The agent holds the instance's lease at that point: a wake takes the lease |
| before it spawns, and a swap keeps it. If the write fails, the generation |
| @@ -197,19 +215,29 @@ build its box holds as failed unless it has already served that build. |
| 1. **`pin` set:** install the pinned build, if §4.1 authorizes it. Bake |
| doesn't apply, because a person chose it. |
| -2. **An `install_request` R above the last one completed** (taken from the |
| - box's own record of consumed requests, or else from the build state |
| - object): |
| - - **If R has already been resolved to a build T,** install T. That |
| - covers a pass that was interrupted, and a request picked up on |
| - another box after a move. |
| +2. **An `install_request` R above the completed watermark.** The |
| + watermark is the **highest completed id known from either source**: the |
| + box's own record (`/opt/carlos/consumed/<host>`) or the build state |
| + object. It is never one source in place of the other, so an older local |
| + marker on a box the instance returns to can't make a newer, durably |
| + completed request look pending. A completed request never enters this |
| + rule again. |
| + - **If R has already been resolved to a build T** (locally or in the |
| + build state), install T. That covers a pass that was interrupted, and |
| + a request picked up on another box. |
| - **Otherwise**, resolve R now: take the channel head that this box's |
| own bake check says is eligible, or the current head if the request |
| asked for `FastTrack`. Write {R, T} to the root-owned file |
| `/opt/carlos/consumed/<host>` before staging. The agent mirrors it |
| into the build state. |
| - - **Completion:** after the stamp, mark R completed, both locally and in |
| - the build state. |
| + - **R stays pending after the install**, until the pre-exec write |
| + (§4.2) records T as executed. A wake or a swap onto T does that. |
| + - **While R is pending, T outranks the held target**: rule 3 below and |
| + the wake check both use T, not `last_executed`. |
| + - **If T fails** (a failure record for T, or a `hold` failure, under the |
| + zero-downtime spec), R is recorded as **completed, failed**, and that |
| + is reported. It doesn't retry in a loop. "Update now" again makes a |
| + new request. |
| - `manual` and `major` are ignored for this one move. |
| - **If the head is still baking**, R stays unresolved and the reason is |
| reported. |
| @@ -230,7 +258,9 @@ build its box holds as failed unless it has already served that build. |
| head. |
| **The wake checks too.** Before it spawns a held instance, the agent |
| - compares the installed build with the build state's `last_executed`. If |
| + compares the installed build with its target. The target is a pending |
| + request's T if there is one (rule 2), and otherwise the build state's |
| + `last_executed`. If |
| they differ, it doesn't run the installed build. It runs |
| `last_executed` from its staged path if the box has it. Otherwise it |
| touches the stage relay and holds the request until root has installed |
| @@ -349,8 +379,16 @@ build whatever these settings say. |
| stale live link installs `last_executed`. The wake never runs the stale |
| link. |
| - **Evidence:** a pin of a pruned and reused version name, with a replayed |
| - pointer, is refused, because it lacks a matching promotion record. |
| - Backfill writes records for builds still in history. |
| + pointer, is refused, because it lacks a matching promotion record. A |
| + build promoted before the feature is recovered from a staged copy, is |
| + refused elsewhere, or is accepted after `carlos releases authorize`. |
| + Nothing is backfilled. |
| +- **Completion:** a request completes at the pre-exec write, not at |
| + install. A held sleeper with a pending request wakes onto T, not |
| + `last_executed`. A failed T completes the request as failed. |
| +- **Watermark merge:** a box the instance returns to, with an older local |
| + completion, doesn't replay a request the build state shows as |
| + completed. |
| - **Root pass:** |
| - a manual or major-held instance on a fresh box installs the build |
| state's installed build, not the head; |
| @@ -390,3 +428,11 @@ build whatever these settings say. |
| | S2-2 (partial), 2 | The build state is written after execution can begin | §4.2: `last_executed` is written under the lease before any generation runs, and a failed write blocks it; cold installs don't touch it; losing a resolution before T ran is harmless | |
| | S2-4 (partial) | A locally resolved request is lost before it is mirrored | §4.2 *Lost requests*: T never ran, so the replacement resolves again | |
| | 3 | Returning to a previous box keeps a stale installation | §4.3 rule 3: the held target is `last_executed`, root reinstalls on a mismatch, and the wake refuses a stale link | |
| + |
| +## 12. Astra S4 findings (rev 5), answered by rev 6 |
| + |
| +| # | Finding | Answer | |
| +|---|---------|--------| |
| +| S3-1 (partial), 1 | Backfill signs whatever is in the bucket now | §4.1: no automatic backfill. Older builds are recovered from a staged copy, refused elsewhere, or authorized explicitly by an app member (`carlos releases authorize`) | |
| +| S2-4 (partial), 2 | An older local marker overrides a newer durable completion | §4.3 rule 2: the watermark is the highest id from either source; completed requests never re-enter | |
| +| 3 | Completing at install loses the update for held instances | §4.2: completion is the pre-exec write; §4.3: a pending T outranks the held target in root and at wake; a failed T completes as failed | |
diff --git a/docs/superpowers/specs/CODEX-FINDINGS-SETTINGS-S4.md b/docs/superpowers/specs/CODEX-FINDINGS-SETTINGS-S4.md| new file mode 100644 |
| index 000000000..88a386dc1 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-FINDINGS-SETTINGS-S4.md |
| @@ -0,0 +1,30 @@ |
| +REJECT |
| + |
| +S3-1 / carried S2-1: **PARTIAL** — Direct pointer substitution is closed for added installs; backfill introduces finding 1 below. |
| +S3-2 / carried S2-2: **CLOSED** — The lease-held, mandatory pre-exec write closes the execution-ordering gap. |
| +S3-3: **CLOSED** — Root and wake now reject the stale installation identified in that finding. |
| +Carried S2-4: **PARTIAL** — Re-resolving a lost, unexecuted request is explicitly accepted; returning boxes can still replay completed requests through finding 2. |
| + |
| +1. **HIGH — §4.1, Backfill: The console signs historical builds without evidence binding their bytes.** |
| + |
| + **Code:** [internal/release/release.go:119](/home/paulca/amadan.net/carlos/platform-design-zero-downtime-deploys/internal/release/release.go:119), [release.go:656](/home/paulca/amadan.net/carlos/platform-design-zero-downtime-deploys/internal/release/release.go:656), [infra/modules/region-host/main.tf:166](/home/paulca/amadan.net/carlos/platform-design-zero-downtime-deploys/infra/modules/region-host/main.tf:166). |
| + |
| + **Failure:** V is in a current pointer’s history. Before backfill, a compromised box replaces V’s unsigned manifest and artifacts with matching malicious bytes. The pointer remains untouched and verifies correctly. Backfill signs the replacement manifest’s digest, after which a pin or recovery passes every new authorization check. No publisher promoted those bytes. This introduces a signing path for historical content; it requires neither pointer replay nor ordinary head adoption. |
| + |
| + **Smallest fix:** Backfill only from independently trusted promotion-time digests or artifacts. Where that evidence is unavailable, require explicit publisher reauthorization instead of automatically signing the current bucket contents. |
| + |
| +2. **HIGH — §4.3 rule 2: An older local completion marker overrides newer durable completion on return.** |
| + |
| + **Code:** [internal/host/movespass.go:491](/home/paulca/amadan.net/carlos/platform-design-zero-downtime-deploys/internal/host/movespass.go:491) purges database artifacts on departure; [internal/registry/registry.go:832](/home/paulca/amadan.net/carlos/platform-design-zero-downtime-deploys/internal/registry/registry.go:832) removes the route without clearing files under `/opt`. |
| + |
| + **Failure:** X retains completed request 1. H moves to Y, completes request 2 targeting B, then runs pinned C and clears its pin while remaining manual. Durable state now contains `last_executed=C` and completed `{2,B}`. On returning to X, rule 2 takes X’s local completion watermark **“or else”** the bucket’s: request 2 therefore appears pending and selects B before rule 3 can select C. If H has already woken C through the new staged-path wake check, the resulting installation feeds a swap back to B against C’s database. No forged state is involved. |
| + |
| + **Smallest fix:** Merge completion monotonically across local and durable state before selecting a target. A newer durable completion must supersede an older local marker; completed requests must never enter the resolution/install branch. |
| + |
| +3. **MEDIUM — §§4.2–4.3, §6: Completing at installation causes held instances to discard explicit updates before execution.** |
| + |
| + **Code:** [internal/release/adopt.go:1258](/home/paulca/amadan.net/carlos/platform-design-zero-downtime-deploys/internal/release/adopt.go:1258) stamps installation independently of execution; [adopt.go:1273](/home/paulca/amadan.net/carlos/platform-design-zero-downtime-deploys/internal/release/adopt.go:1273) leaves sleeping instances to run the build on their next wake. |
| + |
| + **Failure:** Manual H sleeps with `last_executed=A`. “Update now” installs B and marks the request completed after stamping. Cold installation leaves `last_executed=A`. The next root pass therefore reinstalls A under rule 3; even if wake happens first, the new held-instance check explicitly runs A instead of B. The update reports completion but never executes. An awake instance whose swap is queued can likewise lose B before spawning it. |
| + |
| + **Smallest fix:** Keep the explicit target pending until execution is admitted. Let that pending target outrank held recovery during wake, and record completion with the mandatory pre-exec build-state write. |
| \ No newline at end of file |