amadan
spec: region HA spec 1, Astra round 1 findings (unedited)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+110
−0
docs/superpowers/specs/CODEX-FINDINGS-REGION-HA-1.md+110 −0
diff --git a/docs/superpowers/specs/CODEX-FINDINGS-REGION-HA-1.md b/docs/superpowers/specs/CODEX-FINDINGS-REGION-HA-1.md| new file mode 100644 |
| index 000000000..22dc77460 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-FINDINGS-REGION-HA-1.md |
| @@ -0,0 +1,110 @@ |
| +Reviewed [spec 1 at `2f5e5d0c`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/region-ha/docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md), against claim rev 6.3 (`8faa89a4`) and moves rev 7.5 (`cdc98845`). H1–H6 remain fixed. |
| + |
| +1. **P1 — §§2, 5.1, 5.9; K6: termination evidence is not bound to the executor’s identity.** |
| + |
| + **Problem:** Legacy claims retain labels, but the only specified label-to-instance mapping is the edges file. Reading EC2 independently verifies the supplied instance’s state; it does not prove that instance is the claim’s executor. The fence also omits the AWS account and deployment identity. |
| + |
| + **Failing sequence:** A legacy executor `annie-welding` is alive but partitioned. A confused or forged edges file associates that label with a different, terminated instance. The placer obtains genuine `terminated` evidence for that instance and transfers execution. Both the original executor and its replacement can run. |
| + |
| + **Fix:** Persist an authenticated, immutable binding between the claim’s box identity and its AWS partition/account/region/instance ID. Validate the fence against that binding, the current operation and the expected executor. Apply `lost` consistently to legacy aliases. A positive terminal observation can satisfy H1; an unverified identity mapping cannot. |
| + |
| +2. **P1 — §§5.6–5.8; K8: relocation selects a new generation without populating it.** |
| + |
| + **Problem:** Step 3 changes `holder.generation`; step 4 restores the *current* generation. No step copies or reconstructs the predecessor’s durable database and files into that generation. Moves §5.1 explicitly forbids restoring another generation and requires a verified installation and generation marker. |
| + |
| + **Failing sequence:** A has a complete replica in generation G. A terminates. The placer changes the claim to B/G2. B restores G2, which contains no database snapshot or file baseline. It must refuse, or incorrectly start empty. |
| + |
| + **Fix:** Preserve G and its selected object versions in the operation journal. Build and verify G2 from that pinned recovery set, including files, env and required markers, before publishing it as the installed generation. Extend K8 to seed file durability on every move/relocation installation; unchanged files must remain recoverable after the next failure. |
| + |
| +3. **P1 — §5.8: replay from `{from, to, reason}` does not protect an installation that has already written.** |
| + |
| + **Problem:** The relocation operation has no durable phase/admission table or “never reinstall after probing” boundary. Naming a recovery owner does not supply these rules. Claim C2 requires replay rules; moves explicitly journals this boundary. |
| + |
| + **Failing sequence:** B restores, starts its probe and replicates new writes. The placer dies before releasing the claim to `serving`. Its replacement replays installation from `intent` and restores the original transfer over B’s newer state. A paused earlier worker has the same problem if it resumes an installation step within the still-current operation. |
| + |
| + **Fix:** Define the relocation phases, allowed actors and admission permissions. Persist installation completion before probing, make replay preserve subsequent writes, and require phase-specific revalidation and worker exclusion. Define recovery when a participant terminates during relocation, since a second relocation cannot start from `relocating`. |
| + |
| +4. **P1 — §5.6: versioning does not establish file-update or tombstone order.** |
| + |
| + **Problem:** “Latest snapshot, then any continuous objects newer than it” has no defined logical ordering or committed restore manifest. S3 version IDs identify versions; they do not establish which application update supersedes another. |
| + |
| + **Failing sequence:** Upload U1 for `instance.key` stalls. U2 uploads the replacement successfully. U1 then completes and becomes the current object. A later restore selects the old key. Likewise, a delayed upload can overtake a deletion tombstone and resurrect a file. Snapshot upload time also need not equal the point at which its contents were captured. |
| + |
| + **Fix:** Specify ordered mutations, immutable content versions and a conditionally published manifest/checkpoint that names the winning versions and tombstones. Give each snapshot a mutation boundary. Define the drain flush against that boundary, rather than object timestamps. |
| + |
| +5. **P2 — §§3 HA3, 5.5: `active_stamp` is not proof that local data finished installing.** |
| + |
| + **Problem:** Claim §6.4 initializes or refreshes `active_stamp` from the current claim, including after loss of the local record. That authorizes local work; it does not attest to the database and files on disk. |
| + |
| + **Failing sequence:** B initializes its local record for a new installation, then crashes before replacing its previous files. On restart, `active_stamp == holder.stamp`, so §5.5 skips restore. Existing moves markers may refuse the eventual spawn, but the proposed warm-cache decision still incorrectly accepts the copy. |
| + |
| + **Fix:** Require a durable installation-completion record tied to the incarnation, generation, database and file markers. Write it only after installation is durable. Keep authorization-stamp initialization separate from evidence that cached data is usable. |
| + |
| +6. **P2 — §5.3; K2: executor-only reconciliation conflicts with participant rows, and CC10 does not remove former executors.** |
| + |
| + **Problem:** Moves requires held rows for participants that are not yet executor. Conversely, claim CC10 only opens a stale hold on the **current executor of a serving host**. It cannot release the former executor’s row as §5.3 claims. |
| + |
| + **Failing sequence:** During A→B preparation, B is a participant but `exec` still names A. The executor-only mint rule prevents B’s required held row. After relocation, A’s row also cannot be removed through CC10 because A is no longer executor. |
| + |
| + **Fix:** Distinguish permission to keep a held participant row from permission to execute. Specify former-executor row removal separately from local retirement and stale-hold reopening. Add the changed reconciliation contract—including the slow-sweep exception to C4—to §9. |
| + |
| +7. **P2 — §5.7: timer state cannot all be reconstructed from schedules.** |
| + |
| + **Problem:** The existing system also has dynamically registered once-timers, delivery/retry state and application-provided `NextDue`. Moves §§6.2 and 9.4 carry these explicitly. They are not all manifest schedules. |
| + |
| + **Failing sequence:** An app registers a once-timer successfully, then its box dies hours later. The replacement reconstructs declared schedules but has no record of that timer. It never fires. Even a declared cron occurrence during the outage is skipped by today’s converger, which computes `Next(now)`. |
| + |
| + **Fix:** Define durable storage and restoration for timer registrations, pending occurrences and wake indexes. Preserve them during graceful relocation and specify failover recovery. “At least once” permits duplicate delivery; it does not cover permanently losing the registration. |
| + |
| +8. **P2 — §§3 HA1/HA5, 5.7–5.8: the rebuild contract does not cover unit- and socket-backed hosts.** |
| + |
| + **Problem:** The scope includes these hosts, but rendering config and materializing a release does not recreate an externally installed service or socket unit. The existing record contains a unit name, not its definition; moves deliberately refuses such routes. |
| + |
| + **Failing sequence:** The console’s box dies. Its route names `console.service`. A survivor restores the listed artifacts and registry row but has no corresponding unit definition or launch configuration. The console remains down despite successful placement. |
| + |
| + **Fix:** Specify the reconstructible process and unit configuration for each admitted serving class, with its installation and readiness contract. Require that evidence before treating a box as eligible for that host. |
| + |
| +9. **P2 — §8; K5: adoption alone is insufficient to admit a second box.** |
| + |
| + **Problem:** The rollout checks that claims exist but does not require an HA capability floor or activation of `CARLOS_REGION_HA`. Claim-aware software need not understand relocation or stamped home selection. |
| + |
| + **Failing sequence:** Every host is adopted. A second box passes the join check while an ingress box still uses the old sorted-name home map. After relocation, that ingress continues choosing the terminated box’s stale file over the replacement’s stamped file. |
| + |
| + **Fix:** Gate multi-box admission on the required HA capabilities being enabled across executors, placers and ingress readers. Enforce the corresponding executable floor on rollback and boot recovery. Define handling of legacy unstamped files within that rollout. |
| + |
| +10. **P2 — §9: moves still lacks a multi-box target-selection contract.** |
| + |
| + **Problem:** K1 updates create placement, while K7 and K8 update move fencing and files. None specifies how a move chooses and durably binds its target box among several eligible boxes. Moves names source/target participants but assumes the region identifies the target box. |
| + |
| + **Failing sequence:** A move targets UK, where B and C are eligible. One worker performs readiness against B; after restart another resolves the target to C. Reusing B’s readiness inputs for C is invalid, while refusing the mismatch leaves recovery without a specified choice. |
| + |
| + **Fix:** Add a moves change requiring one chosen target identity in the starting claim CAS, with readiness, evidence, installation and recovery bound to it. Target replacement must be an explicit fenced transition. |
| + |
| +11. **P2 — §5.9: termination evidence can disappear before the placer observes it.** |
| + |
| + **Problem:** EC2 does not retain terminated instances indefinitely. AWS says they normally remain visible for less than an hour; `NotFound` is not termination evidence. [DescribeInstances](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html), [EC2 eventual consistency](https://docs.aws.amazon.com/ec2/latest/devguide/eventual-consistency.html). |
| + |
| + **Failing sequence:** The sole box terminates and the replacement placer becomes operational after the old instance disappears from EC2’s results. No termination fence was persisted. The required positive observation can no longer be obtained, so automatic recovery remains blocked. |
| + |
| + **Fix:** Specify durable capture of verified termination evidence and a defined recovery path when capture was missed. Explicitly refuse missing results, API errors and wrong-account/region lookups as evidence. |
| + |
| +12. **P2 — §6: the proposed Describe permissions cannot be restricted by resource tags as written.** |
| + |
| + **Problem:** `ec2:DescribeInstances` and `autoscaling:DescribeAutoScalingGroups` do not support the proposed resource-tag authorization. AWS lists no resource type for either action; EC2 exposes `ec2:Region` for its Describe action. [EC2 authorization reference](https://docs.aws.amazon.com/service-authorization/latest/reference/list_ec2.html), [Auto Scaling authorization reference](https://docs.aws.amazon.com/service-authorization/latest/reference/list_autoscaling.html). |
| + |
| + **Failing sequence:** The policy grants these actions only when the deployment resource tag matches. The required condition is unavailable, so the placer receives authorization failures and cannot establish fences. |
| + |
| + **Fix:** Specify supported read permissions and region restrictions. Validate deployment membership in the verifier against trusted identity records and returned metadata; describe that separately from IAM enforcement. |
| + |
| +13. **P3 — §§1, 4, 5.4: three code citations need correction.** |
| + |
| + **Problem and failing sequence:** Following the cited lines reaches blank lines or an unrelated struct comment rather than the claimed declaration/writer. |
| + |
| + **Fix:** |
| + - `internal/lease/lease.go:38` → `:37` for `LeaseTTL`. |
| + - `internal/provision/reconcile.go:195` → `:202` for `everywhereClass`. |
| + - `internal/host/edgesfile.go:185` → `:272` for `edgesFilePass`, or `:326` for publishing `role: "home"`. |
| + |
| + The other code citations support their stated context. |
| + |
| +**Verdict: not ready.** |
| \ No newline at end of file |