amadan

carlos / platform Public

plan: region HA 1a rev 2, answering Astra plan round 1, with dry-run code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Paul Campbell pushed by claude d4e7d2df729eabba8b656b3cb8e087562b48d44b parent 014b0ce
2 files changed, +360 −55
  • docs/superpowers/plans/2026-10-05-region-ha-1a-foundations.md +314 −55
  • docs/superpowers/plans/CODEX-PLAN-REGION-HA-1A-BRIEF-2.md +46 −0

CI passed — run details

diff --git a/docs/superpowers/plans/2026-10-05-region-ha-1a-foundations.md b/docs/superpowers/plans/2026-10-05-region-ha-1a-foundations.md
index 3321aed1e..34fe9eab8 100644
--- a/docs/superpowers/plans/2026-10-05-region-ha-1a-foundations.md
+++ b/docs/superpowers/plans/2026-10-05-region-ha-1a-foundations.md
@@ -28,6 +28,9 @@ Spec 1 sits on the host lifecycle claim, which exists only as a spec (rev 6.12 o
- No user-facing copy is added here. Log lines are operator-facing and follow the surrounding `agent: …` / `edges: …` style. If a task finds it must add console or CLI copy, stop: that needs the copy-review gate.
- Commit messages: lowercase `area:` prefix and a sentence stating the behaviour, for example `boxid: a box record verifies only under AWS's signature`. End each with `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`.
- Go test environment on this machine: use the default Go build cache. Do not set `GOCACHE`, `GOMODCACHE` or `GOTMPDIR` under `/tmp`. Scratch files go under `$TMPDIR`. Unix-socket tests need a short `TMPDIR`.
+- The branch already contains `origin/main` as of 2026-10-05 (merge 014b0ce1). Before that merge `internal/console`'s tests did not build on this branch; if they fail to build again, merge main first.
+- Local test runs need `GOFLAGS=-buildvcs=false` (some console tests build a binary and fail on VCS stamping in a worktree) and, inside the Claude sandbox, `GOCACHE=$TMPDIR/gocache`. Tests that open Unix sockets (`internal/activator`, `internal/edge`) fail inside the sandbox with `socket: operation not permitted`: that is the sandbox, not the change. Run those packages with the sandbox off, or rely on CI.
+- This plan's code was dry-run on 2026-10-05: every new package and both patch tasks were built, cross-built for darwin and windows, and tested, then removed. Two things the dry run could not do are called out where they matter: the six real-inotify tests (Task 7, step 10) and the IMDS signature check (Task 1, step 7).
- Gate before each commit: `go build ./... && go vet ./... && gofmt -l . && go test ./internal/<pkg>/... -count=1`. Full gate before the branch lands: `make test`.
- Exact values from the spec: continuous deadline **1 second** from the first unpublished modification; repair scan **10 minutes**; polling fallback **1 second**; snapshot interval default **15 minutes**; `files_durability` default **`snapshot`** (H8); capability string **`region-ha-1`**; termination fence accepts **only** state `terminated`.
@@ -107,12 +110,14 @@ Each keeps the spec's property and is recorded in Task 11 as a spec note.
}
func (v Verifier) Verify(rec Record, raw []byte, bind *Bind) (Identity, error)
func Publish(ctx context.Context, s release.Store, rec Record) error // create-only; an identical existing record is success
- func Load(ctx context.Context, s release.Store, box string) (Record, []byte, *Bind, error)
+ func Load(ctx context.Context, s release.Store, box string) (Record, []byte, *Bind, error) // ErrUnverified if the stored record names another box
func EmbeddedCerts() (map[string]*x509.Certificate, error)
var ErrUnverified = errors.New("boxid: record does not verify")
```
`Bind.RecordSHA256` replaces the spec's `record_version_id` (see "Three deliberate departures").
+**Two bindings, both needed.** `Verify` binds the record's `box` field to the signed instance. `Load` binds the storage key to the record's `box` field. Without the second, box A's own valid record copied to `control/boxes/B.json` would load as B and verify as A. `Verify` does not check the region against an expectation: it proves which region the box is in, and the caller compares (Task 3's `eligibleBoxes`, Task 9's `termfence.Verify`).
+
- [ ] **Step 1: Write the failing tests**
`internal/boxid/boxid_test.go`:
@@ -646,6 +651,9 @@ Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
func edgesFileBoxID(instanceID, label string) string
func edgesFileCapabilities(getenv func(string) string) []string
func readHeadroom(dataDir string, awake int) *edgeHeadroom
+ func awakeCount(reg *registry.Registry) int
+ // edgesFilePass keeps its signature and calls this with refresh false:
+ func edgesFileWrite(ctx context.Context, s release.Store, reg *registry.Registry, ident edgesFile, eip string, last *[]byte, refresh bool) error
```
and in `internal/console`, on `edgesFilePlacementView`:
```go
@@ -659,7 +667,7 @@ Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
} `json:"headroom"`
```
-Headroom changes constantly, so it must not defeat the pass's "nothing changed, skip the write" gate. The gate compares a marshalled file with `Generation` blanked; this task blanks `Headroom` in the gate copy too, so headroom is refreshed only when the file is written for another reason or on the loop's five-minute tick.
+Headroom changes constantly, so it must not defeat the pass's "nothing changed, skip the write" gate. The gate compares a marshalled file with `Generation` blanked; this task blanks `Headroom` in the gate copy too. That alone would leave headroom stale for ever on a quiet box, because the loop's timed pass goes through the same gate. So the pass gains one input, `refresh`, true only on the timed pass: a file that carries headroom is then written even if nothing else changed. A box that publishes no headroom behaves exactly as before.
- [ ] **Step 1: Write the failing tests**
@@ -757,6 +765,66 @@ func TestEdgesFileHeadroomAloneDoesNotRewriteTheFile(t *testing.T) {
t.Fatalf("puts = %d, want 1: a headroom change alone is not a reason to write", st.puts)
}
}
+
+// The loop's timed pass refreshes headroom even when nothing else moved.
+func TestEdgesFileTimedPassRefreshesHeadroom(t *testing.T) {
+ st := &putCountingStore{Store: release.NewDirStore(t.TempDir())}
+ reg := edgesTestRegistry(t)
+ seedRoute(t, reg, "app.example.com")
+ var last []byte
+ ident := edgesFile{Box: "annie", Headroom: &edgeHeadroom{MemFreeMiB: 100}}
+ if err := edgesFileWrite(context.Background(), st, reg, ident, "1.2.3.4", &last, false); err != nil {
+ t.Fatal(err)
+ }
+ ident.Headroom = &edgeHeadroom{MemFreeMiB: 42}
+ if err := edgesFileWrite(context.Background(), st, reg, ident, "1.2.3.4", &last, true); err != nil {
+ t.Fatal(err)
+ }
+ if st.puts != 2 {
+ t.Fatalf("puts = %d, want 2: the timed pass must publish the new reading", st.puts)
+ }
+ body, _, err := st.Get(context.Background(), "control/edges/annie.json")
+ if err != nil {
+ t.Fatal(err)
+ }
+ var f edgesFile
+ if err := json.Unmarshal(body, &f); err != nil || f.Headroom == nil || f.Headroom.MemFreeMiB != 42 {
+ t.Fatalf("published headroom = %+v, err %v", f.Headroom, err)
+ }
+ // A box that publishes no headroom keeps today's behaviour exactly:
+ // a timed pass with nothing changed writes nothing.
+ plain := &putCountingStore{Store: release.NewDirStore(t.TempDir())}
+ var plainLast []byte
+ for i := 0; i < 2; i++ {
+ if err := edgesFileWrite(context.Background(), plain, reg, edgesFile{Box: "betty"}, "1.2.3.4", &plainLast, true); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if plain.puts != 1 {
+ t.Fatalf("puts = %d, want 1 for a box with no headroom", plain.puts)
+ }
+}
+
+// A restart must not rewrite an unchanged file just because the stored
+// copy carries a headroom reading.
+func TestEdgesFileSeededGateIgnoresStoredHeadroom(t *testing.T) {
+ st := &putCountingStore{Store: release.NewDirStore(t.TempDir())}
+ reg := edgesTestRegistry(t)
+ seedRoute(t, reg, "app.example.com")
+ var last []byte
+ ident := edgesFile{Box: "annie", Headroom: &edgeHeadroom{MemFreeMiB: 100}}
+ if err := edgesFilePass(context.Background(), st, reg, ident, "1.2.3.4", &last); err != nil {
+ t.Fatal(err)
+ }
+ seeded := seedEdgesFileGate(context.Background(), st, "annie")
+ ident.Headroom = &edgeHeadroom{MemFreeMiB: 7}
+ if err := edgesFilePass(context.Background(), st, reg, ident, "1.2.3.4", &seeded); err != nil {
+ t.Fatal(err)
+ }
+ if st.puts != 1 {
+ t.Fatalf("puts = %d, want 1", st.puts)
+ }
+}
```
Create `internal/host/headroom_test.go`:
@@ -813,12 +881,18 @@ func TestEdgesViewParsesBoxEvidenceAndToleratesItsAbsence(t *testing.T) {
- [ ] **Step 2: Run them to see them fail**
-Run: `go test ./internal/host/ -run 'EdgesFile(BoxID|Capability|RawWireBodyCarries|OmitsBoxEvidence|Headroom)|ParseMemAvailable' -count=1`
+Run: `go test ./internal/host/ -run 'EdgesFile|ParseMemAvailable' -count=1`
Expected: FAIL to build, `undefined: edgesFileBoxID`.
- [ ] **Step 3: Add the fields and helpers to `internal/host/edgesfile.go`**
-After `SecretsKeyFP` (`:228`) in `edgesFile`:
+Add the import (the file does not have it):
+
+```go
+ "github.com/carlosframework/platform/internal/status"
+```
+
+After `SecretsKeyFP` in `edgesFile`:
```go
// BoxID is the name claims use for this box (region HA spec 1 §2): the
@@ -837,7 +911,7 @@ After `SecretsKeyFP` (`:228`) in `edgesFile`:
Headroom *edgeHeadroom `json:"headroom,omitempty"`
```
-Above `edgesFile`:
+Above `edgesFile`'s doc comment:
```go
// edgeHeadroom is this box's spare capacity at the time the file was
@@ -868,16 +942,48 @@ func edgesFileCapabilities(getenv func(string) string) []string {
}
return []string{capRegionHA1}
}
+
+// awakeCount is how many routes this box has running now. An error reads
+// as zero: headroom is advisory.
+func awakeCount(reg *registry.Registry) int {
+ routes, err := reg.List()
+ if err != nil {
+ return 0
+ }
+ n := 0
+ for _, r := range routes {
+ if r.Status == "ready" && !r.Hibernate {
+ n++
+ }
+ }
+ return n
+}
```
-In `edgesFilePass`, change the gate (`:485`) so headroom is not part of it:
+`Hibernate` true means the route sleeps between requests, so a ready route with `Hibernate` false is always running. This under-counts hibernating routes that happen to be awake. That is acceptable here: the field is advisory and spec 2 decides what the chooser really needs.
+
+Split the pass in two. Keep `edgesFilePass`'s signature, so none of its many callers in tests change, and move its body into `edgesFileWrite`:
+
+```go
+func edgesFilePass(ctx context.Context, s release.Store, reg *registry.Registry, ident edgesFile, eip string, last *[]byte) error {
+ return edgesFileWrite(ctx, s, reg, ident, eip, last, false)
+}
+
+// edgesFileWrite is edgesFilePass with one more input. refresh is true on
+// the loop's own timed pass: the file is then written even if nothing but
+// its headroom reading changed, so headroom is never more than one timer
+// period stale. On every other pass a headroom change alone is not a
+// reason to write.
+func edgesFileWrite(ctx context.Context, s release.Store, reg *registry.Registry, ident edgesFile, eip string, last *[]byte, refresh bool) error {
+ // ... the existing body of edgesFilePass, unchanged except for the gate below
+```
+
+In that body, replace the gate:
```go
f := ident
f.V, f.EIP, f.Hosts, f.Generation = 1, eip, hosts, ""
- gateCopy := f
- gateCopy.Headroom = nil // a changing reading is not a reason to write
- gate, err := json.Marshal(gateCopy)
+ gate, err := json.Marshal(f)
if err != nil {
return fmt.Errorf("edges: marshal: %w", err)
}
@@ -886,44 +992,49 @@ In `edgesFilePass`, change the gate (`:485`) so headroom is not part of it:
}
```
-Leave the rest of the function as it is: `body` is still marshalled from `f`, which keeps its headroom.
+with:
-`seedEdgesFileGate` (`:529`) rebuilds the gate from the stored file by blanking `Generation`. Make it blank `Headroom` too, the same way, so a restart does not rewrite an unchanged file. Read the function and add `prior.Headroom = nil` beside the line that clears `Generation`.
+```go
+ f := ident
+ f.V, f.EIP, f.Hosts, f.Generation = 1, eip, hosts, ""
+ gateCopy := f
+ gateCopy.Headroom = nil // a changing reading is not, by itself, a reason to write
+ gate, err := json.Marshal(gateCopy)
+ if err != nil {
+ return fmt.Errorf("edges: marshal: %w", err)
+ }
+ if bytes.Equal(gate, *last) && !(refresh && f.Headroom != nil) {
+ return nil
+ }
+```
+
+Leave the rest as it is: `body` is still marshalled from `f`, which keeps its headroom, and `*last = gate` still stores the headroom-free gate.
-In `edgesFileLoop`, where `ident` is built (`:638`), add the three static fields:
+In `seedEdgesFileGate`, beside the line that clears `Generation`:
```go
- BoxID: edgesFileBoxID(status.InstanceID(), box),
- Capabilities: edgesFileCapabilities(os.Getenv),
+ prior.Generation = ""
+ prior.Headroom = nil // the gate never includes headroom: see edgesFileWrite
```
-and, immediately before each `edgesFilePass` call in the loop, refresh the reading:
+In `edgesFileLoop`, add two fields to the `ident` literal and read the data directory once after it:
```go
- ident.Headroom = readHeadroom(provisionTickConfig(os.Getenv).DataDir, awakeCount(reg))
+ SecretsKeyFP: edgesFileSecretsFP(os.Getenv),
+ BoxID: edgesFileBoxID(status.InstanceID(), box),
+ Capabilities: edgesFileCapabilities(os.Getenv),
+ }
+ dataDir := provisionTickConfig(os.Getenv).DataDir
```
-Add beside the loop:
+and replace the loop's one call, `err = edgesFilePass(ctx, store, reg, ident, eip, &last)`, with:
```go
-// awakeCount is how many routes this box has running now. An error reads
-// as zero: headroom is advisory.
-func awakeCount(reg *registry.Registry) int {
- routes, err := reg.List()
- if err != nil {
- return 0
- }
- n := 0
- for _, r := range routes {
- if r.Status == "ready" && !r.Hibernate {
- n++
- }
- }
- return n
-}
+ ident.Headroom = readHeadroom(dataDir, awakeCount(reg))
+ err = edgesFileWrite(ctx, store, reg, ident, eip, &last, sched.timed)
```
-`Hibernate` true means the route sleeps between requests, so a ready route with `Hibernate` false is always running. This under-counts hibernating routes that happen to be awake. That is acceptable here: the field is advisory and spec 2 decides what the chooser really needs.
+`sched.timed` is the schedule's own flag for "this pass is the timer's" (`edgesFileSchedule`, set in `due`). The loop already reads it two lines above to decide whether to re-read the public IP.
- [ ] **Step 4: Write the headroom readers**
@@ -1031,6 +1142,8 @@ In `internal/console/instancesplacement.go`, after `SecretsKeyFP` in `edgesFileP
} `json:"headroom"`
```
+Run `gofmt -w internal/console/instancesplacement.go`: the new fields change the struct's column alignment, and the gate checks formatting.
+
- [ ] **Step 6: Check the other parsers**
Run: `grep -rn "control/edges\|edgesFileView\|edgesFilePrefix" --include=*.go internal/edgesfile internal/steering internal/edge | grep -v _test`
@@ -1070,8 +1183,9 @@ Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
Awake int
}
// verified maps box id to its verified identity; a box absent from it is never eligible.
+ // awsRegion is the AWS region the caller expects for this placement; "" offers nothing.
// lost is the set of box ids a claim has quarantined.
- func eligibleBoxes(snap edgesSnapshot, verified map[string]boxid.Identity, placement, homeName, expectedFP string, lost map[string]bool) []boxCandidate
+ func eligibleBoxes(snap edgesSnapshot, verified map[string]boxid.Identity, placement, awsRegion, homeName, expectedFP string, lost map[string]bool) []boxCandidate
// running is, per box id, how many hosts claims place there and which apps ("acct/app") they belong to.
type boxLoad struct {
Hosts int
@@ -1083,6 +1197,10 @@ Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
The eligibility predicate is nearest-region's, applied to one box, plus spec 1's additions: a verified record whose instance matches `box_id`, the capability, not draining, not lost.
+**The region is checked against the signature, not the file.** A box's placement code (`uk`) is its own configuration. Its AWS region is in its signed identity. So the caller passes the AWS region it expects for the placement, and a box is eligible only if its signed region and its file's `region` both equal it. Plan 1b decides where the expected region comes from (nearest-region's `regionCandidate.AWSRegion` is a majority vote over the same unsigned files, which is not enough on its own).
+
+**`chooseBox` follows spec §5.2 exactly:** a box that runs no instance of the app is preferred; among boxes equal on that, the fewest claimed hosts; then the lowest box id. It compares whether a box runs the app at all, not how many of it.
+
- [ ] **Step 1: Write the failing tests**
`internal/console/boxchoice_test.go`:
@@ -1386,14 +1504,16 @@ Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
- Test: `internal/edge/homemap_test.go`, `internal/host/edgesfile_test.go`
**Interfaces:**
-- Produces: `edgeHost.Stamp int64 \`json:"stamp,omitempty"\`` (writer), `edgesHostView.Stamp int64 \`json:"stamp,omitempty"\`` (reader), `homeEntry.stamp int64`.
+- Produces: `edgeHost.Stamp int64 \`json:"stamp,omitempty"\`` (writer), `edgesHostView.Stamp int64 \`json:"stamp,omitempty"\`` (reader), `homeEntry.stamp int64`, and a changed signature: `foldHomes(files map[string]edgesFileView, self string) (map[string]homeEntry, []string)`, where the second result is the sorted list of hosts two boxes claim at the same positive stamp. Its one caller is `(*homeMap).Refresh`.
- Nothing sets `Stamp` yet: plan 1b sets it from the claim's `holder.stamp`. Until then every entry is unstamped and today's rule applies unchanged. That is what makes this task safe to land alone.
The rule (spec HA4, §5.4), applied when two files claim one host as a dial target:
1. A stamped entry beats an unstamped one.
2. Between two stamped entries, the higher stamp wins.
3. Between two unstamped entries, today's rule: a route beats an alias, any dial target beats none, otherwise the first box in sorted order.
-4. Equal non-zero stamps cannot happen for two different boxes (a stamp is one claim revision naming one executor). Treat it as rule 3 and log once per refresh.
+4. Equal non-zero stamps cannot happen for two different boxes (a stamp is one claim revision naming one executor). Treat it as rule 3, and log it once per refresh: `foldHomes` returns the hosts it happened to.
+
+**No alert for two unstamped entries.** Spec §5.4 asks for one, on the reasoning that it means the multi-box gate was bypassed. That is wrong for today's files: a static site or a public address route runs on every box, so every box publishes it as `home`, unstamped, and that is normal. The edge cannot tell those from relocatable hosts. Alerting here would fire for every static site on every refresh. This plan reports only equal positive stamps; Task 11 records the correction in the spec, and plan 1b decides whether the publisher should mark relocatable entries so the edge can alert on those alone.
- [ ] **Step 1: Write the failing tests**
@@ -1466,6 +1586,25 @@ func TestHomeMapAStampedEntryWithNoDialAddressDoesNotWin(t *testing.T) {
t.Fatalf("Home = %q %q %v, want annie: a box nothing can dial is skipped today and stays skipped", eip, box, ok)
}
}
+
+func TestHomeMapReportsTwoBoxesAtTheSameStamp(t *testing.T) {
+ files := map[string]edgesFileView{
+ "control/edges/annie.json": {V: 1, Box: "annie", EIP: "1.1.1.1", Hosts: map[string]edgesHostView{
+ "tied.test": {Role: "home", Stamp: 7}, "fine.test": {Role: "home", Stamp: 7}, "plain.test": {Role: "home"}}},
+ "control/edges/betty.json": {V: 1, Box: "betty", EIP: "2.2.2.2", Hosts: map[string]edgesHostView{
+ "tied.test": {Role: "home", Stamp: 7}, "fine.test": {Role: "home", Stamp: 9}, "plain.test": {Role: "home"}}},
+ }
+ hosts, tied := foldHomes(files, "carol")
+ if len(tied) != 1 || tied[0] != "tied.test" {
+ t.Fatalf("tied = %v, want [tied.test]: unstamped duplicates are normal (static sites run on every box) and are not reported", tied)
+ }
+ if hosts["tied.test"].box != "annie" {
+ t.Fatalf("a tie keeps the first box in sorted order; got %q", hosts["tied.test"].box)
+ }
+ if hosts["fine.test"].box != "betty" {
+ t.Fatalf("fine.test box = %q", hosts["fine.test"].box)
+ }
+}
```
Append to `internal/host/edgesfile_test.go`:
@@ -1485,8 +1624,8 @@ func TestEdgesFileStampIsOmittedWhenZero(t *testing.T) {
- [ ] **Step 2: Run them to see them fail**
-Run: `go test ./internal/edge/ -run 'HomeMap(HigherStamp|StampedBeats|UnstampedKeeps|AStamped)' -count=1`
-Expected: `TestHomeMapHigherStampWinsWhateverTheSortOrder` and `TestHomeMapStampedBeatsUnstamped` FAIL (annie wins by sort order); the other three PASS already. `go test ./internal/host/ -run StampIsOmitted` fails to build, `unknown field Stamp`.
+Run: `go test ./internal/edge/ -run 'HomeMap' -count=1`
+Expected: FAIL to build (`TestHomeMapReportsTwoBoxesAtTheSameStamp` uses `Stamp` and `foldHomes`'s second result). Once the field exists and before the fold changes, `TestHomeMapHigherStampWinsWhateverTheSortOrder` and `TestHomeMapStampedBeatsUnstamped` fail (annie wins by sort order) and the other three stamp tests already pass. `go test ./internal/host/ -run StampIsOmitted` fails to build, `unknown field Stamp`.
- [ ] **Step 3: Add the field on both sides**
@@ -1516,7 +1655,20 @@ and in `homeEntry` after `account, app`:
- [ ] **Step 4: Change the fold**
-In `foldHomes`, set the stamp on the incoming entry. Change:
+`foldHomes` gains a second result. Change its signature:
+
+```go
+func foldHomes(files map[string]edgesFileView, self string) (map[string]homeEntry, []string) {
+```
+
+and, where it declares `hosts`, declare the set of ties beside it:
+
+```go
+ hosts := make(map[string]homeEntry)
+ tiedSet := map[string]bool{}
+```
+
+Set the stamp on the incoming entry. Change:
```go
in := homeEntry{eip: f.EIP, box: f.Box, platform: h.Platform && !h.Alias && !h.Wildcard, alias: h.Alias, wildcard: h.Wildcard, dial: h.Role == edgeRoleHome,
@@ -1530,7 +1682,7 @@ to:
account: h.Account, app: h.App, stamp: h.Stamp}
```
-Then replace the merge condition:
+Replace the merge condition:
```go
if in.dial && (!prev.dial || (prev.alias && !in.alias)) {
@@ -1542,15 +1694,26 @@ Then replace the merge condition:
with:
```go
+ if in.dial && prev.dial && in.alias == prev.alias && in.stamp != 0 && in.stamp == prev.stamp && in.box != prev.box {
+ tiedSet[host] = true
+ }
if dialBeats(in, prev) {
merged.eip, merged.box, merged.alias, merged.dial = in.eip, in.box, in.alias, true
merged.account, merged.app, merged.stamp = in.account, in.app, in.stamp
}
```
-and add below `foldHomes`:
+Replace the function's final `return hosts` with:
```go
+ tied := make([]string, 0, len(tiedSet))
+ for h := range tiedSet {
+ tied = append(tied, h)
+ }
+ slices.Sort(tied)
+ return hosts, tied
+}
+
// dialBeats reports whether a later file's entry takes the dial from the
// one already folded. In order:
//
@@ -1577,7 +1740,26 @@ func dialBeats(in, prev homeEntry) bool {
}
```
-An entry with no dial address never reaches `dialBeats` as a dial target: the existing `case edgeRoleHome: if f.EIP == "" { … continue }` skips it first. That is what `TestHomeMapAStampedEntryWithNoDialAddressDoesNotWin` pins.
+In `(*homeMap).Refresh`, replace the one call:
+
+```go
+ snap := &homeSnapshot{files: next, hosts: foldHomes(next, m.self)}
+```
+
+with:
+
+```go
+ hosts, tied := foldHomes(next, m.self)
+ if len(tied) > 0 {
+ // One claim revision names one executor, so two boxes publishing
+ // one host at the same stamp is a fault upstream. The fold stays
+ // deterministic (first box in sorted order); say so once a refresh.
+ log.Printf("edge: home map: %d host(s) claimed by two boxes at the same installation stamp, e.g. %s: keeping the first box in sorted order", len(tied), tied[0])
+ }
+ snap := &homeSnapshot{files: next, hosts: hosts}
+```
+
+`homemap.go` already imports `log` and `slices`. An entry with no dial address never reaches `dialBeats` as a dial target: the existing `case edgeRoleHome: if f.EIP == "" { … continue }` skips it first. That is what `TestHomeMapAStampedEntryWithNoDialAddressDoesNotWin` pins.
- [ ] **Step 5: Run the tests**
@@ -4656,11 +4838,12 @@ func LoadWake(ctx context.Context, s release.Store, prefix string, m Manifest) (
- [ ] **Step 7: Check the registry facts the code relies on**
-Run: `grep -n "func ValidReplication" -A8 internal/registry/registry.go; grep -n "func (r \*Registry) SetReplication" -A12 internal/registry/registry.go`
+The dry run confirmed both, against main at 014b0ce1; confirm them again if `registry.go` has changed since:
+
+- `registry.ValidReplication("")` is true: `""` is the default rung.
+- `(*Registry).SetReplication(host, "")` succeeds and clears the class. `ImportWake` calls it unconditionally, so importing the default class onto a route that had `daily` leaves it on the default. `TestImportWakeOfTheDefaultClassReplacesAnotherClass` pins that.
-Confirm two things and adjust if either is false:
-- `ValidReplication("")` returns true (`""` is the pre-ladder default). If it returns false, change the check in `ImportWake` to `ws.Replication != "" && !registry.ValidReplication(ws.Replication)`.
-- `SetReplication` on a route with `Hibernate` false: the report says Upsert clears `Replication` when `!t.Hibernate`. If `SetReplication` errors for such a route, `TestWakeStateRoundTripsThroughTheManifest` already seeds `Hibernate: true`; keep it, and make `ImportWake` skip `SetReplication` when the route's `Hibernate` is false (read it with `reg.Get`), with a comment saying why.
+Run: `grep -n "func ValidReplication" -A8 internal/registry/registry.go; grep -n "func (r \*Registry) SetReplication" -A14 internal/registry/registry.go`
- [ ] **Step 8: Run the tests**
@@ -5382,18 +5565,35 @@ resource "aws_cloudwatch_event_target" "terminations" {
# Only this rule may put a message on the queue. A message is evidence, so
# who can write one matters as much as who can read one.
+#
+# The Allow alone is not enough: a principal in this account with its own
+# sqs:SendMessage grant (an admin role, a wildcard policy) could still post
+# a forged event, because an identity policy and a queue policy add up.
+# The Deny closes that. It matches every send whose source is not this
+# rule, including one with no source at all, which is what a direct
+# SendMessage from a role looks like.
resource "aws_sqs_queue_policy" "terminations" {
queue_url = aws_sqs_queue.terminations.id
policy = jsonencode({
Version = "2012-10-17"
- Statement = [{
- Sid = "OnlyTheTerminationRule"
- Effect = "Allow"
- Principal = { Service = "events.amazonaws.com" }
- Action = "sqs:SendMessage"
- Resource = aws_sqs_queue.terminations.arn
- Condition = { ArnEquals = { "aws:SourceArn" = aws_cloudwatch_event_rule.terminations.arn } }
- }]
+ Statement = [
+ {
+ Sid = "TheTerminationRuleMaySend"
+ Effect = "Allow"
+ Principal = { Service = "events.amazonaws.com" }
+ Action = "sqs:SendMessage"
+ Resource = aws_sqs_queue.terminations.arn
+ Condition = { ArnEquals = { "aws:SourceArn" = aws_cloudwatch_event_rule.terminations.arn } }
+ },
+ {
+ Sid = "NobodyElseMaySend"
+ Effect = "Deny"
+ Principal = "*"
+ Action = "sqs:SendMessage"
+ Resource = aws_sqs_queue.terminations.arn
+ Condition = { ArnNotEquals = { "aws:SourceArn" = aws_cloudwatch_event_rule.terminations.arn } }
+ },
+ ]
})
}
@@ -5413,6 +5613,12 @@ resource "aws_iam_role_policy" "host_terminations" {
First run `grep -n 'data "aws_caller_identity"' infra/modules/region-host/*.tf`. If the module already declares one, delete the declaration above (it is not otherwise used here) rather than adding a second.
+`tofu validate` cannot prove a policy's effect. Add this to the module README's new section (Step 6) as the check to run once after the first apply, from an administrator's session, and record its result on the discussion that tracks the roll:
+
+ aws sqs send-message --queue-url <termination_queue_url> --message-body '{"forged":true}'
+
+Expected: `AccessDenied`. If it succeeds, the queue is not evidence: stop, and do not enable the placer (spec 3) against it. Then terminate a throwaway instance in the region and confirm its event arrives with `aws sqs receive-message`.
+
The box role already has `ec2:DescribeInstances` on `*` for the infra inventory (`main.tf`, the read-only statement in `host_buckets`). Spec §6 asks for an `ec2:Region` condition on it. Do **not** narrow the existing statement in this task: the inventory reads other regions. Record it as an open item in the README text below; plan 1b decides it with the placer.
- [ ] **Step 3: Add the outputs**
@@ -5525,6 +5731,14 @@ event whenever an EC2 instance in this account and region reaches
instances run elsewhere. Only the module's own EventBridge rule can write
to the queue, and the box role can only read and delete from it.
+After the first apply, check that nobody but the rule can write to it.
+From an administrator's session:
+
+ aws sqs send-message --queue-url <termination_queue_url> --message-body '{"forged":true}'
+
+must fail with `AccessDenied`. Then terminate a throwaway instance in the
+region and confirm its event arrives.
+
The root volume is deleted when the instance is terminated. Region HA
depends on that: a terminated box must leave no disk behind.
@@ -5563,6 +5777,22 @@ and changes three mechanisms:
therefore has no `base` or `base_seq`, and restore reads one manifest
and the objects it names.
+Two corrections to the spec, found while planning:
+
+- **§5.4's alert for two unstamped entries is dropped.** Static sites and
+ public address routes run on every box, so every box publishes them as
+ `home`, unstamped: that is normal, and the edge cannot tell them from
+ relocatable hosts. The edge reports only two boxes at the same positive
+ stamp. Plan 1b decides whether the publisher marks relocatable entries
+ so the edge can alert on those alone.
+- **§5.1's region check has two halves.** `boxid.Verify` proves which
+ region a box is in. Whoever uses that compares it with the region it
+ expects: `eligibleBoxes` takes the placement's AWS region and refuses a
+ box whose signed region differs, and `termfence.Verify` compares the
+ observation's region with the signed one. Plan 1b must supply the
+ expected region from something better than a majority vote of the
+ unsigned edges files.
+
Two limits to lift in plan 1b: a file over 64 MiB is recorded in the
manifest as skipped and is not published (the store takes a whole body in
memory), and the third termination-evidence source (Auto Scaling activity
@@ -5608,4 +5838,33 @@ Written when `internal/claim` and `internal/hostguard` are on main, after re-rea
6. The agent publishing its box record at start, and `carlos ops box bind` (§5.1), with `docs/site/cli.md`.
7. The `relocating` operation, the release sequence and the pinned recovery set (§5.8).
8. The multi-box gate and the HA floor (§8).
-9. A streaming put, lifting the 64 MiB limit; the data bucket's conditional-write policy for the files prefix (§6).
+9. A streaming put, lifting the 64 MiB limit; the data bucket's conditional-write policy for the files prefix (§6), in place before the publisher is switched on for any host.
+10. Where the expected AWS region for a placement comes from (Task 3), and whether relocatable entries are marked so the edge can alert on unstamped duplicates of those alone (Task 4).
+
+## Astra plan round 1: dispositions
+
+Findings are in `CODEX-PLAN-REGION-HA-1A-R1.md`. All nineteen are accepted. After them the plan's code was dry-run: built, cross-built and tested, then removed.
+
+| # | Finding | Disposition |
+|---|---|---|
+| 1 | P1 Task 2: `status` not imported | fixed: the import is a named step |
+| 2 | P1 Task 1: storage key not bound to the record | fixed: `Load` refuses a record that names another box; `TestLoadRefusesAnotherBoxesRecordUnderThisKey` |
+| 3 | P1 Tasks 1, 3: a wrong but supported region accepted | fixed: `eligibleBoxes` takes the expected AWS region and compares the signed one; test with two certified regions |
+| 4 | P1 Task 6: `readStable` follows a swapped ancestor, blocks on a swapped FIFO | fixed: `os.Root`, a non-blocking open, identity check after open; deterministic tests through a seam |
+| 5 | P1 Task 5: restore writes or deletes through a symlink | fixed: every restore operation goes through an `os.Root`; symlinked ancestors are replaced or left alone, never followed |
+| 6 | P1 Task 11: queue policy does not exclude other senders | fixed: an explicit Deny for any other source; a post-apply check in the README |
+| 7 | P2 Task 5: directory and file transitions | fixed: tombstones first, deepest first; obstructions removed; tested both ways into empty and previously restored directories |
+| 8 | P2 Task 7: overflow leaves a directory unwatched | fixed: overflow marks coverage degraded until `Repair` succeeds; tested |
+| 9 | P2 Task 7: a dead watcher disables polling | fixed: a closed channel switches the loop to polling for good; `Repair` fails on a dead watcher; tested |
+| 10 | P2 Task 2: headroom never refreshed on the timer | fixed: `edgesFileWrite(…, refresh)`; the timed pass writes when the file carries headroom |
+| 11 | P2 Task 8: default replication class not applied | fixed: `SetReplication` is always called; tested |
+| 12 | P2 Task 3: `chooseBox` used a different spread rule | fixed: runs-the-app or not, then total hosts, then id |
+| 13 | P2 Task 6: a skipped directory's children tombstoned | fixed: `skipped` checks every ancestor in both passes; tested |
+| 14 | P2 Task 7: the first scan masked the tests | fixed: `startRun` waits for the first scan; event tests run with scans and polling an hour apart and assert nothing is published before the event |
+| 15 | P2 Task 6: Review Focus 2 and 3 partly tested | fixed: FIFO, swapped FIFO, swapped ancestor, symlinked directory; the oversize test counts reads and expects none |
+| 16 | P2 Task 8: the all-or-nothing test never reached the transaction | fixed: duplicate names fail the second insert inside the transaction |
+| 17 | P2 Task 9: a test passed for the wrong reason | fixed: a found, terminated, all-empty observation against an empty identity |
+| 18 | P2 Task 4: conflict diagnostics missing | fixed for equal positive stamps (`foldHomes` returns them, `Refresh` logs once). The unstamped alert is dropped as wrong, with the reason, and recorded as a spec correction |
+| 19 | P3 Task 5: storage protection attributed to Task 11 | fixed: it is plan 1b's, and must precede switching the publisher on |
+
+**What the dry run could not check**, and the plan says so at each place: the six tests that need a real inotify instance (the development machine is at its per-user limit of 128), and the IMDS signature against a real box.
diff --git a/docs/superpowers/plans/CODEX-PLAN-REGION-HA-1A-BRIEF-2.md b/docs/superpowers/plans/CODEX-PLAN-REGION-HA-1A-BRIEF-2.md
new file mode 100644
index 000000000..06d49e331
--- /dev/null
+++ b/docs/superpowers/plans/CODEX-PLAN-REGION-HA-1A-BRIEF-2.md
@@ -0,0 +1,46 @@
+# Review brief: region HA plan 1a (foundations), round 2
+
+Read `CODEX-PLAN-REGION-HA-1A-BRIEF-1.md` first: its "Fixed: do not
+re-argue" and "What to check" sections still apply, to the plan at the
+commit named in the prompt.
+
+## Round 2
+
+This revision answers your round 1 (`CODEX-PLAN-REGION-HA-1A-R1.md`); the
+plan ends with a dispositions table. Since round 1 the plan's code was
+dry-run by its author: every new package and both patch tasks were
+written into the tree, built, cross-built for darwin and windows and
+tested, then removed. The code blocks for `internal/boxid`,
+`internal/hostfiles`, `internal/termfence`, `internal/activator/installed*`
+and `internal/console/boxchoice*` are those files verbatim. So a
+"would not compile" finding should be rare: if you raise one, quote the
+line and say exactly what is undefined.
+
+First give a disposition table for your nineteen round 1 findings:
+resolved, partly resolved (say what is left), or not resolved. Then
+report only new findings, including any the fixes introduce. In
+particular look at:
+
+- `readStable`, `syncRoot`, `Restore`, `removeInside`, `writeInside` and
+ their use of `os.Root` (Go 1.26): any remaining way out of the root,
+ any way to block, any path where a tombstone or a write is skipped
+ that should not be, or the reverse.
+- The run loop's degraded, repair and dead-watcher handling, and whether
+ each test can pass with its mechanism disabled.
+- `edgesFileWrite`'s `refresh` and the seeded gate.
+- `foldHomes`'s tie reporting and `dialBeats` against every existing
+ home-map rule.
+- The SQS queue policy's Deny: does it block the EventBridge rule's own
+ delivery, or anything else that must work?
+- The plan's claim that an alert on two unstamped home entries would be
+ wrong today. Check it against `edgesFilePass` and `everywhereClass`.
+
+Two things the author could not run and says so: the six real-inotify
+tests in `watch_linux_test.go` (read `watch_linux.go` with extra care)
+and the IMDS signature check.
+
+Do not re-raise anything you mark resolved.
+
+## Output
+
+As round 1. End with a verdict: ready to build, or not ready.