carlos / platform Public

plan: region HA 1b answers Astra round 1: stamped aliases, deferred participation, validated volume records, fresh claim reads

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Paul Campbell pushed by claude agent, operated by paul@keymail.dev 161728abef1ffa96bf4139a8e72001cb1f343f2c parent debca27
2 files changed, +592 −65
  • docs/superpowers/plans/2026-10-06-region-ha-1b-claim-readers.md +522 −65
  • docs/superpowers/plans/CODEX-PLAN-REGION-HA-1B-BRIEF-2.md +70 −0

CI passed — run details

diff --git a/docs/superpowers/plans/2026-10-06-region-ha-1b-claim-readers.md b/docs/superpowers/plans/2026-10-06-region-ha-1b-claim-readers.md
index 1467803e8..821c32955 100644
--- a/docs/superpowers/plans/2026-10-06-region-ha-1b-claim-readers.md
+++ b/docs/superpowers/plans/2026-10-06-region-ha-1b-claim-readers.md
@@ -49,9 +49,9 @@ Three of the candidates the brief named were checked against the spec and main a
- **Go environment.** Run Go with `GOCACHE=$TMPDIR/gocache GOFLAGS="-mod=mod -buildvcs=false"` (every command below assumes both are exported). Tests that open Unix sockets fail inside the Claude sandbox with `socket: operation not permitted`; on this branch's base those are, in `internal/host`, `TestSleepingInstanceRearmsItsNextDeliveryBeforeReturning`, `TestScheduledTickWakesASleepingInstanceAndItHibernatesAgain`, `TestDeliveryReusesTheConnectionForASocket`, `TestArmScheduleFiresOnceTimersWithNoDeployStore`, `TestProxyCacheIsStableAndBounded`, `TestDeliveryDoesNotHoldTheInstanceAwake`, `TestDeliveryStillCarriesScheduleAtHeader` and `TestSocketServingProbe`; in `internal/activator`, the `TestExecBackendStart…`, `TestStartUnit…`, `TestUnitProc…`, `TestSeedUnit…`, `TestStop…` and `TestPrepareRemovesAStaleSocketFile` tests; and `internal/edge` as a whole. That is the sandbox, not the change: run those packages with the sandbox off, or rely on CI. Scratch files go under `$TMPDIR`. Never use `git stash`, `reset --hard`, `checkout --` or `clean`.
- **OpenTofu.** `tofu test` starts the AWS provider over a local socket, so it needs the sandbox off, and `tofu init` needs network access to `registry.opentofu.org` and GitHub.
- Gate before each commit: `go build ./... && go vet ./<pkgs>/ && gofmt -l <dirs> && go test ./<pkgs>/ -run '<tests>' -count=1`, then `GOOS=darwin go build ./... && GOOS=windows go build ./...`. Full gate before the branch lands: `go build ./... && make test`, and the three cross-builds.
-- **Exact values.** Claim read cache on the edges file **10 minutes** (H9's sweep); at most **256** claim reads per edges pass; a volume record at least once an **hour** when nothing changes; the first volume record must be taken within **1 hour** of enrollment; the newest within **2 hours** before the termination observation; capability **`region-ha-1`**; fence state **`terminated`** only; root's marker **`/opt/carlos/system/guard/update-begun.json`**; controller records **`<DataDir>/.hostexec/`**; volume records **`control/box-volumes/<enrollment>/<observed_at>.json`**.
+- **Exact values.** Claim read cache on the edges file **10 minutes** (H9's sweep); at most **256** claim reads per edges pass, with an extra pass every **30 seconds** while reads are due; a volume record at least once an **hour** when nothing changes; the first volume record must be taken within **1 hour** of enrollment; the newest within **2 hours** before the termination observation; capability **`region-ha-1`**; fence state **`terminated`** only; root's marker **`/opt/carlos/system/guard/update-begun.json`**; controller records **`<DataDir>/.hostexec/`**; volume records **`control/box-volumes/<enrollment>/<observed_at>.json`**.
- Work on a branch cut from `origin/main` (suggested name `region-ha-1b`). Merging is Paul's call, through `amadan branch merge`.
-- This plan's code was dry-run on 2026-10-06: every task was applied in order to a clean copy of `origin/main` (bf813540), built, vetted, cross-built for darwin, windows and linux/arm64, and its tests run; the five Review Focus tests and five others were each watched to fail under a mutation of the code they pin; Task 8's tofu test was run against the AWS provider's mock and watched to fail against main's policy. Then the code was removed. What the dry run could not do is listed at the end.
+- This plan's code was dry-run on 2026-10-06: every task was applied in order to a clean copy of `origin/main` (bf813540), built, vetted, cross-built for darwin, windows and linux/arm64, and its tests run; the five Review Focus tests and five others were each watched to fail under a mutation of the code they pin; Task 8's tofu test was run against the AWS provider's mock and watched to fail against main's policy. After Astra's plan round 1 the plan was dry-run again from its own text (every task applied in order to a clean tree by a script that reads the plan), and each round 1 fix's test was watched to fail with its mechanism removed, including five mutations of the bucket policy. Then the code was removed. What the dry run could not do is listed at the end.
## Review Focus
@@ -1973,10 +1973,11 @@ Spec §5.9 under H19: regional workers record each enrolled box's block device m
func Discharge(records []Record, e Entry, enrolledAt, terminatedAt time.Time) Verdict
```
-**Four choices the spec leaves to the plan.**
+**Five choices the spec leaves to the plan.**
- A `terminated` read is never recorded. Its mappings are empty (the dev test of 2026-10-06), and a record of it would list every volume as removed.
- "The first record taken at enrollment" is checked as: the earliest record for this enrollment must be observed within `EnrollmentWindow` (one hour, the refresh period) of the roster's enrollment time. A later first record cannot speak for a volume attached and removed before it.
- The newest record must be no more than `LastRecordWindow` (2 hours) older than `terminatedAt`, the fence's `observed_at` (controller ruling 1). A recorder that stopped long before the box died cannot speak for its last hours.
+- `Discharge` checks every stored record of this enrollment before using it (Astra plan R1-6): a version other than 1, a state that carries no mapping (`terminated`, or none), an empty mapping, or a volume with no id or device keeps the chores open. `List` returns whatever is stored, so a record `Write` would never produce is still met here.
- `Discharge` recomputes removals from consecutive records as well as reading `removed_volumes`, so a record one worker failed to write cannot hide a removal.
Keys use a fixed-width UTC time with `-` in place of `:`, so they sort in observation order.
@@ -2198,6 +2199,61 @@ func TestDischargeNeedsARecentLastRecord(t *testing.T) {
t.Fatalf("a newest record older than the window discharged: %+v", v)
}
}
+
+// Astra plan R1-6: what List returns is whatever is stored. A stored record
+// that is not evidence of a running instance's mappings keeps the chores
+// open, however it got there. Each bad record sits beside a good one so
+// that no removal is seen between them: only the record's own check can
+// refuse it.
+func TestDischargeRefusesStoredRecordsThatAreNotEvidence(t *testing.T) {
+ ctx := context.Background()
+ good := records(t, []vol{root})
+ const before = `"enrollment":"enr-annie-1","cloud":{"provider":"aws","account":"111122223333","region":"eu-west-1","instance_id":"i-0abc"},"observed_at":"2026-10-06T08:55:00Z"`
+ const after = `"enrollment":"enr-annie-1","cloud":{"provider":"aws","account":"111122223333","region":"eu-west-1","instance_id":"i-0abc"},"observed_at":"2026-10-06T09:05:00Z"`
+ type bad struct {
+ body string
+ at time.Duration // from t0; the good record is at t0
+ }
+ cases := map[string]bad{
+ // Before the good record: an empty mapping followed by one volume is
+ // an attach, not a removal.
+ "only the binding fields": {`{"v":1,` + before + `}`, -5 * time.Minute},
+ "a terminated read": {`{"v":1,` + before + `,"instance_state":"terminated","block_devices":[],"removed_volumes":[]}`, -5 * time.Minute},
+ // After it, keeping vol-root, so nothing is removed.
+ "a volume with no id": {`{"v":1,` + after + `,"instance_state":"running","block_devices":[{"device":"/dev/xvda","volume_id":"vol-root","delete_on_termination":true,"status":"attached"},{"device":"/dev/xvdb","volume_id":"","delete_on_termination":true,"status":"attached"}],"removed_volumes":[]}`, 5 * time.Minute},
+ "a version 2 record": {`{"v":2,` + after + `,"instance_state":"running","block_devices":[{"device":"/dev/xvda","volume_id":"vol-root","delete_on_termination":true,"status":"attached"}],"removed_volumes":[]}`, 5 * time.Minute},
+ }
+ for name, tc := range cases {
+ t.Run(name, func(t *testing.T) {
+ st := release.NewDirStore(t.TempDir())
+ if _, err := Write(ctx, st, good[0]); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := st.Put(ctx, Key(entry.Enrollment, t0.Add(tc.at)), []byte(tc.body), ""); err != nil {
+ t.Fatal(err)
+ }
+ recs, err := List(ctx, st, entry.Enrollment)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(recs) != 2 {
+ t.Fatalf("listed %d records, want 2", len(recs))
+ }
+ if v := Discharge(recs, entry, t0, t0.Add(time.Hour)); v.Discharge || v.Reason == "" {
+ t.Fatalf("verdict = %+v, want the chores left open", v)
+ }
+ })
+ }
+ // The good record alone, stored and listed, still discharges.
+ st := release.NewDirStore(t.TempDir())
+ if _, err := Write(ctx, st, good[0]); err != nil {
+ t.Fatal(err)
+ }
+ recs, _ := List(ctx, st, entry.Enrollment)
+ if v := Discharge(recs, entry, t0, t0.Add(time.Hour)); !v.Discharge {
+ t.Fatalf("a stored good record: %+v", v)
+ }
+}
```
- [ ] **Step 2: Run them to see them fail**
@@ -2486,6 +2542,25 @@ func List(ctx context.Context, s release.Store, enrollment string) ([]Record, er
return out, nil
}
+// incomplete says why a stored record is not evidence of a running
+// instance's mappings, or "" when it is. Write never produces one of
+// these; a record that is one anyway (an older writer, a hand edit, a
+// bug) keeps the chores open rather than being read as "nothing attached".
+func incomplete(r Record) string {
+ if !recordable[r.InstanceState] {
+ return fmt.Sprintf("has state %q, which carries no mapping", r.InstanceState)
+ }
+ if len(r.BlockDevices) == 0 {
+ return "lists no EBS volume" // a platform box always has its root volume
+ }
+ for _, d := range r.BlockDevices {
+ if d.VolumeID == "" || d.Device == "" {
+ return "lists a volume with no id or device"
+ }
+ }
+ return ""
+}
+
// Verdict is whether a terminated box's park-local chores may be
// discharged without retirement, and if not, why.
type Verdict struct {
@@ -2498,7 +2573,9 @@ type Verdict struct {
// EnrollmentWindow of enrolledAt to the last, taken within LastRecordWindow
// of terminatedAt (the fence's observed_at), was DeleteOnTermination, with
// none ever removed (spec §5.9, H19). Records for another enrollment or
-// another instance are ignored. Anything short of that keeps the chores
+// another instance are ignored; a record of this enrollment's that is
+// incomplete, carries no mapping, or has a version this binary does not
+// read keeps the chores open. Anything short of that keeps the chores
// open for an operator, which costs cleanup and never recovery.
func Discharge(records []Record, e Entry, enrolledAt, terminatedAt time.Time) Verdict {
no := func(format string, a ...any) Verdict { return Verdict{Reason: fmt.Sprintf(format, a...)} }
@@ -2511,9 +2588,20 @@ func Discharge(records []Record, e Entry, enrolledAt, terminatedAt time.Time) Ve
}
var mine []stamped
for _, r := range records {
- if r.V != 1 || r.Enrollment != e.Enrollment || r.Cloud != e.Cloud {
+ if r.Enrollment != e.Enrollment {
continue
}
+ // A version this binary does not read could hold the adverse
+ // record; it is never skipped.
+ if r.V != 1 {
+ return no("a record for %s has version %d", e.Enrollment, r.V)
+ }
+ if r.Cloud != e.Cloud {
+ continue // another instance's (spec §5.9: ignored)
+ }
+ if reason := incomplete(r); reason != "" {
+ return no("the record observed at %q %s", r.ObservedAt, reason)
+ }
at, err := time.Parse(time.RFC3339Nano, r.ObservedAt)
if err != nil {
return no("record with unreadable observed_at %q", r.ObservedAt)
@@ -2552,7 +2640,7 @@ func Discharge(records []Record, e Entry, enrolledAt, terminatedAt time.Time) Ve
- [ ] **Step 4: Run the tests**
Run: `go test ./internal/boxvolumes/ -count=1 -v`
-Expected: PASS, including `TestObserveRefusesWhatIsNotTheInstancesMapping/terminated`, `TestDischargeSeesARemovalNoRecordListed` and `TestDischargeNeedsARecentLastRecord`.
+Expected: PASS, including `TestObserveRefusesWhatIsNotTheInstancesMapping/terminated`, `TestDischargeSeesARemovalNoRecordListed`, `TestDischargeNeedsARecentLastRecord` and `TestDischargeRefusesStoredRecordsThatAreNotEvidence`.
- [ ] **Step 5: Gate and commit**
@@ -2575,14 +2663,16 @@ Spec §5.4: "The edges-file pass publishes a host as `home` only when this box i
- **Which rows.** Only relocatable hosts (spec §2): a provisioned `instance` row that the activator runs as its own exec child (`Backing == ""`, no `Unit`, no `Addr`/`Addrs`, a `Socket`). Unit-, addr- and socket-only routes, static sites and services keep their box and are published as before. Sidecars are never published (unchanged).
- **The decision.** A claim whose `op.exec` is this box's reference: home, stamped with `holder.stamp`. A claim whose `op.exec` names another box, or none, or which has no holder: withheld. No claim: home, unstamped, exactly as before; §8's gate admits a second box only once every host has a claim.
- **Errors.** A read that fails keeps the last decision (claim C6: an error is never absence); a host never read publishes as before.
-- **Cost.** Each read stands 10 minutes (H9), at most 256 reads a pass, never-read hosts first. About 288 thousand GETs a day per 2,000 hosts, under $0.12.
+- **Cost.** Each read stands 10 minutes (H9), at most 256 reads a pass, never-attempted hosts first, then oldest attempt first. A failed read counts as an attempt, so a claim that keeps failing goes to the back of the queue and cannot starve the rest (Astra plan R1-4). About 288 thousand GETs a day per 2,000 hosts, under $0.12.
+- **Refresh.** The registry timer alone runs a pass every five to six minutes, which at 256 reads a pass would take 40 minutes to cover 2,000 hosts (Astra plan R1-5). So, with the flag on, the loop also runs a pass every 30 seconds while any claim read is due (`edgesLoopState`). Every claim is then re-read within about 14 minutes; the edges file is still written only when its body changes. With the flag off, `edgesLoopState` is exactly `edgesFileSchedule`.
+- **Aliases and catch-alls carry their primary's stamp** (Astra plan R1-2). An attached domain and a served `*.<parent>` catch-all are dialled through the primary's installation. Unstamped, a dead predecessor's alias would tie with the successor's at 0 and the edge (`dialBeats`) would keep the box that sorts first.
- **§18's open question.** A positive stamp is the mark: only relocatable hosts with a claim carry one, so the edge's report of two boxes at one positive stamp already concerns them alone. No separate marker field.
-The withholding is applied before owners and aliases are computed, so an alias of a withheld host is withheld with it.
+The withholding and stamping are applied before owners and aliases are computed, so an alias of a withheld host is withheld with it, and an alias of a stamped host reads its stamp from the map.
**Files:**
- Create: `internal/host/edgesclaims.go`
-- Test: `internal/host/edgesclaims_test.go`
+- Test: `internal/host/edgesclaims_test.go`, `internal/edge/homemap_alias_stamp_test.go`
- Modify: `internal/host/edgesfile.go`
**Interfaces:**
@@ -2591,7 +2681,13 @@ The withholding is applied before owners and aliases are computed, so an alias o
```go
const edgesClaimTTL = 10 * time.Minute
const edgesClaimReadsPerPass = 256
- type edgesClaims struct{ /* self, read, now, entries */ }
+ const edgesClaimPassEvery = 30 * time.Second
+ type edgesClaims struct{ /* self, read, now, entries, hosts */ }
+ func (c *edgesClaims) backlog(now time.Time) bool
+ type edgesLoopState struct{ sched *edgesFileSchedule; claims *edgesClaims; claimsNext time.Time; schedDue bool }
+ func (s *edgesLoopState) due(now time.Time, fp string, fpOK bool) bool
+ func (s *edgesLoopState) timed() bool
+ func (s *edgesLoopState) done(now time.Time, ok bool)
func newEdgesClaims(self string, read func(context.Context, string) (hostclaim.Observed, error)) *edgesClaims
func relocatableRow(r registry.Route) bool
func applyEdgesClaims(ctx context.Context, claims *edgesClaims, routes []registry.Route, hosts map[string]edgeHost)
@@ -2788,11 +2884,132 @@ func TestEdgesClaimsBoundReadsPerPassNeverReadFirst(t *testing.T) {
t.Fatalf("second pass read %d claims, want the 10 never read", len(book.reads))
}
}
+
+// Astra plan R1-2: an alias and a served catch-all are dialled through
+// their primary's installation, so they carry its stamp. Unstamped, a dead
+// predecessor's alias would tie with the successor's and the edge would
+// keep the first box by name.
+func TestEdgesFileStampsAliasesAndCatchAllsWithTheirPrimary(t *testing.T) {
+ reg := edgesTestRegistry(t)
+ seedInstance(t, reg, "mine.bab.test")
+ for _, a := range []registry.RouteAlias{
+ {Alias: "www.customer.com", Host: "mine.bab.test", Account: "bab", App: "jam", DNSStatus: "pointing"},
+ {Alias: "*.customer.com", Host: "mine.bab.test", Account: "bab", App: "jam", DNSStatus: "pointing"},
+ } {
+ if err := reg.UpsertAlias(a); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if err := reg.UpsertWildcardDomain(registry.WildcardDomain{Host: "customer.com", Account: "bab", App: "jam"}); err != nil {
+ t.Fatal(err)
+ }
+ book := &claimsBook{claims: map[string]*hostclaim.Claim{"mine.bab.test": serving("annie", 57)}}
+ hosts := publishWithClaims(t, reg, newEdgesClaims("annie", book.read))
+ for _, key := range []string{"mine.bab.test", "www.customer.com", "*.customer.com"} {
+ if h, ok := hosts[key]; !ok || h.Stamp != 57 || h.Role != "home" {
+ t.Errorf("%s = %+v, %v; want home at stamp 57", key, h, ok)
+ }
+ }
+ // With no claim reader the file is what it always was: no stamps.
+ for key, h := range publishWithClaims(t, reg, nil) {
+ if h.Stamp != 0 {
+ t.Errorf("%s stamped %d with region HA off", key, h.Stamp)
+ }
+ }
+}
+
+// Astra plan R1-4: claims that keep failing go to the back of the queue,
+// so they cannot hold every read of every pass.
+func TestEdgesClaimsAFailingClaimDoesNotStarveTheRest(t *testing.T) {
+ book := &claimsBook{claims: map[string]*hostclaim.Claim{"zz-healthy.bab.test": serving("annie", 7)}, errs: map[string]bool{}}
+ var hosts []string
+ for i := 0; i < edgesClaimReadsPerPass; i++ {
+ h := fmt.Sprintf("a%04d.bab.test", i)
+ book.errs[h] = true
+ hosts = append(hosts, h)
+ }
+ hosts = append(hosts, "zz-healthy.bab.test")
+ clock := time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
+ c := newEdgesClaims("annie", book.read)
+ c.now = func() time.Time { return clock }
+ ctx := context.Background()
+ c.decide(ctx, hosts) // the 256 failing claims sort first and all fail
+ clock = clock.Add(edgesClaimPassEvery)
+ if d, ok := c.decide(ctx, hosts)["zz-healthy.bab.test"]; !ok || d.stamp != 7 {
+ t.Fatalf("healthy claim = %+v, %v: starved by claims that cannot be read", d, ok)
+ }
+}
+
+// Astra plan R1-5: the loop's own schedule, with a stable registry, keeps
+// every claim of a 2,000-host box fresh. Timer passes alone come every
+// five to six minutes and read 256 claims each.
+func TestEdgesLoopRefreshesEveryClaimWithinTheTTL(t *testing.T) {
+ const n = 2000
+ book := &claimsBook{claims: map[string]*hostclaim.Claim{}}
+ var hosts []string
+ for i := 0; i < n; i++ {
+ h := fmt.Sprintf("h%04d.bab.test", i)
+ book.claims[h] = serving("annie", uint64(i+1))
+ hosts = append(hosts, h)
+ }
+ start := time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
+ clock := start
+ claims := newEdgesClaims("annie", book.read)
+ claims.now = func() time.Time { return clock }
+ state := &edgesLoopState{sched: newEdgesFileSchedule(start, func() time.Duration { return 5*time.Minute + 30*time.Second }), claims: claims}
+ lastRead := map[string]time.Time{}
+ worst := time.Duration(0)
+ for clock = start; clock.Before(start.Add(3 * time.Hour)); clock = clock.Add(edgesFileWatchInterval) {
+ if !state.due(clock, "fp", true) {
+ continue
+ }
+ book.reads = nil
+ claims.decide(context.Background(), hosts)
+ for _, h := range book.reads {
+ lastRead[h] = clock
+ }
+ state.done(clock, true)
+ if clock.Sub(start) < time.Hour {
+ continue // warm-up: the first reads of every host
+ }
+ for _, h := range hosts {
+ if age := clock.Sub(lastRead[h]); age > worst {
+ worst = age
+ }
+ }
+ }
+ if limit := edgesClaimTTL + 5*time.Minute; worst > limit {
+ t.Fatalf("a claim went %s without a read, want at most %s", worst, limit)
+ }
+}
+```
+
+And create `internal/edge/homemap_alias_stamp_test.go`, which pins the edge's half: with stamped aliases, the successor's alias wins over a dead box's that sorts first. It passes on main already (`dialBeats` compares stamps after the alias rule); it is here so the reason for stamping aliases is tested where the decision is made.
+
+```go
+package edge
+
+import "testing"
+
+// Region HA plan 1b (Astra plan R1-2): after a failover the dead box's file
+// is stale but still listed. Its alias carries the old installation's
+// stamp and the successor's the new one, so the successor wins however the
+// boxes sort.
+func TestHomeMapAnAliasFollowsTheNewerInstallation(t *testing.T) {
+ s := meshTestStore(t)
+ putJSON(t, s, "control/edges/annie.json", `{"v":1,"box":"annie","eip":"1.1.1.1","hosts":{"shop.test":{"role":"home","stamp":41},"www.customer.com":{"role":"home","alias":true,"stamp":41}}}`)
+ putJSON(t, s, "control/edges/betty.json", `{"v":1,"box":"betty","eip":"2.2.2.2","hosts":{"shop.test":{"role":"home","stamp":57},"www.customer.com":{"role":"home","alias":true,"stamp":57}}}`)
+ m := newHomeMap(s, "carol")
+ refreshHomeMap(t, m)
+ if eip, box, ok := m.Home("www.customer.com"); !ok || box != "betty" || eip != "2.2.2.2" {
+ t.Fatalf("Home(www.customer.com) = %q %q %v, want betty", eip, box, ok)
+ }
+}
```
- [ ] **Step 2: Run them to see them fail**
-Run: `go test ./internal/host/ -run 'EdgesFileFollows|EdgesFileWithholds|EdgesFileKeeps|EdgesClaims' -count=1`
+Run: `go test ./internal/host/ -run 'EdgesFileFollows|EdgesFileWithholds|EdgesFileKeeps|EdgesFileStamps|EdgesClaims|EdgesLoop' -count=1`
Expected: FAIL to build, `undefined: edgesClaims`.
- [ ] **Step 3: Write the claim cache**
@@ -2825,10 +3042,16 @@ const (
// a request to a box that then refuses to spawn.
edgesClaimTTL = 10 * time.Minute
// edgesClaimReadsPerPass bounds the GETs one pass makes, so a box with
- // thousands of hosts spreads its first reads over several passes
- // instead of stalling one. A host not read yet publishes as it did
- // before this file existed.
+ // thousands of hosts spreads its reads over several passes instead of
+ // stalling one. A host not read yet publishes as it did before this
+ // file existed.
edgesClaimReadsPerPass = 256
+ // edgesClaimPassEvery is how often the loop runs an extra pass while
+ // claim reads are due. The registry timer alone (5 to 6 minutes) would
+ // take 40 minutes to refresh 2,000 hosts at 256 a pass; with these
+ // passes every host is re-read within about edgesClaimTTL plus four
+ // minutes.
+ edgesClaimPassEvery = 30 * time.Second
)
// edgesDecision is what one claim says about this box and one host.
@@ -2837,9 +3060,14 @@ type edgesDecision struct {
stamp int64 // holder.stamp when this box is the executor
}
+// edgesClaimEntry is one host's last decision and last read attempt. A
+// failed read moves attempted and keeps the decision, so a claim that
+// cannot be read goes to the back of the queue instead of holding its
+// place at the front (claim C6: an error is never absence).
type edgesClaimEntry struct {
- decision edgesDecision
- readAt time.Time
+ decision edgesDecision
+ decided bool // some read of this claim has succeeded
+ attempted time.Time
}
// edgesClaims caches one decision per host. It is used only by the edges
@@ -2849,6 +3077,7 @@ type edgesClaims struct {
read func(ctx context.Context, host string) (hostclaim.Observed, error)
now func() time.Time
entries map[string]edgesClaimEntry
+ hosts []string // the relocatable hosts of the last pass
}
func newEdgesClaims(self string, read func(context.Context, string) (hostclaim.Observed, error)) *edgesClaims {
@@ -2882,18 +3111,34 @@ func (c *edgesClaims) decideClaim(obs hostclaim.Observed) edgesDecision {
return edgesDecision{stamp: int64(stamp)}
}
-// decide returns a decision for each host it has one for. Hosts whose
-// entry is older than edgesClaimTTL are re-read, oldest first, up to
-// edgesClaimReadsPerPass. A read that fails keeps the previous decision
-// (claim C6: an error is never absence); a host with neither a previous
-// decision nor a successful read has none, and publishes as before.
+func (c *edgesClaims) dueAt(h string, now time.Time) bool {
+ e, ok := c.entries[h]
+ return !ok || now.Sub(e.attempted) >= edgesClaimTTL
+}
+
+// backlog reports whether any host of the last pass has a read due.
+func (c *edgesClaims) backlog(now time.Time) bool {
+ for _, h := range c.hosts {
+ if c.dueAt(h, now) {
+ return true
+ }
+ }
+ return false
+}
+
+// decide returns a decision for each host that has one. Hosts whose last
+// attempt is older than edgesClaimTTL are read, never-attempted first and
+// then oldest attempt first, up to edgesClaimReadsPerPass. A read that
+// fails keeps the previous decision; a host with no successful read has
+// none, and publishes as before.
func (c *edgesClaims) decide(ctx context.Context, hosts []string) map[string]edgesDecision {
now := c.now()
+ c.hosts = append(c.hosts[:0], hosts...)
want := make(map[string]bool, len(hosts))
var due []string
for _, h := range hosts {
want[h] = true
- if e, ok := c.entries[h]; !ok || now.Sub(e.readAt) >= edgesClaimTTL {
+ if c.dueAt(h, now) {
due = append(due, h)
}
}
@@ -2906,10 +3151,10 @@ func (c *edgesClaims) decide(ctx context.Context, hosts []string) map[string]edg
ei, iok := c.entries[due[i]]
ej, jok := c.entries[due[j]]
if iok != jok {
- return !iok // never read first
+ return !iok // never attempted first
}
- if !ei.readAt.Equal(ej.readAt) {
- return ei.readAt.Before(ej.readAt)
+ if !ei.attempted.Equal(ej.attempted) {
+ return ei.attempted.Before(ej.attempted)
}
return due[i] < due[j]
})
@@ -2917,22 +3162,26 @@ func (c *edgesClaims) decide(ctx context.Context, hosts []string) map[string]edg
due = due[:edgesClaimReadsPerPass]
}
for _, h := range due {
- obs, err := c.read(ctx, h)
- if err != nil {
- continue
+ e := c.entries[h]
+ e.attempted = now
+ if obs, err := c.read(ctx, h); err == nil {
+ e.decision, e.decided = c.decideClaim(obs), true
}
- c.entries[h] = edgesClaimEntry{decision: c.decideClaim(obs), readAt: now}
+ c.entries[h] = e
}
out := make(map[string]edgesDecision, len(c.entries))
for h, e := range c.entries {
- out[h] = e.decision
+ if e.decided {
+ out[h] = e.decision
+ }
}
return out
}
// applyEdgesClaims withholds or stamps every relocatable row in hosts
// (edgesFileWrite's served map, before aliases join it, so an alias of a
-// withheld host is withheld with it). claims nil leaves hosts untouched.
+// withheld host is withheld with it and an alias of a stamped host takes
+// its stamp). claims nil leaves hosts untouched.
func applyEdgesClaims(ctx context.Context, claims *edgesClaims, routes []registry.Route, hosts map[string]edgeHost) {
if claims == nil {
return
@@ -2956,6 +3205,36 @@ func applyEdgesClaims(ctx context.Context, claims *edgesClaims, routes []registr
hosts[host] = h
}
}
+
+// edgesLoopState is edgesFileLoop's per-tick decision: the registry and
+// timer schedule, plus, with region HA on, an extra pass every
+// edgesClaimPassEvery while claim reads are due.
+type edgesLoopState struct {
+ sched *edgesFileSchedule
+ claims *edgesClaims
+ claimsNext time.Time
+ schedDue bool
+}
+
+// due reports whether to run a pass on this tick.
+func (s *edgesLoopState) due(now time.Time, fp string, fpOK bool) bool {
+ s.schedDue = s.sched.due(now, fp, fpOK)
+ claimPass := s.claims != nil && !now.Before(s.claimsNext) && s.claims.backlog(now)
+ return s.schedDue || claimPass
+}
+
+// timed reports whether the pass due just allowed is the timer's.
+func (s *edgesLoopState) timed() bool { return s.schedDue && s.sched.timed }
+
+// done records the pass's outcome.
+func (s *edgesLoopState) done(now time.Time, ok bool) {
+ if s.schedDue {
+ s.sched.done(now, ok)
+ }
+ if s.claims != nil {
+ s.claimsNext = now.Add(edgesClaimPassEvery)
+ }
+}
```
- [ ] **Step 4: Wire it into the publisher**
@@ -3029,31 +3308,132 @@ with:
log.Printf("agent: edges file loop armed (5min + jitter, and %s after a registry change, %s)", edgesFileWatchInterval, edgesFileKey(box))
```
-and replace the pass call:
+Then, where the loop builds its schedule, replace:
+
+```go
+ sched := newEdgesFileSchedule(time.Now(), func() time.Duration {
+ return 5*time.Minute + rand.N(time.Minute)
+ })
+ tick := time.NewTicker(edgesFileWatchInterval)
+```
+
+with:
```go
+ sched := newEdgesFileSchedule(time.Now(), func() time.Duration {
+ return 5*time.Minute + rand.N(time.Minute)
+ })
+ state := &edgesLoopState{sched: sched, claims: claims}
+ tick := time.NewTicker(edgesFileWatchInterval)
+```
+
+Then, in the loop body, replace:
+
+```go
+ if !sched.due(now, fp, err == nil) {
+ continue
+ }
+ if sched.timed {
+ eip = readPublicIP()
+ }
+ ident.Headroom = headroomFor(ident.Capabilities, dataDir, reg)
err = edgesFileWrite(ctx, store, reg, ident, eip, &last, sched.timed)
```
with:
```go
- err = edgesFileWriteClaims(ctx, store, reg, ident, eip, &last, sched.timed, claims)
+ if !state.due(now, fp, err == nil) {
+ continue
+ }
+ if state.timed() {
+ eip = readPublicIP()
+ }
+ ident.Headroom = headroomFor(ident.Capabilities, dataDir, reg)
+ err = edgesFileWriteClaims(ctx, store, reg, ident, eip, &last, state.timed(), claims)
+```
+
+Then, at the end of the loop body, replace:
+
+```go
+ sched.done(now, err == nil)
+```
+
+with:
+
+```go
+ state.done(now, err == nil)
+```
+
+Then, in the alias loop, replace:
+
+```go
+ acct, app := owners[a.Host].owner()
+ hosts[a.Alias] = edgeHost{Role: "home", Alias: true, Account: acct, App: app}
+```
+
+with:
+
+```go
+ acct, app := owners[a.Host].owner()
+ // The alias is served by its primary's installation, so it carries
+ // the primary's stamp (region HA HA4): an unstamped alias would tie
+ // with a dead predecessor's and the edge would keep the first box
+ // by name.
+ hosts[a.Alias] = edgeHost{Role: "home", Alias: true, Account: acct, App: app, Stamp: hosts[a.Host].Stamp}
+```
+
+Then, in the delegated wildcard loop, replace:
+
+```go
+ role, alias := "wildcard", false
+ var acct, app string
+ if primary, served := catchallServed[wd.Host]; served {
+ role, alias = "home", true
+```
+
+with:
+
+```go
+ role, alias := "wildcard", false
+ var acct, app string
+ var stamp int64
+ if primary, served := catchallServed[wd.Host]; served {
+ role, alias = "home", true
+ stamp = hosts[primary].Stamp // the dial half is the primary's installation
```
+Then, at the end of the same loop, replace:
+
+```go
+ hosts[key] = edgeHost{Role: role, Wildcard: true, Alias: alias, Account: acct, App: app}
+```
+
+with:
+
+```go
+ hosts[key] = edgeHost{Role: role, Wildcard: true, Alias: alias, Account: acct, App: app, Stamp: stamp}
+```
+
+With region HA off every primary's stamp is 0, so every alias and catch-all is published exactly as before.
+
`box` here is `metricsBoxLabel(…)`, the same value the file publishes as `box_id`, which is the reference claims name (spec §2).
- [ ] **Step 5: Run the tests**
-Run: `go test ./internal/host/ -run 'EdgesFile|EdgesClaims' -count=1 -v`
-Expected: PASS: the five new tests and every existing edges-file test.
+Run:
+```bash
+go test ./internal/host/ -run 'EdgesFile|EdgesClaims|EdgesLoop' -count=1 -v
+go test ./internal/edge/ -run 'TestHomeMap' -count=1
+```
+Expected: PASS: the eight new host tests (including `TestEdgesFileStampsAliasesAndCatchAllsWithTheirPrimary`, `TestEdgesClaimsAFailingClaimDoesNotStarveTheRest` and `TestEdgesLoopRefreshesEveryClaimWithinTheTTL`), every existing edges-file test, and the home map tests. The `internal/edge` run of `TestHomeMap` alone needs no socket.
- [ ] **Step 6: Gate and commit**
```bash
-go build ./... && go vet ./internal/host/ && gofmt -l internal/host
+go build ./... && go vet ./internal/host/ ./internal/edge/ && gofmt -l internal/host internal/edge
GOOS=darwin go build ./... && GOOS=windows go build ./...
-git add internal/host/edgesclaims.go internal/host/edgesclaims_test.go internal/host/edgesfile.go
+git add internal/host/edgesclaims.go internal/host/edgesclaims_test.go internal/host/edgesfile.go internal/edge/homemap_alias_stamp_test.go
git commit -m "edges: with region HA on, a relocatable host is home only on the box its claim names, stamped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
@@ -3063,7 +3443,7 @@ Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
### Task 7: Root refuses an update while the box takes part in a claim operation (`internal/host/systemparticipation.go`)
-Spec §5.10, root's side (H14): under root's update lock, before an executable change, write `/opt/carlos/system/guard/update-begun.json` durably, then scan `<DataDir>/.hostexec/` completely. Any record there, in any state, or anything root cannot read, list or parse, removes the marker and refuses the update. Otherwise the update proceeds and the marker stays until it finishes or is recovered.
+Spec §5.10, root's side (H14): under root's update lock, before an executable change, write `/opt/carlos/system/guard/update-begun.json` durably, then scan `<DataDir>/.hostexec/` completely. Any record there, in any state, or anything root cannot read, list or parse, removes the marker and refuses the update, to be retried later. Root records that refusal as `deferred`, not `refused` (Astra plan R1-3): `refused` is terminal for that promote (`systemResult.terminal`, `systemleg.go`), and the agent would never ask again; `deferred` is re-requested on the agent's backoff ladder, at most every five minutes, until the operation ends. Otherwise the update proceeds and the marker stays until it finishes or is recovered.
On main every executable change goes through one path, `systemApplier.install` (an operator's `carlos system rollback` re-points the channel, and the box applies it the same way); the guarded activation and the console's activation reservation are claim D1 and D7 and not on main. Recovery of an update already begun needs no check (spec §5.10: no participant acknowledged meanwhile). The participant's half (write `pending`, then refuse to acknowledge while the marker exists) belongs to claim D6's controller and waits for L6.
@@ -3075,7 +3455,7 @@ Nothing writes `.hostexec/` on main, so with the flag on this changes nothing bu
- Modify: `internal/host/systemapply.go` (`systemApplier`, `RunSystemApply`, `Run`, `install`)
**Interfaces:**
-- Consumes: `regionHAEnabled` (Task 3); `writeJSONDurable`, `syncDir`, `readSystemJournal`, `errSystemRefused`, `systemApplier.record` (existing).
+- Consumes: `regionHAEnabled` (Task 3); `writeJSONDurable`, `syncDir`, `readSystemJournal`, `errSystemDeferred`, `systemApplier.record` (existing).
- Produces:
```go
func systemGuardDir(optDir string) string
@@ -3115,8 +3495,11 @@ func hostexec(t *testing.T, f *applyFixture) string {
func assertNotUpdated(t *testing.T, f *applyFixture, why string) {
t.Helper()
r := f.result()
- if r == nil || r.Outcome != systemOutcomeRefused || !strings.Contains(r.Reason, why) {
- t.Fatalf("result = %+v, want refused naming %q", r, why)
+ if r == nil || r.Outcome != systemOutcomeDeferred || !strings.Contains(r.Reason, why) {
+ t.Fatalf("result = %+v, want deferred naming %q", r, why)
+ }
+ if r.terminal() {
+ t.Fatal("the agent would never retry this promote")
}
if bin, _ := os.ReadFile(systemBinaryPath(f.a.optDir)); !strings.Contains(string(bin), "carlos v1") {
t.Fatal("the binary changed under a refusal")
@@ -3270,6 +3653,30 @@ func TestTheMarkerStaysWhileAJournalIsOpen(t *testing.T) {
t.Fatalf("marker cleared while the journal is still open: %v", err)
}
}
+
+// Astra plan R1-3: participation defers, it does not refuse. Once the
+// operation ends and its record is cleared, the same promote applies. The
+// agent's side, re-requesting a deferred apply, is
+// TestSystemLegReusesAVerifiedCacheOnARetry.
+func TestApplyDeferredForParticipationAppliesOnceItEnds(t *testing.T) {
+ f, req := applyHappy(t)
+ f.a.regionHA = true
+ rec := filepath.Join(hostexec(t, f), "shop.bab.test.json")
+ os.WriteFile(rec, []byte(`{"state":"active"}`), 0o600)
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ if r := f.result(); r == nil || r.Outcome != systemOutcomeDeferred || r.terminal() {
+ t.Fatalf("result = %+v, want deferred, which the agent retries", r)
+ }
+ os.Remove(rec)
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ if r := f.result(); r == nil || r.Outcome != systemOutcomeApplied || !r.matches("stable", "v2", req.PromotedAt) {
+ t.Fatalf("result after the operation ended = %+v, want applied", r)
+ }
+}
```
- [ ] **Step 2: Run them to see them fail**
@@ -3366,8 +3773,11 @@ func hostexecParticipation(dataDir string) error {
// beginExecutableChange is §5.10's root side, run under the apply lock
// just before the first box-changing step: write the marker durably, then
// look for participation. On a refusal the marker is removed again and the
-// error says why; the caller records it as a refusal, so the console's
-// reservation stays held and the update is retried later.
+// error, wrapping errSystemDeferred, says why. The caller records it as
+// "deferred", never "refused": a refusal is terminal for that promote, and
+// this one must be retried once the operation ends (spec §5.10: the
+// update is refused and retried later). The agent's leg re-requests a
+// deferred apply on its backoff ladder, at most every five minutes.
func (a *systemApplier) beginExecutableChange(target string, promotedAt time.Time) error {
if !a.regionHA {
return nil
@@ -3382,7 +3792,7 @@ func (a *systemApplier) beginExecutableChange(target string, promotedAt time.Tim
if err := a.clearUpdateBegun(); err != nil {
a.ops.logf("system apply: %v", err)
}
- return fmt.Errorf("%w: %v", errSystemRefused, reason)
+ return fmt.Errorf("%w: %v", errSystemDeferred, reason)
}
return nil
}
@@ -3449,22 +3859,22 @@ In `install`, immediately before `// PromotedAt rides along for the whole transa
```go
// Region HA (spec 1 §5.10): the marker, then the participation check,
- // before the first step that changes the box. A refusal leaves the box
- // exactly as it was.
+ // before the first step that changes the box. Either failing defers the
+ // update, which the agent retries; the box is left exactly as it was.
if err := a.beginExecutableChange(target, req.PromotedAt); err != nil {
- if !errors.Is(err, errSystemRefused) {
- err = fmt.Errorf("%w: update-begun marker: %v", errSystemRefused, err)
+ if !errors.Is(err, errSystemDeferred) {
+ err = fmt.Errorf("%w: update-begun marker: %v", errSystemDeferred, err)
}
return a.record(req, err)
}
```
-A refusal is recorded in `result.json` like every other refusal, so the agent's next request retries it.
+The deferral is recorded in `result.json`; the agent's leg re-requests a deferred apply (pinned on the agent's side by `TestSystemLegReusesAVerifiedCacheOnARetry`).
- [ ] **Step 5: Run the tests**
-Run: `go test ./internal/host/ -run 'Apply|Recover|Rollback|Prev|Layout|UpdateBegun|MarkerStays|TakesPart|CannotTell|CannotList' -count=1 -v`
-Expected: PASS: the seven new tests and every existing apply and recovery test.
+Run: `go test ./internal/host/ -run 'Apply|Recover|Rollback|Prev|Layout|UpdateBegun|MarkerStays|TakesPart|CannotTell|CannotList|Deferred' -count=1 -v`
+Expected: PASS: the eight new tests (including `TestApplyDeferredForParticipationAppliesOnceItEnds`) and every existing apply and recovery test.
- [ ] **Step 6: Gate and commit**
@@ -3497,7 +3907,10 @@ Create `infra/modules/deployment-buckets/tests/evidence.tftest.hcl`:
```hcl
# Region HA spec 1 §6: every box-evidence prefix is create-only and never
# deleted, and the Denies name those prefixes and nothing wider (a broad
-# Deny on this bucket once broke every lease release).
+# Deny on this bucket once broke every lease release). Each statement is
+# checked whole: effect, principal, actions, condition and resources, so
+# turning a Deny into an Allow, dropping an action or loosening the
+# conditional-write condition fails here.
mock_provider "aws" {}
@@ -3515,16 +3928,43 @@ run "evidence_prefixes_are_create_only_and_never_deleted" {
command = plan
assert {
- condition = alltrue([
- for sid in ["BoxRecordsAreCreateOnly", "BoxRecordsAreNeverDeleted"] :
- toset([for s in jsondecode(aws_s3_bucket_policy.deployment_box_records.policy).Statement : s if s.Sid == sid][0].Resource) == toset([
- "arn:aws:s3:::dep/control/boxes/*",
- "arn:aws:s3:::dep/control/box-volumes/*",
- "arn:aws:s3:::dep/control/box-enrollment/*",
- "arn:aws:s3:::dep/control/box-fences/*",
- ])
- ])
- error_message = "the create-only and never-deleted Denies must name exactly the four box-evidence prefixes"
+ condition = jsonencode([
+ for s in jsondecode(aws_s3_bucket_policy.deployment_box_records.policy).Statement : s if s.Sid == "BoxRecordsAreCreateOnly"
+ ]) == jsonencode([{
+ Action = "s3:PutObject"
+ Condition = {
+ Bool = { "s3:ObjectCreationOperation" = "true" }
+ Null = { "s3:if-none-match" = "true" }
+ }
+ Effect = "Deny"
+ Principal = "*"
+ Resource = [
+ "arn:aws:s3:::dep/control/boxes/*",
+ "arn:aws:s3:::dep/control/box-volumes/*",
+ "arn:aws:s3:::dep/control/box-enrollment/*",
+ "arn:aws:s3:::dep/control/box-fences/*",
+ ]
+ Sid = "BoxRecordsAreCreateOnly"
+ }])
+ error_message = "BoxRecordsAreCreateOnly must Deny, to everyone, every PutObject without If-None-Match on exactly the four box-evidence prefixes"
+ }
+
+ assert {
+ condition = jsonencode([
+ for s in jsondecode(aws_s3_bucket_policy.deployment_box_records.policy).Statement : s if s.Sid == "BoxRecordsAreNeverDeleted"
+ ]) == jsonencode([{
+ Action = ["s3:DeleteObject", "s3:DeleteObjectVersion"]
+ Effect = "Deny"
+ Principal = "*"
+ Resource = [
+ "arn:aws:s3:::dep/control/boxes/*",
+ "arn:aws:s3:::dep/control/box-volumes/*",
+ "arn:aws:s3:::dep/control/box-enrollment/*",
+ "arn:aws:s3:::dep/control/box-fences/*",
+ ]
+ Sid = "BoxRecordsAreNeverDeleted"
+ }])
+ error_message = "BoxRecordsAreNeverDeleted must Deny, to everyone, both delete actions on exactly the four box-evidence prefixes"
}
}
```
@@ -3535,7 +3975,7 @@ Run (sandbox off; `tofu init` needs the network once):
```bash
cd infra/modules/deployment-buckets && tofu init -backend=false -input=false && tofu test; cd -
```
-Expected: `Failure! 0 passed, 1 failed.`, naming the four prefixes. Afterwards remove `.terraform/` and `.terraform.lock.hcl` from the module directory, as no module here commits them.
+Expected: `Failure! 0 passed, 1 failed.`, both assertions naming their statement. Leave `.terraform/` and `.terraform.lock.hcl` in place until Step 4 has run: Step 4 needs the initialized provider.
- [ ] **Step 3: Extend the policy**
@@ -3628,8 +4068,11 @@ The Sids are kept, so a reader of the bucket policy's history sees one statement
- [ ] **Step 4: Run the module test**
-Run: `cd infra/modules/deployment-buckets && tofu fmt -check && tofu validate && tofu test; cd -` (sandbox off).
-Expected: `Success! The configuration is valid.` and `Success! 1 passed, 0 failed.` Remove `.terraform/` and `.terraform.lock.hcl` again.
+Run (sandbox off; this re-runs `tofu init`, which is a no-op if Step 2's initialization is still there):
+```bash
+cd infra/modules/deployment-buckets && tofu init -backend=false -input=false && tofu fmt -check && tofu validate && tofu test; cd -
+```
+Expected: `Success! The configuration is valid.` and `Success! 1 passed, 0 failed.` Only now remove `.terraform/` and `.terraform.lock.hcl` from the module directory, as no module here commits them.
- [ ] **Step 5: Record this plan's decisions in the spec**
@@ -3773,3 +4216,17 @@ Allow could not take that away. Region HA keeps to the box's own region
in code instead: every describe it makes for a fence or a volume record
is pinned to the region the box was enrolled in.
```
+
+## Astra plan round 1: dispositions
+
+Findings are in `CODEX-PLAN-REGION-HA-1B-R1.md` (reviewed at 856767d8). All seven were checked against the tree and accepted. The fixes were dry-run again from the plan's own text: a script applied Tasks 1 to 8 in order to a clean copy of bf813540 and each built, vetted and cross-built; the tests of every task passed (with the sandbox's eight known socket failures in `internal/host`, unchanged); and each new test failed with its mechanism removed. No controller ruling was needed.
+
+| # | Finding | Disposition |
+|---|---|---|
+| 1 | P1 Task 8: Step 2's cleanup leaves Step 4 uninitialized | fixed: Step 2 keeps `.terraform/`; Step 4 runs `tofu init` again (a no-op when still initialized) before `validate` and `test`, and only then removes it |
+| 2 | P2 Task 6: aliases and served catch-alls lose the stamp, so a dead box's alias ties and can win | fixed: both take their primary's stamp from the served map; `TestEdgesFileStampsAliasesAndCatchAllsWithTheirPrimary` (killed by removing either stamp), and `internal/edge`'s `TestHomeMapAnAliasFollowsTheNewerInstallation` pins the edge's half. With the flag off every stamp is 0, so the file is unchanged |
+| 3 | P2 Task 7: `refused` is terminal for the promote, so participation suppresses the release for good | fixed: participation and a marker failure are recorded as `deferred`, which the agent's leg re-requests on its backoff ladder; `TestApplyDeferredForParticipationAppliesOnceItEnds` (deferred, not terminal, then applied once the record is gone; killed when recorded as refused) |
+| 4 | P2 Task 6: failing reads keep their place at the front and starve healthy claims | fixed: an entry records its last attempt separately from its last decision; a failed read moves the attempt and keeps the decision, so it goes to the back of the queue; `TestEdgesClaimsAFailingClaimDoesNotStarveTheRest` (256 failing claims then a healthy one; killed by the old bookkeeping) |
+| 5 | P2 Task 6: 256 reads a pass at the timer's 5 to 6 minutes cannot refresh 2,000 hosts in 10 minutes | fixed: `edgesLoopState` adds a pass every 30 seconds while claim reads are due (flag on only); `TestEdgesLoopRefreshesEveryClaimWithinTheTTL` simulates the loop's schedule for three hours with 2,000 hosts and requires every claim re-read within 15 minutes (killed with the extra passes off) |
+| 6 | P2 Task 5: incomplete, non-recordable or unknown-version records can authorize discharge | fixed: `Discharge` validates every stored record of the enrollment (version 1, a state that carries mappings, a non-empty mapping, every volume with an id and device); anything else keeps the chores open. `TestDischargeRefusesStoredRecordsThatAreNotEvidence` writes each bad record through the store and lists it back, placed so that no removal masks it; each case killed by removing its check |
+| 7 | P2 Task 8: the policy test checks resources only | fixed: each statement is compared whole (effect, principal, actions, condition, resources); five mutations (Deny to Allow, a dropped delete action, the wrong condition key, a narrower principal, a wider prefix) and main's policy each fail it |
diff --git a/docs/superpowers/plans/CODEX-PLAN-REGION-HA-1B-BRIEF-2.md b/docs/superpowers/plans/CODEX-PLAN-REGION-HA-1B-BRIEF-2.md
new file mode 100644
index 000000000..35e401ae6
--- /dev/null
+++ b/docs/superpowers/plans/CODEX-PLAN-REGION-HA-1B-BRIEF-2.md
@@ -0,0 +1,70 @@
+# Review brief: region HA plan 1b, round 2
+
+You are an independent second reader. Review the implementation plan
+`docs/superpowers/plans/2026-10-06-region-ha-1b-claim-readers.md` at the
+commit named in the prompt, against the tree at the working directory
+(Go, module github.com/carlosframework/platform) and the approved spec
+`docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md`
+(rev 5.4; rulings H1–H19 in §0). Your round 1 findings are in
+`CODEX-PLAN-REGION-HA-1B-R1.md`; the author's answers are the table
+"Astra plan round 1: dispositions" at the end of the plan.
+
+## Fixed: do not re-argue
+
+- The spec's rulings H1 to H19, and Paul's approval of rev 5.4.
+- The scope rule: plan 1b builds only what needs none of the spec's
+ L1–L20 claim/moves changes; everything listed under "Waits for the
+ claim" is out of scope. Do not ask 1b to build those.
+- The controller rulings recorded in the plan's header (six, unchanged
+ this round).
+- Everything stays dormant behind CARLOS_REGION_HA and changes nothing on
+ a one-box region.
+- Round 1 findings the dispositions table marks fixed are not to be
+ raised again unless the fix itself is wrong; say which, and why.
+
+## What changed since round 1
+
+- Task 5: `Discharge` validates every stored record of the enrollment
+ before using it (version, a state carrying mappings, a non-empty
+ mapping, volume ids and devices); new test
+ `TestDischargeRefusesStoredRecordsThatAreNotEvidence`.
+- Task 6: aliases and served catch-alls take their primary's stamp (two
+ new replacements in `edgesfile.go`); entries track the last attempt
+ separately from the last decision; `edgesLoopState` adds a pass every
+ 30 seconds while claim reads are due, replacing the loop's direct use of
+ `edgesFileSchedule`; three new host tests and one new `internal/edge`
+ test.
+- Task 7: participation and marker failures are recorded as `deferred`
+ (not `refused`); new test
+ `TestApplyDeferredForParticipationAppliesOnceItEnds`.
+- Task 8: the tofu test compares each statement whole; Steps 2 and 4 keep
+ the module initialized until validation has run.
+
+## What to check
+
+1. The dispositions table first: for each of R1-1 to R1-7, is the finding
+ resolved, partly resolved (say what is left), or not resolved?
+2. Then only new findings, in the changed parts above or anything the
+ changes break elsewhere. In particular:
+ - `edgesLoopState`: with CARLOS_REGION_HA off, is the loop's behaviour
+ identical to before (passes, `timed`, retry backoff, headroom
+ refresh)? With it on, can the 30-second passes interfere with the
+ schedule's change detection or backoff?
+ - The `deferred` outcome: does anything in the agent's leg or root's
+ apply treat a repeated deferral badly (log volume, the replay
+ watermark in `seen.json`, the rolled-back stickiness)?
+ - `Discharge`'s validation: could it refuse a record `Write` produces
+ today from a real describe (for example a box mid-`stopping`)?
+ - Alias stamps: with the flag on, can an alias of a withheld primary
+ or a non-relocatable primary ever be published with a stale stamp?
+3. As in round 1: does the code compile and do the tests pass as written,
+ does each new test fail with its mechanism removed, and do task
+ boundaries still build in order?
+
+## Output
+
+Findings ranked P1 (would not compile, a test fails, or unsafe), P2
+(wrong or missing), P3 (clarity). Each: task and step, the problem, the
+evidence (file:line, or the plan's own line), and a concrete fix. Start
+with the round 1 table, then new findings only. End with a verdict:
+ready to build, or not ready.