carlos / platform Public

branch:

Clone

git clone https://amadan.net/carlos/platform

Plain git — no account needed to clone.

Download

Download this file

1	# Region HA 1b: Evidence and Claim Readers Implementation Plan
2
3	> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
4
5	Goal: Build every part of region HA spec 1 rev 5.4 that needs none of the claim and moves changes L1 to L20: enrollment evidence from `internal/boxid`, §19's corrections to plan 1a, the agent publishing its identity record, the termination fence bound to the roster's `cloud`, volume records and the discharge check, the edges file following the claim's executor, and root's refusal to update a box that takes part in a claim operation. All of it dormant behind `CARLOS_REGION_HA`.
6
7	Architecture: Eight tasks. Four are pure libraries that read and decide but write nothing a decision rests on (`boxid.Enroll`, `termfence.Verify`, `boxvolumes`, `chooseBox`/`eligibleBoxes`). Three change what a box does, each only with `CARLOS_REGION_HA` on: the agent publishes its identity record once, the edges file publishes a relocatable host as home only on the box its claim's `op.exec` names, and root refuses an executable update while `<DataDir>/.hostexec/` holds anything. One task is storage policy. Nothing adds a claim op kind, changes `hostclaim`'s admission rules, or changes moves; claims are only read, through `hostclaim.Read`.
8
9	Tech Stack: Go 1.26, module `github.com/carlosframework/platform`; `release.Store` (create-only and ETag compare-and-swap puts); `aws-sdk-go-v2/service/ec2` and `smithy-go` (already direct dependencies); SQLite registry; OpenTofu modules under `infra/modules/`.
10
11	Spec: `docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md`, rev 5.4 (approved by Paul 2026-10-06). Executors read §0 (rulings H1 to H19), §2, §5.1, §5.4, §5.9, §5.10, §6, §8, §9 (the L-table), §18 and §19 first. Plan 1a (`docs/superpowers/plans/2026-10-05-region-ha-1a-foundations.md`, merged as aadb860) built the pieces this plan corrects; read its "After the build" section.
12
13	## Controller rulings, 2026-10-06
14
15	Made overnight by the controller on Paul's behalf, on the six points the first draft of this plan left open. Each names what it costs if it is wrong.
16
17	1. Discharge also needs a recent last record. The newest running-time volume record must be no more than 2 hours older than the termination observation (`boxvolumes.LastRecordWindow`); otherwise the dead box's `park-local` chores stay open for the operator path. Pinned by `TestDischargeNeedsARecentLastRecord` (Task 5). Cost if wrong: a recorder outage longer than two hours before a death sends that box's cleanup to an operator; nothing about recovery waits on it (H19).
18	2. §5.4 is followed literally. The edges file withholds a relocatable host whose claim's `op.exec` names another box or none (Task 6). Cost if wrong: with the flag on in a one-box region, a host whose claim briefly names no executor (a move's cut, once moves exist) is missing from remote edges for that window and answers there as unknown; moves are not built, so nothing is affected today.
19	3. Root's refusal stays behind `CARLOS_REGION_HA` in 1b (Task 7). It becomes unconditional with claim D6 (see "Waits for the claim"). Cost if wrong: until D6 lands nothing writes `.hostexec/`, so no update is wrongly allowed today; the risk is forgetting to lift the flag when D6 lands, which the waits table records.
20	4. The stricter enrollment refusal stands: one instance is never enrolled under two references (Task 1). Cost if wrong: re-enrolling a machine under a new label needs the old entry handled by an operator first.
21	5. The bind statement and `operator_role_arns` stay in `deployment-buckets` (Task 8). Retiring both is a follow-up once no consuming root sets the variable. Cost if wrong: one harmless Deny on a prefix nothing writes.
22	6. No README change in this plan. The region-host README paragraph needs Paul's copy review, which waits for the morning; its proposed text is under "Copy waiting for Paul" at the end, and Task 8 does not touch the README. Cost if wrong: the module README keeps saying region HA depends on root volumes being deleted, which H19 made untrue, until the copy lands.
23
24	## Why this is plan 1b, and what it leaves out
25
26	The spec asks the host lifecycle claim and move-any-app sessions for twenty changes (§9, L1 to L20). Neither session has accepted them. So this plan builds only what needs none of them, and stops at every point where the next step would write a claim, add an op kind, depend on the roster (`control/host-actors.json`, claim D7, not on main), or change moves. The section "Waits for the claim", after the tasks, maps everything left to its L-number.
27
28	Three of the candidates the brief named were checked against the spec and main and left out:
29
30	- Reading the termination queue. The spec gives the placer loop that reads the queue to spec 3 (§12), and an event read is only useful to a writer of fence records (L2 to L4). A reader that cannot write a fence could only leave messages on the queue or delete evidence. It also needs `service/sqs`, which is not a dependency.
31	- The Auto Scaling membership confirmation (H16, H17). It reads the target region's group ARN from the roster's `regions` entry (claim P5-D2's extension) and lists the roster's enrollments in that region. There is no roster and no Auto Scaling group yet (spec 4), and the console may accept the record only once L19 is accepted. It needs `service/autoscaling` too.
32	- IAM for those two. Nothing in this plan calls SQS or Auto Scaling, and this repository grants nothing for later. `ec2:DescribeInstances`, the one call this plan's code makes, is already allowed in every region by the inventory statement (`infra/modules/region-host/main.tf:213`): a second, narrower Allow could not take that away, and the code pins every describe to the box's enrolled region instead.
33
34	## Global Constraints
35
36	- Dormant and inert. Nothing in this plan may change behaviour on a box where `CARLOS_REGION_HA` is unset or `0`, or on a one-box region (spec §4, §8 step 2: "Ship §5 dormant behind `CARLOS_REGION_HA` … With one box per region, every rule is a no-op"). The flag is read by one helper, `regionHAEnabled` (Task 3).
37	- Scope (the brief). Do not add a claim op kind, change `hostclaim`'s admission rules, or change moves. Read claims only through `hostclaim.Read`. Use `hostguard`, `stopevidence` and `genselect` as they are.
38	- H19, verbatim: "The failover fence rests on the `terminated` read alone; disk evidence only decides discharge of local chores." Nothing in this plan lets a volume record gate execution, admission, relocation, recovery, row minting or a box's start (spec §8, B4-1).
39	- Absence is never evidence (spec §5.9): "`stopped` and `stopping` do not count … A missing instance, `InvalidInstanceID.NotFound`, an API error, a partial reply, or a lookup in another account or region is never evidence." Only state `terminated` fences.
40	- Box references are labels (#212 option B, spec §2): one DNS label (`controlname.ValidLabel`), so never containing `.` or `/`. The instance id is private and lives in the roster entry's `cloud`.
41	- Wire-shape changes are additive (AGENTS.md). This plan adds no edges-file field (`stamp` exists since plan 1a). Records it defines are new objects under new prefixes.
42	- Do not add a method to `release.Store` (eleven implementations). `Put(ctx, key, body, "")` is create-only; a lost race is `release.ErrCASConflict`.
43	- No new Go module dependencies. `service/ec2`, `aws/middleware` and `smithy-go` are already direct. Do not add `service/sqs` or `service/autoscaling`.
44	- AWS is behind interfaces. Every EC2 call goes through `termfence.EC2`; tests use fakes and never call AWS. IMDS is reached through an `*http.Client` whose transport tests replace, so no test opens a socket.
45	- Linux-only syscalls live in `*_linux.go` with a `!linux` stub. This plan adds none: `systemparticipation.go` uses `os` only. CI cross-compiles darwin, windows and linux/arm64.
46	- Tests use `release.NewDirStore(t.TempDir())`; inject failures with a struct embedding `release.Store` that overrides one method.
47	- Copy. No console, CLI or docs copy is added (controller ruling 6). Log lines are operator-facing and follow the surrounding `agent: …` / `system apply: …` style. If any task finds it needs user-facing copy, stop: it waits for Paul's copy review.
48	- Commit messages: lowercase `area:` prefix and a sentence stating the behaviour, then a blank line and `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`.
49	- Go environment. Run Go with `GOCACHE=$TMPDIR/gocache GOFLAGS="-mod=mod -buildvcs=false"` (every command below assumes both are exported). Tests that open Unix sockets fail inside the Claude sandbox with `socket: operation not permitted`; on this branch's base those are, in `internal/host`, `TestSleepingInstanceRearmsItsNextDeliveryBeforeReturning`, `TestScheduledTickWakesASleepingInstanceAndItHibernatesAgain`, `TestDeliveryReusesTheConnectionForASocket`, `TestArmScheduleFiresOnceTimersWithNoDeployStore`, `TestProxyCacheIsStableAndBounded`, `TestDeliveryDoesNotHoldTheInstanceAwake`, `TestDeliveryStillCarriesScheduleAtHeader` and `TestSocketServingProbe`; in `internal/activator`, the `TestExecBackendStart…`, `TestStartUnit…`, `TestUnitProc…`, `TestSeedUnit…`, `TestStop…` and `TestPrepareRemovesAStaleSocketFile` tests; and `internal/edge` as a whole. That is the sandbox, not the change: run those packages with the sandbox off, or rely on CI. Scratch files go under `$TMPDIR`. Never use `git stash`, `reset --hard`, `checkout --` or `clean`.
50	- OpenTofu. `tofu test` starts the AWS provider over a local socket, so it needs the sandbox off, and `tofu init` needs network access to `registry.opentofu.org` and GitHub.
51	- Gate before each commit: `go build ./... && go vet ./<pkgs>/ && gofmt -l <dirs> && go test ./<pkgs>/ -run '<tests>' -count=1`, then `GOOS=darwin go build ./... && GOOS=windows go build ./...`. Full gate before the branch lands: `go build ./... && make test`, and the three cross-builds.
52	- Exact values. Claim read cache on the edges file 10 minutes (H9's sweep); at most 256 claim reads per edges pass, with an extra pass every 30 seconds while reads are due; a volume record at least once an hour when nothing changes; the first volume record must be taken within 1 hour of enrollment; the newest within 2 hours before the termination observation; capability `region-ha-1`; fence state `terminated` only; root's marker `/opt/carlos/system/guard/update-begun.json`; controller records `<DataDir>/.hostexec/`; volume records `control/box-volumes/<enrollment>/<observed_at>.json`.
53	- Work on a branch cut from `origin/main` (suggested name `region-ha-1b`). Merging is Paul's call, through `amadan branch merge`.
54	- This plan's code was dry-run on 2026-10-06: every task was applied in order to a clean copy of `origin/main` (bf813540), built, vetted, cross-built for darwin, windows and linux/arm64, and its tests run; the five Review Focus tests and five others were each watched to fail under a mutation of the code they pin; Task 8's tofu test was run against the AWS provider's mock and watched to fail against main's policy. After Astra's plan round 1 the plan was dry-run again from its own text (every task applied in order to a clean tree by a script that reads the plan), and each round 1 fix's test was watched to fail with its mechanism removed, including five mutations of the bucket policy. Then the code was removed. What the dry run could not do is listed at the end.
55
56	## Review Focus
57
58	1. A copied identity record. Box A's own record, validly signed, written under box B's reference with its `box` field rewritten. The `box` field is not signed, so the signature still verifies; enrollment must refuse because the instance is already enrolled as A (Task 1, `TestEnrollRefusesAnInstanceAlreadyEnrolledUnderAnotherReference`).
59	2. A claim that cannot be read. A store error on a claim read must neither take a served host off this box's edges file nor publish a host this box was told not to; a host never read yet publishes as it did before claims (Task 6, `TestEdgesFileKeepsTheLastDecisionWhenAClaimCannotBeRead`).
60	3. A `terminated` describe offered as a volume observation. Its `BlockDeviceMappings` is empty, so recorded it would list every volume as removed. It must not be recordable; and a removal that no record listed must still keep the chores open (Task 5, `TestObserveRefusesWhatIsNotTheInstancesMapping`, `TestDischargeSeesARemovalNoRecordListed`).
61	4. A controller directory root cannot read. A temporary file, a record that does not parse, a symlink or a file in place of `.hostexec`, or a directory root cannot list: each must refuse the update, never pass it (Task 7, `TestApplyRefusesWhenItCannotTellWhetherThisBoxTakesPart`, `TestApplyRefusesWhenItCannotListTheControllerDirectory`).
62	5. A fence read from another region. A worker enrolled in another AWS region reading a perfect `terminated` observation must not fence the box (spec §6: the fence refuses any observation not made in the box's own region) (Task 4, `TestAWorkerInAnotherRegionNeverFences`).
63
64	## File Structure
65
66	\| Path \| Responsibility \|
67	\|---\|---\|
68	\| `internal/boxid/record.go` \| The identity record, `ValidBox`, `Key`, create-only `Publish`, `Load` (exact bytes) \|
69	\| `internal/boxid/verify.go` \| `Cloud`, `Verifier.Enroll`: the record as enrollment evidence \|
70	\| `internal/console/boxchoice.go` \| `eligibleBoxes` over the roster's proven cloud; `chooseBox` with P5-D2's tie-break \|
71	\| `internal/activator/installed.go`, `activator.go` \| `WarmCopyTrusted` removed; comments point at claim R6 \|
72	\| `internal/host/boxrecord.go` \| `regionHAEnabled`; reading IMDS; publishing this box's record at start \|
73	\| `internal/termfence/fence.go` \| Observations, `DescribeInstance`, `Verify` bound to the roster entry, §5.9's record shape \|
74	\| `internal/boxvolumes/volumes.go` \| Volume records: observe, decide, write create-only, list, decide discharge \|
75	\| `internal/host/edgesclaims.go` \| The edges file's claim cache and the home/stamp decision \|
76	\| `internal/host/edgesfile.go` \| Wiring the claim cache into the publisher \|
77	\| `internal/host/systemparticipation.go` \| Root's update-begun marker and `.hostexec` check \|
78	\| `internal/host/systemapply.go` \| Wiring them into the apply \|
79	\| `infra/modules/deployment-buckets/main.tf`, `tests/evidence.tftest.hcl` \| Create-only, never-deleted evidence prefixes \|
80
81	---
82
83	### Task 1: Enrollment evidence (`internal/boxid`)
84
85	Spec §5.1 and §19: `control/boxes/<ref>.json` is no longer an authority. It is the evidence enrollment reads, and enrollment records the proven account, region and instance id in the roster entry's `cloud`. The operator bind is superseded (its tofu check moves into enrollment), and box references are labels.
86
87	Files:
88	- Modify (replace whole file): `internal/boxid/record.go`, `internal/boxid/verify.go`
89	- Test (replace whole file): `internal/boxid/boxid_test.go`
90
91	Interfaces:
92	- Consumes: `release.Store`, `controlname.ValidLabel`.
93	- Produces:
94	```go
95	package boxid
96	const Prefix = "control/boxes/"
97	func ValidBox(ref string) bool
98	func Key(box string) string
99	type Record struct{ V int; Box, IdentityDocument, IdentitySignature, WrittenAt string } // json as before
100	func Publish(ctx context.Context, s release.Store, rec Record) error // create-only; same bytes again is success
101	func Load(ctx context.Context, s release.Store, box string) ([]byte, error) // exact bytes; os.ErrNotExist-wrapped when absent
102	var ErrUnverified error
103	type Identity struct{ AccountID, Region, InstanceID string } // kept only until Task 4
104	type Cloud struct{ Provider, Account, Region, InstanceID string } // json: provider, account, region, instance_id
105	func (c Cloud) Complete() bool
106	type Verifier struct{ Account string; Certs map[string]*x509.Certificate }
107	type Enrolled struct{ Box string; Cloud Cloud }
108	type Expect struct{ Box, AWSRegion, TofuInstanceID string; Roster []Enrolled }
109	type Evidence struct{ Box string; Cloud Cloud; RecordSHA256 string }
110	func (v Verifier) Enroll(raw []byte, e Expect) (Evidence, error)
111	func EmbeddedCerts() (map[string]*x509.Certificate, error) // unchanged
112	```
113	Removed: `BindKey`, `Bind`, `Verifier.Verify`, and `Load`'s record and bind returns. Nothing outside `internal/boxid` used them.
114
115	Why `Enroll` takes raw bytes. The hash enrollment records must be of the bytes it verified. Decoding inside `Enroll` makes that one value, where a `Record` and a separately supplied `[]byte` could disagree.
116
117	Why the roster check runs both ways. The spec asks enrollment to "refuse a reference already enrolled to a different instance". The record's `box` field is not covered by AWS's signature, so a box can copy another box's record to its own key, rewrite `box`, and still verify. `Enroll` therefore also refuses an instance the roster already holds under another reference. Entries in every state count: a retired reference never names a working box again, and a terminated instance never returns.
118
119	- [ ] Step 1: Write the failing tests
120
121	Replace `internal/boxid/boxid_test.go` with:
122
123	```go
124	package boxid
125
126	import (
127	"context"
128	"crypto"
129	"crypto/rand"
130	"crypto/rsa"
131	"crypto/sha256"
132	"crypto/x509"
133	"crypto/x509/pkix"
134	"encoding/base64"
135	"encoding/hex"
136	"encoding/json"
137	"errors"
138	"math/big"
139	"os"
140	"testing"
141	"time"
142
143	"github.com/carlosframework/platform/internal/release"
144	)
145
146	// signer stands in for AWS: one key and certificate per test.
147	type signer struct {
148	key *rsa.PrivateKey
149	cert *x509.Certificate
150	}
151
152	func newSigner(t *testing.T) signer {
153	t.Helper()
154	key, err := rsa.GenerateKey(rand.Reader, 2048)
155	if err != nil {
156	t.Fatal(err)
157	}
158	tmpl := &x509.Certificate{SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "test"},
159	NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour)}
160	der, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key)
161	if err != nil {
162	t.Fatal(err)
163	}
164	cert, err := x509.ParseCertificate(der)
165	if err != nil {
166	t.Fatal(err)
167	}
168	return signer{key, cert}
169	}
170
171	func (s signer) record(t *testing.T, box, account, region, instance string) (Record, []byte) {
172	t.Helper()
173	doc, _ := json.Marshal(map[string]string{"accountId": account, "region": region, "instanceId": instance})
174	sum := sha256.Sum256(doc)
175	sig, err := rsa.SignPKCS1v15(rand.Reader, s.key, crypto.SHA256, sum[:])
176	if err != nil {
177	t.Fatal(err)
178	}
179	rec := Record{V: 1, Box: box, IdentityDocument: base64.StdEncoding.EncodeToString(doc),
180	IdentitySignature: base64.StdEncoding.EncodeToString(sig), WrittenAt: "2026-10-06T00:00:00Z"}
181	raw, _ := json.Marshal(rec)
182	return rec, raw
183	}
184
185	// verifier certifies eu-west-1 and eu-west-2 with the same key, so a test
186	// can show that a supported but wrong region is still refused.
187	func (s signer) verifier() Verifier {
188	return Verifier{Account: "111122223333", Certs: map[string]*x509.Certificate{"eu-west-1": s.cert, "eu-west-2": s.cert}}
189	}
190
191	func expect(box string) Expect { return Expect{Box: box, AWSRegion: "eu-west-1"} }
192
193	func TestEnrollProvesTheCloudOfALabel(t *testing.T) {
194	s := newSigner(t)
195	_, raw := s.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
196	ev, err := s.verifier().Enroll(raw, expect("annie-welding"))
197	if err != nil {
198	t.Fatal(err)
199	}
200	sum := sha256.Sum256(raw)
201	want := Evidence{Box: "annie-welding", RecordSHA256: hex.EncodeToString(sum[:]),
202	Cloud: Cloud{Provider: "aws", Account: "111122223333", Region: "eu-west-1", InstanceID: "i-0abc"}}
203	if ev != want {
204	t.Fatalf("evidence = %+v\nwant %+v", ev, want)
205	}
206	if !ev.Cloud.Complete() {
207	t.Fatal("a proven cloud is not complete")
208	}
209	}
210
211	func TestEnrollRefusals(t *testing.T) {
212	s, other := newSigner(t), newSigner(t)
213	good := func() []byte {
214	_, raw := s.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
215	return raw
216	}
217	cases := map[string]struct {
218	raw []byte
219	e Expect
220	}{
221	"wrong account": {func() []byte {
222	_, raw := s.record(t, "annie-welding", "999999999999", "eu-west-1", "i-0abc")
223	return raw
224	}(), expect("annie-welding")},
225	"region with no certificate": {func() []byte {
226	_, raw := s.record(t, "annie-welding", "111122223333", "ap-south-1", "i-0abc")
227	return raw
228	}(), Expect{Box: "annie-welding", AWSRegion: "ap-south-1"}},
229	"a supported region that is not the placement's": {func() []byte {
230	_, raw := s.record(t, "annie-welding", "111122223333", "eu-west-2", "i-0abc")
231	return raw
232	}(), expect("annie-welding")},
233	"signed by someone else": {func() []byte {
234	_, raw := other.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
235	return raw
236	}(), expect("annie-welding")},
237	"document altered after signing": {func() []byte {
238	r, _ := s.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
239	doc, _ := json.Marshal(map[string]string{"accountId": "111122223333", "region": "eu-west-1", "instanceId": "i-0zzz"})
240	r.IdentityDocument = base64.StdEncoding.EncodeToString(doc)
241	raw, _ := json.Marshal(r)
242	return raw
243	}(), expect("annie-welding")},
244	"record names another box": {good(), expect("bert-sawing")},
245	"reference is not a label": {good(), expect("annie.welding")},
246	"no expected region": {good(), Expect{Box: "annie-welding"}},
247	"tofu names another instance": {good(), Expect{Box: "annie-welding", AWSRegion: "eu-west-1", TofuInstanceID: "i-0zzz"}},
248	"unknown field in the record": {[]byte(`{"v":1,"box":"annie-welding","extra":1}`), expect("annie-welding")},
249	"trailing data": {append(good(), []byte(`{}`)...), expect("annie-welding")},
250	"version 2": {func() []byte {
251	r, _ := s.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
252	r.V = 2
253	raw, _ := json.Marshal(r)
254	return raw
255	}(), expect("annie-welding")},
256	"reference enrolled elsewhere": {good(), Expect{Box: "annie-welding", AWSRegion: "eu-west-1", Roster: []Enrolled{{Box: "annie-welding", Cloud: Cloud{InstanceID: "i-0old"}}}}},
257	}
258	for name, tc := range cases {
259	t.Run(name, func(t *testing.T) {
260	if ev, err := s.verifier().Enroll(tc.raw, tc.e); !errors.Is(err, ErrUnverified) {
261	t.Fatalf("evidence = %+v, err = %v; want ErrUnverified", ev, err)
262	}
263	})
264	}
265	}
266
267	// Review Focus 1: box A's record, intact and validly signed, copied under
268	// box B's key with its box field rewritten. The signature still verifies,
269	// because the box field is not signed. Only the roster check refuses it.
270	func TestEnrollRefusesAnInstanceAlreadyEnrolledUnderAnotherReference(t *testing.T) {
271	s := newSigner(t)
272	_, raw := s.record(t, "bert-sawing", "111122223333", "eu-west-1", "i-0abc")
273	roster := []Enrolled{{Box: "annie-welding", Cloud: Cloud{Provider: "aws", Account: "111122223333", Region: "eu-west-1", InstanceID: "i-0abc"}}}
274	if _, err := s.verifier().Enroll(raw, Expect{Box: "bert-sawing", AWSRegion: "eu-west-1", Roster: roster}); !errors.Is(err, ErrUnverified) {
275	t.Fatalf("err = %v, want ErrUnverified", err)
276	}
277	// The same box re-enrolled as the same instance is not a conflict.
278	roster[0].Box = "bert-sawing"
279	if _, err := s.verifier().Enroll(raw, Expect{Box: "bert-sawing", AWSRegion: "eu-west-1", Roster: roster}); err != nil {
280	t.Fatalf("re-enrolling the same instance: %v", err)
281	}
282	}
283
284	func TestEnrollAcceptsTheTofuInstanceForALegacyReference(t *testing.T) {
285	s := newSigner(t)
286	_, raw := s.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
287	if _, err := s.verifier().Enroll(raw, Expect{Box: "annie-welding", AWSRegion: "eu-west-1", TofuInstanceID: "i-0abc"}); err != nil {
288	t.Fatal(err)
289	}
290	}
291
292	func TestPublishIsCreateOnlyAndIdempotent(t *testing.T) {
293	ctx, st, s := context.Background(), release.NewDirStore(t.TempDir()), newSigner(t)
294	rec, _ := s.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
295	if err := Publish(ctx, st, rec); err != nil {
296	t.Fatal(err)
297	}
298	if err := Publish(ctx, st, rec); err != nil {
299	t.Fatalf("republishing the same record: %v", err)
300	}
301	other := rec
302	other.WrittenAt = "2027-01-01T00:00:00Z"
303	if err := Publish(ctx, st, other); !errors.Is(err, release.ErrCASConflict) {
304	t.Fatalf("a different record for the same box: err = %v, want ErrCASConflict", err)
305	}
306	raw, err := Load(ctx, st, "annie-welding")
307	if err != nil {
308	t.Fatal(err)
309	}
310	if _, err := s.verifier().Enroll(raw, expect("annie-welding")); err != nil {
311	t.Fatalf("the published record does not enroll: %v", err)
312	}
313	}
314
315	func TestPublishAndLoadRefuseAReferenceThatIsNotALabel(t *testing.T) {
316	ctx, st, s := context.Background(), release.NewDirStore(t.TempDir()), newSigner(t)
317	rec, _ := s.record(t, "x.bind", "111122223333", "eu-west-1", "i-0abc")
318	if err := Publish(ctx, st, rec); err == nil {
319	t.Fatal("published under a reference with a dot")
320	}
321	if _, err := Load(ctx, st, "../edges/x"); !errors.Is(err, ErrUnverified) {
322	t.Fatalf("err = %v, want ErrUnverified", err)
323	}
324	if _, err := Load(ctx, st, "nobody-here"); !errors.Is(err, os.ErrNotExist) {
325	t.Fatalf("err = %v, want os.ErrNotExist", err)
326	}
327	}
328
329	func TestEmbeddedCertsCoverTheFlagshipRegions(t *testing.T) {
330	certs, err := EmbeddedCerts()
331	if err != nil {
332	t.Fatal(err)
333	}
334	for _, region := range []string{"eu-west-1", "eu-west-2", "us-east-1", "ap-southeast-2"} {
335	c := certs[region]
336	if c == nil {
337	t.Errorf("no certificate for %s", region)
338	continue
339	}
340	if c.PublicKeyAlgorithm != x509.RSA {
341	t.Errorf("%s: %v key; the base64-signature certificate is RSA (the PKCS7 one is DSA: wrong tab)", region, c.PublicKeyAlgorithm)
342	}
343	}
344	}
345	```
346
347	- [ ] Step 2: Run them to see them fail
348
349	Run: `go test ./internal/boxid/ -count=1`
350	Expected: FAIL to build, `undefined: Expect` and `s.verifier().Enroll undefined`.
351
352	- [ ] Step 3: Write the record
353
354	Replace `internal/boxid/record.go` with:
355
356	```go
357	// Package boxid is the evidence a platform box presents to be enrolled in
358	// the roster (region HA spec 1 §5.1, ruling H11): the box's own AWS-signed
359	// instance identity document, published once under its box reference. The
360	// record grants nothing by itself. Enrollment verifies it (Verifier.Enroll)
361	// and copies the proven account, region and instance id into the roster
362	// entry's cloud field; from then on the roster, never this record, is what
363	// binds a reference to an instance.
364	package boxid
365
366	import (
367	"bytes"
368	"context"
369	"encoding/json"
370	"errors"
371	"fmt"
372
373	"github.com/carlosframework/platform/internal/controlname"
374	"github.com/carlosframework/platform/internal/release"
375	)
376
377	const Prefix = "control/boxes/"
378
379	// ValidBox reports whether ref can be a box reference: one DNS label, so
380	// it can never contain "." or "/" and no two references share a key.
381	func ValidBox(ref string) bool { return controlname.ValidLabel(ref) }
382
383	// Key is where a box's record lives. Callers check ValidBox first.
384	func Key(box string) string { return Prefix + box + ".json" }
385
386	type Record struct {
387	V int `json:"v"`
388	Box string `json:"box"`
389	IdentityDocument string `json:"identity_document"`
390	IdentitySignature string `json:"identity_signature"`
391	WrittenAt string `json:"written_at"`
392	}
393
394	// Publish writes a box's record create-only. Writing the same bytes again
395	// succeeds; a different record for the same box is ErrCASConflict, because
396	// a record is never replaced.
397	func Publish(ctx context.Context, s release.Store, rec Record) error {
398	if !ValidBox(rec.Box) {
399	return fmt.Errorf("boxid: %q is not a box reference", rec.Box)
400	}
401	body, err := json.Marshal(rec)
402	if err != nil {
403	return fmt.Errorf("boxid: marshal: %w", err)
404	}
405	_, err = s.Put(ctx, Key(rec.Box), body, "")
406	if !errors.Is(err, release.ErrCASConflict) {
407	return err
408	}
409	cur, _, gerr := s.Get(ctx, Key(rec.Box))
410	if gerr != nil {
411	return fmt.Errorf("boxid: %s exists and cannot be read: %w", rec.Box, gerr)
412	}
413	if bytes.Equal(cur, body) {
414	return nil
415	}
416	return fmt.Errorf("boxid: %s already has a different record: %w", rec.Box, release.ErrCASConflict)
417	}
418
419	// Load returns the exact bytes of a box's record, which is what enrollment
420	// verifies and hashes. A missing record is os.ErrNotExist-wrapped.
421	func Load(ctx context.Context, s release.Store, box string) ([]byte, error) {
422	if !ValidBox(box) {
423	return nil, fmt.Errorf("%w: %q is not a box reference", ErrUnverified, box)
424	}
425	raw, _, err := s.Get(ctx, Key(box))
426	return raw, err
427	}
428	```
429
430	- [ ] Step 4: Write enrollment
431
432	Replace `internal/boxid/verify.go` with:
433
434	```go
435	package boxid
436
437	import (
438	"bytes"
439	"crypto/sha256"
440	"crypto/x509"
441	"encoding/base64"
442	"encoding/hex"
443	"encoding/json"
444	"errors"
445	"fmt"
446	"io"
447	)
448
449	// ErrUnverified wraps every reason a record is not evidence for enrollment.
450	var ErrUnverified = errors.New("boxid: record does not verify")
451
452	// Identity is what an identity document proves. termfence binds to it
453	// until it moves to Cloud; nothing else uses it.
454	type Identity struct{ AccountID, Region, InstanceID string }
455
456	// Cloud is the roster entry's cloud field (spec §2, §5.1): the instance a
457	// box reference is enrolled as. Enrollment fills it from a verified record;
458	// every later check (the termination fence, volume records) compares
459	// against it and never against anything a box says about itself.
460	type Cloud struct {
461	Provider string `json:"provider"`
462	Account string `json:"account"`
463	Region string `json:"region"`
464	InstanceID string `json:"instance_id"`
465	}
466
467	// Complete reports whether every field is set and the provider is AWS.
468	func (c Cloud) Complete() bool {
469	return c.Provider == "aws" && c.Account != "" && c.Region != "" && c.InstanceID != ""
470	}
471
472	type Verifier struct {
473	Account string // the deployment's AWS account id
474	Certs map[string]*x509.Certificate // by AWS region
475	}
476
477	// Enrolled is one roster entry as enrollment needs to see it: its
478	// reference and the instance it was enrolled as. Entries in every state
479	// count, retired ones included: a retired reference never names a working
480	// box again (spec §5.9), and a terminated instance never comes back.
481	type Enrolled struct {
482	Box string
483	Cloud Cloud
484	}
485
486	// Expect is what the enrolling side knows before it reads the record.
487	type Expect struct {
488	Box string // the reference being enrolled
489	AWSRegion string // the AWS region of the placement it is enrolled into
490	// TofuInstanceID is the region's tofu region-host instance_id output.
491	// The caller sets it when Box already appears in claims or edges files,
492	// as a box predating region HA does, and leaves it "" otherwise.
493	TofuInstanceID string
494	Roster []Enrolled
495	}
496
497	// Evidence is what enrollment records: the roster entry's cloud, and the
498	// SHA-256 of the exact record bytes it was proven from.
499	type Evidence struct {
500	Box string
501	Cloud Cloud
502	RecordSHA256 string
503	}
504
505	func refuse(format string, a ...any) error {
506	return fmt.Errorf("%w: %s", ErrUnverified, fmt.Sprintf(format, a...))
507	}
508
509	func decodeRecord(raw []byte) (Record, error) {
510	dec := json.NewDecoder(bytes.NewReader(raw))
511	dec.DisallowUnknownFields()
512	var rec Record
513	if err := dec.Decode(&rec); err != nil {
514	return Record{}, refuse("record is not JSON: %v", err)
515	}
516	if _, err := dec.Token(); err != io.EOF {
517	return Record{}, refuse("trailing data after the record")
518	}
519	return rec, nil
520	}
521
522	// Enroll checks raw, the exact bytes stored at Key(e.Box), as enrollment
523	// evidence (spec §5.1, H11). In order: the reference, the record's own box
524	// field, AWS's signature under the certificate for the document's region,
525	// the deployment account, the AWS region expected for the placement, the
526	// tofu instance id for a reference predating region HA, and the roster:
527	// the reference enrolled as another instance, or the instance enrolled
528	// under another reference, is refused. A record copied from one box to
529	// another's key carries the first box's signed instance id, so the last
530	// check is what keeps one machine from holding two references.
531	func (v Verifier) Enroll(raw []byte, e Expect) (Evidence, error) {
532	if !ValidBox(e.Box) {
533	return Evidence{}, refuse("%q is not a box reference", e.Box)
534	}
535	if e.AWSRegion == "" {
536	return Evidence{}, refuse("no expected AWS region for %s", e.Box)
537	}
538	rec, err := decodeRecord(raw)
539	if err != nil {
540	return Evidence{}, err
541	}
542	if rec.V != 1 {
543	return Evidence{}, refuse("record version %d", rec.V)
544	}
545	if rec.Box != e.Box {
546	return Evidence{}, refuse("the record stored for %q names %q", e.Box, rec.Box)
547	}
548	doc, err := base64.StdEncoding.DecodeString(rec.IdentityDocument)
549	if err != nil {
550	return Evidence{}, refuse("document is not base64")
551	}
552	sig, err := base64.StdEncoding.DecodeString(rec.IdentitySignature)
553	if err != nil {
554	return Evidence{}, refuse("signature is not base64")
555	}
556	var d struct {
557	AccountID string `json:"accountId"`
558	Region string `json:"region"`
559	InstanceID string `json:"instanceId"`
560	}
561	if err := json.Unmarshal(doc, &d); err != nil {
562	return Evidence{}, refuse("document is not JSON")
563	}
564	cert := v.Certs[d.Region]
565	if cert == nil {
566	return Evidence{}, refuse("no signing certificate for region %q", d.Region)
567	}
568	if err := cert.CheckSignature(x509.SHA256WithRSA, doc, sig); err != nil {
569	return Evidence{}, refuse("signature: %v", err)
570	}
571	switch {
572	case d.InstanceID == "":
573	return Evidence{}, refuse("document names no instance")
574	case v.Account == "" \|\| d.AccountID != v.Account:
575	return Evidence{}, refuse("account %q is not the deployment's", d.AccountID)
576	case d.Region != e.AWSRegion:
577	return Evidence{}, refuse("the box is in %q, the placement is in %q", d.Region, e.AWSRegion)
578	case e.TofuInstanceID != "" && e.TofuInstanceID != d.InstanceID:
579	return Evidence{}, refuse("tofu records instance %q for this region, the record %q", e.TofuInstanceID, d.InstanceID)
580	}
581	for _, en := range e.Roster {
582	if en.Box == e.Box && en.Cloud.InstanceID != d.InstanceID {
583	return Evidence{}, refuse("%s is enrolled as instance %q", e.Box, en.Cloud.InstanceID)
584	}
585	if en.Box != e.Box && en.Cloud.InstanceID == d.InstanceID {
586	return Evidence{}, refuse("instance %s is enrolled as %s", d.InstanceID, en.Box)
587	}
588	}
589	sum := sha256.Sum256(raw)
590	return Evidence{
591	Box: e.Box,
592	Cloud: Cloud{Provider: "aws", Account: d.AccountID, Region: d.Region, InstanceID: d.InstanceID},
593	RecordSHA256: hex.EncodeToString(sum[:]),
594	}, nil
595	}
596	```
597
598	- [ ] Step 5: Run the tests
599
600	Run: `go test ./internal/boxid/ -count=1 -v`
601	Expected: PASS, including `TestEnrollRefusesAnInstanceAlreadyEnrolledUnderAnotherReference` and every subtest of `TestEnrollRefusals`.
602
603	- [ ] Step 6: Gate and commit
604
605	```bash
606	go build ./... && go vet ./internal/boxid/ && gofmt -l internal/boxid
607	go test ./internal/boxid/ ./internal/termfence/ ./internal/console/ -run 'Enroll\|Publish\|Load\|Embedded\|Eligible\|ChooseBox\|Fence\|Lookalike\|Event\|Legacy\|Unverified' -count=1
608	GOOS=darwin go build ./... && GOOS=windows go build ./...
609	git add internal/boxid
610	git commit -m "boxid: a box record is enrollment evidence, and one instance never holds two references
611
612	Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
613	```
614
615	Expected: `gofmt -l` prints nothing; every package `ok`.
616
617	---
618
619	### Task 2: §19's corrections to choice and the warm copy
620
621	Spec §19: `eligibleBoxes` takes its identities from the roster, not from loading records; `chooseBox`'s last tie-break becomes claim P5-D2's greatest SHA-256 of `host + "\x00" + reference` (H12); `WarmCopyTrusted` and the activator's warm path are withdrawn (H13), and the activator's one-box comment points at claim R6.
622
623	Files:
624	- Modify (replace whole file): `internal/console/boxchoice.go`
625	- Test (replace whole file): `internal/console/boxchoice_test.go`
626	- Modify: `internal/activator/installed.go`, `internal/activator/installed_test.go`, `internal/activator/activator.go:1164-1168`
627
628	Interfaces:
629	- Consumes: `boxid.Cloud` (Task 1).
630	- Produces:
631	```go
632	// package console
633	type rosterBox struct{ Placement string; Cloud boxid.Cloud }
634	func eligibleBoxes(snap edgesSnapshot, roster map[string]rosterBox, placement, awsRegion, homeName, expectedFP string, lost map[string]bool) []boxCandidate
635	func chooseBox(host string, cands []boxCandidate, running map[string]boxLoad, app string) (boxCandidate, bool)
636	```
637	Neither has a caller outside its tests yet: wiring them into create is L8. Removed from `activator`: `Installation`, `WarmCopyTrusted`.
638
639	- [ ] Step 1: Write the failing tests
640
641	Replace `internal/console/boxchoice_test.go` with:
642
643	```go
644	package console
645
646	import (
647	"crypto/sha256"
648	"encoding/hex"
649	"encoding/json"
650	"testing"
651
652	"github.com/carlosframework/platform/internal/boxid"
653	)
654
655	// haBox is an eligible box's edges file in placement "uk".
656	func haBox(id string, mut func(m map[string]any)) string {
657	m := map[string]any{
658	"v": 1, "box": id, "box_id": id, "eip": "1.2.3.4", "region": "eu-west-2", "placement": "uk",
659	"hosts": map[string]any{}, "hibernation_configured": true, "serves_shared_pool": true,
660	"accepts_inbound": true, "secrets_key_fp": testKeyFP, "capabilities": []string{"region-ha-1"},
661	"headroom": map[string]any{"mem_free_mib": 1000, "disk_free_mib": 1000, "awake": 1},
662	}
663	if mut != nil {
664	mut(m)
665	}
666	b, _ := json.Marshal(m)
667	return string(b)
668	}
669
670	func ids(cs []boxCandidate) []string {
671	out := make([]string, len(cs))
672	for i, c := range cs {
673	out[i] = c.BoxID
674	}
675	return out
676	}
677
678	// rosterFor enrolls each box in placement "uk" as an instance in eu-west-2.
679	func rosterFor(boxes ...string) map[string]rosterBox {
680	out := map[string]rosterBox{}
681	for _, b := range boxes {
682	out[b] = rosterBox{Placement: "uk", Cloud: boxid.Cloud{Provider: "aws", Account: "1", Region: "eu-west-2", InstanceID: "i-" + b}}
683	}
684	return out
685	}
686
687	func TestEligibleBoxesAppliesThePredicateOneBoxAtATime(t *testing.T) {
688	del := func(k string) func(map[string]any) { return func(m map[string]any) { delete(m, k) } }
689	set := func(k string, v any) func(map[string]any) { return func(m map[string]any) { m[k] = v } }
690	cases := []struct {
691	name string
692	mut func(map[string]any)
693	want bool
694	}{
695	{"all present", nil, true},
696	{"no dial address", set("eip", ""), false},
697	{"hibernation unconfigured", set("hibernation_configured", false), false},
698	{"hibernation unknown", del("hibernation_configured"), false},
699	{"refuses the shared pool", set("serves_shared_pool", false), false},
700	{"refuses inbound", set("accepts_inbound", false), false},
701	{"wrong secrets key", set("secrets_key_fp", "sha256:other"), false},
702	{"no capability", del("capabilities"), false},
703	{"draining", set("draining", true), false},
704	{"another placement", set("placement", "us"), false},
705	{"box_id names no roster entry", set("box_id", "i-0zzz"), false},
706	}
707	for _, tc := range cases {
708	t.Run(tc.name, func(t *testing.T) {
709	snap := views(t, haBox("i-0aaa", tc.mut))
710	got := eligibleBoxes(snap, rosterFor("i-0aaa"), "uk", "eu-west-2", "ie", testKeyFP, nil)
711	if (len(got) == 1) != tc.want {
712	t.Fatalf("eligible = %v, want %v", ids(got), tc.want)
713	}
714	})
715	}
716	}
717
718	func TestEligibleBoxesNeedsARosterEntryFreshnessAndNotLost(t *testing.T) {
719	snap := views(t, haBox("i-0aaa", nil), haBox("i-0bbb", nil), haBox("i-0ccc", nil))
720	snap = markStale(snap, "i-0bbb")
721	got := eligibleBoxes(snap, rosterFor("i-0aaa", "i-0bbb"), "uk", "eu-west-2", "ie", testKeyFP, nil)
722	if len(got) != 1 \|\| got[0].BoxID != "i-0aaa" {
723	t.Fatalf("got %v, want only i-0aaa (i-0bbb is stale, i-0ccc has no roster entry)", ids(got))
724	}
725	got = eligibleBoxes(snap, rosterFor("i-0aaa"), "uk", "eu-west-2", "ie", testKeyFP, map[string]bool{"i-0aaa": true})
726	if len(got) != 0 {
727	t.Fatalf("a lost box is eligible: %v", ids(got))
728	}
729	}
730
731	// TestEligibilityNeverCombinesBoxes is nearest-region's rule for boxes: one
732	// box holding the key and another accepting inbound do not add up to one.
733	func TestEligibleBoxesNeverCombinesTwoHalfBoxes(t *testing.T) {
734	snap := views(t,
735	haBox("i-0aaa", func(m map[string]any) { m["secrets_key_fp"] = "" }),
736	haBox("i-0bbb", func(m map[string]any) { m["accepts_inbound"] = false }))
737	if got := eligibleBoxes(snap, rosterFor("i-0aaa", "i-0bbb"), "uk", "eu-west-2", "ie", testKeyFP, nil); len(got) != 0 {
738	t.Fatalf("got %v, want none", ids(got))
739	}
740	}
741
742	func TestEligibleBoxesAnIncompleteSnapshotOffersNothing(t *testing.T) {
743	snap := views(t, haBox("i-0aaa", nil))
744	snap.Complete = false
745	if got := eligibleBoxes(snap, rosterFor("i-0aaa"), "uk", "eu-west-2", "ie", testKeyFP, nil); len(got) != 0 {
746	t.Fatalf("got %v from an incomplete snapshot", ids(got))
747	}
748	}
749
750	func TestEligibleBoxesHomePlacementMatchesTheEmptyCode(t *testing.T) {
751	home := haBox("i-0aaa", func(m map[string]any) { m["placement"] = "" })
752	named := haBox("i-0bbb", func(m map[string]any) { m["placement"] = "ie" })
753	snap := views(t, home, named)
754	roster := rosterFor("i-0aaa", "i-0bbb")
755	for _, b := range []string{"i-0aaa", "i-0bbb"} {
756	rb := roster[b]
757	rb.Placement = "ie"
758	roster[b] = rb
759	}
760	got := eligibleBoxes(snap, roster, "", "eu-west-2", "ie", testKeyFP, nil)
761	if len(got) != 2 {
762	t.Fatalf("got %v, want both home boxes", ids(got))
763	}
764	}
765
766	func TestChooseBox(t *testing.T) {
767	a, b := boxCandidate{BoxID: "i-0aaa"}, boxCandidate{BoxID: "i-0bbb"}
768	load := func(hosts int, apps ...string) boxLoad {
769	l := boxLoad{Hosts: hosts, Apps: map[string]int{}}
770	for _, x := range apps {
771	l.Apps[x]++
772	}
773	return l
774	}
775	const host = "shop.bab.oncarlos.com"
776	if _, ok := chooseBox(host, nil, nil, "bab/jam"); ok {
777	t.Fatal("chose from nothing")
778	}
779	got, _ := chooseBox(host, []boxCandidate{a, b}, map[string]boxLoad{"i-0aaa": load(5), "i-0bbb": load(2)}, "bab/jam")
780	if got.BoxID != "i-0bbb" {
781	t.Fatalf("fewest hosts: got %s", got.BoxID)
782	}
783	got, _ = chooseBox(host, []boxCandidate{a, b}, map[string]boxLoad{"i-0aaa": load(5), "i-0bbb": load(2, "bab/jam")}, "bab/jam")
784	if got.BoxID != "i-0aaa" {
785	t.Fatalf("spread beats load: got %s", got.BoxID)
786	}
787	got, ok := chooseBox(host, []boxCandidate{a, b}, map[string]boxLoad{"i-0aaa": load(5, "bab/jam"), "i-0bbb": load(2, "bab/jam", "bab/jam")}, "bab/jam")
788	if !ok \|\| got.BoxID != "i-0bbb" {
789	t.Fatalf("every box runs the app: fewest hosts overall, never a refusal; got %s ok=%v", got.BoxID, ok)
790	}
791	}
792
793	// H12, claim P5-D2: an equal load breaks by the greatest SHA-256 of
794	// host + "\x00" + reference, so the choice differs host by host instead
795	// of always landing on the lowest name, and does not depend on the order
796	// the candidates arrive in.
797	func TestChooseBoxBreaksATieByTheHostsHash(t *testing.T) {
798	cands := []boxCandidate{{BoxID: "annie-welding"}, {BoxID: "bert-sawing"}, {BoxID: "cleo-mixing"}}
799	want := func(host string) string {
800	best, bestSum := "", ""
801	for _, c := range cands {
802	sum := sha256.Sum256([]byte(host + "\x00" + c.BoxID))
803	if h := hex.EncodeToString(sum[:]); h > bestSum {
804	best, bestSum = c.BoxID, h
805	}
806	}
807	return best
808	}
809	seen := map[string]bool{}
810	for _, host := range []string{"a.bab.oncarlos.com", "b.bab.oncarlos.com", "c.bab.oncarlos.com", "d.bab.oncarlos.com", "e.bab.oncarlos.com", "f.bab.oncarlos.com"} {
811	forward, _ := chooseBox(host, cands, nil, "bab/jam")
812	backward, _ := chooseBox(host, []boxCandidate{cands[2], cands[1], cands[0]}, nil, "bab/jam")
813	if forward.BoxID != want(host) \|\| backward.BoxID != want(host) {
814	t.Fatalf("%s: chose %s and %s, want %s", host, forward.BoxID, backward.BoxID, want(host))
815	}
816	seen[forward.BoxID] = true
817	}
818	if len(seen) < 2 {
819	t.Fatalf("six hosts all chose %v: the tie-break is not per host", seen)
820	}
821	}
822
823	// A box enrolled in another AWS region than the placement's is not in that
824	// placement, even when its own file says it is.
825	func TestEligibleBoxesRefusesABoxEnrolledInAnotherRegion(t *testing.T) {
826	snap := views(t, haBox("i-0aaa", nil), haBox("i-0bbb", nil))
827	roster := rosterFor("i-0aaa", "i-0bbb")
828	roster["i-0bbb"] = rosterBox{Placement: "uk", Cloud: boxid.Cloud{Provider: "aws", Account: "1", Region: "us-east-1", InstanceID: "i-0bbb"}}
829	got := eligibleBoxes(snap, roster, "uk", "eu-west-2", "ie", testKeyFP, nil)
830	if len(got) != 1 \|\| got[0].BoxID != "i-0aaa" {
831	t.Fatalf("got %v, want only i-0aaa", ids(got))
832	}
833	// And a file that advertises another AWS region than the one expected.
834	snap = views(t, haBox("i-0aaa", func(m map[string]any) { m["region"] = "us-east-1" }))
835	if got := eligibleBoxes(snap, rosterFor("i-0aaa"), "uk", "eu-west-2", "ie", testKeyFP, nil); len(got) != 0 {
836	t.Fatalf("got %v, want none", ids(got))
837	}
838	if got := eligibleBoxes(views(t, haBox("i-0aaa", nil)), rosterFor("i-0aaa"), "uk", "", "ie", testKeyFP, nil); len(got) != 0 {
839	t.Fatalf("got %v with no expected region", ids(got))
840	}
841	}
842
843	// The roster's enrolled placement is the enrollment's word and must agree
844	// with the file's; an incomplete cloud is no identity at all.
845	func TestEligibleBoxesNeedsTheRosterToAgree(t *testing.T) {
846	snap := views(t, haBox("i-0aaa", nil), haBox("i-0bbb", nil))
847	roster := rosterFor("i-0aaa", "i-0bbb")
848	roster["i-0aaa"] = rosterBox{Placement: "us", Cloud: roster["i-0aaa"].Cloud}
849	roster["i-0bbb"] = rosterBox{Placement: "uk", Cloud: boxid.Cloud{Provider: "aws", Region: "eu-west-2", InstanceID: "i-0bbb"}}
850	if got := eligibleBoxes(snap, roster, "uk", "eu-west-2", "ie", testKeyFP, nil); len(got) != 0 {
851	t.Fatalf("got %v, want none", ids(got))
852	}
853	}
854	```
855
856	- [ ] Step 2: Run them to see them fail
857
858	Run: `go test ./internal/console/ -run 'EligibleBoxes\|ChooseBox' -count=1`
859	Expected: FAIL to build, `undefined: rosterBox` and `too many arguments in call to chooseBox`.
860
861	- [ ] Step 3: Write the choice
862
863	Replace `internal/console/boxchoice.go` with:
864
865	```go
866	package console
867
868	import (
869	"crypto/sha256"
870	"encoding/hex"
871	"slices"
872	"sort"
873
874	"github.com/carlosframework/platform/internal/boxid"
875	)
876
877	// boxCandidate is one box that may be named as a host's executor (region HA
878	// spec 1 §2, "eligible box").
879	type boxCandidate struct {
880	BoxID string
881	Region string
882	MemFreeMiB, DiskFreeMiB int64
883	Awake int
884	}
885
886	// rosterBox is what the roster (control/host-actors.json, claim D7) says
887	// about one active box: the placement it is enrolled in and the cloud
888	// identity enrollment proved from its boxid record (spec §5.1, H11). The
889	// caller passes active entries only.
890	type rosterBox struct {
891	Placement string
892	Cloud boxid.Cloud
893	}
894
895	// eligibleBoxes is regionCandidates' predicate applied to one box at a time,
896	// plus what a multi-box region adds: a roster entry for the file's box_id,
897	// enrolled in this placement, whose proven cloud is in awsRegion (the AWS
898	// region the roster records for this placement), the HA capability, not
899	// draining, not lost. Two boxes that each satisfy half of it never add up
900	// to one. The edges file is evidence of liveness and configuration only:
901	// identity and region come from the roster (spec §5.1).
902	//
903	// An incomplete snapshot offers nothing: an unread file could be the one
904	// box that would change the answer.
905	func eligibleBoxes(snap edgesSnapshot, roster map[string]rosterBox, placement, awsRegion, homeName, expectedFP string, lost map[string]bool) []boxCandidate {
906	if !snap.complete() \|\| expectedFP == "" \|\| awsRegion == "" {
907	return nil
908	}
909	home := func(code string) string {
910	if homeName != "" && code == homeName {
911	return ""
912	}
913	return code
914	}
915	var out []boxCandidate
916	for _, f := range snap.Files {
917	want := home(placement)
918	rb, ok := roster[f.BoxID]
919	switch {
920	case home(f.Placement) != want:
921	case !f.Fresh, f.EIP == "", f.BoxID == "":
922	case !ok \|\| !rb.Cloud.Complete():
923	case home(rb.Placement) != want:
924	// The file's placement is the box's own word; the roster's is
925	// the enrollment's. Both must agree.
926	case rb.Cloud.Region != awsRegion, f.Region != awsRegion:
927	// The AWS region was proven at enrollment: a box enrolled in
928	// another region is not in this placement, whatever its file
929	// says.
930	case !isTrue(f.HibernationConfigured), !isTrue(f.ServesSharedPool), !isTrue(f.AcceptsInbound):
931	case f.SecretsKeyFP != expectedFP:
932	case !slices.Contains(f.Capabilities, "region-ha-1"):
933	case f.Draining, lost[f.BoxID]:
934	default:
935	c := boxCandidate{BoxID: f.BoxID, Region: rb.Cloud.Region}
936	if f.Headroom != nil {
937	c.MemFreeMiB, c.DiskFreeMiB, c.Awake = f.Headroom.MemFreeMiB, f.Headroom.DiskFreeMiB, f.Headroom.Awake
938	}
939	out = append(out, c)
940	}
941	}
942	sort.Slice(out, func(i, j int) bool { return out[i].BoxID < out[j].BoxID })
943	return out
944	}
945
946	// boxLoad is what claims already place on a box.
947	type boxLoad struct {
948	Hosts int
949	Apps map[string]int // "acct/app" -> hosts of that app
950	}
951
952	// chooseBox is spec 1 §5.2's choose() until spec 2 (H12): prefer a box
953	// that runs no instance of this app (soft spread: it never refuses), then
954	// the fewest hosts overall, then the greatest hexadecimal SHA-256 of
955	// host + "\x00" + box reference (claim P5-D2), then the smallest reference
956	// if two hashes are equal.
957	func chooseBox(host string, cands []boxCandidate, running map[string]boxLoad, app string) (boxCandidate, bool) {
958	if len(cands) == 0 {
959	return boxCandidate{}, false
960	}
961	tie := func(ref string) string {
962	sum := sha256.Sum256([]byte(host + "\x00" + ref))
963	return hex.EncodeToString(sum[:])
964	}
965	best := cands[0]
966	for _, c := range cands[1:] {
967	cl, bl := running[c.BoxID], running[best.BoxID]
968	cRuns, bRuns := cl.Apps[app] > 0, bl.Apps[app] > 0
969	switch {
970	case cRuns != bRuns:
971	if !cRuns {
972	best = c
973	}
974	case cl.Hosts != bl.Hosts:
975	if cl.Hosts < bl.Hosts {
976	best = c
977	}
978	case tie(c.BoxID) != tie(best.BoxID):
979	if tie(c.BoxID) > tie(best.BoxID) {
980	best = c
981	}
982	case c.BoxID < best.BoxID:
983	best = c
984	}
985	}
986	return best, true
987	}
988	```
989
990	- [ ] Step 4: Remove the warm-copy check
991
992	In `internal/activator/installed.go`, replace the `InstallMarker` doc comment:
993
994	```go
995	// InstallMarker is a box's own record that it finished installing one
996	// generation of a host: the database restored and synced, the files
997	// applied, the wake state imported (region HA spec 1 §5.5). It is written
998	// last, so its presence vouches for all of them.
999	//
1000	// It is evidence, not authority. Authority is the claim naming this box,
1001	// which the box holds as its active stamp. A box that crashed mid-install
1002	// has authority and no marker, and must restore.
1003	```
1004
1005	with:
1006
1007	```go
1008	// InstallMarker is a box's own note that it finished a local install of
1009	// one generation of a host. Nothing reads it to trust a local database:
1010	// claim ruling R6 has every admitted writer purge and restore from the
1011	// certified remote (region HA spec 1 H13, §5.5 withdrawn). It is kept for
1012	// relocation's local install journal (§5.8), which plan 1b does not build.
1013	```
1014
1015	Delete the `Installation` type (the block starting `// Installation is what the claim says is current for a host.`) and the whole of `WarmCopyTrusted` with its doc comment, from `// WarmCopyTrusted is invariant HA3` to the end of the file. In `internal/activator/installed_test.go`, delete `TestWarmCopyTrusted`, from `func TestWarmCopyTrusted(t *testing.T) {` to the end of the file.
1016
1017	In `internal/activator/activator.go`, in `doWake`, replace:
1018
1019	```go
1020	// S3 state written by the true holder). When it's still on disk it is the
1021	// warm cache: on one box, with the lease held, the local copy can only be
1022	// what we ourselves checkpointed at the last sleep.
1023	```
1024
1025	with:
1026
1027	```go
1028	// S3 state written by the true holder). When it's still on disk it is the
1029	// warm cache: on one box, with the lease held, the local copy can only be
1030	// what we ourselves checkpointed at the last sleep. That holds only while
1031	// a region has one box. Claim ruling R6 ends it (region HA spec 1 H13):
1032	// once wakes are admitted by hostclaim's AdmitWriter, every admitted
1033	// writer purges its local copy and restores from the certified remote,
1034	// and no marker is ever read to skip that.
1035	```
1036
1037	The behaviour of `doWake` does not change: the warm cache stays correct while every region has one box, and replacing it is the claim's `AdmitWriter` wiring, not this plan's.
1038
1039	- [ ] Step 5: Run the tests
1040
1041	Run:
1042	```bash
1043	go test ./internal/console/ -run 'EligibleBoxes\|ChooseBox\|CandidateLocation' -count=1 -v
1044	go test ./internal/activator/ -run 'InstallMarker\|DamagedMarker' -count=1
1045	```
1046	Expected: PASS, including `TestChooseBoxBreaksATieByTheHostsHash` and `TestEligibleBoxesNeedsTheRosterToAgree`.
1047
1048	- [ ] Step 6: Gate and commit
1049
1050	```bash
1051	go build ./... && go vet ./internal/console/ ./internal/activator/ && gofmt -l internal/console internal/activator
1052	GOOS=darwin go build ./... && GOOS=windows go build ./...
1053	git add internal/console/boxchoice.go internal/console/boxchoice_test.go internal/activator/installed.go internal/activator/installed_test.go internal/activator/activator.go
1054	git commit -m "console: eligible boxes come from the roster's proven cloud, and a tie breaks by the host's hash; activator: no local copy is trusted
1055
1056	Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
1057	```
1058
1059	---
1060
1061	### Task 3: The agent publishes its identity record (`internal/host/boxrecord.go`)
1062
1063	Spec §19: "Item 6 becomes publishing the enrollment evidence at start." Plan 1a's "After the build" item 13: the record is written under the box's label. With `CARLOS_REGION_HA` on, the agent reads IMDS's identity document and signature once and publishes them create-only. The record grants nothing; enrollment reads it later.
1064
1065	Files:
1066	- Create: `internal/host/boxrecord.go`
1067	- Test: `internal/host/boxrecord_test.go`
1068	- Modify: `internal/host/edgesfile.go` (`edgesFileCapabilities`), `internal/host/agent.go:524`
1069
1070	Interfaces:
1071	- Consumes: `boxid.Record`, `boxid.ValidBox`, `boxid.Load`, `boxid.Publish` (Task 1); `openDeployStore`, `metricsBoxLabel`, `readMachineID`, `shouldSkipPublisherElection` (existing).
1072	- Produces:
1073	```go
1074	func regionHAEnabled(getenv func(string) string) bool // used by Tasks 6 and 7
1075	func readIdentityRecord(client *http.Client, base, box string, now time.Time) (boxid.Record, error)
1076	func publishBoxRecord(ctx context.Context, s release.Store, box string, read func() (boxid.Record, error)) (done bool, err error)
1077	func boxRecordLoop()
1078	```
1079
1080	Two things IMDS does that matter. The identity document is signed byte for byte, so it is stored exactly as served: `imdsLeaf` trims, and is not used. The signature may arrive wrapped across lines, which Go's base64 decoder refuses, so whitespace is removed from it.
1081
1082	- [ ] Step 1: Write the failing tests
1083
1084	Create `internal/host/boxrecord_test.go`:
1085
1086	```go
1087	package host
1088
1089	import (
1090	"context"
1091	"encoding/base64"
1092	"errors"
1093	"io"
1094	"net/http"
1095	"strings"
1096	"testing"
1097	"time"
1098
1099	"github.com/carlosframework/platform/internal/boxid"
1100	"github.com/carlosframework/platform/internal/release"
1101	)
1102
1103	// fakeIMDS answers IMDSv2 in-process: no socket, so it runs in any sandbox.
1104	type fakeIMDS struct {
1105	doc, sig string
1106	status int // for the document and signature; 0 = 200
1107	calls []string
1108	}
1109
1110	func (f fakeIMDS) RoundTrip(r http.Request) (*http.Response, error) {
1111	f.calls = append(f.calls, r.Method+" "+r.URL.Path)
1112	answer := func(code int, body string) (*http.Response, error) {
1113	return &http.Response{StatusCode: code, Body: io.NopCloser(strings.NewReader(body)), Header: http.Header{}, Request: r}, nil
1114	}
1115	if r.Method == http.MethodPut && r.URL.Path == "/latest/api/token" {
1116	return answer(200, "tok\n")
1117	}
1118	if r.Header.Get("X-aws-ec2-metadata-token") != "tok" {
1119	return answer(401, "")
1120	}
1121	code := f.status
1122	if code == 0 {
1123	code = 200
1124	}
1125	switch r.URL.Path {
1126	case imdsIdentityDocument:
1127	return answer(code, f.doc)
1128	case imdsIdentitySignature:
1129	return answer(code, f.sig)
1130	}
1131	return answer(404, "")
1132	}
1133
1134	func imdsClient(f fakeIMDS) http.Client { return &http.Client{Transport: f} }
1135
1136	var recordNow = time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
1137
1138	// The document is signed byte for byte, so its trailing newline must
1139	// survive; the signature arrives wrapped and must not.
1140	func TestReadIdentityRecordKeepsTheDocumentsExactBytes(t *testing.T) {
1141	doc := "{\n \"accountId\" : \"111122223333\",\n \"instanceId\" : \"i-0abc\"\n}\n"
1142	sig := base64.StdEncoding.EncodeToString([]byte(strings.Repeat("s", 256)))
1143	wrapped := sig[:64] + "\n" + sig[64:] + "\n"
1144	f := &fakeIMDS{doc: doc, sig: wrapped}
1145	rec, err := readIdentityRecord(imdsClient(f), "http://imds.test", "annie-welding", recordNow)
1146	if err != nil {
1147	t.Fatal(err)
1148	}
1149	got, _ := base64.StdEncoding.DecodeString(rec.IdentityDocument)
1150	if string(got) != doc {
1151	t.Fatalf("document = %q, want the bytes IMDS served, %q", got, doc)
1152	}
1153	if rec.IdentitySignature != sig \|\| rec.V != 1 \|\| rec.Box != "annie-welding" \|\| rec.WrittenAt != "2026-10-06T09:00:00Z" {
1154	t.Fatalf("record = %+v", rec)
1155	}
1156	}
1157
1158	func TestReadIdentityRecordRefusesWhatIsNotAnAnswer(t *testing.T) {
1159	for name, f := range map[string]*fakeIMDS{
1160	"document missing": {doc: "{}", sig: "c2lnCg==", status: 404},
1161	"signature not base64": {doc: "{}", sig: "not base64!"},
1162	"signature empty": {doc: "{}", sig: "\n"},
1163	} {
1164	t.Run(name, func(t *testing.T) {
1165	if rec, err := readIdentityRecord(imdsClient(f), "http://imds.test", "annie-welding", recordNow); err == nil {
1166	t.Fatalf("record = %+v, want an error", rec)
1167	}
1168	})
1169	}
1170	}
1171
1172	func TestPublishBoxRecordWritesOnceAndNeverRewrites(t *testing.T) {
1173	ctx, st := context.Background(), release.NewDirStore(t.TempDir())
1174	reads := 0
1175	read := func() (boxid.Record, error) {
1176	reads++
1177	return boxid.Record{V: 1, Box: "annie-welding", IdentityDocument: "ZG9j", IdentitySignature: "c2ln", WrittenAt: "2026-10-06T09:00:00Z"}, nil
1178	}
1179	if done, err := publishBoxRecord(ctx, st, "annie-welding", read); !done \|\| err != nil {
1180	t.Fatalf("first publish: done=%v err=%v", done, err)
1181	}
1182	if done, err := publishBoxRecord(ctx, st, "annie-welding", read); !done \|\| err != nil {
1183	t.Fatalf("second publish: done=%v err=%v", done, err)
1184	}
1185	if reads != 1 {
1186	t.Fatalf("IMDS read %d times; an existing record must not be re-read or rewritten", reads)
1187	}
1188	}
1189
1190	func TestPublishBoxRecordRetriesAReadFailureAndStopsOnABadLabel(t *testing.T) {
1191	ctx, st := context.Background(), release.NewDirStore(t.TempDir())
1192	failing := func() (boxid.Record, error) { return boxid.Record{}, errors.New("IMDS down") }
1193	if done, err := publishBoxRecord(ctx, st, "annie-welding", failing); done \|\| err == nil {
1194	t.Fatalf("done=%v err=%v: a read failure is retried", done, err)
1195	}
1196	if done, err := publishBoxRecord(ctx, st, "Annie.Welding", failing); !done \|\| err == nil {
1197	t.Fatalf("done=%v err=%v: a reference that is not a label is not retried", done, err)
1198	}
1199	}
1200
1201	// A store that refuses the read with something other than not-found must
1202	// not be taken as "no record yet": that would publish over an outage.
1203	type unreadableBoxStore struct{ release.Store }
1204
1205	func (unreadableBoxStore) Get(context.Context, string) ([]byte, string, error) {
1206	return nil, "", errors.New("AccessDenied")
1207	}
1208
1209	func TestPublishBoxRecordDoesNotPublishWhenItCannotRead(t *testing.T) {
1210	ctx := context.Background()
1211	st := unreadableBoxStore{release.NewDirStore(t.TempDir())}
1212	read := func() (boxid.Record, error) {
1213	t.Fatal("IMDS read although the store could not say whether a record exists")
1214	return boxid.Record{}, nil
1215	}
1216	if done, err := publishBoxRecord(ctx, st, "annie-welding", read); done \|\| err == nil {
1217	t.Fatalf("done=%v err=%v", done, err)
1218	}
1219	}
1220
1221	func TestRegionHAEnabled(t *testing.T) {
1222	for v, want := range map[string]bool{"": false, "0": false, " 0 ": false, "1": true, "yes": true} {
1223	if got := regionHAEnabled(func(string) string { return v }); got != want {
1224	t.Errorf("CARLOS_REGION_HA=%q: %v, want %v", v, got, want)
1225	}
1226	}
1227	}
1228	```
1229
1230	- [ ] Step 2: Run them to see them fail
1231
1232	Run: `go test ./internal/host/ -run 'IdentityRecord\|BoxRecord\|RegionHAEnabled' -count=1`
1233	Expected: FAIL to build, `undefined: readIdentityRecord`.
1234
1235	- [ ] Step 3: Write the publisher
1236
1237	Create `internal/host/boxrecord.go`:
1238
1239	```go
1240	package host
1241
1242	import (
1243	"context"
1244	"encoding/base64"
1245	"errors"
1246	"fmt"
1247	"io"
1248	"log"
1249	"net/http"
1250	"os"
1251	"strings"
1252	"time"
1253
1254	"github.com/carlosframework/platform/internal/boxid"
1255	"github.com/carlosframework/platform/internal/release"
1256	)
1257
1258	// regionHAEnabled is CARLOS_REGION_HA: on for any value but "" and "0".
1259	// Every region HA behaviour on a box is behind it (spec 1 §8 step 2).
1260	func regionHAEnabled(getenv func(string) string) bool {
1261	switch strings.TrimSpace(getenv("CARLOS_REGION_HA")) {
1262	case "", "0":
1263	return false
1264	}
1265	return true
1266	}
1267
1268	const (
1269	imdsBase = "http://169.254.169.254"
1270	imdsIdentityDocument = "/latest/dynamic/instance-identity/document"
1271	imdsIdentitySignature = "/latest/dynamic/instance-identity/signature"
1272	boxRecordRetry = time.Minute
1273	)
1274
1275	// imdsGetRaw reads one IMDSv2 path and returns its body untouched. The
1276	// identity document is signed byte for byte, so unlike imdsLeaf nothing is
1277	// trimmed.
1278	func imdsGetRaw(client *http.Client, base, token, path string) ([]byte, error) {
1279	req, err := http.NewRequest(http.MethodGet, base+path, nil)
1280	if err != nil {
1281	return nil, err
1282	}
1283	req.Header.Set("X-aws-ec2-metadata-token", token)
1284	resp, err := client.Do(req)
1285	if err != nil {
1286	return nil, err
1287	}
1288	defer resp.Body.Close()
1289	body, err := io.ReadAll(io.LimitReader(resp.Body, 64<<10))
1290	if err != nil {
1291	return nil, err
1292	}
1293	if resp.StatusCode != http.StatusOK {
1294	return nil, fmt.Errorf("IMDS %s answered %d", path, resp.StatusCode)
1295	}
1296	return body, nil
1297	}
1298
1299	// readIdentityRecord builds this box's record (spec §5.1) from IMDS: the
1300	// identity document exactly as served, and its base64 RSA-SHA256
1301	// signature with any line breaks removed (base64 ignores them; Go's
1302	// decoder does not).
1303	func readIdentityRecord(client *http.Client, base, box string, now time.Time) (boxid.Record, error) {
1304	req, err := http.NewRequest(http.MethodPut, base+"/latest/api/token", nil)
1305	if err != nil {
1306	return boxid.Record{}, err
1307	}
1308	req.Header.Set("X-aws-ec2-metadata-token-ttl-seconds", "60")
1309	resp, err := client.Do(req)
1310	if err != nil {
1311	return boxid.Record{}, fmt.Errorf("IMDS token: %w", err)
1312	}
1313	tok, _ := io.ReadAll(io.LimitReader(resp.Body, 4<<10))
1314	resp.Body.Close()
1315	if resp.StatusCode != http.StatusOK \|\| len(strings.TrimSpace(string(tok))) == 0 {
1316	return boxid.Record{}, fmt.Errorf("IMDS token answered %d", resp.StatusCode)
1317	}
1318	token := strings.TrimSpace(string(tok))
1319	doc, err := imdsGetRaw(client, base, token, imdsIdentityDocument)
1320	if err != nil {
1321	return boxid.Record{}, err
1322	}
1323	sig, err := imdsGetRaw(client, base, token, imdsIdentitySignature)
1324	if err != nil {
1325	return boxid.Record{}, err
1326	}
1327	cleaned := strings.Join(strings.Fields(string(sig)), "")
1328	if _, err := base64.StdEncoding.DecodeString(cleaned); err != nil \|\| cleaned == "" {
1329	return boxid.Record{}, errors.New("IMDS signature is not base64")
1330	}
1331	return boxid.Record{V: 1, Box: box, IdentityDocument: base64.StdEncoding.EncodeToString(doc),
1332	IdentitySignature: cleaned, WrittenAt: now.UTC().Format(time.RFC3339)}, nil
1333	}
1334
1335	// publishBoxRecord makes one attempt to publish this box's enrollment
1336	// evidence. done is true once there is nothing left to retry: the record
1337	// is in place, or a different one already holds this reference, which
1338	// only an operator can sort out and which is logged loudly.
1339	func publishBoxRecord(ctx context.Context, s release.Store, box string, read func() (boxid.Record, error)) (done bool, err error) {
1340	if !boxid.ValidBox(box) {
1341	return true, fmt.Errorf("box reference %q is not a label; no enrollment evidence is published", box)
1342	}
1343	if _, err := boxid.Load(ctx, s, box); err == nil {
1344	return true, nil // create-only: an existing record is never rewritten
1345	} else if !errors.Is(err, os.ErrNotExist) {
1346	return false, err
1347	}
1348	rec, err := read()
1349	if err != nil {
1350	return false, err
1351	}
1352	err = boxid.Publish(ctx, s, rec)
1353	if errors.Is(err, release.ErrCASConflict) {
1354	return true, fmt.Errorf("another record already holds %s: %w", box, err)
1355	}
1356	return err == nil, err
1357	}
1358
1359	// boxRecordLoop publishes this box's enrollment evidence once, at start,
1360	// when CARLOS_REGION_HA is on, retrying each minute until it is in place.
1361	// The record grants nothing: enrollment (an operator, or for Auto Scaling
1362	// boxes the console with a membership confirmation) reads it later.
1363	// Platform-box job: skipped on fleet boxes.
1364	func boxRecordLoop() {
1365	if !regionHAEnabled(os.Getenv) \|\| shouldSkipPublisherElection(os.Getenv) {
1366	return
1367	}
1368	ctx := context.Background()
1369	store, err := openDeployStore(ctx)
1370	if err != nil {
1371	log.Printf("agent: box record not published: %v", err)
1372	return
1373	}
1374	box := metricsBoxLabel(os.Getenv, readMachineID())
1375	client := &http.Client{Timeout: 2 * time.Second}
1376	read := func() (boxid.Record, error) { return readIdentityRecord(client, imdsBase, box, time.Now()) }
1377	for attempt := 1; ; attempt++ {
1378	done, err := publishBoxRecord(ctx, store, box, read)
1379	if done {
1380	if err != nil {
1381	log.Printf("agent: box record for %s: %v (region HA cannot enroll this box until an operator resolves it)", box, err)
1382	}
1383	return
1384	}
1385	if attempt == 1 \|\| attempt%60 == 0 {
1386	log.Printf("agent: box record for %s not published yet (#%d): %v", box, attempt, err)
1387	}
1388	time.Sleep(boxRecordRetry)
1389	}
1390	}
1391	```
1392
1393	- [ ] Step 4: Read the flag in one place
1394
1395	In `internal/host/edgesfile.go`, replace the body of `edgesFileCapabilities`:
1396
1397	```go
1398	func edgesFileCapabilities(getenv func(string) string) []string {
1399	switch strings.TrimSpace(getenv("CARLOS_REGION_HA")) {
1400	case "", "0":
1401	return nil
1402	}
1403	return []string{capRegionHA1}
1404	}
1405	```
1406
1407	with:
1408
1409	```go
1410	func edgesFileCapabilities(getenv func(string) string) []string {
1411	if !regionHAEnabled(getenv) {
1412	return nil
1413	}
1414	return []string{capRegionHA1}
1415	}
1416	```
1417
1418	- [ ] Step 5: Start the loop
1419
1420	In `internal/host/agent.go`, in `RunAgent`, replace:
1421
1422	```go
1423	go edgesFileLoop(reg)
1424	```
1425
1426	with:
1427
1428	```go
1429	go edgesFileLoop(reg)
1430	// Region HA enrollment evidence (spec 1 §5.1): once, at start, and only
1431	// with CARLOS_REGION_HA on. See boxrecord.go.
1432	go boxRecordLoop()
1433	```
1434
1435	- [ ] Step 6: Run the tests
1436
1437	Run: `go test ./internal/host/ -run 'IdentityRecord\|BoxRecord\|RegionHAEnabled\|EdgesFileCapability\|Headroom' -count=1 -v`
1438	Expected: PASS.
1439
1440	- [ ] Step 7: Gate and commit
1441
1442	```bash
1443	go build ./... && go vet ./internal/host/ && gofmt -l internal/host
1444	GOOS=darwin go build ./... && GOOS=windows go build ./...
1445	git add internal/host/boxrecord.go internal/host/boxrecord_test.go internal/host/edgesfile.go internal/host/agent.go
1446	git commit -m "agent: with region HA on, a box publishes its AWS-signed identity once, under its label
1447
1448	Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
1449	```
1450
1451	---
1452
1453	### Task 4: The fence binds to the roster entry (`internal/termfence`)
1454
1455	Spec §19: "`Verify` binds to a `boxid.Identity`; it must bind to the roster entry's `cloud` and produce §5.9's record. It takes no volume input (H19) … `FromDescribe` and `FromEvent` stay as the two terminated reads; a `shutting-down` observation is never accepted." §6 adds that the fence refuses any observation not made in the box's own region. This task also gives the describe a seam: one call, pinned to the box's region, with `InvalidInstanceID.NotFound` read as absence.
1456
1457	Writing the record to `control/box-fences/<enrollment>.json`, and citing it in a claim CAS, are L2 to L4 and are not built.
1458
1459	Files:
1460	- Modify (replace whole file): `internal/termfence/fence.go`
1461	- Test (replace whole file): `internal/termfence/fence_test.go`
1462	- Modify: `internal/boxid/verify.go` (remove `Identity`)
1463
1464	Interfaces:
1465	- Consumes: `boxid.Cloud` (Task 1).
1466	- Produces:
1467	```go
1468	package termfence
1469	const ViaDescribe, ViaEvent = "describe", "event"
1470	var ErrNotEvidence error
1471	type Observation struct{ Via string; Found bool; InstanceID, Account, Region, State, EventID, RequestID string }
1472	type Box struct{ Ref, Enrollment string; Cloud boxid.Cloud }
1473	type WrittenBy struct{ Box, Enrollment string } // json: box, enrollment
1474	type Record struct{ V int; Box, Enrollment string; Cloud boxid.Cloud; State, ObservedAt, Via, RequestID, EventID string; WrittenBy WrittenBy } // §5.9
1475	type EC2 interface{ DescribeInstances(ctx, ec2.DescribeInstancesInput, ...func(ec2.Options)) (*ec2.DescribeInstancesOutput, error) }
1476	func FromDescribe(out *ec2.DescribeInstancesOutput, instanceID, region string) Observation
1477	func FromEvent(body []byte) (Observation, error)
1478	func DescribeInstance(ctx context.Context, api EC2, c boxid.Cloud) (*ec2.DescribeInstancesOutput, error) // used by Task 5's callers too
1479	func Verify(box, worker Box, o Observation, now time.Time) (Record, error)
1480	```
1481	Removed: `Fence`, and `Verify`'s `(string, boxid.Identity, …)` form.
1482
1483	- [ ] Step 1: Write the failing tests
1484
1485	Replace `internal/termfence/fence_test.go` with:
1486
1487	```go
1488	package termfence
1489
1490	import (
1491	"context"
1492	"errors"
1493	"testing"
1494	"time"
1495
1496	"github.com/aws/aws-sdk-go-v2/aws"
1497	awsmiddleware "github.com/aws/aws-sdk-go-v2/aws/middleware"
1498	"github.com/aws/aws-sdk-go-v2/service/ec2"
1499	"github.com/aws/aws-sdk-go-v2/service/ec2/types"
1500	"github.com/aws/smithy-go"
1501	smithymiddleware "github.com/aws/smithy-go/middleware"
1502
1503	"github.com/carlosframework/platform/internal/boxid"
1504	)
1505
1506	var (
1507	box = Box{Ref: "annie-welding", Enrollment: "enr-annie-1",
1508	Cloud: boxid.Cloud{Provider: "aws", Account: "111122223333", Region: "eu-west-1", InstanceID: "i-0abc"}}
1509	worker = Box{Ref: "bert-sawing", Enrollment: "enr-bert-1",
1510	Cloud: boxid.Cloud{Provider: "aws", Account: "111122223333", Region: "eu-west-1", InstanceID: "i-0bbb"}}
1511	now = time.Date(2026, 10, 6, 12, 0, 0, 0, time.UTC)
1512	)
1513
1514	func describe(owner, instance string, state types.InstanceStateName) *ec2.DescribeInstancesOutput {
1515	return &ec2.DescribeInstancesOutput{Reservations: []types.Reservation{{
1516	OwnerId: aws.String(owner),
1517	Instances: []types.Instance{{InstanceId: aws.String(instance), State: &types.InstanceState{Name: state}}},
1518	}}}
1519	}
1520
1521	func withRequestID(out ec2.DescribeInstancesOutput, id string) ec2.DescribeInstancesOutput {
1522	var md smithymiddleware.Metadata
1523	awsmiddleware.SetRequestIDMetadata(&md, id)
1524	out.ResultMetadata = md
1525	return out
1526	}
1527
1528	func TestATerminatedInstanceOfThisBoxIsAFence(t *testing.T) {
1529	out := withRequestID(describe("111122223333", "i-0abc", types.InstanceStateNameTerminated), "req-1")
1530	rec, err := Verify(box, worker, FromDescribe(out, "i-0abc", "eu-west-1"), now)
1531	if err != nil {
1532	t.Fatal(err)
1533	}
1534	want := Record{V: 1, Box: "annie-welding", Enrollment: "enr-annie-1", Cloud: box.Cloud, State: "terminated",
1535	ObservedAt: "2026-10-06T12:00:00Z", Via: "describe", RequestID: "req-1",
1536	WrittenBy: WrittenBy{Box: "bert-sawing", Enrollment: "enr-bert-1"}}
1537	if rec != want {
1538	t.Fatalf("record = %+v\nwant %+v", rec, want)
1539	}
1540	}
1541
1542	// Every lookalike is refused.
1543	func TestLookalikesAreNotEvidence(t *testing.T) {
1544	cases := map[string]Observation{
1545	"stopped": FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameStopped), "i-0abc", "eu-west-1"),
1546	"stopping": FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameStopping), "i-0abc", "eu-west-1"),
1547	"shutting-down": FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameShuttingDown), "i-0abc", "eu-west-1"),
1548	"running": FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameRunning), "i-0abc", "eu-west-1"),
1549	"another account": FromDescribe(describe("999999999999", "i-0abc", types.InstanceStateNameTerminated), "i-0abc", "eu-west-1"),
1550	"another region": FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameTerminated), "i-0abc", "us-east-1"),
1551	"another instance": FromDescribe(describe("111122223333", "i-0zzz", types.InstanceStateNameTerminated), "i-0zzz", "eu-west-1"),
1552	"not in the reply": FromDescribe(describe("111122223333", "i-0zzz", types.InstanceStateNameTerminated), "i-0abc", "eu-west-1"),
1553	"empty reply": FromDescribe(&ec2.DescribeInstancesOutput{}, "i-0abc", "eu-west-1"),
1554	"nil reply": FromDescribe(nil, "i-0abc", "eu-west-1"),
1555	"no state": FromDescribe(&ec2.DescribeInstancesOutput{Reservations: []types.Reservation{{OwnerId: aws.String("111122223333"), Instances: []types.Instance{{InstanceId: aws.String("i-0abc")}}}}}, "i-0abc", "eu-west-1"),
1556	"unknown source": {Via: "cloudtrail", Found: true, InstanceID: "i-0abc", Account: "111122223333", Region: "eu-west-1", State: "terminated"},
1557	"zero observation": {},
1558	}
1559	for name, o := range cases {
1560	t.Run(name, func(t *testing.T) {
1561	if rec, err := Verify(box, worker, o, now); !errors.Is(err, ErrNotEvidence) {
1562	t.Fatalf("record = %+v, err = %v; want ErrNotEvidence", rec, err)
1563	}
1564	})
1565	}
1566	}
1567
1568	// Review Focus 5: the fence is a regional worker's own read in the box's
1569	// own region. A worker enrolled elsewhere is refused even when what it
1570	// read is a perfect terminated observation.
1571	func TestAWorkerInAnotherRegionNeverFences(t *testing.T) {
1572	far := worker
1573	far.Cloud.Region = "us-east-1"
1574	o := FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameTerminated), "i-0abc", "eu-west-1")
1575	if _, err := Verify(box, far, o, now); !errors.Is(err, ErrNotEvidence) {
1576	t.Fatalf("err = %v, want ErrNotEvidence", err)
1577	}
1578	}
1579
1580	func TestAnIncompleteRosterEntryIsNeverFenced(t *testing.T) {
1581	// A found, terminated observation whose every coordinate is empty, so
1582	// it MATCHES an empty cloud field for field. Only the guard on the
1583	// entry itself can refuse it.
1584	o := Observation{Via: ViaDescribe, Found: true, State: "terminated"}
1585	for name, b := range map[string]Box{
1586	"empty cloud": {Ref: "annie-welding", Enrollment: "enr-annie-1"},
1587	"no enrollment": {Ref: "annie-welding", Cloud: box.Cloud},
1588	"no reference": {Enrollment: "enr-annie-1", Cloud: box.Cloud},
1589	"not aws": {Ref: "annie-welding", Enrollment: "enr-annie-1", Cloud: boxid.Cloud{Provider: "gcp", Account: "111122223333", Region: "eu-west-1", InstanceID: "i-0abc"}},
1590	} {
1591	t.Run(name, func(t *testing.T) {
1592	full := Observation{Via: ViaDescribe, Found: true, InstanceID: b.Cloud.InstanceID, Account: b.Cloud.Account, Region: b.Cloud.Region, State: "terminated"}
1593	for _, obs := range []Observation{o, full} {
1594	if _, err := Verify(b, worker, obs, now); !errors.Is(err, ErrNotEvidence) {
1595	t.Fatalf("err = %v, want ErrNotEvidence", err)
1596	}
1597	}
1598	})
1599	}
1600	if _, err := Verify(box, Box{Ref: "bert-sawing"}, FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameTerminated), "i-0abc", "eu-west-1"), now); !errors.Is(err, ErrNotEvidence) {
1601	t.Fatalf("a worker with no proven identity: err = %v", err)
1602	}
1603	}
1604
1605	const terminatedEvent = `{"version":"0","id":"ev-123","detail-type":"EC2 Instance State-change Notification","source":"aws.ec2","account":"111122223333","time":"2026-10-06T11:59:00Z","region":"eu-west-1","resources":["arn:aws:ec2:eu-west-1:111122223333:instance/i-0abc"],"detail":{"instance-id":"i-0abc","state":"terminated"}}`
1606
1607	func TestATerminatedEventIsAFence(t *testing.T) {
1608	o, err := FromEvent([]byte(terminatedEvent))
1609	if err != nil {
1610	t.Fatal(err)
1611	}
1612	rec, err := Verify(box, worker, o, now)
1613	if err != nil {
1614	t.Fatal(err)
1615	}
1616	if rec.Via != "event" \|\| rec.EventID != "ev-123" \|\| rec.State != "terminated" \|\| rec.RequestID != "" {
1617	t.Fatalf("record = %+v", rec)
1618	}
1619	}
1620
1621	func TestEventsThatAreNotEC2StateChangesAreRefused(t *testing.T) {
1622	for name, body := range map[string]string{
1623	"not JSON": `terminated`,
1624	"another source": `{"id":"e","detail-type":"EC2 Instance State-change Notification","source":"my.app","account":"111122223333","region":"eu-west-1","detail":{"instance-id":"i-0abc","state":"terminated"}}`,
1625	"another detail-type": `{"id":"e","detail-type":"EC2 Spot Instance Interruption Warning","source":"aws.ec2","account":"111122223333","region":"eu-west-1","detail":{"instance-id":"i-0abc","state":"terminated"}}`,
1626	"no event id": `{"detail-type":"EC2 Instance State-change Notification","source":"aws.ec2","account":"111122223333","region":"eu-west-1","detail":{"instance-id":"i-0abc","state":"terminated"}}`,
1627	} {
1628	t.Run(name, func(t *testing.T) {
1629	if _, err := FromEvent([]byte(body)); err == nil {
1630	t.Fatal("accepted")
1631	}
1632	})
1633	}
1634	}
1635
1636	func TestAShuttingDownEventIsNotEvidence(t *testing.T) {
1637	body := `{"id":"e","detail-type":"EC2 Instance State-change Notification","source":"aws.ec2","account":"111122223333","region":"eu-west-1","detail":{"instance-id":"i-0abc","state":"shutting-down"}}`
1638	o, err := FromEvent([]byte(body))
1639	if err != nil {
1640	t.Fatal(err)
1641	}
1642	if _, err := Verify(box, worker, o, now); !errors.Is(err, ErrNotEvidence) {
1643	t.Fatalf("err = %v, want ErrNotEvidence", err)
1644	}
1645	}
1646
1647	// fakeEC2 answers DescribeInstances and records the region each call was
1648	// forced into.
1649	type fakeEC2 struct {
1650	out *ec2.DescribeInstancesOutput
1651	err error
1652	regions []string
1653	ids [][]string
1654	}
1655
1656	func (f fakeEC2) DescribeInstances(_ context.Context, in ec2.DescribeInstancesInput, optFns ...func(ec2.Options)) (ec2.DescribeInstancesOutput, error) {
1657	o := ec2.Options{Region: "client-default"}
1658	for _, fn := range optFns {
1659	fn(&o)
1660	}
1661	f.regions = append(f.regions, o.Region)
1662	f.ids = append(f.ids, in.InstanceIds)
1663	return f.out, f.err
1664	}
1665
1666	func TestDescribeInstanceAsksInTheBoxsOwnRegion(t *testing.T) {
1667	f := &fakeEC2{out: describe("111122223333", "i-0abc", types.InstanceStateNameTerminated)}
1668	out, err := DescribeInstance(context.Background(), f, box.Cloud)
1669	if err != nil {
1670	t.Fatal(err)
1671	}
1672	if len(f.regions) != 1 \|\| f.regions[0] != "eu-west-1" \|\| len(f.ids[0]) != 1 \|\| f.ids[0][0] != "i-0abc" {
1673	t.Fatalf("called in %v for %v", f.regions, f.ids)
1674	}
1675	if _, err := Verify(box, worker, FromDescribe(out, "i-0abc", "eu-west-1"), now); err != nil {
1676	t.Fatal(err)
1677	}
1678	}
1679
1680	func TestDescribeInstanceNotFoundIsAbsenceAndOtherErrorsAreErrors(t *testing.T) {
1681	notFound := &fakeEC2{err: &smithy.GenericAPIError{Code: "InvalidInstanceID.NotFound", Message: "gone"}}
1682	out, err := DescribeInstance(context.Background(), notFound, box.Cloud)
1683	if err != nil {
1684	t.Fatalf("NotFound: err = %v", err)
1685	}
1686	if _, err := Verify(box, worker, FromDescribe(out, "i-0abc", "eu-west-1"), now); !errors.Is(err, ErrNotEvidence) {
1687	t.Fatalf("NotFound fenced the box: err = %v", err)
1688	}
1689	denied := &fakeEC2{err: &smithy.GenericAPIError{Code: "UnauthorizedOperation", Message: "no"}}
1690	if _, err := DescribeInstance(context.Background(), denied, box.Cloud); err == nil {
1691	t.Fatal("an access error read as an answer")
1692	}
1693	if _, err := DescribeInstance(context.Background(), &fakeEC2{}, boxid.Cloud{InstanceID: "i-0abc"}); err == nil {
1694	t.Fatal("described an incomplete cloud")
1695	}
1696	}
1697	```
1698
1699	- [ ] Step 2: Run them to see them fail
1700
1701	Run: `go test ./internal/termfence/ -count=1`
1702	Expected: FAIL to build, `undefined: Box` and `undefined: DescribeInstance`.
1703
1704	- [ ] Step 3: Write the fence
1705
1706	Replace `internal/termfence/fence.go` with:
1707
1708	```go
1709	// Package termfence decides whether an observation proves that a box's
1710	// machine has been terminated (region HA spec 1 §5.9, invariants HA7 and
1711	// H19).
1712	//
1713	// Only the state "terminated" counts, only for the instance, account and
1714	// region the box's roster entry records under cloud, and only when the
1715	// worker that read it is in that same region. Silence is never evidence,
1716	// and no disk condition applies: volume records decide only whether the
1717	// dead box's local chores are discharged (internal/boxvolumes).
1718	//
1719	// This package reads and decides. Writing the fence record and citing it
1720	// in a claim CAS wait for the claim's acceptance of L2 to L5.
1721	package termfence
1722
1723	import (
1724	"context"
1725	"encoding/json"
1726	"errors"
1727	"fmt"
1728	"time"
1729
1730	"github.com/aws/aws-sdk-go-v2/aws"
1731	awsmiddleware "github.com/aws/aws-sdk-go-v2/aws/middleware"
1732	"github.com/aws/aws-sdk-go-v2/service/ec2"
1733	"github.com/aws/smithy-go"
1734
1735	"github.com/carlosframework/platform/internal/boxid"
1736	)
1737
1738	const (
1739	ViaDescribe = "describe"
1740	ViaEvent = "event"
1741
1742	stateTerminated = "terminated"
1743	eventSource = "aws.ec2"
1744	eventDetailType = "EC2 Instance State-change Notification"
1745	)
1746
1747	// ErrNotEvidence wraps every reason an observation does not fence a box.
1748	var ErrNotEvidence = errors.New("termfence: not evidence of termination")
1749
1750	// Observation is what one source said about one instance. Found is false
1751	// when the source said nothing about it: an empty reply, another instance.
1752	type Observation struct {
1753	Via string
1754	Found bool
1755	InstanceID string
1756	Account string
1757	Region string
1758	State string
1759	EventID string
1760	RequestID string
1761	}
1762
1763	// Box is a roster entry as the fence needs it: the reference, the
1764	// enrollment, and the cloud identity enrollment proved (boxid.Enroll).
1765	type Box struct {
1766	Ref string
1767	Enrollment string
1768	Cloud boxid.Cloud
1769	}
1770
1771	func (b Box) complete() bool { return b.Ref != "" && b.Enrollment != "" && b.Cloud.Complete() }
1772
1773	// WrittenBy names the regional worker that made the read.
1774	type WrittenBy struct {
1775	Box string `json:"box"`
1776	Enrollment string `json:"enrollment"`
1777	}
1778
1779	// Record is §5.9's fence record. Its key (control/box-fences/<enrollment>.json)
1780	// and its writer belong to the claim's L4 and are not built here.
1781	type Record struct {
1782	V int `json:"v"`
1783	Box string `json:"box"`
1784	Enrollment string `json:"enrollment"`
1785	Cloud boxid.Cloud `json:"cloud"`
1786	State string `json:"state"`
1787	ObservedAt string `json:"observed_at"`
1788	Via string `json:"via"`
1789	RequestID string `json:"request_id,omitempty"`
1790	EventID string `json:"event_id,omitempty"`
1791	WrittenBy WrittenBy `json:"written_by"`
1792	}
1793
1794	// FromDescribe reads one instance out of a DescribeInstances reply. region
1795	// is the region the call was made in: the reply does not carry it. A reply
1796	// that does not mention instanceID yields Found false, which Verify
1797	// refuses: AWS forgets a terminated instance within about an hour, so
1798	// absence proves nothing.
1799	func FromDescribe(out *ec2.DescribeInstancesOutput, instanceID, region string) Observation {
1800	o := Observation{Via: ViaDescribe, Region: region}
1801	if out == nil \|\| instanceID == "" {
1802	return o
1803	}
1804	if id, ok := awsmiddleware.GetRequestIDMetadata(out.ResultMetadata); ok {
1805	o.RequestID = id
1806	}
1807	for _, r := range out.Reservations {
1808	for _, inst := range r.Instances {
1809	if aws.ToString(inst.InstanceId) != instanceID {
1810	continue
1811	}
1812	o.Found, o.InstanceID, o.Account = true, instanceID, aws.ToString(r.OwnerId)
1813	if inst.State != nil {
1814	o.State = string(inst.State.Name)
1815	}
1816	return o
1817	}
1818	}
1819	return o
1820	}
1821
1822	// FromEvent reads an EventBridge "EC2 Instance State-change Notification".
1823	// It refuses anything else: the queue's policy admits only the rule that
1824	// forwards these, and this is the second check.
1825	func FromEvent(body []byte) (Observation, error) {
1826	var ev struct {
1827	ID string `json:"id"`
1828	DetailType string `json:"detail-type"`
1829	Source string `json:"source"`
1830	Account string `json:"account"`
1831	Region string `json:"region"`
1832	Detail struct {
1833	InstanceID string `json:"instance-id"`
1834	State string `json:"state"`
1835	} `json:"detail"`
1836	}
1837	if err := json.Unmarshal(body, &ev); err != nil {
1838	return Observation{}, fmt.Errorf("termfence: event is not JSON: %w", err)
1839	}
1840	switch {
1841	case ev.Source != eventSource:
1842	return Observation{}, fmt.Errorf("termfence: event source %q is not %s", ev.Source, eventSource)
1843	case ev.DetailType != eventDetailType:
1844	return Observation{}, fmt.Errorf("termfence: event detail-type %q is not a state change", ev.DetailType)
1845	case ev.ID == "":
1846	return Observation{}, errors.New("termfence: event has no id")
1847	}
1848	return Observation{Via: ViaEvent, Found: ev.Detail.InstanceID != "", InstanceID: ev.Detail.InstanceID,
1849	Account: ev.Account, Region: ev.Region, State: ev.Detail.State, EventID: ev.ID}, nil
1850	}
1851
1852	// EC2 is the one call this package makes. *ec2.Client satisfies it; tests
1853	// use a fake. It is a read: the box role holds no EC2 write right.
1854	type EC2 interface {
1855	DescribeInstances(ctx context.Context, in ec2.DescribeInstancesInput, optFns ...func(ec2.Options)) (*ec2.DescribeInstancesOutput, error)
1856	}
1857
1858	// DescribeInstance asks EC2 about c.InstanceID alone, in c.Region whatever
1859	// region the client was built for. InvalidInstanceID.NotFound comes back
1860	// as an empty reply, which FromDescribe turns into Found false: absence,
1861	// never termination. Every other error is returned.
1862	func DescribeInstance(ctx context.Context, api EC2, c boxid.Cloud) (*ec2.DescribeInstancesOutput, error) {
1863	if !c.Complete() {
1864	return nil, errors.New("termfence: describe of an incomplete cloud identity")
1865	}
1866	region := c.Region
1867	out, err := api.DescribeInstances(ctx, &ec2.DescribeInstancesInput{InstanceIds: []string{c.InstanceID}},
1868	func(o *ec2.Options) { o.Region = region })
1869	var apiErr smithy.APIError
1870	if errors.As(err, &apiErr) && apiErr.ErrorCode() == "InvalidInstanceID.NotFound" {
1871	return &ec2.DescribeInstancesOutput{}, nil
1872	}
1873	return out, err
1874	}
1875
1876	func refuse(format string, a ...any) error {
1877	return fmt.Errorf("%w: %s", ErrNotEvidence, fmt.Sprintf(format, a...))
1878	}
1879
1880	// Verify returns box's fence record when o proves that the instance its
1881	// roster entry records is terminated, read by worker, a regional worker of
1882	// the same AWS region (spec §6: the fence refuses any observation not made
1883	// in the box's own region). Whether worker is active in the roster is the
1884	// caller's check, made before it reads.
1885	func Verify(box, worker Box, o Observation, now time.Time) (Record, error) {
1886	c := box.Cloud
1887	switch {
1888	case !box.complete():
1889	return Record{}, refuse("the box has no proven cloud identity")
1890	case !worker.complete():
1891	return Record{}, refuse("the worker has no proven cloud identity")
1892	case worker.Cloud.Region != c.Region:
1893	return Record{}, refuse("worker %s is in %q, the box in %q", worker.Ref, worker.Cloud.Region, c.Region)
1894	case o.Via != ViaDescribe && o.Via != ViaEvent:
1895	return Record{}, refuse("source %q is not one this binary accepts", o.Via)
1896	case !o.Found:
1897	return Record{}, refuse("the source said nothing about %s; absence is not termination", c.InstanceID)
1898	case o.InstanceID != c.InstanceID:
1899	return Record{}, refuse("observed %s, the box is %s", o.InstanceID, c.InstanceID)
1900	case o.Account != c.Account:
1901	return Record{}, refuse("observed in account %q, the box is in %q", o.Account, c.Account)
1902	case o.Region != c.Region:
1903	return Record{}, refuse("observed in region %q, the box is in %q", o.Region, c.Region)
1904	case o.State != stateTerminated:
1905	return Record{}, refuse("state is %q; only %q counts", o.State, stateTerminated)
1906	}
1907	return Record{V: 1, Box: box.Ref, Enrollment: box.Enrollment, Cloud: c, State: o.State,
1908	ObservedAt: now.UTC().Format(time.RFC3339), Via: o.Via, RequestID: o.RequestID, EventID: o.EventID,
1909	WrittenBy: WrittenBy{Box: worker.Ref, Enrollment: worker.Enrollment}}, nil
1910	}
1911	```
1912
1913	- [ ] Step 4: Remove `boxid.Identity`
1914
1915	Nothing uses it now. In `internal/boxid/verify.go`, delete:
1916
1917	```go
1918	// Identity is what an identity document proves. termfence binds to it
1919	// until it moves to Cloud; nothing else uses it.
1920	type Identity struct{ AccountID, Region, InstanceID string }
1921
1922	```
1923
1924	Then check: `grep -rn "boxid.Identity" --include=*.go .` prints nothing.
1925
1926	- [ ] Step 5: Run the tests
1927
1928	Run: `go test ./internal/termfence/ ./internal/boxid/ -count=1 -v`
1929	Expected: PASS, including `TestAWorkerInAnotherRegionNeverFences` and `TestDescribeInstanceNotFoundIsAbsenceAndOtherErrorsAreErrors`.
1930
1931	- [ ] Step 6: Gate and commit
1932
1933	```bash
1934	go build ./... && go vet ./internal/termfence/ ./internal/boxid/ && gofmt -l internal/termfence internal/boxid
1935	GOOS=darwin go build ./... && GOOS=windows go build ./...
1936	git add internal/termfence internal/boxid/verify.go
1937	git commit -m "termfence: a fence binds to the roster entry's cloud and is read by a worker in the box's own region
1938
1939	Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
1940	```
1941
1942	---
1943
1944	### Task 5: Volume records and the discharge check (`internal/boxvolumes`)
1945
1946	Spec §5.9 under H19: regional workers record each enrolled box's block device mappings while it runs, create-only, on change and at least hourly; a removal or any `false` flag rules out automatic discharge for good. A separate function decides discharge from the records. This task builds the record, the decision to write one, the writer, the reader and the discharge check. The loop that describes every enrolled box in the region needs the roster's enrollments (L7), and the discharge CAS is L3: both wait.
1947
1948	Files:
1949	- Create: `internal/boxvolumes/volumes.go`
1950	- Test: `internal/boxvolumes/volumes_test.go`
1951
1952	Interfaces:
1953	- Consumes: `boxid.Cloud` (Task 1); `release.Store`. Callers get the describe reply from `termfence.DescribeInstance` (Task 4).
1954	- Produces:
1955	```go
1956	package boxvolumes
1957	const Prefix = "control/box-volumes/"
1958	const RefreshEvery, EnrollmentWindow = time.Hour, time.Hour
1959	type Device struct{ Device, VolumeID string; DeleteOnTermination bool; Status string }
1960	type WrittenBy struct{ Box, Enrollment string }
1961	type Record struct{ V int; Box, Enrollment string; Cloud boxid.Cloud; InstanceState string; BlockDevices []Device; RemovedVolumes []string; ObservedAt, RequestID string; WrittenBy WrittenBy } // §5.9
1962	type Entry struct{ Box, Enrollment string; Cloud boxid.Cloud }
1963	type Observation struct{ State string; Devices []Device; RequestID string }
1964	var ErrNotRecordable error
1965	func ValidEnrollment(e string) bool
1966	func Key(enrollment string, t time.Time) string
1967	func Observe(out *ec2.DescribeInstancesOutput, c boxid.Cloud) (Observation, error)
1968	func Next(prev *Record, e Entry, writer WrittenBy, o Observation, now time.Time) (Record, bool)
1969	func Write(ctx context.Context, s release.Store, rec Record) (string, error)
1970	func List(ctx context.Context, s release.Store, enrollment string) ([]Record, error)
1971	type Verdict struct{ Discharge bool; Reason string }
1972	const LastRecordWindow = 2 * time.Hour
1973	func Discharge(records []Record, e Entry, enrolledAt, terminatedAt time.Time) Verdict
1974	```
1975
1976	Five choices the spec leaves to the plan.
1977	- A `terminated` read is never recorded. Its mappings are empty (the dev test of 2026-10-06), and a record of it would list every volume as removed.
1978	- "The first record taken at enrollment" is checked as: the earliest record for this enrollment must be observed within `EnrollmentWindow` (one hour, the refresh period) of the roster's enrollment time. A later first record cannot speak for a volume attached and removed before it.
1979	- The newest record must be no more than `LastRecordWindow` (2 hours) older than `terminatedAt`, the fence's `observed_at` (controller ruling 1). A recorder that stopped long before the box died cannot speak for its last hours.
1980	- `Discharge` checks every stored record of this enrollment before using it (Astra plan R1-6, R2-2): a version other than 1, a state that carries no mapping (`terminated`, or none), an empty mapping, or a volume with no id or no device keeps the chores open. The test has one stored case per check, each failing that check alone. `List` returns whatever is stored, so a record `Write` would never produce is still met here.
1981	- `Discharge` recomputes removals from consecutive records as well as reading `removed_volumes`, so a record one worker failed to write cannot hide a removal.
1982
1983	Keys use a fixed-width UTC time with `-` in place of `:`, so they sort in observation order.
1984
1985	- [ ] Step 1: Write the failing tests
1986
1987	Create `internal/boxvolumes/volumes_test.go`:
1988
1989	```go
1990	package boxvolumes
1991
1992	import (
1993	"context"
1994	"errors"
1995	"testing"
1996	"time"
1997
1998	"github.com/aws/aws-sdk-go-v2/aws"
1999	awsmiddleware "github.com/aws/aws-sdk-go-v2/aws/middleware"
2000	"github.com/aws/aws-sdk-go-v2/service/ec2"
2001	"github.com/aws/aws-sdk-go-v2/service/ec2/types"
2002	smithymiddleware "github.com/aws/smithy-go/middleware"
2003
2004	"github.com/carlosframework/platform/internal/boxid"
2005	"github.com/carlosframework/platform/internal/release"
2006	)
2007
2008	var (
2009	cloud = boxid.Cloud{Provider: "aws", Account: "111122223333", Region: "eu-west-1", InstanceID: "i-0abc"}
2010	entry = Entry{Box: "annie-welding", Enrollment: "enr-annie-1", Cloud: cloud}
2011	writer = WrittenBy{Box: "bert-sawing", Enrollment: "enr-bert-1"}
2012	t0 = time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
2013	)
2014
2015	type vol struct {
2016	dev, id string
2017	dot bool
2018	}
2019
2020	func reply(owner string, state types.InstanceStateName, vols ...vol) *ec2.DescribeInstancesOutput {
2021	var maps []types.InstanceBlockDeviceMapping
2022	for _, v := range vols {
2023	maps = append(maps, types.InstanceBlockDeviceMapping{DeviceName: aws.String(v.dev),
2024	Ebs: &types.EbsInstanceBlockDevice{VolumeId: aws.String(v.id), DeleteOnTermination: aws.Bool(v.dot), Status: types.AttachmentStatusAttached}})
2025	}
2026	var md smithymiddleware.Metadata
2027	awsmiddleware.SetRequestIDMetadata(&md, "req-1")
2028	return &ec2.DescribeInstancesOutput{ResultMetadata: md, Reservations: []types.Reservation{{OwnerId: aws.String(owner),
2029	Instances: []types.Instance{{InstanceId: aws.String("i-0abc"), State: &types.InstanceState{Name: state}, BlockDeviceMappings: maps}}}}}
2030	}
2031
2032	var root, data, scratch = vol{"/dev/xvda", "vol-root", true}, vol{"/dev/xvdb", "vol-data", true}, vol{"/dev/xvdc", "vol-keep", false}
2033
2034	func observe(t testing.T, out ec2.DescribeInstancesOutput) Observation {
2035	t.Helper()
2036	o, err := Observe(out, cloud)
2037	if err != nil {
2038	t.Fatal(err)
2039	}
2040	return o
2041	}
2042
2043	func TestObserveReadsEBSMappingsInVolumeOrder(t *testing.T) {
2044	out := reply("111122223333", types.InstanceStateNameRunning, root, data)
2045	out.Reservations[0].Instances[0].BlockDeviceMappings = append(out.Reservations[0].Instances[0].BlockDeviceMappings,
2046	types.InstanceBlockDeviceMapping{DeviceName: aws.String("/dev/sdz")}) // instance store: no Ebs
2047	o := observe(t, out)
2048	want := []Device{{"/dev/xvdb", "vol-data", true, "attached"}, {"/dev/xvda", "vol-root", true, "attached"}}
2049	if o.State != "running" \|\| o.RequestID != "req-1" \|\| len(o.Devices) != 2 \|\| o.Devices[0] != want[0] \|\| o.Devices[1] != want[1] {
2050	t.Fatalf("observation = %+v", o)
2051	}
2052	}
2053
2054	// Review Focus 3: a terminated instance's reply has no mappings. Recorded,
2055	// it would claim every volume was removed; it must not be recordable.
2056	func TestObserveRefusesWhatIsNotTheInstancesMapping(t *testing.T) {
2057	for name, out := range map[string]*ec2.DescribeInstancesOutput{
2058	"terminated": reply("111122223333", types.InstanceStateNameTerminated),
2059	"another owner": reply("999999999999", types.InstanceStateNameRunning, root),
2060	"another instance": {Reservations: []types.Reservation{{OwnerId: aws.String("111122223333"), Instances: []types.Instance{{InstanceId: aws.String("i-0zzz")}}}}},
2061	"empty reply": {},
2062	"nil reply": nil,
2063	} {
2064	t.Run(name, func(t *testing.T) {
2065	if o, err := Observe(out, cloud); !errors.Is(err, ErrNotRecordable) {
2066	t.Fatalf("observation = %+v, err = %v; want ErrNotRecordable", o, err)
2067	}
2068	})
2069	}
2070	// A shutting-down read still lists the mappings and may be recorded.
2071	if o := observe(t, reply("111122223333", types.InstanceStateNameShuttingDown, root)); len(o.Devices) != 1 {
2072	t.Fatalf("shutting-down: %+v", o)
2073	}
2074	}
2075
2076	func TestNextWritesOnChangeAndAtLeastHourly(t *testing.T) {
2077	first, ok := Next(nil, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameRunning, root)), t0)
2078	if !ok \|\| first.ObservedAt != "2026-10-06T09:00:00Z" \|\| len(first.RemovedVolumes) != 0 \|\| first.RemovedVolumes == nil {
2079	t.Fatalf("first = %+v ok=%v", first, ok)
2080	}
2081	same := observe(t, reply("111122223333", types.InstanceStateNameRunning, root))
2082	if _, ok := Next(&first, entry, writer, same, t0.Add(59*time.Minute)); ok {
2083	t.Fatal("wrote an unchanged mapping inside the hour")
2084	}
2085	if _, ok := Next(&first, entry, writer, same, t0.Add(time.Hour)); !ok {
2086	t.Fatal("no record after an hour")
2087	}
2088	grown, ok := Next(&first, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameRunning, root, scratch)), t0.Add(time.Minute))
2089	if !ok \|\| len(grown.BlockDevices) != 2 {
2090	t.Fatalf("an attached volume was not recorded: %+v", grown)
2091	}
2092	shrunk, ok := Next(&grown, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameRunning, root)), t0.Add(2*time.Minute))
2093	if !ok \|\| len(shrunk.RemovedVolumes) != 1 \|\| shrunk.RemovedVolumes[0] != "vol-keep" {
2094	t.Fatalf("a detached volume is not in removed_volumes: %+v", shrunk)
2095	}
2096	if _, ok := Next(&first, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameShuttingDown, root)), t0.Add(time.Minute)); !ok {
2097	t.Fatal("a state change was not recorded")
2098	}
2099	}
2100
2101	func TestWriteIsCreateOnlyAndListReturnsObservationOrder(t *testing.T) {
2102	ctx, st := context.Background(), release.NewDirStore(t.TempDir())
2103	a, _ := Next(nil, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameRunning, root)), t0)
2104	b, _ := Next(&a, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameRunning, root, data)), t0.Add(10*time.Minute))
2105	for _, r := range []Record{b, a} {
2106	if _, err := Write(ctx, st, r); err != nil {
2107	t.Fatal(err)
2108	}
2109	}
2110	if _, err := Write(ctx, st, a); err != nil {
2111	t.Fatalf("the same record twice: %v", err)
2112	}
2113	other := a
2114	other.RequestID = "req-other"
2115	if _, err := Write(ctx, st, other); !errors.Is(err, release.ErrCASConflict) {
2116	t.Fatalf("a different record under the same key: err = %v", err)
2117	}
2118	got, err := List(ctx, st, entry.Enrollment)
2119	if err != nil {
2120	t.Fatal(err)
2121	}
2122	if len(got) != 2 \|\| got[0].ObservedAt != a.ObservedAt \|\| got[1].ObservedAt != b.ObservedAt {
2123	t.Fatalf("list = %+v", got)
2124	}
2125	if _, err := Write(ctx, st, Record{Enrollment: "../edges", ObservedAt: a.ObservedAt}); err == nil {
2126	t.Fatal("wrote outside the prefix")
2127	}
2128	}
2129
2130	func TestListFailsOnARecordItCannotRead(t *testing.T) {
2131	ctx, st := context.Background(), release.NewDirStore(t.TempDir())
2132	if _, err := st.Put(ctx, Key(entry.Enrollment, t0), []byte(`{"v":1,"surprise":true}`), ""); err != nil {
2133	t.Fatal(err)
2134	}
2135	if _, err := List(ctx, st, entry.Enrollment); err == nil {
2136	t.Fatal("an unreadable record read as a gap")
2137	}
2138	}
2139
2140	func records(t *testing.T, steps ...[]vol) []Record {
2141	t.Helper()
2142	var out []Record
2143	var prev *Record
2144	for i, vs := range steps {
2145	rec, _ := Next(prev, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameRunning, vs...)), t0.Add(time.Duration(i)10time.Minute))
2146	out = append(out, rec)
2147	prev = &out[len(out)-1]
2148	}
2149	return out
2150	}
2151
2152	func TestDischarge(t *testing.T) {
2153	enrolled := t0.Add(-5 * time.Minute)
2154	terminated := t0.Add(30 * time.Minute)
2155	if v := Discharge(records(t, []vol{root}, []vol{root, data}, []vol{root, data}), entry, enrolled, terminated); !v.Discharge {
2156	t.Fatalf("all true, nothing removed: %+v", v)
2157	}
2158	cases := map[string]struct {
2159	recs []Record
2160	at time.Time
2161	}{
2162	"no record at all": {nil, enrolled},
2163	"a false flag": {records(t, []vol{root}, []vol{root, scratch}), enrolled},
2164	"a removed volume": {records(t, []vol{root, data}, []vol{root}), enrolled},
2165	"a replaced root volume": {records(t, []vol{root}, []vol{{"/dev/xvda", "vol-root2", true}}), enrolled},
2166	"first record long after": {records(t, []vol{root}), t0.Add(-2 * time.Hour)},
2167	"only another enrollment's": {func() []Record { r := records(t, []vol{root}); r[0].Enrollment = "enr-other"; return r }(), enrolled},
2168	"only another instance's": {func() []Record { r := records(t, []vol{root}); r[0].Cloud.InstanceID = "i-0zzz"; return r }(), enrolled},
2169	}
2170	for name, tc := range cases {
2171	t.Run(name, func(t *testing.T) {
2172	if v := Discharge(tc.recs, entry, tc.at, terminated); v.Discharge \|\| v.Reason == "" {
2173	t.Fatalf("verdict = %+v, want the chores left open with a reason", v)
2174	}
2175	})
2176	}
2177	}
2178
2179	// A removal is caught from the sequence even when the record that should
2180	// have listed it was never written (two workers, one missed write).
2181	func TestDischargeSeesARemovalNoRecordListed(t *testing.T) {
2182	recs := records(t, []vol{root, data}, []vol{root})
2183	recs[1].RemovedVolumes = []string{}
2184	if v := Discharge(recs, entry, t0, t0.Add(time.Hour)); v.Discharge {
2185	t.Fatal("discharged across a silent removal")
2186	}
2187	}
2188
2189	// Controller ruling, 2026-10-06: the newest record must be no more than
2190	// LastRecordWindow older than the termination observation. A recorder that
2191	// stopped long before the box died cannot speak for its last hours.
2192	func TestDischargeNeedsARecentLastRecord(t *testing.T) {
2193	recs := records(t, []vol{root}, []vol{root}) // newest at t0+10m
2194	newest := t0.Add(10 * time.Minute)
2195	if v := Discharge(recs, entry, t0, newest.Add(LastRecordWindow)); !v.Discharge {
2196	t.Fatalf("exactly at the window: %+v", v)
2197	}
2198	if v := Discharge(recs, entry, t0, newest.Add(LastRecordWindow+time.Second)); v.Discharge \|\| v.Reason == "" {
2199	t.Fatalf("a newest record older than the window discharged: %+v", v)
2200	}
2201	}
2202
2203	// Astra plan R1-6: what List returns is whatever is stored. A stored record
2204	// that is not evidence of a running instance's mappings keeps the chores
2205	// open, however it got there. Each bad record sits beside a good one so
2206	// that no removal is seen between them: only the record's own check can
2207	// refuse it.
2208	func TestDischargeRefusesStoredRecordsThatAreNotEvidence(t *testing.T) {
2209	ctx := context.Background()
2210	good := records(t, []vol{root})
2211	const before = `"enrollment":"enr-annie-1","cloud":{"provider":"aws","account":"111122223333","region":"eu-west-1","instance_id":"i-0abc"},"observed_at":"2026-10-06T08:55:00Z"`
2212	const after = `"enrollment":"enr-annie-1","cloud":{"provider":"aws","account":"111122223333","region":"eu-west-1","instance_id":"i-0abc"},"observed_at":"2026-10-06T09:05:00Z"`
2213	type bad struct {
2214	body string
2215	at time.Duration // from t0; the good record is at t0
2216	}
2217	cases := map[string]bad{
2218	// Before the good record: an empty mapping followed by one volume is
2219	// an attach, not a removal.
2220	"only the binding fields": {`{"v":1,` + before + `}`, -5 * time.Minute},
2221	"a terminated read": {`{"v":1,` + before + `,"instance_state":"terminated","block_devices":[],"removed_volumes":[]}`, -5 * time.Minute},
2222	// After it, keeping vol-root, so nothing is removed.
2223	"a volume with no id": {`{"v":1,` + after + `,"instance_state":"running","block_devices":[{"device":"/dev/xvda","volume_id":"vol-root","delete_on_termination":true,"status":"attached"},{"device":"/dev/xvdb","volume_id":"","delete_on_termination":true,"status":"attached"}],"removed_volumes":[]}`, 5 * time.Minute},
2224	"a version 2 record": {`{"v":2,` + after + `,"instance_state":"running","block_devices":[{"device":"/dev/xvda","volume_id":"vol-root","delete_on_termination":true,"status":"attached"}],"removed_volumes":[]}`, 5 * time.Minute},
2225	// Astra plan R2-2: one case per check, each failing only that check.
2226	"a running read with no volume": {`{"v":1,` + before + `,"instance_state":"running","block_devices":[],"removed_volumes":[]}`, -5 * time.Minute},
2227	"a terminated read with its root mapping": {`{"v":1,` + after + `,"instance_state":"terminated","block_devices":[{"device":"/dev/xvda","volume_id":"vol-root","delete_on_termination":true,"status":"attached"}],"removed_volumes":[]}`, 5 * time.Minute},
2228	"a volume with no device": {`{"v":1,` + after + `,"instance_state":"running","block_devices":[{"device":"/dev/xvda","volume_id":"vol-root","delete_on_termination":true,"status":"attached"},{"device":"","volume_id":"vol-data","delete_on_termination":true,"status":"attached"}],"removed_volumes":[]}`, 5 * time.Minute},
2229	}
2230	for name, tc := range cases {
2231	t.Run(name, func(t *testing.T) {
2232	st := release.NewDirStore(t.TempDir())
2233	if _, err := Write(ctx, st, good[0]); err != nil {
2234	t.Fatal(err)
2235	}
2236	if _, err := st.Put(ctx, Key(entry.Enrollment, t0.Add(tc.at)), []byte(tc.body), ""); err != nil {
2237	t.Fatal(err)
2238	}
2239	recs, err := List(ctx, st, entry.Enrollment)
2240	if err != nil {
2241	t.Fatal(err)
2242	}
2243	if len(recs) != 2 {
2244	t.Fatalf("listed %d records, want 2", len(recs))
2245	}
2246	if v := Discharge(recs, entry, t0, t0.Add(time.Hour)); v.Discharge \|\| v.Reason == "" {
2247	t.Fatalf("verdict = %+v, want the chores left open", v)
2248	}
2249	})
2250	}
2251	// The good record alone, stored and listed, still discharges.
2252	st := release.NewDirStore(t.TempDir())
2253	if _, err := Write(ctx, st, good[0]); err != nil {
2254	t.Fatal(err)
2255	}
2256	recs, _ := List(ctx, st, entry.Enrollment)
2257	if v := Discharge(recs, entry, t0, t0.Add(time.Hour)); !v.Discharge {
2258	t.Fatalf("a stored good record: %+v", v)
2259	}
2260	}
2261	```
2262
2263	- [ ] Step 2: Run them to see them fail
2264
2265	Run: `go test ./internal/boxvolumes/ -count=1`
2266	Expected: FAIL, `no non-test Go files` or `undefined: Observe`.
2267
2268	- [ ] Step 3: Write the package
2269
2270	Create `internal/boxvolumes/volumes.go`:
2271
2272	```go
2273	// Package boxvolumes records a platform box's block device mappings while
2274	// it runs (region HA spec 1 §5.9, ruling H19), and decides from those
2275	// records whether a terminated box's park-local chores may be discharged
2276	// without retirement (claim D5's destroyed-local exception, L3).
2277	//
2278	// Nothing here is a termination fence, and nothing about execution,
2279	// admission, relocation or recovery reads it (spec §8, B4-1). A record that
2280	// is wrong or missing can leave a disk unretired for an operator; it can
2281	// never admit a writer.
2282	//
2283	// This package builds, writes and reads records. The loop that describes
2284	// every enrolled box in the region needs the roster's enrollments, and the
2285	// discharge CAS needs L3: both wait for the claim.
2286	package boxvolumes
2287
2288	import (
2289	"bytes"
2290	"context"
2291	"encoding/json"
2292	"errors"
2293	"fmt"
2294	"io"
2295	"slices"
2296	"sort"
2297	"strings"
2298	"time"
2299
2300	"github.com/aws/aws-sdk-go-v2/aws"
2301	awsmiddleware "github.com/aws/aws-sdk-go-v2/aws/middleware"
2302	"github.com/aws/aws-sdk-go-v2/service/ec2"
2303
2304	"github.com/carlosframework/platform/internal/boxid"
2305	"github.com/carlosframework/platform/internal/release"
2306	)
2307
2308	const (
2309	Prefix = "control/box-volumes/"
2310
2311	// RefreshEvery is the longest a running box goes without a new record
2312	// when nothing changes (spec §5.9: "at least once an hour").
2313	RefreshEvery = time.Hour
2314	// EnrollmentWindow is how soon after enrollment the first record must
2315	// be taken for it to stand for the launch-time mappings.
2316	EnrollmentWindow = time.Hour
2317	// LastRecordWindow is the most the newest record may predate the
2318	// termination observation (controller ruling, 2026-10-06). An older one
2319	// cannot speak for a volume attached after it.
2320	LastRecordWindow = 2 * time.Hour
2321
2322	keyTime = "2006-01-02T15-04-05.000000000Z"
2323	)
2324
2325	// Device is one EBS volume in the instance's block device mappings.
2326	type Device struct {
2327	Device string `json:"device"`
2328	VolumeID string `json:"volume_id"`
2329	DeleteOnTermination bool `json:"delete_on_termination"`
2330	Status string `json:"status"`
2331	}
2332
2333	// WrittenBy names the regional worker that made the read.
2334	type WrittenBy struct {
2335	Box string `json:"box"`
2336	Enrollment string `json:"enrollment"`
2337	}
2338
2339	// Record is §5.9's volume record,
2340	// control/box-volumes/<enrollment>/<observed_at>.json.
2341	type Record struct {
2342	V int `json:"v"`
2343	Box string `json:"box"`
2344	Enrollment string `json:"enrollment"`
2345	Cloud boxid.Cloud `json:"cloud"`
2346	InstanceState string `json:"instance_state"`
2347	BlockDevices []Device `json:"block_devices"`
2348	RemovedVolumes []string `json:"removed_volumes"`
2349	ObservedAt string `json:"observed_at"`
2350	RequestID string `json:"request_id"`
2351	WrittenBy WrittenBy `json:"written_by"`
2352	}
2353
2354	// Entry is the roster entry a record describes.
2355	type Entry struct {
2356	Box string
2357	Enrollment string
2358	Cloud boxid.Cloud
2359	}
2360
2361	// Observation is one DescribeInstances read of one enrolled box.
2362	type Observation struct {
2363	State string
2364	Devices []Device
2365	RequestID string
2366	}
2367
2368	var ErrNotRecordable = errors.New("boxvolumes: not a recordable observation")
2369
2370	// ValidEnrollment reports whether e can be a key segment: 1 to 128 of
2371	// [A-Za-z0-9._-], never "." or "..".
2372	func ValidEnrollment(e string) bool {
2373	if e == "" \|\| len(e) > 128 \|\| e == "." \|\| e == ".." {
2374	return false
2375	}
2376	for _, r := range e {
2377	ok := r >= 'a' && r <= 'z' \|\| r >= 'A' && r <= 'Z' \|\| r >= '0' && r <= '9' \|\| r == '.' \|\| r == '_' \|\| r == '-'
2378	if !ok {
2379	return false
2380	}
2381	}
2382	return true
2383	}
2384
2385	// Key is where a record observed at t lives. The time is fixed-width UTC,
2386	// so keys sort in observation order.
2387	func Key(enrollment string, t time.Time) string {
2388	return Prefix + enrollment + "/" + t.UTC().Format(keyTime) + ".json"
2389	}
2390
2391	// recordable is every state in which EC2 still reports the instance's
2392	// mappings. "terminated" is not one: its BlockDeviceMappings is empty (the
2393	// dev test of 2026-10-06), which would read as every volume removed.
2394	var recordable = map[string]bool{"pending": true, "running": true, "shutting-down": true, "stopping": true, "stopped": true}
2395
2396	// Observe reads the entry's instance out of a DescribeInstances reply (see
2397	// termfence.DescribeInstance for the call). It refuses a reply that does
2398	// not name the instance, names it under another owner, or reports a state
2399	// whose mappings are not the instance's.
2400	func Observe(out *ec2.DescribeInstancesOutput, c boxid.Cloud) (Observation, error) {
2401	if out == nil \|\| !c.Complete() {
2402	return Observation{}, fmt.Errorf("%w: no reply, or no proven cloud", ErrNotRecordable)
2403	}
2404	var o Observation
2405	if id, ok := awsmiddleware.GetRequestIDMetadata(out.ResultMetadata); ok {
2406	o.RequestID = id
2407	}
2408	for _, r := range out.Reservations {
2409	for _, inst := range r.Instances {
2410	if aws.ToString(inst.InstanceId) != c.InstanceID {
2411	continue
2412	}
2413	if aws.ToString(r.OwnerId) != c.Account {
2414	return Observation{}, fmt.Errorf("%w: %s is owned by %q, the roster says %q", ErrNotRecordable, c.InstanceID, aws.ToString(r.OwnerId), c.Account)
2415	}
2416	if inst.State != nil {
2417	o.State = string(inst.State.Name)
2418	}
2419	if !recordable[o.State] {
2420	return Observation{}, fmt.Errorf("%w: state %q", ErrNotRecordable, o.State)
2421	}
2422	for _, m := range inst.BlockDeviceMappings {
2423	if m.Ebs == nil {
2424	continue // instance store dies with the instance
2425	}
2426	o.Devices = append(o.Devices, Device{Device: aws.ToString(m.DeviceName), VolumeID: aws.ToString(m.Ebs.VolumeId),
2427	DeleteOnTermination: aws.ToBool(m.Ebs.DeleteOnTermination), Status: string(m.Ebs.Status)})
2428	}
2429	sort.Slice(o.Devices, func(i, j int) bool { return o.Devices[i].VolumeID < o.Devices[j].VolumeID })
2430	return o, nil
2431	}
2432	}
2433	return Observation{}, fmt.Errorf("%w: the reply does not name %s", ErrNotRecordable, c.InstanceID)
2434	}
2435
2436	func volumeIDs(ds []Device) []string {
2437	out := make([]string, 0, len(ds))
2438	for _, d := range ds {
2439	out = append(out, d.VolumeID)
2440	}
2441	return out
2442	}
2443
2444	func removed(prev, cur []Device) []string {
2445	have := volumeIDs(cur)
2446	var out []string
2447	for _, id := range volumeIDs(prev) {
2448	if !slices.Contains(have, id) {
2449	out = append(out, id)
2450	}
2451	}
2452	sort.Strings(out)
2453	return out
2454	}
2455
2456	// Next decides whether o, made by writer at now, is a new record after
2457	// prev (nil when the enrollment has none). It is one when there is no
2458	// record yet, when the mapping or the instance state differs from prev,
2459	// and otherwise once RefreshEvery has passed.
2460	func Next(prev *Record, e Entry, writer WrittenBy, o Observation, now time.Time) (Record, bool) {
2461	rec := Record{V: 1, Box: e.Box, Enrollment: e.Enrollment, Cloud: e.Cloud, InstanceState: o.State,
2462	BlockDevices: o.Devices, RemovedVolumes: []string{}, ObservedAt: now.UTC().Format(time.RFC3339Nano),
2463	RequestID: o.RequestID, WrittenBy: writer}
2464	if rec.BlockDevices == nil {
2465	rec.BlockDevices = []Device{}
2466	}
2467	if prev == nil {
2468	return rec, true
2469	}
2470	if r := removed(prev.BlockDevices, o.Devices); len(r) > 0 {
2471	rec.RemovedVolumes = r
2472	}
2473	if !slices.Equal(prev.BlockDevices, rec.BlockDevices) \|\| prev.InstanceState != o.State {
2474	return rec, true
2475	}
2476	at, err := time.Parse(time.RFC3339Nano, prev.ObservedAt)
2477	return rec, err != nil \|\| now.Sub(at) >= RefreshEvery
2478	}
2479
2480	// Write stores rec create-only at its key. Records are never replaced or
2481	// deleted (spec §6); the same bytes written twice are one record.
2482	func Write(ctx context.Context, s release.Store, rec Record) (string, error) {
2483	if !ValidEnrollment(rec.Enrollment) {
2484	return "", fmt.Errorf("boxvolumes: %q is not an enrollment", rec.Enrollment)
2485	}
2486	at, err := time.Parse(time.RFC3339Nano, rec.ObservedAt)
2487	if err != nil {
2488	return "", fmt.Errorf("boxvolumes: observed_at: %w", err)
2489	}
2490	body, err := json.Marshal(rec)
2491	if err != nil {
2492	return "", err
2493	}
2494	key := Key(rec.Enrollment, at)
2495	_, err = s.Put(ctx, key, body, "")
2496	if errors.Is(err, release.ErrCASConflict) {
2497	cur, _, gerr := s.Get(ctx, key)
2498	if gerr == nil && bytes.Equal(cur, body) {
2499	return key, nil
2500	}
2501	return "", fmt.Errorf("boxvolumes: %s already holds another record: %w", key, err)
2502	}
2503	return key, err
2504	}
2505
2506	func decode(body []byte) (Record, error) {
2507	dec := json.NewDecoder(bytes.NewReader(body))
2508	dec.DisallowUnknownFields()
2509	var rec Record
2510	if err := dec.Decode(&rec); err != nil {
2511	return Record{}, err
2512	}
2513	if _, err := dec.Token(); err != io.EOF {
2514	return Record{}, errors.New("trailing data")
2515	}
2516	return rec, nil
2517	}
2518
2519	// List returns every record stored for an enrollment, in key order. Any
2520	// record that cannot be read or parsed is an error, never a gap: a gap
2521	// could hide the record that shows a removal.
2522	func List(ctx context.Context, s release.Store, enrollment string) ([]Record, error) {
2523	if !ValidEnrollment(enrollment) {
2524	return nil, fmt.Errorf("boxvolumes: %q is not an enrollment", enrollment)
2525	}
2526	keys, err := s.List(ctx, Prefix+enrollment+"/")
2527	if err != nil {
2528	return nil, err
2529	}
2530	sort.Strings(keys)
2531	out := make([]Record, 0, len(keys))
2532	for _, k := range keys {
2533	if !strings.HasSuffix(k, ".json") {
2534	continue
2535	}
2536	body, _, err := s.Get(ctx, k)
2537	if err != nil {
2538	return nil, fmt.Errorf("boxvolumes: %s: %w", k, err)
2539	}
2540	rec, err := decode(body)
2541	if err != nil {
2542	return nil, fmt.Errorf("boxvolumes: %s: %w", k, err)
2543	}
2544	out = append(out, rec)
2545	}
2546	return out, nil
2547	}
2548
2549	// incomplete says why a stored record is not evidence of a running
2550	// instance's mappings, or "" when it is. Write never produces one of
2551	// these; a record that is one anyway (an older writer, a hand edit, a
2552	// bug) keeps the chores open rather than being read as "nothing attached".
2553	func incomplete(r Record) string {
2554	if !recordable[r.InstanceState] {
2555	return fmt.Sprintf("has state %q, which carries no mapping", r.InstanceState)
2556	}
2557	if len(r.BlockDevices) == 0 {
2558	return "lists no EBS volume" // a platform box always has its root volume
2559	}
2560	for _, d := range r.BlockDevices {
2561	if d.VolumeID == "" \|\| d.Device == "" {
2562	return "lists a volume with no id or device"
2563	}
2564	}
2565	return ""
2566	}
2567
2568	// Verdict is whether a terminated box's park-local chores may be
2569	// discharged without retirement, and if not, why.
2570	type Verdict struct {
2571	Discharge bool
2572	Reason string
2573	}
2574
2575	// Discharge decides from records (as List returns them) whether every EBS
2576	// volume e's instance had attached, from the first record taken within
2577	// EnrollmentWindow of enrolledAt to the last, taken within LastRecordWindow
2578	// of terminatedAt (the fence's observed_at), was DeleteOnTermination, with
2579	// none ever removed (spec §5.9, H19). Records for another enrollment or
2580	// another instance are ignored; a record of this enrollment's that is
2581	// incomplete, carries no mapping, or has a version this binary does not
2582	// read keeps the chores open. Anything short of that keeps the chores
2583	// open for an operator, which costs cleanup and never recovery.
2584	func Discharge(records []Record, e Entry, enrolledAt, terminatedAt time.Time) Verdict {
2585	no := func(format string, a ...any) Verdict { return Verdict{Reason: fmt.Sprintf(format, a...)} }
2586	if !e.Cloud.Complete() \|\| e.Enrollment == "" {
2587	return no("the roster entry has no proven cloud")
2588	}
2589	type stamped struct {
2590	at time.Time
2591	rec Record
2592	}
2593	var mine []stamped
2594	for _, r := range records {
2595	if r.Enrollment != e.Enrollment {
2596	continue
2597	}
2598	// A version this binary does not read could hold the adverse
2599	// record; it is never skipped.
2600	if r.V != 1 {
2601	return no("a record for %s has version %d", e.Enrollment, r.V)
2602	}
2603	if r.Cloud != e.Cloud {
2604	continue // another instance's (spec §5.9: ignored)
2605	}
2606	if reason := incomplete(r); reason != "" {
2607	return no("the record observed at %q %s", r.ObservedAt, reason)
2608	}
2609	at, err := time.Parse(time.RFC3339Nano, r.ObservedAt)
2610	if err != nil {
2611	return no("record with unreadable observed_at %q", r.ObservedAt)
2612	}
2613	mine = append(mine, stamped{at, r})
2614	}
2615	if len(mine) == 0 {
2616	return no("no volume record for %s", e.Enrollment)
2617	}
2618	sort.SliceStable(mine, func(i, j int) bool { return mine[i].at.Before(mine[j].at) })
2619	if first := mine[0].at; first.After(enrolledAt.Add(EnrollmentWindow)) {
2620	return no("the first record (%s) is more than %s after enrollment", first.Format(time.RFC3339), EnrollmentWindow)
2621	}
2622	if last := mine[len(mine)-1].at; terminatedAt.Sub(last) > LastRecordWindow {
2623	return no("the newest record (%s) is more than %s before the termination", last.Format(time.RFC3339), LastRecordWindow)
2624	}
2625	for i, s := range mine {
2626	for _, d := range s.rec.BlockDevices {
2627	if !d.DeleteOnTermination {
2628	return no("volume %s was not DeleteOnTermination at %s", d.VolumeID, s.rec.ObservedAt)
2629	}
2630	}
2631	if len(s.rec.RemovedVolumes) > 0 {
2632	return no("volumes %v were removed by %s", s.rec.RemovedVolumes, s.rec.ObservedAt)
2633	}
2634	if i > 0 {
2635	if r := removed(mine[i-1].rec.BlockDevices, s.rec.BlockDevices); len(r) > 0 {
2636	return no("volumes %v left the mapping by %s", r, s.rec.ObservedAt)
2637	}
2638	}
2639	}
2640	return Verdict{Discharge: true}
2641	}
2642	```
2643
2644	- [ ] Step 4: Run the tests
2645
2646	Run: `go test ./internal/boxvolumes/ -count=1 -v`
2647	Expected: PASS, including `TestObserveRefusesWhatIsNotTheInstancesMapping/terminated`, `TestDischargeSeesARemovalNoRecordListed`, `TestDischargeNeedsARecentLastRecord` and `TestDischargeRefusesStoredRecordsThatAreNotEvidence`.
2648
2649	- [ ] Step 5: Gate and commit
2650
2651	```bash
2652	go build ./... && go vet ./internal/boxvolumes/ && gofmt -l internal/boxvolumes
2653	GOOS=darwin go build ./... && GOOS=windows go build ./...
2654	git add internal/boxvolumes
2655	git commit -m "boxvolumes: a running box's volumes are recorded create-only, and only an unbroken all-true record discharges its chores
2656
2657	Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
2658	```
2659
2660	---
2661
2662	### Task 6: The edges file follows the claim's executor (`internal/host/edgesclaims.go`)
2663
2664	Spec §5.4: "The edges-file pass publishes a host as `home` only when this box is its executor and its row is `ready`, and adds the installation's stamp." Plan 1a built the edge's side (`foldHomes` believes the higher stamp). This task builds the publisher's side, with `CARLOS_REGION_HA` on.
2665
2666	Decisions.
2667	- Which rows. Only relocatable hosts (spec §2): a provisioned `instance` row that the activator runs as its own exec child (`Backing == ""`, no `Unit`, no `Addr`/`Addrs`, a `Socket`). Unit-, addr- and socket-only routes, static sites and services keep their box and are published as before. Sidecars are never published (unchanged).
2668	- The decision. A claim whose `op.exec` is this box's reference: home, stamped with `holder.stamp`. A claim whose `op.exec` names another box, or none, or which has no holder: withheld. No claim: home, unstamped, exactly as before; §8's gate admits a second box only once every host has a claim.
2669	- Errors. A read that fails keeps the last decision (claim C6: an error is never absence); a host never read publishes as before.
2670	- Cost. Each read stands 10 minutes (H9), at most 256 reads a pass, never-attempted hosts first, then oldest attempt first. A failed read counts as an attempt, so a claim that keeps failing goes to the back of the queue and cannot starve the rest (Astra plan R1-4). About 288 thousand GETs a day per 2,000 hosts, under $0.12.
2671	- Refresh. The registry timer alone runs a pass every five to six minutes, which at 256 reads a pass would take 40 minutes to cover 2,000 hosts (Astra plan R1-5). So, with the flag on, the loop also runs a pass every 30 seconds while any claim read is due (`edgesLoopState`). Every claim is then re-read within about 14 minutes; the edges file is still written only when its body changes. A claim pass is a publication like any other (Astra plan R2-1): it waits out the schedule's retry backoff after a failed write, and its own outcome goes through `edgesFileSchedule.done`, so a bucket that refuses every write is asked on the existing 2-second-to-5-minute ladder, not every 30 seconds. With the flag off, `edgesLoopState` is exactly `edgesFileSchedule`.
2672	- Aliases and catch-alls carry their primary's stamp (Astra plan R1-2). An attached domain and a served `*.<parent>` catch-all are dialled through the primary's installation. Unstamped, a dead predecessor's alias would tie with the successor's at 0 and the edge (`dialBeats`) would keep the box that sorts first.
2673	- §18's open question. A positive stamp is the mark: only relocatable hosts with a claim carry one, so the edge's report of two boxes at one positive stamp already concerns them alone. No separate marker field.
2674
2675	The withholding and stamping are applied before owners and aliases are computed, so an alias of a withheld host is withheld with it, and an alias of a stamped host reads its stamp from the map.
2676
2677	Files:
2678	- Create: `internal/host/edgesclaims.go`
2679	- Test: `internal/host/edgesclaims_test.go`, `internal/edge/homemap_alias_stamp_test.go`
2680	- Modify: `internal/host/edgesfile.go`
2681
2682	Interfaces:
2683	- Consumes: `hostclaim.Read`, `hostclaim.Observed`, `hostclaim.Claim` (main); `regionHAEnabled` (Task 3); `edgeHost.Stamp` (plan 1a).
2684	- Produces:
2685	```go
2686	const edgesClaimTTL = 10 * time.Minute
2687	const edgesClaimReadsPerPass = 256
2688	const edgesClaimPassEvery = 30 * time.Second
2689	type edgesClaims struct{ /* self, read, now, entries, hosts */ }
2690	func (c *edgesClaims) backlog(now time.Time) bool
2691	type edgesLoopState struct{ sched edgesFileSchedule; claims edgesClaims; claimsNext time.Time; schedDue, claimDue bool }
2692	func (s *edgesLoopState) due(now time.Time, fp string, fpOK bool) bool
2693	func (s *edgesLoopState) timed() bool
2694	func (s *edgesLoopState) done(now time.Time, ok bool)
2695	func newEdgesClaims(self string, read func(context.Context, string) (hostclaim.Observed, error)) *edgesClaims
2696	func relocatableRow(r registry.Route) bool
2697	func applyEdgesClaims(ctx context.Context, claims *edgesClaims, routes []registry.Route, hosts map[string]edgeHost)
2698	func edgesFileWriteClaims(ctx context.Context, s release.Store, reg registry.Registry, ident edgesFile, eip string, last []byte, refresh bool, claims *edgesClaims) error
2699	```
2700	`edgesFileWrite` keeps its signature and calls `edgesFileWriteClaims` with `nil`.
2701
2702	- [ ] Step 1: Write the failing tests
2703
2704	Create `internal/host/edgesclaims_test.go`:
2705
2706	```go
2707	package host
2708
2709	import (
2710	"context"
2711	"encoding/json"
2712	"errors"
2713	"fmt"
2714	"testing"
2715	"time"
2716
2717	"github.com/carlosframework/platform/internal/hostclaim"
2718	"github.com/carlosframework/platform/internal/registry"
2719	"github.com/carlosframework/platform/internal/release"
2720	)
2721
2722	// seedInstance upserts a ready, provisioned exec instance: a relocatable
2723	// host (spec 1 §2).
2724	func seedInstance(t testing.T, reg registry.Registry, host string) {
2725	t.Helper()
2726	if err := reg.Upsert(registry.Route{Host: host, Kind: "instance", Socket: "/run/carlos/" + host + ".sock",
2727	Provisioned: true, App: "jam", Account: "bab", Status: "ready"}); err != nil {
2728	t.Fatal(err)
2729	}
2730	}
2731
2732	// claimsBook answers claim reads from a map; a missing host is an absent
2733	// claim, a host in errs fails.
2734	type claimsBook struct {
2735	claims map[string]*hostclaim.Claim
2736	errs map[string]bool
2737	reads []string
2738	}
2739
2740	func (b *claimsBook) read(_ context.Context, host string) (hostclaim.Observed, error) {
2741	b.reads = append(b.reads, host)
2742	if b.errs[host] {
2743	return hostclaim.Observed{}, errors.New("SlowDown")
2744	}
2745	c, ok := b.claims[host]
2746	if !ok {
2747	return hostclaim.Observed{Host: host, Absent: true}, nil
2748	}
2749	return hostclaim.Observed{Host: host, Claim: c, ETag: "e"}, nil
2750	}
2751
2752	func serving(exec string, stamp uint64) *hostclaim.Claim {
2753	return &hostclaim.Claim{V: 2, Gen: 7, Holder: &hostclaim.Holder{Stamp: stamp},
2754	Op: hostclaim.Op{Kind: hostclaim.KindServing, ID: "op-1", Exec: exec}}
2755	}
2756
2757	func publishWithClaims(t testing.T, reg registry.Registry, claims *edgesClaims) map[string]edgeHost {
2758	t.Helper()
2759	st := release.NewDirStore(t.TempDir())
2760	var last []byte
2761	if err := edgesFileWriteClaims(context.Background(), st, reg, edgesFile{Box: "annie", BoxID: "annie"}, "1.2.3.4", &last, false, claims); err != nil {
2762	t.Fatal(err)
2763	}
2764	body, _, err := st.Get(context.Background(), edgesFileKey("annie"))
2765	if err != nil {
2766	t.Fatal(err)
2767	}
2768	var f edgesFile
2769	if err := json.Unmarshal(body, &f); err != nil {
2770	t.Fatal(err)
2771	}
2772	return f.Hosts
2773	}
2774
2775	func TestEdgesFileFollowsTheClaimsExecutor(t *testing.T) {
2776	reg := edgesTestRegistry(t)
2777	for _, h := range []string{"mine.bab.test", "theirs.bab.test", "between.bab.test", "unclaimed.bab.test"} {
2778	seedInstance(t, reg, h)
2779	}
2780	seedRoute(t, reg, "static.example.com") // not relocatable: never read, always home
2781	book := &claimsBook{claims: map[string]*hostclaim.Claim{
2782	"mine.bab.test": serving("annie", 41),
2783	"theirs.bab.test": serving("bert", 42),
2784	"between.bab.test": serving("", 43), // op.exec none: nobody serves it
2785	}}
2786	hosts := publishWithClaims(t, reg, newEdgesClaims("annie", book.read))
2787	if h, ok := hosts["mine.bab.test"]; !ok \|\| h.Stamp != 41 \|\| h.Role != "home" {
2788	t.Fatalf("executor's host = %+v, %v; want home stamped 41", h, ok)
2789	}
2790	for _, h := range []string{"theirs.bab.test", "between.bab.test"} {
2791	if _, ok := hosts[h]; ok {
2792	t.Errorf("%s published by a box its claim does not name", h)
2793	}
2794	}
2795	if h, ok := hosts["unclaimed.bab.test"]; !ok \|\| h.Stamp != 0 {
2796	t.Errorf("a host with no claim = %+v, %v; want home, unstamped, as before", h, ok)
2797	}
2798	if h, ok := hosts["static.example.com"]; !ok \|\| h.Stamp != 0 {
2799	t.Errorf("a service route = %+v, %v; want home, unstamped", h, ok)
2800	}
2801	for _, r := range book.reads {
2802	if r == "static.example.com" {
2803	t.Error("read a claim for a route that is not relocatable")
2804	}
2805	}
2806	}
2807
2808	func TestEdgesFileWithholdsTheAliasOfAWithheldHost(t *testing.T) {
2809	reg := edgesTestRegistry(t)
2810	seedInstance(t, reg, "theirs.bab.test")
2811	if err := reg.UpsertAlias(registry.RouteAlias{Alias: "www.customer.com", Host: "theirs.bab.test", Account: "bab", App: "jam", DNSStatus: "pointing"}); err != nil {
2812	t.Fatal(err)
2813	}
2814	book := &claimsBook{claims: map[string]*hostclaim.Claim{"theirs.bab.test": serving("bert", 42)}}
2815	hosts := publishWithClaims(t, reg, newEdgesClaims("annie", book.read))
2816	if len(hosts) != 0 {
2817	t.Fatalf("hosts = %v, want none", hosts)
2818	}
2819	}
2820
2821	// Review Focus 2: a claim that cannot be read never moves a host. It keeps
2822	// the last decision, or, never read, publishes as it did before claims.
2823	func TestEdgesFileKeepsTheLastDecisionWhenAClaimCannotBeRead(t *testing.T) {
2824	reg := edgesTestRegistry(t)
2825	seedInstance(t, reg, "mine.bab.test")
2826	seedInstance(t, reg, "theirs.bab.test")
2827	book := &claimsBook{claims: map[string]*hostclaim.Claim{"mine.bab.test": serving("annie", 41), "theirs.bab.test": serving("bert", 42)}}
2828	clock := time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
2829	claims := newEdgesClaims("annie", book.read)
2830	claims.now = func() time.Time { return clock }
2831	publishWithClaims(t, reg, claims)
2832	clock = clock.Add(edgesClaimTTL)
2833	book.errs = map[string]bool{"mine.bab.test": true, "theirs.bab.test": true}
2834	hosts := publishWithClaims(t, reg, claims)
2835	if h, ok := hosts["mine.bab.test"]; !ok \|\| h.Stamp != 41 {
2836	t.Fatalf("mine = %+v, %v: an unreadable claim dropped or unstamped a served host", h, ok)
2837	}
2838	if _, ok := hosts["theirs.bab.test"]; ok {
2839	t.Fatal("an unreadable claim published a host this box was told not to")
2840	}
2841
2842	fresh := newEdgesClaims("annie", book.read)
2843	if h, ok := publishWithClaims(t, reg, fresh)["mine.bab.test"]; !ok \|\| h.Stamp != 0 {
2844	t.Fatalf("never read = %+v, %v; want home, unstamped", h, ok)
2845	}
2846	}
2847
2848	func TestEdgesClaimsRereadOnlyAfterTheTTL(t *testing.T) {
2849	book := &claimsBook{claims: map[string]*hostclaim.Claim{"mine.bab.test": serving("annie", 41)}}
2850	clock := time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
2851	c := newEdgesClaims("annie", book.read)
2852	c.now = func() time.Time { return clock }
2853	ctx := context.Background()
2854	c.decide(ctx, []string{"mine.bab.test"})
2855	clock = clock.Add(edgesClaimTTL - time.Second)
2856	c.decide(ctx, []string{"mine.bab.test"})
2857	if len(book.reads) != 1 {
2858	t.Fatalf("%d reads inside the TTL, want 1", len(book.reads))
2859	}
2860	book.claims["mine.bab.test"] = serving("bert", 50)
2861	clock = clock.Add(time.Second)
2862	if d := c.decide(ctx, []string{"mine.bab.test"})["mine.bab.test"]; !d.withhold {
2863	t.Fatalf("decision after the TTL = %+v; the move to another box was not seen", d)
2864	}
2865	c.decide(ctx, nil)
2866	if len(c.entries) != 0 {
2867	t.Fatalf("a host no longer served kept its entry: %v", c.entries)
2868	}
2869	}
2870
2871	func TestEdgesClaimsBoundReadsPerPassNeverReadFirst(t *testing.T) {
2872	book := &claimsBook{claims: map[string]*hostclaim.Claim{}}
2873	clock := time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
2874	c := newEdgesClaims("annie", book.read)
2875	c.now = func() time.Time { return clock }
2876	var hosts []string
2877	for i := 0; i < edgesClaimReadsPerPass+10; i++ {
2878	hosts = append(hosts, fmt.Sprintf("h%04d.bab.test", i))
2879	}
2880	ctx := context.Background()
2881	c.decide(ctx, hosts)
2882	if len(book.reads) != edgesClaimReadsPerPass {
2883	t.Fatalf("first pass read %d claims, want %d", len(book.reads), edgesClaimReadsPerPass)
2884	}
2885	book.reads = nil
2886	c.decide(ctx, hosts)
2887	if len(book.reads) != 10 {
2888	t.Fatalf("second pass read %d claims, want the 10 never read", len(book.reads))
2889	}
2890	}
2891
2892	// Astra plan R1-2: an alias and a served catch-all are dialled through
2893	// their primary's installation, so they carry its stamp. Unstamped, a dead
2894	// predecessor's alias would tie with the successor's and the edge would
2895	// keep the first box by name.
2896	func TestEdgesFileStampsAliasesAndCatchAllsWithTheirPrimary(t *testing.T) {
2897	reg := edgesTestRegistry(t)
2898	seedInstance(t, reg, "mine.bab.test")
2899	for _, a := range []registry.RouteAlias{
2900	{Alias: "www.customer.com", Host: "mine.bab.test", Account: "bab", App: "jam", DNSStatus: "pointing"},
2901	{Alias: "*.customer.com", Host: "mine.bab.test", Account: "bab", App: "jam", DNSStatus: "pointing"},
2902	} {
2903	if err := reg.UpsertAlias(a); err != nil {
2904	t.Fatal(err)
2905	}
2906	}
2907	if err := reg.UpsertWildcardDomain(registry.WildcardDomain{Host: "customer.com", Account: "bab", App: "jam"}); err != nil {
2908	t.Fatal(err)
2909	}
2910	book := &claimsBook{claims: map[string]*hostclaim.Claim{"mine.bab.test": serving("annie", 57)}}
2911	hosts := publishWithClaims(t, reg, newEdgesClaims("annie", book.read))
2912	for _, key := range []string{"mine.bab.test", "www.customer.com", "*.customer.com"} {
2913	if h, ok := hosts[key]; !ok \|\| h.Stamp != 57 \|\| h.Role != "home" {
2914	t.Errorf("%s = %+v, %v; want home at stamp 57", key, h, ok)
2915	}
2916	}
2917	// With no claim reader the file is what it always was: no stamps.
2918	for key, h := range publishWithClaims(t, reg, nil) {
2919	if h.Stamp != 0 {
2920	t.Errorf("%s stamped %d with region HA off", key, h.Stamp)
2921	}
2922	}
2923	}
2924
2925	// Astra plan R1-4: claims that keep failing go to the back of the queue,
2926	// so they cannot hold every read of every pass.
2927	func TestEdgesClaimsAFailingClaimDoesNotStarveTheRest(t *testing.T) {
2928	book := &claimsBook{claims: map[string]*hostclaim.Claim{"zz-healthy.bab.test": serving("annie", 7)}, errs: map[string]bool{}}
2929	var hosts []string
2930	for i := 0; i < edgesClaimReadsPerPass; i++ {
2931	h := fmt.Sprintf("a%04d.bab.test", i)
2932	book.errs[h] = true
2933	hosts = append(hosts, h)
2934	}
2935	hosts = append(hosts, "zz-healthy.bab.test")
2936	clock := time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
2937	c := newEdgesClaims("annie", book.read)
2938	c.now = func() time.Time { return clock }
2939	ctx := context.Background()
2940	c.decide(ctx, hosts) // the 256 failing claims sort first and all fail
2941	clock = clock.Add(edgesClaimPassEvery)
2942	if d, ok := c.decide(ctx, hosts)["zz-healthy.bab.test"]; !ok \|\| d.stamp != 7 {
2943	t.Fatalf("healthy claim = %+v, %v: starved by claims that cannot be read", d, ok)
2944	}
2945	}
2946
2947	// Astra plan R1-5: the loop's own schedule, with a stable registry, keeps
2948	// every claim of a 2,000-host box fresh. Timer passes alone come every
2949	// five to six minutes and read 256 claims each.
2950	func TestEdgesLoopRefreshesEveryClaimWithinTheTTL(t *testing.T) {
2951	const n = 2000
2952	book := &claimsBook{claims: map[string]*hostclaim.Claim{}}
2953	var hosts []string
2954	for i := 0; i < n; i++ {
2955	h := fmt.Sprintf("h%04d.bab.test", i)
2956	book.claims[h] = serving("annie", uint64(i+1))
2957	hosts = append(hosts, h)
2958	}
2959	start := time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
2960	clock := start
2961	claims := newEdgesClaims("annie", book.read)
2962	claims.now = func() time.Time { return clock }
2963	state := &edgesLoopState{sched: newEdgesFileSchedule(start, func() time.Duration { return 5time.Minute + 30time.Second }), claims: claims}
2964	lastRead := map[string]time.Time{}
2965	worst := time.Duration(0)
2966	for clock = start; clock.Before(start.Add(3 * time.Hour)); clock = clock.Add(edgesFileWatchInterval) {
2967	if !state.due(clock, "fp", true) {
2968	continue
2969	}
2970	book.reads = nil
2971	claims.decide(context.Background(), hosts)
2972	for _, h := range book.reads {
2973	lastRead[h] = clock
2974	}
2975	state.done(clock, true)
2976	if clock.Sub(start) < time.Hour {
2977	continue // warm-up: the first reads of every host
2978	}
2979	for _, h := range hosts {
2980	if age := clock.Sub(lastRead[h]); age > worst {
2981	worst = age
2982	}
2983	}
2984	}
2985	if limit := edgesClaimTTL + 5*time.Minute; worst > limit {
2986	t.Fatalf("a claim went %s without a read, want at most %s", worst, limit)
2987	}
2988	}
2989
2990	// Astra plan R2-1: a claim pass is a publication. While the bucket refuses
2991	// every write, claim passes wait out the schedule's backoff and their own
2992	// failures move it; once writes succeed, the 30-second claim passes resume.
2993	func TestEdgesLoopClaimPassesObeyTheRetryBackoff(t *testing.T) {
2994	book := &claimsBook{claims: map[string]*hostclaim.Claim{}}
2995	var hosts []string
2996	for i := 0; i < 4000; i++ { // enough that reads stay due throughout
2997	h := fmt.Sprintf("h%04d.bab.test", i)
2998	book.claims[h] = serving("annie", 1)
2999	hosts = append(hosts, h)
3000	}
3001	start := time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
3002	clock := start
3003	claims := newEdgesClaims("annie", book.read)
3004	claims.now = func() time.Time { return clock }
3005	state := &edgesLoopState{sched: newEdgesFileSchedule(start, func() time.Duration { return 5time.Minute + 30time.Second }), claims: claims}
3006	failUntil := start.Add(20 * time.Minute)
3007	var during, after []time.Time
3008	for clock = start; clock.Before(failUntil.Add(8 * time.Minute)); clock = clock.Add(edgesFileWatchInterval) {
3009	if !state.due(clock, "fp", true) {
3010	continue
3011	}
3012	claims.decide(context.Background(), hosts)
3013	ok := !clock.Before(failUntil)
3014	if ok {
3015	after = append(after, clock)
3016	} else {
3017	during = append(during, clock)
3018	}
3019	state.done(clock, ok)
3020	}
3021	// Unthrottled, a claim pass every 30 seconds would be 40 attempts in
3022	// 20 minutes. The backoff (2 s doubling to 5 min) plus the timer allows
3023	// about a dozen.
3024	t.Logf("%d attempts while failing, %d after", len(during), len(after))
3025	if len(during) > 16 {
3026	t.Fatalf("%d publication attempts while every write failed, want at most 16: %v", len(during), during)
3027	}
3028	// Recovery waits out the backoff (at most five minutes), then the
3029	// 30-second claim passes resume.
3030	if len(after) < 3 \|\| after[len(after)-1].Sub(after[len(after)-2]) != edgesClaimPassEvery {
3031	t.Fatalf("passes after writes recovered = %v, want the 30-second claim passes back within eight minutes", after)
3032	}
3033	}
3034	```
3035
3036	And create `internal/edge/homemap_alias_stamp_test.go`, which pins the edge's half: with stamped aliases, the successor's alias wins over a dead box's that sorts first. It passes on main already (`dialBeats` compares stamps after the alias rule); it is here so the reason for stamping aliases is tested where the decision is made.
3037
3038	```go
3039	package edge
3040
3041	import "testing"
3042
3043	// Region HA plan 1b (Astra plan R1-2): after a failover the dead box's file
3044	// is stale but still listed. Its alias carries the old installation's
3045	// stamp and the successor's the new one, so the successor wins however the
3046	// boxes sort.
3047	func TestHomeMapAnAliasFollowsTheNewerInstallation(t *testing.T) {
3048	s := meshTestStore(t)
3049	putJSON(t, s, "control/edges/annie.json", `{"v":1,"box":"annie","eip":"1.1.1.1","hosts":{"shop.test":{"role":"home","stamp":41},"www.customer.com":{"role":"home","alias":true,"stamp":41}}}`)
3050	putJSON(t, s, "control/edges/betty.json", `{"v":1,"box":"betty","eip":"2.2.2.2","hosts":{"shop.test":{"role":"home","stamp":57},"www.customer.com":{"role":"home","alias":true,"stamp":57}}}`)
3051	m := newHomeMap(s, "carol")
3052	refreshHomeMap(t, m)
3053	if eip, box, ok := m.Home("www.customer.com"); !ok \|\| box != "betty" \|\| eip != "2.2.2.2" {
3054	t.Fatalf("Home(www.customer.com) = %q %q %v, want betty", eip, box, ok)
3055	}
3056	}
3057	```
3058
3059	- [ ] Step 2: Run them to see them fail
3060
3061	Run: `go test ./internal/host/ -run 'EdgesFileFollows\|EdgesFileWithholds\|EdgesFileKeeps\|EdgesFileStamps\|EdgesClaims\|EdgesLoop' -count=1`
3062	Expected: FAIL to build, `undefined: edgesClaims`.
3063
3064	- [ ] Step 3: Write the claim cache
3065
3066	Create `internal/host/edgesclaims.go`:
3067
3068	```go
3069	package host
3070
3071	import (
3072	"context"
3073	"math"
3074	"sort"
3075	"time"
3076
3077	"github.com/carlosframework/platform/internal/hostclaim"
3078	"github.com/carlosframework/platform/internal/registry"
3079	)
3080
3081	// Region HA spec 1 §5.4: with CARLOS_REGION_HA on, a relocatable host is
3082	// published as home only by the box its claim's op.exec names, carrying
3083	// the claim's holder.stamp so the edge's home map believes the newest
3084	// installation (HA4). The edge never reads claims; this publisher does, on
3085	// a slow cache, because the safety gate is the activator's fresh read at
3086	// every spawn (claim C4), not this file.
3087
3088	const (
3089	// edgesClaimTTL is how long one claim read stands: reconcile's sweep
3090	// (H9). A held row's entry can be stale this long, which can only send
3091	// a request to a box that then refuses to spawn.
3092	edgesClaimTTL = 10 * time.Minute
3093	// edgesClaimReadsPerPass bounds the GETs one pass makes, so a box with
3094	// thousands of hosts spreads its reads over several passes instead of
3095	// stalling one. A host not read yet publishes as it did before this
3096	// file existed.
3097	edgesClaimReadsPerPass = 256
3098	// edgesClaimPassEvery is how often the loop runs an extra pass while
3099	// claim reads are due. The registry timer alone (5 to 6 minutes) would
3100	// take 40 minutes to refresh 2,000 hosts at 256 a pass; with these
3101	// passes every host is re-read within about edgesClaimTTL plus four
3102	// minutes.
3103	edgesClaimPassEvery = 30 * time.Second
3104	)
3105
3106	// edgesDecision is what one claim says about this box and one host.
3107	type edgesDecision struct {
3108	withhold bool // the claim names another executor, or none
3109	stamp int64 // holder.stamp when this box is the executor
3110	}
3111
3112	// edgesClaimEntry is one host's last decision and last read attempt. A
3113	// failed read moves attempted and keeps the decision, so a claim that
3114	// cannot be read goes to the back of the queue instead of holding its
3115	// place at the front (claim C6: an error is never absence).
3116	type edgesClaimEntry struct {
3117	decision edgesDecision
3118	decided bool // some read of this claim has succeeded
3119	attempted time.Time
3120	}
3121
3122	// edgesClaims caches one decision per host. It is used only by the edges
3123	// file loop's goroutine.
3124	type edgesClaims struct {
3125	self string // this box's reference: the edges file's box_id
3126	read func(ctx context.Context, host string) (hostclaim.Observed, error)
3127	now func() time.Time
3128	entries map[string]edgesClaimEntry
3129	hosts []string // the relocatable hosts of the last pass
3130	}
3131
3132	func newEdgesClaims(self string, read func(context.Context, string) (hostclaim.Observed, error)) *edgesClaims {
3133	return &edgesClaims{self: self, read: read, now: time.Now, entries: map[string]edgesClaimEntry{}}
3134	}
3135
3136	// relocatableRow is spec 1 §2's relocatable host as a registry row shows
3137	// it: a provisioned instance whose process is the activator's own exec
3138	// child. Unit-backed, addr-backed, static and service routes keep their
3139	// box and are published as before; sidecars are never published at all.
3140	func relocatableRow(r registry.Route) bool {
3141	return r.Provisioned && r.Kind == "instance" && r.Backing == "" && r.Unit == "" &&
3142	r.Addr == "" && r.Addrs == "" && r.Socket != ""
3143	}
3144
3145	// decideClaim turns one observation into a decision. An absent claim
3146	// leaves the host as it was before claims existed: a region can only be
3147	// given a second box once every host has one (spec §8).
3148	func (c *edgesClaims) decideClaim(obs hostclaim.Observed) edgesDecision {
3149	if obs.Absent \|\| obs.Claim == nil {
3150	return edgesDecision{}
3151	}
3152	cl := obs.Claim
3153	if cl.Holder == nil \|\| cl.Op.Exec != c.self {
3154	return edgesDecision{withhold: true}
3155	}
3156	stamp := cl.Holder.Stamp
3157	if stamp > math.MaxInt64 {
3158	stamp = 0
3159	}
3160	return edgesDecision{stamp: int64(stamp)}
3161	}
3162
3163	func (c *edgesClaims) dueAt(h string, now time.Time) bool {
3164	e, ok := c.entries[h]
3165	return !ok \|\| now.Sub(e.attempted) >= edgesClaimTTL
3166	}
3167
3168	// backlog reports whether any host of the last pass has a read due.
3169	func (c *edgesClaims) backlog(now time.Time) bool {
3170	for _, h := range c.hosts {
3171	if c.dueAt(h, now) {
3172	return true
3173	}
3174	}
3175	return false
3176	}
3177
3178	// decide returns a decision for each host that has one. Hosts whose last
3179	// attempt is older than edgesClaimTTL are read, never-attempted first and
3180	// then oldest attempt first, up to edgesClaimReadsPerPass. A read that
3181	// fails keeps the previous decision; a host with no successful read has
3182	// none, and publishes as before.
3183	func (c *edgesClaims) decide(ctx context.Context, hosts []string) map[string]edgesDecision {
3184	now := c.now()
3185	c.hosts = append(c.hosts[:0], hosts...)
3186	want := make(map[string]bool, len(hosts))
3187	var due []string
3188	for _, h := range hosts {
3189	want[h] = true
3190	if c.dueAt(h, now) {
3191	due = append(due, h)
3192	}
3193	}
3194	for h := range c.entries {
3195	if !want[h] {
3196	delete(c.entries, h)
3197	}
3198	}
3199	sort.Slice(due, func(i, j int) bool {
3200	ei, iok := c.entries[due[i]]
3201	ej, jok := c.entries[due[j]]
3202	if iok != jok {
3203	return !iok // never attempted first
3204	}
3205	if !ei.attempted.Equal(ej.attempted) {
3206	return ei.attempted.Before(ej.attempted)
3207	}
3208	return due[i] < due[j]
3209	})
3210	if len(due) > edgesClaimReadsPerPass {
3211	due = due[:edgesClaimReadsPerPass]
3212	}
3213	for _, h := range due {
3214	e := c.entries[h]
3215	e.attempted = now
3216	if obs, err := c.read(ctx, h); err == nil {
3217	e.decision, e.decided = c.decideClaim(obs), true
3218	}
3219	c.entries[h] = e
3220	}
3221	out := make(map[string]edgesDecision, len(c.entries))
3222	for h, e := range c.entries {
3223	if e.decided {
3224	out[h] = e.decision
3225	}
3226	}
3227	return out
3228	}
3229
3230	// applyEdgesClaims withholds or stamps every relocatable row in hosts
3231	// (edgesFileWrite's served map, before aliases join it, so an alias of a
3232	// withheld host is withheld with it and an alias of a stamped host takes
3233	// its stamp). claims nil leaves hosts untouched.
3234	func applyEdgesClaims(ctx context.Context, claims *edgesClaims, routes []registry.Route, hosts map[string]edgeHost) {
3235	if claims == nil {
3236	return
3237	}
3238	var reloc []string
3239	for _, r := range routes {
3240	if _, served := hosts[r.Host]; served && relocatableRow(r) {
3241	reloc = append(reloc, r.Host)
3242	}
3243	}
3244	for host, d := range claims.decide(ctx, reloc) {
3245	h, ok := hosts[host]
3246	if !ok {
3247	continue
3248	}
3249	if d.withhold {
3250	delete(hosts, host)
3251	continue
3252	}
3253	h.Stamp = d.stamp
3254	hosts[host] = h
3255	}
3256	}
3257
3258	// edgesLoopState is edgesFileLoop's per-tick decision: the registry and
3259	// timer schedule, plus, with region HA on, an extra pass every
3260	// edgesClaimPassEvery while claim reads are due. A claim pass is a
3261	// publication like any other: it waits out the schedule's retry backoff
3262	// after a failed write, and its own outcome moves that backoff, so a bucket
3263	// that refuses every write is not asked every 30 seconds (Astra plan R2-1).
3264	type edgesLoopState struct {
3265	sched *edgesFileSchedule
3266	claims *edgesClaims
3267	claimsNext time.Time
3268	schedDue bool
3269	claimDue bool
3270	}
3271
3272	// due reports whether to run a pass on this tick.
3273	func (s *edgesLoopState) due(now time.Time, fp string, fpOK bool) bool {
3274	s.schedDue = s.sched.due(now, fp, fpOK)
3275	s.claimDue = s.claims != nil && !now.Before(s.claimsNext) && !now.Before(s.sched.retryAt) && s.claims.backlog(now)
3276	return s.schedDue \|\| s.claimDue
3277	}
3278
3279	// timed reports whether the pass due just allowed is the timer's.
3280	func (s *edgesLoopState) timed() bool { return s.schedDue && s.sched.timed }
3281
3282	// done records the pass's outcome. Every pass, claim passes included, goes
3283	// through the schedule: a failure advances its backoff, and a success
3284	// clears it and records the fingerprint the pass published. A claim pass
3285	// is never the timer's (due cleared timed), so the timer does not move.
3286	func (s *edgesLoopState) done(now time.Time, ok bool) {
3287	if s.schedDue \|\| s.claimDue {
3288	s.sched.done(now, ok)
3289	}
3290	if s.claims != nil {
3291	s.claimsNext = now.Add(edgesClaimPassEvery)
3292	}
3293	}
3294	```
3295
3296	- [ ] Step 4: Wire it into the publisher
3297
3298	In `internal/host/edgesfile.go`, add the import (keep the group sorted):
3299
3300	```go
3301	"github.com/carlosframework/platform/internal/appconfig"
3302	"github.com/carlosframework/platform/internal/hostclaim"
3303	"github.com/carlosframework/platform/internal/registry"
3304	```
3305
3306	Replace the head of `edgesFileWrite`:
3307
3308	```go
3309	func edgesFileWrite(ctx context.Context, s release.Store, reg registry.Registry, ident edgesFile, eip string, last []byte, refresh bool) error {
3310	box := ident.Box
3311	```
3312
3313	with:
3314
3315	```go
3316	func edgesFileWrite(ctx context.Context, s release.Store, reg registry.Registry, ident edgesFile, eip string, last []byte, refresh bool) error {
3317	return edgesFileWriteClaims(ctx, s, reg, ident, eip, last, refresh, nil)
3318	}
3319
3320	// edgesFileWriteClaims is edgesFileWrite with the claim reader region HA
3321	// adds (edgesclaims.go). claims is nil unless CARLOS_REGION_HA is on, and
3322	// nil publishes exactly what edgesFileWrite always has.
3323	func edgesFileWriteClaims(ctx context.Context, s release.Store, reg registry.Registry, ident edgesFile, eip string, last []byte, refresh bool, claims *edgesClaims) error {
3324	box := ident.Box
3325	```
3326
3327	After the routes loop, replace:
3328
3329	```go
3330	hosts[r.Host] = edgeHost{Role: "home", Platform: r.Platform \|\| r.Provisioned, Replica: r.Replica}
3331	}
3332	```
3333
3334	with:
3335
3336	```go
3337	hosts[r.Host] = edgeHost{Role: "home", Platform: r.Platform \|\| r.Provisioned, Replica: r.Replica}
3338	}
3339	// Region HA (spec 1 §5.4): a relocatable host is home only on the box
3340	// its claim names, stamped with the installation. Before owners and
3341	// aliases, so an alias of a withheld host is withheld with it.
3342	applyEdgesClaims(ctx, claims, routes, hosts)
3343	```
3344
3345	In `edgesFileLoop`, replace:
3346
3347	```go
3348	dataDir := provisionTickConfig(os.Getenv).DataDir
3349	log.Printf("agent: edges file loop armed (5min + jitter, and %s after a registry change, %s)", edgesFileWatchInterval, edgesFileKey(box))
3350	```
3351
3352	with:
3353
3354	```go
3355	dataDir := provisionTickConfig(os.Getenv).DataDir
3356	// Region HA reads each relocatable host's claim (spec 1 §5.4); off, the
3357	// file is exactly what it always was.
3358	var claims *edgesClaims
3359	if regionHAEnabled(os.Getenv) {
3360	claims = newEdgesClaims(box, func(ctx context.Context, host string) (hostclaim.Observed, error) {
3361	return hostclaim.Read(ctx, store, host)
3362	})
3363	}
3364	log.Printf("agent: edges file loop armed (5min + jitter, and %s after a registry change, %s)", edgesFileWatchInterval, edgesFileKey(box))
3365	```
3366
3367	Then, where the loop builds its schedule, replace:
3368
3369	```go
3370	sched := newEdgesFileSchedule(time.Now(), func() time.Duration {
3371	return 5*time.Minute + rand.N(time.Minute)
3372	})
3373	tick := time.NewTicker(edgesFileWatchInterval)
3374	```
3375
3376	with:
3377
3378	```go
3379	sched := newEdgesFileSchedule(time.Now(), func() time.Duration {
3380	return 5*time.Minute + rand.N(time.Minute)
3381	})
3382	state := &edgesLoopState{sched: sched, claims: claims}
3383	tick := time.NewTicker(edgesFileWatchInterval)
3384	```
3385
3386	Then, in the loop body, replace:
3387
3388	```go
3389	if !sched.due(now, fp, err == nil) {
3390	continue
3391	}
3392	if sched.timed {
3393	eip = readPublicIP()
3394	}
3395	ident.Headroom = headroomFor(ident.Capabilities, dataDir, reg)
3396	err = edgesFileWrite(ctx, store, reg, ident, eip, &last, sched.timed)
3397	```
3398
3399	with:
3400
3401	```go
3402	if !state.due(now, fp, err == nil) {
3403	continue
3404	}
3405	if state.timed() {
3406	eip = readPublicIP()
3407	}
3408	ident.Headroom = headroomFor(ident.Capabilities, dataDir, reg)
3409	err = edgesFileWriteClaims(ctx, store, reg, ident, eip, &last, state.timed(), claims)
3410	```
3411
3412	Then, at the end of the loop body, replace:
3413
3414	```go
3415	sched.done(now, err == nil)
3416	```
3417
3418	with:
3419
3420	```go
3421	state.done(now, err == nil)
3422	```
3423
3424	Then, in the alias loop, replace:
3425
3426	```go
3427	acct, app := owners[a.Host].owner()
3428	hosts[a.Alias] = edgeHost{Role: "home", Alias: true, Account: acct, App: app}
3429	```
3430
3431	with:
3432
3433	```go
3434	acct, app := owners[a.Host].owner()
3435	// The alias is served by its primary's installation, so it carries
3436	// the primary's stamp (region HA HA4): an unstamped alias would tie
3437	// with a dead predecessor's and the edge would keep the first box
3438	// by name.
3439	hosts[a.Alias] = edgeHost{Role: "home", Alias: true, Account: acct, App: app, Stamp: hosts[a.Host].Stamp}
3440	```
3441
3442	Then, in the delegated wildcard loop, replace:
3443
3444	```go
3445	role, alias := "wildcard", false
3446	var acct, app string
3447	if primary, served := catchallServed[wd.Host]; served {
3448	role, alias = "home", true
3449	```
3450
3451	with:
3452
3453	```go
3454	role, alias := "wildcard", false
3455	var acct, app string
3456	var stamp int64
3457	if primary, served := catchallServed[wd.Host]; served {
3458	role, alias = "home", true
3459	stamp = hosts[primary].Stamp // the dial half is the primary's installation
3460	```
3461
3462	Then, at the end of the same loop, replace:
3463
3464	```go
3465	hosts[key] = edgeHost{Role: role, Wildcard: true, Alias: alias, Account: acct, App: app}
3466	```
3467
3468	with:
3469
3470	```go
3471	hosts[key] = edgeHost{Role: role, Wildcard: true, Alias: alias, Account: acct, App: app, Stamp: stamp}
3472	```
3473
3474	With region HA off every primary's stamp is 0, so every alias and catch-all is published exactly as before.
3475
3476	`box` here is `metricsBoxLabel(…)`, the same value the file publishes as `box_id`, which is the reference claims name (spec §2).
3477
3478	- [ ] Step 5: Run the tests
3479
3480	Run:
3481	```bash
3482	go test ./internal/host/ -run 'EdgesFile\|EdgesClaims\|EdgesLoop' -count=1 -v
3483	go test ./internal/edge/ -run 'TestHomeMap' -count=1
3484	```
3485	Expected: PASS: the nine new host tests (including `TestEdgesFileStampsAliasesAndCatchAllsWithTheirPrimary`, `TestEdgesClaimsAFailingClaimDoesNotStarveTheRest`, `TestEdgesLoopRefreshesEveryClaimWithinTheTTL` and `TestEdgesLoopClaimPassesObeyTheRetryBackoff`), every existing edges-file test, and the home map tests. The `internal/edge` run of `TestHomeMap` alone needs no socket.
3486
3487	- [ ] Step 6: Gate and commit
3488
3489	```bash
3490	go build ./... && go vet ./internal/host/ ./internal/edge/ && gofmt -l internal/host internal/edge
3491	GOOS=darwin go build ./... && GOOS=windows go build ./...
3492	git add internal/host/edgesclaims.go internal/host/edgesclaims_test.go internal/host/edgesfile.go internal/edge/homemap_alias_stamp_test.go
3493	git commit -m "edges: with region HA on, a relocatable host is home only on the box its claim names, stamped
3494
3495	Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
3496	```
3497
3498	---
3499
3500	### Task 7: Root refuses an update while the box takes part in a claim operation (`internal/host/systemparticipation.go`)
3501
3502	Spec §5.10, root's side (H14): under root's update lock, before an executable change, write `/opt/carlos/system/guard/update-begun.json` durably, then scan `<DataDir>/.hostexec/` completely. Any record there, in any state, or anything root cannot read, list or parse, removes the marker and refuses the update, to be retried later. Root records that refusal as `deferred`, not `refused` (Astra plan R1-3): `refused` is terminal for that promote (`systemResult.terminal`, `systemleg.go`), and the agent would never ask again; `deferred` is re-requested on the agent's backoff ladder, at most every five minutes, until the operation ends. Otherwise the update proceeds and the marker stays until it finishes or is recovered.
3503
3504	On main every executable change goes through one path, `systemApplier.install` (an operator's `carlos system rollback` re-points the channel, and the box applies it the same way); the guarded activation and the console's activation reservation are claim D1 and D7 and not on main. Recovery of an update already begun needs no check (spec §5.10: no participant acknowledged meanwhile). The participant's half (write `pending`, then refuse to acknowledge while the marker exists) belongs to claim D6's controller and waits for L6.
3505
3506	Nothing writes `.hostexec/` on main, so with the flag on this changes nothing but a marker file during an update.
3507
3508	Files:
3509	- Create: `internal/host/systemparticipation.go`
3510	- Test: `internal/host/systemparticipation_test.go`
3511	- Modify: `internal/host/systemapply.go` (`systemApplier`, `RunSystemApply`, `Run`, `install`)
3512
3513	Interfaces:
3514	- Consumes: `regionHAEnabled` (Task 3); `writeJSONDurable`, `syncDir`, `readSystemJournal`, `errSystemDeferred`, `systemApplier.record` (existing).
3515	- Produces:
3516	```go
3517	func systemGuardDir(optDir string) string
3518	func updateBegunPath(optDir string) string
3519	func hostexecParticipation(dataDir string) error
3520	func (a *systemApplier) beginExecutableChange(target string, promotedAt time.Time) error
3521	func (a *systemApplier) clearUpdateBegun() error
3522	func (a *systemApplier) settleUpdateBegun()
3523	// systemApplier gains: regionHA bool
3524	```
3525
3526	- [ ] Step 1: Write the failing tests
3527
3528	Create `internal/host/systemparticipation_test.go`:
3529
3530	```go
3531	package host
3532
3533	import (
3534	"context"
3535	"os"
3536	"path/filepath"
3537	"runtime"
3538	"strings"
3539	"testing"
3540	)
3541
3542	func hostexec(t testing.T, f applyFixture) string {
3543	t.Helper()
3544	dir := filepath.Join(f.a.dataDir, hostexecDir)
3545	if err := os.MkdirAll(dir, 0o755); err != nil {
3546	t.Fatal(err)
3547	}
3548	return dir
3549	}
3550
3551	func assertNotUpdated(t testing.T, f applyFixture, why string) {
3552	t.Helper()
3553	r := f.result()
3554	if r == nil \|\| r.Outcome != systemOutcomeDeferred \|\| !strings.Contains(r.Reason, why) {
3555	t.Fatalf("result = %+v, want deferred naming %q", r, why)
3556	}
3557	if r.terminal() {
3558	t.Fatal("the agent would never retry this promote")
3559	}
3560	if bin, _ := os.ReadFile(systemBinaryPath(f.a.optDir)); !strings.Contains(string(bin), "carlos v1") {
3561	t.Fatal("the binary changed under a refusal")
3562	}
3563	if j, _ := readSystemJournal(f.a.optDir); j != nil {
3564	t.Fatalf("a refusal opened a journal: %+v", j)
3565	}
3566	if _, err := os.Stat(updateBegunPath(f.a.optDir)); !os.IsNotExist(err) {
3567	t.Fatalf("the update-begun marker outlived the refusal: %v", err)
3568	}
3569	if f.called("systemctl stop") \|\| f.called("systemctl restart") {
3570	t.Fatal("a refused update touched a unit")
3571	}
3572	}
3573
3574	// Spec §5.10: any controller record, in any state, refuses the update.
3575	func TestApplyRefusesWhileThisBoxTakesPartInAClaimOperation(t *testing.T) {
3576	for _, state := range []string{"pending", "active", "lost"} {
3577	t.Run(state, func(t *testing.T) {
3578	f, _ := applyHappy(t)
3579	f.a.regionHA = true
3580	os.WriteFile(filepath.Join(hostexec(t, f), "shop.bab.test.json"), []byte(`{"state":"`+state+`"}`), 0o600)
3581	if err := f.a.Run(context.Background(), false); err != nil {
3582	t.Fatal(err)
3583	}
3584	assertNotUpdated(t, f, "takes part in a running claim operation")
3585	})
3586	}
3587	}
3588
3589	// Review Focus 4: what root cannot read is a refusal, never a pass.
3590	func TestApplyRefusesWhenItCannotTellWhetherThisBoxTakesPart(t *testing.T) {
3591	cases := map[string]func(t testing.T, f applyFixture){
3592	"a temporary file": func(t testing.T, f applyFixture) {
3593	os.WriteFile(filepath.Join(hostexec(t, f), ".shop.bab.test.json.tmp"), nil, 0o600)
3594	},
3595	"garbage": func(t testing.T, f applyFixture) {
3596	os.WriteFile(filepath.Join(hostexec(t, f), "shop.bab.test.json"), []byte("{half"), 0o600)
3597	},
3598	"a symlink in place of the directory": func(t testing.T, f applyFixture) {
3599	elsewhere := t.TempDir()
3600	if err := os.Symlink(elsewhere, filepath.Join(f.a.dataDir, hostexecDir)); err != nil {
3601	t.Skip("no symlinks here:", err)
3602	}
3603	},
3604	"a file in place of the directory": func(t testing.T, f applyFixture) {
3605	os.WriteFile(filepath.Join(f.a.dataDir, hostexecDir), nil, 0o600)
3606	},
3607	}
3608	for name, setup := range cases {
3609	t.Run(name, func(t *testing.T) {
3610	f, _ := applyHappy(t)
3611	f.a.regionHA = true
3612	setup(t, f)
3613	if err := f.a.Run(context.Background(), false); err != nil {
3614	t.Fatal(err)
3615	}
3616	assertNotUpdated(t, f, "hostexec")
3617	})
3618	}
3619	}
3620
3621	func TestApplyRefusesWhenItCannotListTheControllerDirectory(t *testing.T) {
3622	if runtime.GOOS == "windows" \|\| os.Geteuid() == 0 {
3623	t.Skip("permission bits do not stop this user")
3624	}
3625	f, _ := applyHappy(t)
3626	f.a.regionHA = true
3627	dir := hostexec(t, f)
3628	os.Chmod(dir, 0)
3629	t.Cleanup(func() { os.Chmod(dir, 0o755) })
3630	if err := f.a.Run(context.Background(), false); err != nil {
3631	t.Fatal(err)
3632	}
3633	assertNotUpdated(t, f, "cannot be listed")
3634	}
3635
3636	// The marker is written before the box changes and is gone once the
3637	// update is done; an empty controller directory is no participation.
3638	func TestApplyHoldsTheUpdateBegunMarkerAcrossTheInstall(t *testing.T) {
3639	f, req := applyHappy(t)
3640	f.a.regionHA = true
3641	hostexec(t, f) // present and empty
3642	seen := false
3643	orig := f.a.ops.systemctl
3644	f.a.ops.systemctl = func(args ...string) (string, error) {
3645	if args[0] == "restart" && args[len(args)-1] == systemEdgeUnit {
3646	_, err := os.Stat(updateBegunPath(f.a.optDir))
3647	seen = err == nil
3648	}
3649	return orig(args...)
3650	}
3651	if err := f.a.Run(context.Background(), false); err != nil {
3652	t.Fatal(err)
3653	}
3654	if r := f.result(); r == nil \|\| r.Outcome != systemOutcomeApplied \|\| !r.matches("stable", "v2", req.PromotedAt) {
3655	t.Fatalf("result = %+v", r)
3656	}
3657	if !seen {
3658	t.Fatal("no update-begun marker while the edge restarted onto the new binary")
3659	}
3660	if _, err := os.Stat(updateBegunPath(f.a.optDir)); !os.IsNotExist(err) {
3661	t.Fatalf("marker left after the update finished: %v", err)
3662	}
3663	}
3664
3665	func TestApplyWithoutRegionHAWritesNoMarkerAndIgnoresControllerRecords(t *testing.T) {
3666	f, _ := applyHappy(t)
3667	os.WriteFile(filepath.Join(hostexec(t, f), "shop.bab.test.json"), []byte(`{}`), 0o600)
3668	if err := f.a.Run(context.Background(), false); err != nil {
3669	t.Fatal(err)
3670	}
3671	if r := f.result(); r == nil \|\| r.Outcome != systemOutcomeApplied {
3672	t.Fatalf("result = %+v: the flag is off, so nothing changes", r)
3673	}
3674	if _, err := os.Stat(systemGuardDir(f.a.optDir)); !os.IsNotExist(err) {
3675	t.Fatalf("the guard directory was created with the flag off: %v", err)
3676	}
3677	}
3678
3679	// A crash between the marker and the journal leaves a marker with no
3680	// update behind it; the next run, --recover included, clears it.
3681	func TestAStaleUpdateBegunMarkerIsClearedByTheNextRun(t *testing.T) {
3682	f := newApplyFixture(t, "stable", "v1")
3683	os.MkdirAll(systemGuardDir(f.a.optDir), 0o755)
3684	os.WriteFile(updateBegunPath(f.a.optDir), []byte(`{"target":"v2"}`), 0o644)
3685	if err := f.a.Run(context.Background(), true); err != nil {
3686	t.Fatal(err)
3687	}
3688	if _, err := os.Stat(updateBegunPath(f.a.optDir)); !os.IsNotExist(err) {
3689	t.Fatalf("stale marker kept: %v", err)
3690	}
3691	}
3692
3693	// A rollback that could not finish keeps its journal for the next run, and
3694	// the marker with it: the update is not over.
3695	func TestTheMarkerStaysWhileAJournalIsOpen(t *testing.T) {
3696	f := newApplyFixture(t, "stable", "v1")
3697	f.a.regionHA = true
3698	os.MkdirAll(systemGuardDir(f.a.optDir), 0o755)
3699	os.WriteFile(updateBegunPath(f.a.optDir), []byte(`{"target":"v2"}`), 0o644)
3700	// A journal recovery cannot act on: no carlos.prev to restore.
3701	if err := writeJSONDurable(f.a.file("journal.json"), systemJournal{Phase: systemPhaseInstalled, Target: "v2"}, 0o644); err != nil {
3702	t.Fatal(err)
3703	}
3704	_ = f.a.Run(context.Background(), true)
3705	if j, _ := readSystemJournal(f.a.optDir); j == nil {
3706	t.Fatal("recovery closed the journal; this test needs one it cannot finish")
3707	}
3708	if _, err := os.Stat(updateBegunPath(f.a.optDir)); err != nil {
3709	t.Fatalf("marker cleared while the journal is still open: %v", err)
3710	}
3711	}
3712
3713	// Astra plan R1-3: participation defers, it does not refuse. Once the
3714	// operation ends and its record is cleared, the same promote applies. The
3715	// agent's side, re-requesting a deferred apply, is
3716	// TestSystemLegReusesAVerifiedCacheOnARetry.
3717	func TestApplyDeferredForParticipationAppliesOnceItEnds(t *testing.T) {
3718	f, req := applyHappy(t)
3719	f.a.regionHA = true
3720	rec := filepath.Join(hostexec(t, f), "shop.bab.test.json")
3721	os.WriteFile(rec, []byte(`{"state":"active"}`), 0o600)
3722	if err := f.a.Run(context.Background(), false); err != nil {
3723	t.Fatal(err)
3724	}
3725	if r := f.result(); r == nil \|\| r.Outcome != systemOutcomeDeferred \|\| r.terminal() {
3726	t.Fatalf("result = %+v, want deferred, which the agent retries", r)
3727	}
3728	os.Remove(rec)
3729	if err := f.a.Run(context.Background(), false); err != nil {
3730	t.Fatal(err)
3731	}
3732	if r := f.result(); r == nil \|\| r.Outcome != systemOutcomeApplied \|\| !r.matches("stable", "v2", req.PromotedAt) {
3733	t.Fatalf("result after the operation ended = %+v, want applied", r)
3734	}
3735	}
3736	```
3737
3738	- [ ] Step 2: Run them to see them fail
3739
3740	Run: `go test ./internal/host/ -run 'TakesPart\|CannotTell\|CannotList\|UpdateBegun\|RegionHAWrites\|MarkerStays' -count=1`
3741	Expected: FAIL to build, `f.a.regionHA undefined` and `undefined: hostexecDir`.
3742
3743	- [ ] Step 3: Write root's check
3744
3745	Create `internal/host/systemparticipation.go`:
3746
3747	```go
3748	package host
3749
3750	import (
3751	"errors"
3752	"fmt"
3753	"os"
3754	"path/filepath"
3755	"sort"
3756	"strings"
3757	"time"
3758	)
3759
3760	// Region HA spec 1 §5.10 (H14), root's half. A box that takes part in a
3761	// running claim operation never takes an executable update. Each side
3762	// records its own intent durably, then checks the other's, so of two that
3763	// race at least one sees the other:
3764	//
3765	// - root writes its update-begun marker, then scans <DataDir>/.hostexec/
3766	// and refuses the update while any controller record is there, in any
3767	// state, or while it cannot tell;
3768	// - a participant writes its .hostexec record (claim D6), then reads the
3769	// marker and refuses to acknowledge while one exists. That half waits
3770	// for the claim's D6 controller and is not built here.
3771	//
3772	// The marker stays until the update finishes or is recovered. Dormant
3773	// unless CARLOS_REGION_HA is on.
3774
3775	const (
3776	updateBegunFile = "update-begun.json"
3777	hostexecDir = ".hostexec"
3778	)
3779
3780	// systemGuardDir is /opt/carlos/system/guard, beside root's update state.
3781	func systemGuardDir(optDir string) string { return filepath.Join(optDir, "system", "guard") }
3782
3783	func updateBegunPath(optDir string) string {
3784	return filepath.Join(systemGuardDir(optDir), updateBegunFile)
3785	}
3786
3787	type updateBegun struct {
3788	Target string `json:"target"`
3789	PromotedAt time.Time `json:"promoted_at"`
3790	At time.Time `json:"at"`
3791	}
3792
3793	// hostexecParticipation reports why this box may still be a participant
3794	// in a claim operation, or nil when it certainly is not. The directory
3795	// absent is the only "not". Anything in it, whatever its name or state,
3796	// and anything root cannot list or that is not a plain directory, is a
3797	// refusal (spec §5.10, B1-5). Only names are read: root never opens a
3798	// record here.
3799	func hostexecParticipation(dataDir string) error {
3800	dir := filepath.Join(dataDir, hostexecDir)
3801	fi, err := os.Lstat(dir)
3802	if errors.Is(err, os.ErrNotExist) {
3803	return nil
3804	}
3805	if err != nil {
3806	return fmt.Errorf("%s cannot be read: %v", dir, err)
3807	}
3808	if !fi.IsDir() {
3809	return fmt.Errorf("%s is not a directory", dir)
3810	}
3811	entries, err := os.ReadDir(dir)
3812	if err != nil {
3813	return fmt.Errorf("%s cannot be listed: %v", dir, err)
3814	}
3815	if len(entries) == 0 {
3816	return nil
3817	}
3818	names := make([]string, 0, len(entries))
3819	for _, e := range entries {
3820	names = append(names, e.Name())
3821	}
3822	sort.Strings(names)
3823	if len(names) > 3 {
3824	names = append(names[:3], fmt.Sprintf("and %d more", len(entries)-3))
3825	}
3826	return fmt.Errorf("this box takes part in a running claim operation (%s holds %s)", dir, strings.Join(names, ", "))
3827	}
3828
3829	// beginExecutableChange is §5.10's root side, run under the apply lock
3830	// just before the first box-changing step: write the marker durably, then
3831	// look for participation. On a refusal the marker is removed again and the
3832	// error, wrapping errSystemDeferred, says why. The caller records it as
3833	// "deferred", never "refused": a refusal is terminal for that promote, and
3834	// this one must be retried once the operation ends (spec §5.10: the
3835	// update is refused and retried later). The agent's leg re-requests a
3836	// deferred apply on its backoff ladder, at most every five minutes.
3837	func (a *systemApplier) beginExecutableChange(target string, promotedAt time.Time) error {
3838	if !a.regionHA {
3839	return nil
3840	}
3841	if err := os.MkdirAll(systemGuardDir(a.optDir), 0o755); err != nil {
3842	return err
3843	}
3844	if err := writeJSONDurable(updateBegunPath(a.optDir), updateBegun{Target: target, PromotedAt: promotedAt, At: a.ops.now().UTC()}, 0o644); err != nil {
3845	return err
3846	}
3847	if reason := hostexecParticipation(a.dataDir); reason != nil {
3848	if err := a.clearUpdateBegun(); err != nil {
3849	a.ops.logf("system apply: %v", err)
3850	}
3851	return fmt.Errorf("%w: %v", errSystemDeferred, reason)
3852	}
3853	return nil
3854	}
3855
3856	// clearUpdateBegun removes the marker durably. A marker that is not there
3857	// is already cleared.
3858	func (a *systemApplier) clearUpdateBegun() error {
3859	err := os.Remove(updateBegunPath(a.optDir))
3860	if errors.Is(err, os.ErrNotExist) {
3861	return nil
3862	}
3863	if err != nil {
3864	return fmt.Errorf("remove %s: %w", updateBegunPath(a.optDir), err)
3865	}
3866	return syncDir(systemGuardDir(a.optDir))
3867	}
3868
3869	// settleUpdateBegun clears the marker once no update is in progress: no
3870	// journal means the update finished, rolled back, was recovered, or never
3871	// got past its first step. A kept journal (a rollback that could not
3872	// finish) keeps the marker with it. Run with or without CARLOS_REGION_HA,
3873	// so a marker left behind after the flag is turned off still goes.
3874	func (a *systemApplier) settleUpdateBegun() {
3875	if j, err := readSystemJournal(a.optDir); err != nil \|\| j != nil {
3876	return
3877	}
3878	if err := a.clearUpdateBegun(); err != nil {
3879	a.ops.logf("system apply: %v", err)
3880	}
3881	}
3882	```
3883
3884	- [ ] Step 4: Wire it into the apply
3885
3886	In `internal/host/systemapply.go`, add the field at the end of `systemApplier`:
3887
3888	```go
3889	platform string // "<goos>-<goarch>"; the caller sets it from runtime
3890	gateWait, soak time.Duration
3891	// regionHA is CARLOS_REGION_HA: root writes its update-begun marker and
3892	// refuses an update while this box takes part in a claim operation
3893	// (region HA spec 1 §5.10; systemparticipation.go).
3894	regionHA bool
3895	}
3896	```
3897
3898	In `RunSystemApply`, set it:
3899
3900	```go
3901	platform: runtime.GOOS + "-" + runtime.GOARCH, gateWait: systemGateWait, soak: systemGateSoak,
3902	regionHA: regionHAEnabled(os.Getenv),
3903	}
3904	```
3905
3906	In `Run`, right after `defer os.RemoveAll(a.stagingDir())`, add:
3907
3908	```go
3909	// Whatever this run does, a finished or recovered update leaves no
3910	// update-begun marker behind (spec §5.10).
3911	defer a.settleUpdateBegun()
3912	```
3913
3914	In `install`, immediately before `// PromotedAt rides along for the whole transaction`, add:
3915
3916	```go
3917	// Region HA (spec 1 §5.10): the marker, then the participation check,
3918	// before the first step that changes the box. Either failing defers the
3919	// update, which the agent retries; the box is left exactly as it was.
3920	if err := a.beginExecutableChange(target, req.PromotedAt); err != nil {
3921	if !errors.Is(err, errSystemDeferred) {
3922	err = fmt.Errorf("%w: update-begun marker: %v", errSystemDeferred, err)
3923	}
3924	return a.record(req, err)
3925	}
3926	```
3927
3928	The deferral is recorded in `result.json`; the agent's leg re-requests a deferred apply (pinned on the agent's side by `TestSystemLegReusesAVerifiedCacheOnARetry`).
3929
3930	- [ ] Step 5: Run the tests
3931
3932	Run: `go test ./internal/host/ -run 'Apply\|Recover\|Rollback\|Prev\|Layout\|UpdateBegun\|MarkerStays\|TakesPart\|CannotTell\|CannotList\|Deferred' -count=1 -v`
3933	Expected: PASS: the eight new tests (including `TestApplyDeferredForParticipationAppliesOnceItEnds`) and every existing apply and recovery test.
3934
3935	- [ ] Step 6: Gate and commit
3936
3937	```bash
3938	go build ./... && go vet ./internal/host/ && gofmt -l internal/host
3939	GOOS=darwin go build ./... && GOOS=windows go build ./... && GOOS=linux GOARCH=arm64 go build ./...
3940	git add internal/host/systemparticipation.go internal/host/systemparticipation_test.go internal/host/systemapply.go
3941	git commit -m "system: with region HA on, root marks an update begun and refuses it while the box takes part in a claim operation
3942
3943	Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
3944	```
3945
3946	---
3947
3948	### Task 8: Storage policy, the spec note and the branch gate
3949
3950	Spec §6: "The control bucket denies `PutObject` without `s3:if-none-match: `, and `DeleteObject` and `DeleteObjectVersion`, on `control/boxes/` (plan 1a's policy) and on `control/box-volumes/`, `control/box-enrollment/` and `control/box-fences/`. Each Deny is scoped to its prefix only." `control/boxes/` is already covered. Task 5's writer uses `control/box-volumes/`; `control/box-enrollment/` and `control/box-fences/` have no writer yet, but a Deny grants nothing, and putting all four in place at once means one apply. The `.bind.json` statement and the `operator_role_arns` variable stay (§19: "harmless and can go at the next apply"): removing the variable would break every root that sets it.
3951
3952	Nothing here is applied. The flagship's root lives in `carlosframework/carloku-infrastructure` and consumes these modules at a release tag. Two cautions bind this task: a whole-root apply of the flagship restarts every box (user-data drift), and this task changes no `user_data`; a broad Deny on these buckets once broke every lease release, and every Deny here names one evidence prefix and nothing wider.
3953
3954	Files:
3955	- Modify: `infra/modules/deployment-buckets/main.tf`
3956	- Create: `infra/modules/deployment-buckets/tests/evidence.tftest.hcl`
3957	- Modify: `docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md` (append §24)
3958
3959	- [ ] Step 1: Write the failing module test
3960
3961	Create `infra/modules/deployment-buckets/tests/evidence.tftest.hcl`:
3962
3963	```hcl
3964	# Region HA spec 1 §6: every box-evidence prefix is create-only and never
3965	# deleted, and the Denies name those prefixes and nothing wider (a broad
3966	# Deny on this bucket once broke every lease release). Each statement is
3967	# checked whole: effect, principal, actions, condition and resources, so
3968	# turning a Deny into an Allow, dropping an action or loosening the
3969	# conditional-write condition fails here.
3970
3971	mock_provider "aws" {}
3972
3973	override_resource {
3974	target = aws_s3_bucket.deployment
3975	values = { arn = "arn:aws:s3:::dep" }
3976	}
3977
3978	variables {
3979	deployment_bucket_name = "dep"
3980	data_bucket_name = "data"
3981	}
3982
3983	run "evidence_prefixes_are_create_only_and_never_deleted" {
3984	command = plan
3985
3986	assert {
3987	condition = jsonencode([
3988	for s in jsondecode(aws_s3_bucket_policy.deployment_box_records.policy).Statement : s if s.Sid == "BoxRecordsAreCreateOnly"
3989	]) == jsonencode([{
3990	Action = "s3:PutObject"
3991	Condition = {
3992	Bool = { "s3:ObjectCreationOperation" = "true" }
3993	Null = { "s3:if-none-match" = "true" }
3994	}
3995	Effect = "Deny"
3996	Principal = "*"
3997	Resource = [
3998	"arn:aws:s3:::dep/control/boxes/*",
3999	"arn:aws:s3:::dep/control/box-volumes/*",
4000	"arn:aws:s3:::dep/control/box-enrollment/*",
4001	"arn:aws:s3:::dep/control/box-fences/*",
4002	]
4003	Sid = "BoxRecordsAreCreateOnly"
4004	}])
4005	error_message = "BoxRecordsAreCreateOnly must Deny, to everyone, every PutObject without If-None-Match on exactly the four box-evidence prefixes"
4006	}
4007
4008	assert {
4009	condition = jsonencode([
4010	for s in jsondecode(aws_s3_bucket_policy.deployment_box_records.policy).Statement : s if s.Sid == "BoxRecordsAreNeverDeleted"
4011	]) == jsonencode([{
4012	Action = ["s3:DeleteObject", "s3:DeleteObjectVersion"]
4013	Effect = "Deny"
4014	Principal = "*"
4015	Resource = [
4016	"arn:aws:s3:::dep/control/boxes/*",
4017	"arn:aws:s3:::dep/control/box-volumes/*",
4018	"arn:aws:s3:::dep/control/box-enrollment/*",
4019	"arn:aws:s3:::dep/control/box-fences/*",
4020	]
4021	Sid = "BoxRecordsAreNeverDeleted"
4022	}])
4023	error_message = "BoxRecordsAreNeverDeleted must Deny, to everyone, both delete actions on exactly the four box-evidence prefixes"
4024	}
4025	}
4026	```
4027
4028	- [ ] Step 2: Run it to see it fail
4029
4030	Run (sandbox off; `tofu init` needs the network once):
4031	```bash
4032	cd infra/modules/deployment-buckets && tofu init -backend=false -input=false && tofu test; cd -
4033	```
4034	Expected: `Failure! 0 passed, 1 failed.`, both assertions naming their statement. Leave `.terraform/` and `.terraform.lock.hcl` in place until Step 4 has run: Step 4 needs the initialized provider.
4035
4036	- [ ] Step 3: Extend the policy
4037
4038	In `infra/modules/deployment-buckets/main.tf`, replace the section comment:
4039
4040	```hcl
4041	# --- Box records (region HA spec 1 §5.1) -----------------------------------
4042	# control/boxes/<id>.json binds a box's name to one AWS instance, and
4043	# control/boxes/<id>.bind.json is an operator's word for a legacy label.
4044	# Both are written once and never changed. Storage enforces that, so no
4045	# bug or stolen box credential can rebind a name to another machine.
4046	#
4047	# EVERY statement here names control/boxes/ and nothing wider. A broad Deny
4048	# on this bucket has broken lease releases before.
4049	```
4050
4051	with:
4052
4053	```hcl
4054	# --- Box evidence (region HA spec 1 §5.1, §5.9, §6) ------------------------
4055	# control/boxes/<ref>.json is a box's AWS-signed identity, the evidence its
4056	# enrollment reads. control/box-volumes/, control/box-enrollment/ and
4057	# control/box-fences/ are the volume records, membership confirmations
4058	# and termination fence records regional workers write. Every one is
4059	# written once and never changed or deleted. Storage enforces that, so no
4060	# bug or stolen box credential can rewrite the evidence behind a decision.
4061	# (control/boxes/*.bind.json is no longer written: enrollment replaced the
4062	# operator bind. Its statement stays harmless until operator_role_arns is
4063	# retired with it.)
4064	#
4065	# EVERY statement here names one of these prefixes and nothing wider. A
4066	# broad Deny on this bucket has broken lease releases before.
4067
4068	locals {
4069	box_evidence_prefixes = ["control/boxes", "control/box-volumes", "control/box-enrollment", "control/box-fences"]
4070	}
4071	```
4072
4073	and replace the first two statements of `aws_s3_bucket_policy.deployment_box_records`:
4074
4075	```hcl
4076	{
4077	# A record or bind may be created, never replaced. The qualifier
4078	# leaves multipart preparation alone and still covers its completion.
4079	Sid = "BoxRecordsAreCreateOnly"
4080	Effect = "Deny"
4081	Principal = "*"
4082	Action = "s3:PutObject"
4083	Resource = "${aws_s3_bucket.deployment.arn}/control/boxes/*"
4084	Condition = {
4085	Null = { "s3:if-none-match" = "true" }
4086	Bool = { "s3:ObjectCreationOperation" = "true" }
4087	}
4088	},
4089	{
4090	Sid = "BoxRecordsAreNeverDeleted"
4091	Effect = "Deny"
4092	Principal = "*"
4093	Action = ["s3:DeleteObject", "s3:DeleteObjectVersion"]
4094	Resource = "${aws_s3_bucket.deployment.arn}/control/boxes/*"
4095	},
4096	```
4097
4098	with:
4099
4100	```hcl
4101	{
4102	# Evidence may be created, never replaced. The qualifier leaves
4103	# multipart preparation alone and still covers its completion.
4104	Sid = "BoxRecordsAreCreateOnly"
4105	Effect = "Deny"
4106	Principal = "*"
4107	Action = "s3:PutObject"
4108	Resource = [for p in local.box_evidence_prefixes : "${aws_s3_bucket.deployment.arn}/${p}/*"]
4109	Condition = {
4110	Null = { "s3:if-none-match" = "true" }
4111	Bool = { "s3:ObjectCreationOperation" = "true" }
4112	}
4113	},
4114	{
4115	Sid = "BoxRecordsAreNeverDeleted"
4116	Effect = "Deny"
4117	Principal = "*"
4118	Action = ["s3:DeleteObject", "s3:DeleteObjectVersion"]
4119	Resource = [for p in local.box_evidence_prefixes : "${aws_s3_bucket.deployment.arn}/${p}/*"]
4120	},
4121	```
4122
4123	The Sids are kept, so a reader of the bucket policy's history sees one statement widened, not two replaced.
4124
4125	- [ ] Step 4: Run the module test
4126
4127	Run (sandbox off; this re-runs `tofu init`, which is a no-op if Step 2's initialization is still there):
4128	```bash
4129	cd infra/modules/deployment-buckets && tofu init -backend=false -input=false && tofu fmt -check && tofu validate && tofu test; cd -
4130	```
4131	Expected: `Success! The configuration is valid.` and `Success! 1 passed, 0 failed.` Only now remove `.terraform/` and `.terraform.lock.hcl` from the module directory, as no module here commits them.
4132
4133	- [ ] Step 5: Record this plan's decisions in the spec
4134
4135	Append to `docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md`:
4136
4137	```markdown
4138
4139	## 24. Notes from plan 1b (2026-10-06)
4140
4141	Plan `docs/superpowers/plans/2026-10-06-region-ha-1b-claim-readers.md`
4142	builds the parts of rev 5.4 that need none of L1 to L20: enrollment
4143	evidence, §19's corrections, the agent publishing its identity record,
4144	the fence bound to the roster's `cloud`, volume records and the discharge
4145	check, the edges file following `op.exec`, and root's half of §5.10. All
4146	of it is dormant behind `CARLOS_REGION_HA`. It keeps every property above
4147	and settles five details:
4148
4149	1. Enrollment refuses one instance under two references (§5.1). The
4150	record's `box` field is not covered by AWS's signature, so box A's
4151	record copied to B's key with that field rewritten still verifies.
4152	`boxid.Verifier.Enroll` refuses an instance the roster holds under another
4153	reference, as well as a reference the roster holds for another
4154	instance, counting entries in every state.
4155	2. A positive stamp is the relocatable mark (§18's open question).
4156	Only relocatable hosts with a claim, and their aliases and served
4157	catch-alls, carry a stamp, so the edge's report
4158	of two boxes at one positive stamp already concerns them alone. No
4159	separate marker.
4160	3. The edges file's claim reads (§5.4). A relocatable host with no
4161	claim is published as before, unstamped: §8's gate admits a second box
4162	only once every host has a claim. A read that fails keeps the last
4163	decision. Each read stands 10 minutes (H9), at most 256 reads a pass.
4164	4. "The first record taken at enrollment" (§5.9) is the earliest
4165	volume record for the enrollment, observed within one hour of the
4166	roster's enrollment time. A later first record cannot speak for a
4167	volume attached and removed before it, so the chores stay open. And
4168	(controller ruling, 2026-10-06) the newest record must be no more
4169	than two hours older than the termination observation; otherwise the
4170	chores stay open for the operator path.
4171	5. No volume record from a `terminated` read. Its mappings are empty
4172	(§8 step 5's dev test), and recording it would list every volume as
4173	removed.
4174
4175	It also removes plan 1a's `Bind`, `BindKey`, `Verifier.Verify` and
4176	`activator.WarmCopyTrusted`, and makes every box-evidence prefix
4177	create-only and never deleted in the deployment bucket's policy.
4178
4179	Controller rulings of 2026-10-06, on Paul's behalf: the edges file
4180	withholds a relocatable host when `op.exec` names another box or none
4181	(§5.4 as written); root's §5.10 refusal stays behind `CARLOS_REGION_HA`
4182	until claim D6 lands, then becomes unconditional; the `*.bind.json`
4183	statement and `operator_role_arns` stay until no root sets the variable.
4184	```
4185
4186	- [ ] Step 6: Run the whole gate
4187
4188	Run: `go build ./... && make test`, then `GOOS=darwin go build ./... && GOOS=windows go build ./... && GOOS=linux GOARCH=arm64 go build ./...`.
4189	Expected: `gofmt -l .` clean and every package `ok`, with the sandbox off (see Global Constraints for the socket tests that fail inside it). `make test` takes up to thirty minutes. If a package unrelated to this plan fails, do not fix it here and do not skip it: run that package alone, confirm it also fails on `origin/main`, and report both results.
4190
4191	- [ ] Step 7: Commit
4192
4193	```bash
4194	git add infra/modules/deployment-buckets docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md
4195	git commit -m "infra: box evidence prefixes are create-only and never deleted; spec: plan 1b's decisions
4196
4197	Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
4198	```
4199
4200	- [ ] Step 8: Push and read CI
4201
4202	```bash
4203	git push origin region-ha-1b
4204	amadan ci status carlos/platform -branch region-ha-1b
4205	```
4206
4207	Expected: exit 0 (pass). Exit 2 is pending: wait and read it again. Exit 1 is a failure: read the log, fix it, and do not ask for the branch to be merged. Merging is Paul's call, through `amadan branch merge`, never a push to `main`.
4208
4209	---
4210
4211	## Waits for the claim
4212
4213	Everything spec 1 rev 5.4 asks for that this plan does not build, with the request it waits on (§9). Item numbers are §19's revised list for plan 1b.
4214
4215	\| What \| Waits for \| Why it cannot be built now \|
4216	\|---\|---\|---\|
4217	\| Relocation: the `relocating` op kind, its phases, start CAS, replacement table, source-loss conversion, two-replacement bound (§5.8; item 7) \| L1 \| a new claim op kind \|
4218	\| The release sequence's writers-stopped check (§5.8 step 1) \| L10 \| changes `stopevidence`'s contract \|
4219	\| Coverage `released` on the first probe admission (§5.8) \| L9 \| changes `hostclaim.AdmitWriter`'s admission rules \|
4220	\| A relocation start despite a regional `park-generation` chore \| L11 \| changes `hostclaim.StartGuard` (CC11) \|
4221	\| Building the prepared generation from a pinned set; the local install receipt \| L16, L17 (and L1) \| shared with moves' landing; changes moves \|
4222	\| Reconcile's row cases, held participant rows, former executor rows, the 10-minute sweep as an exception to C4 (§5.3; item 2) \| L12 \| changes claim C4 and moves §9.6 \|
4223	\| Create names a box: `chooseBox(eligibleBoxes(…))` in the create intent's `selected_executor` (§5.2; item 1) \| L8 (claim P5-D6, the claim's plan 5) \| changes create's step 2 and step 4 CASes \|
4224	\| A move into a multi-box region naming its target box (§5.2) \| L15 \| changes moves \|
4225	\| A target terminated mid-move taken off the move by the fence \| L14 \| changes moves \|
4226	\| Writing fence records to `control/box-fences/`, the `lost` list, the fence CAS (§5.9) \| L2, L4 \| a lost-box fence with no operator is a claim rule change \|
4227	\| Discharging `park-local` chores in the fence CAS from `boxvolumes.Discharge` \| L3 \| a claim D5 change \|
4228	\| The census excluding a fenced box and a console on it; the `compute` binding (§5.9) \| L5 \| a claim D7 change \|
4229	\| A stale console report accepted for box-started relocations (H18) \| L20 \| a claim D7 change \|
4230	\| Root's §5.10 refusal made unconditional, no longer behind `CARLOS_REGION_HA` (controller ruling 3) \| claim D6 (with L6) \| until D6's controller writes `.hostexec/`, the flag is the only thing keeping a one-box region unchanged \|
4231	\| The participant's half of §5.10: write `pending`, then refuse to acknowledge while root's marker exists \| L6 (and claim D6's controller, not on main) \| `.hostexec/` is D6's; nothing writes it yet \|
4232	\| Enrollment itself: the console verifying records with `boxid.Enroll` into `control/host-actors.json`, the tofu check, refusals \| L7 (claim D7's roster, not on main) \| there is no roster to write \|
4233	\| The volume recorder loop: describing every enrolled box in the region on the 10-minute sweep and on `shutting-down` \| L7 \| records are keyed by enrollment, which only the roster assigns \|
4234	\| `eligibleBoxes`' caller, reading the roster and the region's AWS region from it (§18's second correction) \| L7, L8 \| no roster \|
4235	\| Auto Scaling membership confirmations, and the cross-region confirmer for an emptied region (H16, H17), with the cross-region IAM statement \| L19 (and spec 4's group and the roster's `regions` entry) \| the console may accept them only under L19; nothing to confirm yet \|
4236	\| The files publisher as a generation-bound writer, the copied manifest at each cold admission, retirement by `park-generation` (§5.6); `files_durability` on routes and the agent starting `hostfiles.Run` (item 5) \| L13 \| a second ordinary-replication exception to claim C3 \|
4237	\| The multi-box gate's roster half and the HA floor (§8; item 8) \| L7 and claim D1's floor \| no roster, no floor on main \|
4238	\| The joint model with two boxes per region \| L18 \| the model belongs to the claim and moves sessions \|
4239	\| Reading the termination queue and the placer loop \| spec 3 (§12), then L2 to L4 \| nothing can be done with an event before a fence can be written \|
4240
4241	Follow-up, not waiting for the claim. Retire the `OnlyOperatorsWriteBinds` statement and the `operator_role_arns` variable in `deployment-buckets` once no consuming root sets the variable (controller ruling 5).
4242
4243	Not waiting for the claim, and not in this plan. Plan 1a's "After the build" items 9 (a streaming put, lifting the 64 MiB limit, and the data bucket's conditional-write policy for `files/`), 11 (one inotify instance per agent) and 12 (a size cap in `StagePlatform`, a rate limit on `Commit`) need no claim change. They matter only once the files publisher is switched on, which waits for L13, so they belong with that plan.
4244
4245	Turning the flag on. `CARLOS_REGION_HA` must be set in `/etc/carlos/host.env`, where both the agent and root's `carlos-system-update.service` read it. A drop-in for `carlos-edge.service` alone would turn on the edges file's claim reads and the box record but not root's §5.10 refusal. It is on only for a value `strconv.ParseBool` reads as true (`1`, `t`, `true`). Two things must be in place before any box sets it: the per-pass deadline on the edges file's claim reads (built: see "After the build", item 4), and a roster-wide check that no instance holds more than one reference (enrollment's per-record check cannot see two entries the roster already holds; it belongs with L7's roster).
4246
4247	## What the dry run could not check
4248
4249	- `readIdentityRecord` against a real box's IMDS. Plan 1a verified the regional certificates and the signature on the dev box; this plan's reader was tested only against an in-process fake. Before the branch lands, on the dev box with `CARLOS_REGION_HA=1` set for the agent, confirm `control/boxes/<label>.json` appears once and that `boxid.Enroll` accepts its bytes with the dev account and region.
4250	- `termfence.DescribeInstance` and `boxvolumes.Observe` against EC2. Both were tested with fakes and the SDK's own types. The first live use is spec 3's placer.
4251	- Root's marker under a real `carlos-system-update.service`. The apply's tests run the whole `Run` with faked `systemctl`; whether `/etc/carlos/host.env` carries `CARLOS_REGION_HA` on a given box is a deployment fact.
4252	- The tests listed under Global Constraints that open Unix sockets, inside the sandbox.
4253
4254	## Self-review
4255
4256	- Spec coverage. §19's corrections: Tasks 1, 2 and 4 (box records as evidence, the bind's tofu check moved into enrollment, `eligibleBoxes` from the roster, `chooseBox`'s tie-break, `WarmCopyTrusted`, `termfence` on `cloud` with no volume input). §5.1 enrollment evidence: Tasks 1 and 3. §5.4: Task 6. §5.9 under H19: Tasks 4 and 5. §5.10 root's half: Task 7. §6 storage: Task 8. §10's unit tests that need no claim: enrollment evidence (Task 1), `eligibleBoxes` and `choose` (Task 2), termination-fence acceptance (Task 4: NotFound, wrong account, wrong region, stopped, an event alone, a describe alone, `shutting-down` offered as evidence, accepted with no volume record), the volume recorder and chore discharge, including the 2-hour last-record window (Task 5), root's refusal on any controller record (Task 7). Everything else is in "Waits for the claim".
4257	- Placeholders. None: every code step is a whole file or an exact replacement.
4258	- Type consistency. `boxid.Cloud` is defined in Task 1 and used by Tasks 2, 4 and 5; `boxid.Identity` is kept through Task 3 and removed in Task 4, its last user; `regionHAEnabled` is defined in Task 3 and used by Tasks 6 and 7; `termfence.DescribeInstance` (Task 4) returns the `*ec2.DescribeInstancesOutput` that `boxvolumes.Observe` (Task 5) reads. The dry run applied the tasks in order to a clean tree and built each.
4259	- Review Focus. Each of the five has a test in its owning task, named above, and each was watched to fail with its guard removed.
4260
4261	## Copy waiting for Paul
4262
4263	Controller ruling 6 took this out of Task 8 until Paul has reviewed it (the `copy-review` skill). Proposed for `infra/modules/region-host/README.md`, replacing the two paragraphs that begin "The root volume is deleted when the instance is terminated." and "Open item for region HA plan 1b":
4264
4265	```markdown
4266	The root volume is deleted when the instance is terminated. The fence
4267	does not depend on that: a `terminated` read is enough. It decides only
4268	whether a dead box's local cleanup can be skipped automatically, so keep
4269	it, and keep Recycle Bin, Data Lifecycle Manager and AWS Backup rules off
4270	platform box volumes.
4271
4272	The box role's `ec2:DescribeInstances` has no region condition, because
4273	the infrastructure inventory reads every region, and a second, narrower
4274	Allow could not take that away. Region HA keeps to the box's own region
4275	in code instead: every describe it makes for a fence or a volume record
4276	is pinned to the region the box was enrolled in.
4277	```
4278
4279	## Astra plan round 1: dispositions
4280
4281	Findings are in `CODEX-PLAN-REGION-HA-1B-R1.md` (reviewed at 856767d8). All seven were checked against the tree and accepted. The fixes were dry-run again from the plan's own text: a script applied Tasks 1 to 8 in order to a clean copy of bf813540 and each built, vetted and cross-built; the tests of every task passed (with the sandbox's eight known socket failures in `internal/host`, unchanged); and each new test failed with its mechanism removed. No controller ruling was needed.
4282
4283	\| # \| Finding \| Disposition \|
4284	\|---\|---\|---\|
4285	\| 1 \| P1 Task 8: Step 2's cleanup leaves Step 4 uninitialized \| fixed: Step 2 keeps `.terraform/`; Step 4 runs `tofu init` again (a no-op when still initialized) before `validate` and `test`, and only then removes it \|
4286	\| 2 \| P2 Task 6: aliases and served catch-alls lose the stamp, so a dead box's alias ties and can win \| fixed: both take their primary's stamp from the served map; `TestEdgesFileStampsAliasesAndCatchAllsWithTheirPrimary` (killed by removing either stamp), and `internal/edge`'s `TestHomeMapAnAliasFollowsTheNewerInstallation` pins the edge's half. With the flag off every stamp is 0, so the file is unchanged \|
4287	\| 3 \| P2 Task 7: `refused` is terminal for the promote, so participation suppresses the release for good \| fixed: participation and a marker failure are recorded as `deferred`, which the agent's leg re-requests on its backoff ladder; `TestApplyDeferredForParticipationAppliesOnceItEnds` (deferred, not terminal, then applied once the record is gone; killed when recorded as refused) \|
4288	\| 4 \| P2 Task 6: failing reads keep their place at the front and starve healthy claims \| fixed: an entry records its last attempt separately from its last decision; a failed read moves the attempt and keeps the decision, so it goes to the back of the queue; `TestEdgesClaimsAFailingClaimDoesNotStarveTheRest` (256 failing claims then a healthy one; killed by the old bookkeeping) \|
4289	\| 5 \| P2 Task 6: 256 reads a pass at the timer's 5 to 6 minutes cannot refresh 2,000 hosts in 10 minutes \| fixed: `edgesLoopState` adds a pass every 30 seconds while claim reads are due (flag on only); `TestEdgesLoopRefreshesEveryClaimWithinTheTTL` simulates the loop's schedule for three hours with 2,000 hosts and requires every claim re-read within 15 minutes (killed with the extra passes off) \|
4290	\| 6 \| P2 Task 5: incomplete, non-recordable or unknown-version records can authorize discharge \| fixed: `Discharge` validates every stored record of the enrollment (version 1, a state that carries mappings, a non-empty mapping, every volume with an id and device); anything else keeps the chores open. `TestDischargeRefusesStoredRecordsThatAreNotEvidence` writes each bad record through the store and lists it back, placed so that no removal masks it; each case killed by removing its check \|
4291	\| 7 \| P2 Task 8: the policy test checks resources only \| fixed: each statement is compared whole (effect, principal, actions, condition, resources); five mutations (Deny to Allow, a dropped delete action, the wrong condition key, a narrower principal, a wider prefix) and main's policy each fail it \|
4292
4293	## Astra plan round 2: dispositions
4294
4295	Findings are in `CODEX-PLAN-REGION-HA-1B-R2.md` (reviewed at 161728ab). Round 2 marked R1-1 to R1-4 and R1-7 resolved and R1-5 and R1-6 partly resolved, and raised two findings, both checked and accepted. Tasks 5 and 6 were dry-run again from the plan's text, with every task applied in order to a clean copy of bf813540, built, vetted, cross-built and tested; each check below was removed in turn and its test watched to fail.
4296
4297	\| # \| Finding \| Disposition \|
4298	\|---\|---\|---\|
4299	\| R2-1 \| P2 Task 6: claim passes bypass the publication retry backoff \| fixed: a claim pass is gated on the schedule's `retryAt`, and its outcome goes through `edgesFileSchedule.done` (a failure advances the backoff, a success clears it; the timer does not move). `TestEdgesLoopClaimPassesObeyTheRetryBackoff` runs the loop's schedule against a bucket that fails for 20 minutes then recovers: 12 attempts while failing (40 unthrottled), and the 30-second passes back after recovery. It fails with the `retryAt` gate removed, with claim outcomes not recorded, and with claim passes off \|
4300	\| R2-2 \| P2 Task 5: three validation checks survive removal \| fixed: three more stored cases, each failing exactly one check: `running` with no volume (empty mapping), `terminated` with its root mapping (state), a volume with an id but no device (device). Removing each of the five checks (state, empty mapping, volume id, device, version) fails its own case \|
4301
4302	## After the build
4303
4304	The final whole-branch review (2026-10-06) raised seven findings, all ruled by the controller and fixed on the branch:
4305
4306	1. `boxvolumes`: a `shutting-down` record whose DeleteOnTermination volumes show `detaching` (EC2 detaching them on the way to deletion) is evidence, not a refusal. In any other state, and for a volume whose flag is false, only `attached` and `attaching` count.
4307	2. `boxid.Verifier.Enroll`: a roster entry for the same reference with no proven cloud yet does not refuse it; only an entry naming another instance does.
4308	3. `Enroll`'s doc says the record's `box` is not signed, so for a reference the roster does not yet hold the caller must bind instance to reference independently (operator approval, the tofu id, or H16's group confirmation). The agent's `publishBoxRecord` checks a record already under its label against its own identity and logs loudly, once per run, when it is signed for another instance or cannot be read.
4309	4. The edges file's claim reads share a 20-second deadline per pass; a read it cuts short keeps the last decision, and reads not reached wait for a later pass. Failed reads are logged on the first failing pass, every 60th after, and on recovery. A host `hostclaim.Key` rejects is never read and publishes as before.
4310	5. `regionHAEnabled` accepts only what `strconv.ParseBool` reads as true; `""`, `0`, `false`, `off`, `no` and anything unparseable are off.
4311	6. `boxid`'s "unknown field" test uses a fully signed record plus one extra field, so only `DisallowUnknownFields` can refuse it.
4312	7. Spec §24 names `Verifier.Enroll` and includes aliases and served catch-alls among the stamped entries; this plan says where the flag is set and what must exist before it is turned on.
4313