amadan

carlos / platform Public

plan: region HA 1b, the evidence and claim readers that need none of L1 to L20

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Paul Campbell pushed by claude 257438bf33ca201cfaed1112ed250907a58b98b0 parent bf81354
1 file changed, +3741 −0
  • docs/superpowers/plans/2026-10-06-region-ha-1b-claim-readers.md +3741 −0
diff --git a/docs/superpowers/plans/2026-10-06-region-ha-1b-claim-readers.md b/docs/superpowers/plans/2026-10-06-region-ha-1b-claim-readers.md
new file mode 100644
index 000000000..0130922ca
--- /dev/null
+++ b/docs/superpowers/plans/2026-10-06-region-ha-1b-claim-readers.md
@@ -0,0 +1,3741 @@
+# Region HA 1b: Evidence and Claim Readers Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Build every part of region HA spec 1 rev 5.4 that needs none of the claim and moves changes L1 to L20: enrollment evidence from `internal/boxid`, §19's corrections to plan 1a, the agent publishing its identity record, the termination fence bound to the roster's `cloud`, volume records and the discharge check, the edges file following the claim's executor, and root's refusal to update a box that takes part in a claim operation. All of it dormant behind `CARLOS_REGION_HA`.
+
+**Architecture:** Eight tasks. Four are pure libraries that read and decide but write nothing a decision rests on (`boxid.Enroll`, `termfence.Verify`, `boxvolumes`, `chooseBox`/`eligibleBoxes`). Three change what a box does, each only with `CARLOS_REGION_HA` on: the agent publishes its identity record once, the edges file publishes a relocatable host as home only on the box its claim's `op.exec` names, and root refuses an executable update while `<DataDir>/.hostexec/` holds anything. One task is storage policy. Nothing adds a claim op kind, changes `hostclaim`'s admission rules, or changes moves; claims are only read, through `hostclaim.Read`.
+
+**Tech Stack:** Go 1.26, module `github.com/carlosframework/platform`; `release.Store` (create-only and ETag compare-and-swap puts); `aws-sdk-go-v2/service/ec2` and `smithy-go` (already direct dependencies); SQLite registry; OpenTofu modules under `infra/modules/`.
+
+**Spec:** `docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md`, rev 5.4 (approved by Paul 2026-10-06). Executors read §0 (rulings H1 to H19), §2, §5.1, §5.4, §5.9, §5.10, §6, §8, §9 (the L-table), §18 and §19 first. Plan 1a (`docs/superpowers/plans/2026-10-05-region-ha-1a-foundations.md`, merged as aadb860) built the pieces this plan corrects; read its "After the build" section.
+
+## Why this is plan 1b, and what it leaves out
+
+The spec asks the host lifecycle claim and move-any-app sessions for twenty changes (§9, L1 to L20). Neither session has accepted them. So this plan builds only what needs none of them, and stops at every point where the next step would write a claim, add an op kind, depend on the roster (`control/host-actors.json`, claim D7, not on main), or change moves. The section "Waits for the claim", after the tasks, maps everything left to its L-number.
+
+Three of the candidates the brief named were checked against the spec and main and left out:
+
+- **Reading the termination queue.** The spec gives the placer loop that reads the queue to spec 3 (§12), and an event read is only useful to a writer of fence records (L2 to L4). A reader that cannot write a fence could only leave messages on the queue or delete evidence. It also needs `service/sqs`, which is not a dependency.
+- **The Auto Scaling membership confirmation (H16, H17).** It reads the target region's group ARN from the roster's `regions` entry (claim P5-D2's extension) and lists the roster's enrollments in that region. There is no roster and no Auto Scaling group yet (spec 4), and the console may accept the record only once L19 is accepted. It needs `service/autoscaling` too.
+- **IAM for those two.** Nothing in this plan calls SQS or Auto Scaling, and this repository grants nothing for later. `ec2:DescribeInstances`, the one call this plan's code makes, is already allowed in every region by the inventory statement (`infra/modules/region-host/main.tf:213`); Task 8 records why a narrower statement would add nothing.
+
+## Global Constraints
+
+- **Dormant and inert.** Nothing in this plan may change behaviour on a box where `CARLOS_REGION_HA` is unset or `0`, or on a one-box region (spec §4, §8 step 2: "Ship §5 dormant behind `CARLOS_REGION_HA` … With one box per region, every rule is a no-op"). The flag is read by one helper, `regionHAEnabled` (Task 3).
+- **Scope (the brief).** Do not add a claim op kind, change `hostclaim`'s admission rules, or change moves. Read claims only through `hostclaim.Read`. Use `hostguard`, `stopevidence` and `genselect` as they are.
+- **H19, verbatim:** "The failover fence rests on the `terminated` read alone; disk evidence only decides discharge of local chores." Nothing in this plan lets a volume record gate execution, admission, relocation, recovery, row minting or a box's start (spec §8, B4-1).
+- **Absence is never evidence** (spec §5.9): "`stopped` and `stopping` do not count … A missing instance, `InvalidInstanceID.NotFound`, an API error, a partial reply, or a lookup in another account or region is **never** evidence." Only state `terminated` fences.
+- **Box references are labels** (#212 option B, spec §2): one DNS label (`controlname.ValidLabel`), so never containing `.` or `/`. The instance id is private and lives in the roster entry's `cloud`.
+- **Wire-shape changes are additive** (AGENTS.md). This plan adds no edges-file field (`stamp` exists since plan 1a). Records it defines are new objects under new prefixes.
+- **Do not add a method to `release.Store`** (eleven implementations). `Put(ctx, key, body, "")` is create-only; a lost race is `release.ErrCASConflict`.
+- **No new Go module dependencies.** `service/ec2`, `aws/middleware` and `smithy-go` are already direct. Do not add `service/sqs` or `service/autoscaling`.
+- **AWS is behind interfaces.** Every EC2 call goes through `termfence.EC2`; tests use fakes and never call AWS. IMDS is reached through an `*http.Client` whose transport tests replace, so no test opens a socket.
+- Linux-only syscalls live in `*_linux.go` with a `!linux` stub. This plan adds none: `systemparticipation.go` uses `os` only. CI cross-compiles darwin, windows and linux/arm64.
+- Tests use `release.NewDirStore(t.TempDir())`; inject failures with a struct embedding `release.Store` that overrides one method.
+- **Copy.** No console or CLI copy is added. Log lines are operator-facing and follow the surrounding `agent: …` / `system apply: …` style. Task 8's README paragraph is docs prose: before committing it, put it through Paul's copy review (the `copy-review` skill) and apply his edits verbatim. If any other task finds it needs user-facing copy, stop.
+- Commit messages: lowercase `area:` prefix and a sentence stating the behaviour, then a blank line and `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`.
+- **Go environment.** Run Go with `GOCACHE=$TMPDIR/gocache GOFLAGS="-mod=mod -buildvcs=false"` (every command below assumes both are exported). Tests that open Unix sockets fail inside the Claude sandbox with `socket: operation not permitted`; on this branch's base those are, in `internal/host`, `TestSleepingInstanceRearmsItsNextDeliveryBeforeReturning`, `TestScheduledTickWakesASleepingInstanceAndItHibernatesAgain`, `TestDeliveryReusesTheConnectionForASocket`, `TestArmScheduleFiresOnceTimersWithNoDeployStore`, `TestProxyCacheIsStableAndBounded`, `TestDeliveryDoesNotHoldTheInstanceAwake`, `TestDeliveryStillCarriesScheduleAtHeader` and `TestSocketServingProbe`; in `internal/activator`, the `TestExecBackendStart…`, `TestStartUnit…`, `TestUnitProc…`, `TestSeedUnit…`, `TestStop…` and `TestPrepareRemovesAStaleSocketFile` tests; and `internal/edge` as a whole. That is the sandbox, not the change: run those packages with the sandbox off, or rely on CI. Scratch files go under `$TMPDIR`. Never use `git stash`, `reset --hard`, `checkout --` or `clean`.
+- **OpenTofu.** `tofu test` starts the AWS provider over a local socket, so it needs the sandbox off, and `tofu init` needs network access to `registry.opentofu.org` and GitHub.
+- Gate before each commit: `go build ./... && go vet ./<pkgs>/ && gofmt -l <dirs> && go test ./<pkgs>/ -run '<tests>' -count=1`, then `GOOS=darwin go build ./... && GOOS=windows go build ./...`. Full gate before the branch lands: `go build ./... && make test`, and the three cross-builds.
+- **Exact values.** Claim read cache on the edges file **10 minutes** (H9's sweep); at most **256** claim reads per edges pass; a volume record at least once an **hour** when nothing changes; the first volume record must be taken within **1 hour** of enrollment; capability **`region-ha-1`**; fence state **`terminated`** only; root's marker **`/opt/carlos/system/guard/update-begun.json`**; controller records **`<DataDir>/.hostexec/`**; volume records **`control/box-volumes/<enrollment>/<observed_at>.json`**.
+- Work on a branch cut from `origin/main` (suggested name `region-ha-1b`). Merging is Paul's call, through `amadan branch merge`.
+- This plan's code was dry-run on 2026-10-06: every task was applied in order to a clean copy of `origin/main` (bf813540), built, vetted, cross-built for darwin, windows and linux/arm64, and its tests run; the five Review Focus tests and four others were each watched to fail under a mutation of the code they pin; Task 8's tofu test was run against the AWS provider's mock and watched to fail against main's policy. Then the code was removed. What the dry run could not do is listed at the end.
+
+## Review Focus
+
+1. **A copied identity record.** Box A's own record, validly signed, written under box B's reference with its `box` field rewritten. The `box` field is not signed, so the signature still verifies; enrollment must refuse because the instance is already enrolled as A (Task 1, `TestEnrollRefusesAnInstanceAlreadyEnrolledUnderAnotherReference`).
+2. **A claim that cannot be read.** A store error on a claim read must neither take a served host off this box's edges file nor publish a host this box was told not to; a host never read yet publishes as it did before claims (Task 6, `TestEdgesFileKeepsTheLastDecisionWhenAClaimCannotBeRead`).
+3. **A `terminated` describe offered as a volume observation.** Its `BlockDeviceMappings` is empty, so recorded it would list every volume as removed. It must not be recordable; and a removal that no record listed must still keep the chores open (Task 5, `TestObserveRefusesWhatIsNotTheInstancesMapping`, `TestDischargeSeesARemovalNoRecordListed`).
+4. **A controller directory root cannot read.** A temporary file, a record that does not parse, a symlink or a file in place of `.hostexec`, or a directory root cannot list: each must refuse the update, never pass it (Task 7, `TestApplyRefusesWhenItCannotTellWhetherThisBoxTakesPart`, `TestApplyRefusesWhenItCannotListTheControllerDirectory`).
+5. **A fence read from another region.** A worker enrolled in another AWS region reading a perfect `terminated` observation must not fence the box (spec §6: the fence refuses any observation not made in the box's own region) (Task 4, `TestAWorkerInAnotherRegionNeverFences`).
+
+## File Structure
+
+| Path | Responsibility |
+|---|---|
+| `internal/boxid/record.go` | The identity record, `ValidBox`, `Key`, create-only `Publish`, `Load` (exact bytes) |
+| `internal/boxid/verify.go` | `Cloud`, `Verifier.Enroll`: the record as enrollment evidence |
+| `internal/console/boxchoice.go` | `eligibleBoxes` over the roster's proven cloud; `chooseBox` with P5-D2's tie-break |
+| `internal/activator/installed.go`, `activator.go` | `WarmCopyTrusted` removed; comments point at claim R6 |
+| `internal/host/boxrecord.go` | `regionHAEnabled`; reading IMDS; publishing this box's record at start |
+| `internal/termfence/fence.go` | Observations, `DescribeInstance`, `Verify` bound to the roster entry, §5.9's record shape |
+| `internal/boxvolumes/volumes.go` | Volume records: observe, decide, write create-only, list, decide discharge |
+| `internal/host/edgesclaims.go` | The edges file's claim cache and the home/stamp decision |
+| `internal/host/edgesfile.go` | Wiring the claim cache into the publisher |
+| `internal/host/systemparticipation.go` | Root's update-begun marker and `.hostexec` check |
+| `internal/host/systemapply.go` | Wiring them into the apply |
+| `infra/modules/deployment-buckets/main.tf`, `tests/evidence.tftest.hcl` | Create-only, never-deleted evidence prefixes |
+| `infra/modules/region-host/README.md` | The disk and DescribeInstances notes, revised for H19 |
+
+---
+
+### Task 1: Enrollment evidence (`internal/boxid`)
+
+Spec §5.1 and §19: `control/boxes/<ref>.json` is no longer an authority. It is the evidence enrollment reads, and enrollment records the proven account, region and instance id in the roster entry's `cloud`. The operator bind is superseded (its tofu check moves into enrollment), and box references are labels.
+
+**Files:**
+- Modify (replace whole file): `internal/boxid/record.go`, `internal/boxid/verify.go`
+- Test (replace whole file): `internal/boxid/boxid_test.go`
+
+**Interfaces:**
+- Consumes: `release.Store`, `controlname.ValidLabel`.
+- Produces:
+ ```go
+ package boxid
+ const Prefix = "control/boxes/"
+ func ValidBox(ref string) bool
+ func Key(box string) string
+ type Record struct{ V int; Box, IdentityDocument, IdentitySignature, WrittenAt string } // json as before
+ func Publish(ctx context.Context, s release.Store, rec Record) error // create-only; same bytes again is success
+ func Load(ctx context.Context, s release.Store, box string) ([]byte, error) // exact bytes; os.ErrNotExist-wrapped when absent
+ var ErrUnverified error
+ type Identity struct{ AccountID, Region, InstanceID string } // kept only until Task 4
+ type Cloud struct{ Provider, Account, Region, InstanceID string } // json: provider, account, region, instance_id
+ func (c Cloud) Complete() bool
+ type Verifier struct{ Account string; Certs map[string]*x509.Certificate }
+ type Enrolled struct{ Box string; Cloud Cloud }
+ type Expect struct{ Box, AWSRegion, TofuInstanceID string; Roster []Enrolled }
+ type Evidence struct{ Box string; Cloud Cloud; RecordSHA256 string }
+ func (v Verifier) Enroll(raw []byte, e Expect) (Evidence, error)
+ func EmbeddedCerts() (map[string]*x509.Certificate, error) // unchanged
+ ```
+ Removed: `BindKey`, `Bind`, `Verifier.Verify`, and `Load`'s record and bind returns. Nothing outside `internal/boxid` used them.
+
+**Why `Enroll` takes raw bytes.** The hash enrollment records must be of the bytes it verified. Decoding inside `Enroll` makes that one value, where a `Record` and a separately supplied `[]byte` could disagree.
+
+**Why the roster check runs both ways.** The spec asks enrollment to "refuse a reference already enrolled to a different instance". The record's `box` field is not covered by AWS's signature, so a box can copy another box's record to its own key, rewrite `box`, and still verify. `Enroll` therefore also refuses an instance the roster already holds under another reference. Entries in every state count: a retired reference never names a working box again, and a terminated instance never returns.
+
+- [ ] **Step 1: Write the failing tests**
+
+Replace `internal/boxid/boxid_test.go` with:
+
+```go
+package boxid
+
+import (
+ "context"
+ "crypto"
+ "crypto/rand"
+ "crypto/rsa"
+ "crypto/sha256"
+ "crypto/x509"
+ "crypto/x509/pkix"
+ "encoding/base64"
+ "encoding/hex"
+ "encoding/json"
+ "errors"
+ "math/big"
+ "os"
+ "testing"
+ "time"
+
+ "github.com/carlosframework/platform/internal/release"
+)
+
+// signer stands in for AWS: one key and certificate per test.
+type signer struct {
+ key *rsa.PrivateKey
+ cert *x509.Certificate
+}
+
+func newSigner(t *testing.T) signer {
+ t.Helper()
+ key, err := rsa.GenerateKey(rand.Reader, 2048)
+ if err != nil {
+ t.Fatal(err)
+ }
+ tmpl := &x509.Certificate{SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "test"},
+ NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour)}
+ der, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key)
+ if err != nil {
+ t.Fatal(err)
+ }
+ cert, err := x509.ParseCertificate(der)
+ if err != nil {
+ t.Fatal(err)
+ }
+ return signer{key, cert}
+}
+
+func (s signer) record(t *testing.T, box, account, region, instance string) (Record, []byte) {
+ t.Helper()
+ doc, _ := json.Marshal(map[string]string{"accountId": account, "region": region, "instanceId": instance})
+ sum := sha256.Sum256(doc)
+ sig, err := rsa.SignPKCS1v15(rand.Reader, s.key, crypto.SHA256, sum[:])
+ if err != nil {
+ t.Fatal(err)
+ }
+ rec := Record{V: 1, Box: box, IdentityDocument: base64.StdEncoding.EncodeToString(doc),
+ IdentitySignature: base64.StdEncoding.EncodeToString(sig), WrittenAt: "2026-10-06T00:00:00Z"}
+ raw, _ := json.Marshal(rec)
+ return rec, raw
+}
+
+// verifier certifies eu-west-1 and eu-west-2 with the same key, so a test
+// can show that a supported but wrong region is still refused.
+func (s signer) verifier() Verifier {
+ return Verifier{Account: "111122223333", Certs: map[string]*x509.Certificate{"eu-west-1": s.cert, "eu-west-2": s.cert}}
+}
+
+func expect(box string) Expect { return Expect{Box: box, AWSRegion: "eu-west-1"} }
+
+func TestEnrollProvesTheCloudOfALabel(t *testing.T) {
+ s := newSigner(t)
+ _, raw := s.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
+ ev, err := s.verifier().Enroll(raw, expect("annie-welding"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ sum := sha256.Sum256(raw)
+ want := Evidence{Box: "annie-welding", RecordSHA256: hex.EncodeToString(sum[:]),
+ Cloud: Cloud{Provider: "aws", Account: "111122223333", Region: "eu-west-1", InstanceID: "i-0abc"}}
+ if ev != want {
+ t.Fatalf("evidence = %+v\nwant %+v", ev, want)
+ }
+ if !ev.Cloud.Complete() {
+ t.Fatal("a proven cloud is not complete")
+ }
+}
+
+func TestEnrollRefusals(t *testing.T) {
+ s, other := newSigner(t), newSigner(t)
+ good := func() []byte {
+ _, raw := s.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
+ return raw
+ }
+ cases := map[string]struct {
+ raw []byte
+ e Expect
+ }{
+ "wrong account": {func() []byte {
+ _, raw := s.record(t, "annie-welding", "999999999999", "eu-west-1", "i-0abc")
+ return raw
+ }(), expect("annie-welding")},
+ "region with no certificate": {func() []byte {
+ _, raw := s.record(t, "annie-welding", "111122223333", "ap-south-1", "i-0abc")
+ return raw
+ }(), Expect{Box: "annie-welding", AWSRegion: "ap-south-1"}},
+ "a supported region that is not the placement's": {func() []byte {
+ _, raw := s.record(t, "annie-welding", "111122223333", "eu-west-2", "i-0abc")
+ return raw
+ }(), expect("annie-welding")},
+ "signed by someone else": {func() []byte {
+ _, raw := other.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
+ return raw
+ }(), expect("annie-welding")},
+ "document altered after signing": {func() []byte {
+ r, _ := s.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
+ doc, _ := json.Marshal(map[string]string{"accountId": "111122223333", "region": "eu-west-1", "instanceId": "i-0zzz"})
+ r.IdentityDocument = base64.StdEncoding.EncodeToString(doc)
+ raw, _ := json.Marshal(r)
+ return raw
+ }(), expect("annie-welding")},
+ "record names another box": {good(), expect("bert-sawing")},
+ "reference is not a label": {good(), expect("annie.welding")},
+ "no expected region": {good(), Expect{Box: "annie-welding"}},
+ "tofu names another instance": {good(), Expect{Box: "annie-welding", AWSRegion: "eu-west-1", TofuInstanceID: "i-0zzz"}},
+ "unknown field in the record": {[]byte(`{"v":1,"box":"annie-welding","extra":1}`), expect("annie-welding")},
+ "trailing data": {append(good(), []byte(`{}`)...), expect("annie-welding")},
+ "version 2": {func() []byte {
+ r, _ := s.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
+ r.V = 2
+ raw, _ := json.Marshal(r)
+ return raw
+ }(), expect("annie-welding")},
+ "reference enrolled elsewhere": {good(), Expect{Box: "annie-welding", AWSRegion: "eu-west-1", Roster: []Enrolled{{Box: "annie-welding", Cloud: Cloud{InstanceID: "i-0old"}}}}},
+ }
+ for name, tc := range cases {
+ t.Run(name, func(t *testing.T) {
+ if ev, err := s.verifier().Enroll(tc.raw, tc.e); !errors.Is(err, ErrUnverified) {
+ t.Fatalf("evidence = %+v, err = %v; want ErrUnverified", ev, err)
+ }
+ })
+ }
+}
+
+// Review Focus 1: box A's record, intact and validly signed, copied under
+// box B's key with its box field rewritten. The signature still verifies,
+// because the box field is not signed. Only the roster check refuses it.
+func TestEnrollRefusesAnInstanceAlreadyEnrolledUnderAnotherReference(t *testing.T) {
+ s := newSigner(t)
+ _, raw := s.record(t, "bert-sawing", "111122223333", "eu-west-1", "i-0abc")
+ roster := []Enrolled{{Box: "annie-welding", Cloud: Cloud{Provider: "aws", Account: "111122223333", Region: "eu-west-1", InstanceID: "i-0abc"}}}
+ if _, err := s.verifier().Enroll(raw, Expect{Box: "bert-sawing", AWSRegion: "eu-west-1", Roster: roster}); !errors.Is(err, ErrUnverified) {
+ t.Fatalf("err = %v, want ErrUnverified", err)
+ }
+ // The same box re-enrolled as the same instance is not a conflict.
+ roster[0].Box = "bert-sawing"
+ if _, err := s.verifier().Enroll(raw, Expect{Box: "bert-sawing", AWSRegion: "eu-west-1", Roster: roster}); err != nil {
+ t.Fatalf("re-enrolling the same instance: %v", err)
+ }
+}
+
+func TestEnrollAcceptsTheTofuInstanceForALegacyReference(t *testing.T) {
+ s := newSigner(t)
+ _, raw := s.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
+ if _, err := s.verifier().Enroll(raw, Expect{Box: "annie-welding", AWSRegion: "eu-west-1", TofuInstanceID: "i-0abc"}); err != nil {
+ t.Fatal(err)
+ }
+}
+
+func TestPublishIsCreateOnlyAndIdempotent(t *testing.T) {
+ ctx, st, s := context.Background(), release.NewDirStore(t.TempDir()), newSigner(t)
+ rec, _ := s.record(t, "annie-welding", "111122223333", "eu-west-1", "i-0abc")
+ if err := Publish(ctx, st, rec); err != nil {
+ t.Fatal(err)
+ }
+ if err := Publish(ctx, st, rec); err != nil {
+ t.Fatalf("republishing the same record: %v", err)
+ }
+ other := rec
+ other.WrittenAt = "2027-01-01T00:00:00Z"
+ if err := Publish(ctx, st, other); !errors.Is(err, release.ErrCASConflict) {
+ t.Fatalf("a different record for the same box: err = %v, want ErrCASConflict", err)
+ }
+ raw, err := Load(ctx, st, "annie-welding")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if _, err := s.verifier().Enroll(raw, expect("annie-welding")); err != nil {
+ t.Fatalf("the published record does not enroll: %v", err)
+ }
+}
+
+func TestPublishAndLoadRefuseAReferenceThatIsNotALabel(t *testing.T) {
+ ctx, st, s := context.Background(), release.NewDirStore(t.TempDir()), newSigner(t)
+ rec, _ := s.record(t, "x.bind", "111122223333", "eu-west-1", "i-0abc")
+ if err := Publish(ctx, st, rec); err == nil {
+ t.Fatal("published under a reference with a dot")
+ }
+ if _, err := Load(ctx, st, "../edges/x"); !errors.Is(err, ErrUnverified) {
+ t.Fatalf("err = %v, want ErrUnverified", err)
+ }
+ if _, err := Load(ctx, st, "nobody-here"); !errors.Is(err, os.ErrNotExist) {
+ t.Fatalf("err = %v, want os.ErrNotExist", err)
+ }
+}
+
+func TestEmbeddedCertsCoverTheFlagshipRegions(t *testing.T) {
+ certs, err := EmbeddedCerts()
+ if err != nil {
+ t.Fatal(err)
+ }
+ for _, region := range []string{"eu-west-1", "eu-west-2", "us-east-1", "ap-southeast-2"} {
+ c := certs[region]
+ if c == nil {
+ t.Errorf("no certificate for %s", region)
+ continue
+ }
+ if c.PublicKeyAlgorithm != x509.RSA {
+ t.Errorf("%s: %v key; the base64-signature certificate is RSA (the PKCS7 one is DSA: wrong tab)", region, c.PublicKeyAlgorithm)
+ }
+ }
+}
+```
+
+- [ ] **Step 2: Run them to see them fail**
+
+Run: `go test ./internal/boxid/ -count=1`
+Expected: FAIL to build, `undefined: Expect` and `s.verifier().Enroll undefined`.
+
+- [ ] **Step 3: Write the record**
+
+Replace `internal/boxid/record.go` with:
+
+```go
+// Package boxid is the evidence a platform box presents to be enrolled in
+// the roster (region HA spec 1 §5.1, ruling H11): the box's own AWS-signed
+// instance identity document, published once under its box reference. The
+// record grants nothing by itself. Enrollment verifies it (Verifier.Enroll)
+// and copies the proven account, region and instance id into the roster
+// entry's cloud field; from then on the roster, never this record, is what
+// binds a reference to an instance.
+package boxid
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+
+ "github.com/carlosframework/platform/internal/controlname"
+ "github.com/carlosframework/platform/internal/release"
+)
+
+const Prefix = "control/boxes/"
+
+// ValidBox reports whether ref can be a box reference: one DNS label, so
+// it can never contain "." or "/" and no two references share a key.
+func ValidBox(ref string) bool { return controlname.ValidLabel(ref) }
+
+// Key is where a box's record lives. Callers check ValidBox first.
+func Key(box string) string { return Prefix + box + ".json" }
+
+type Record struct {
+ V int `json:"v"`
+ Box string `json:"box"`
+ IdentityDocument string `json:"identity_document"`
+ IdentitySignature string `json:"identity_signature"`
+ WrittenAt string `json:"written_at"`
+}
+
+// Publish writes a box's record create-only. Writing the same bytes again
+// succeeds; a different record for the same box is ErrCASConflict, because
+// a record is never replaced.
+func Publish(ctx context.Context, s release.Store, rec Record) error {
+ if !ValidBox(rec.Box) {
+ return fmt.Errorf("boxid: %q is not a box reference", rec.Box)
+ }
+ body, err := json.Marshal(rec)
+ if err != nil {
+ return fmt.Errorf("boxid: marshal: %w", err)
+ }
+ _, err = s.Put(ctx, Key(rec.Box), body, "")
+ if !errors.Is(err, release.ErrCASConflict) {
+ return err
+ }
+ cur, _, gerr := s.Get(ctx, Key(rec.Box))
+ if gerr != nil {
+ return fmt.Errorf("boxid: %s exists and cannot be read: %w", rec.Box, gerr)
+ }
+ if bytes.Equal(cur, body) {
+ return nil
+ }
+ return fmt.Errorf("boxid: %s already has a different record: %w", rec.Box, release.ErrCASConflict)
+}
+
+// Load returns the exact bytes of a box's record, which is what enrollment
+// verifies and hashes. A missing record is os.ErrNotExist-wrapped.
+func Load(ctx context.Context, s release.Store, box string) ([]byte, error) {
+ if !ValidBox(box) {
+ return nil, fmt.Errorf("%w: %q is not a box reference", ErrUnverified, box)
+ }
+ raw, _, err := s.Get(ctx, Key(box))
+ return raw, err
+}
+```
+
+- [ ] **Step 4: Write enrollment**
+
+Replace `internal/boxid/verify.go` with:
+
+```go
+package boxid
+
+import (
+ "bytes"
+ "crypto/sha256"
+ "crypto/x509"
+ "encoding/base64"
+ "encoding/hex"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+)
+
+// ErrUnverified wraps every reason a record is not evidence for enrollment.
+var ErrUnverified = errors.New("boxid: record does not verify")
+
+// Identity is what an identity document proves. termfence binds to it
+// until it moves to Cloud; nothing else uses it.
+type Identity struct{ AccountID, Region, InstanceID string }
+
+// Cloud is the roster entry's cloud field (spec §2, §5.1): the instance a
+// box reference is enrolled as. Enrollment fills it from a verified record;
+// every later check (the termination fence, volume records) compares
+// against it and never against anything a box says about itself.
+type Cloud struct {
+ Provider string `json:"provider"`
+ Account string `json:"account"`
+ Region string `json:"region"`
+ InstanceID string `json:"instance_id"`
+}
+
+// Complete reports whether every field is set and the provider is AWS.
+func (c Cloud) Complete() bool {
+ return c.Provider == "aws" && c.Account != "" && c.Region != "" && c.InstanceID != ""
+}
+
+type Verifier struct {
+ Account string // the deployment's AWS account id
+ Certs map[string]*x509.Certificate // by AWS region
+}
+
+// Enrolled is one roster entry as enrollment needs to see it: its
+// reference and the instance it was enrolled as. Entries in every state
+// count, retired ones included: a retired reference never names a working
+// box again (spec §5.9), and a terminated instance never comes back.
+type Enrolled struct {
+ Box string
+ Cloud Cloud
+}
+
+// Expect is what the enrolling side knows before it reads the record.
+type Expect struct {
+ Box string // the reference being enrolled
+ AWSRegion string // the AWS region of the placement it is enrolled into
+ // TofuInstanceID is the region's tofu region-host instance_id output.
+ // The caller sets it when Box already appears in claims or edges files,
+ // as a box predating region HA does, and leaves it "" otherwise.
+ TofuInstanceID string
+ Roster []Enrolled
+}
+
+// Evidence is what enrollment records: the roster entry's cloud, and the
+// SHA-256 of the exact record bytes it was proven from.
+type Evidence struct {
+ Box string
+ Cloud Cloud
+ RecordSHA256 string
+}
+
+func refuse(format string, a ...any) error {
+ return fmt.Errorf("%w: %s", ErrUnverified, fmt.Sprintf(format, a...))
+}
+
+func decodeRecord(raw []byte) (Record, error) {
+ dec := json.NewDecoder(bytes.NewReader(raw))
+ dec.DisallowUnknownFields()
+ var rec Record
+ if err := dec.Decode(&rec); err != nil {
+ return Record{}, refuse("record is not JSON: %v", err)
+ }
+ if _, err := dec.Token(); err != io.EOF {
+ return Record{}, refuse("trailing data after the record")
+ }
+ return rec, nil
+}
+
+// Enroll checks raw, the exact bytes stored at Key(e.Box), as enrollment
+// evidence (spec §5.1, H11). In order: the reference, the record's own box
+// field, AWS's signature under the certificate for the document's region,
+// the deployment account, the AWS region expected for the placement, the
+// tofu instance id for a reference predating region HA, and the roster:
+// the reference enrolled as another instance, or the instance enrolled
+// under another reference, is refused. A record copied from one box to
+// another's key carries the first box's signed instance id, so the last
+// check is what keeps one machine from holding two references.
+func (v Verifier) Enroll(raw []byte, e Expect) (Evidence, error) {
+ if !ValidBox(e.Box) {
+ return Evidence{}, refuse("%q is not a box reference", e.Box)
+ }
+ if e.AWSRegion == "" {
+ return Evidence{}, refuse("no expected AWS region for %s", e.Box)
+ }
+ rec, err := decodeRecord(raw)
+ if err != nil {
+ return Evidence{}, err
+ }
+ if rec.V != 1 {
+ return Evidence{}, refuse("record version %d", rec.V)
+ }
+ if rec.Box != e.Box {
+ return Evidence{}, refuse("the record stored for %q names %q", e.Box, rec.Box)
+ }
+ doc, err := base64.StdEncoding.DecodeString(rec.IdentityDocument)
+ if err != nil {
+ return Evidence{}, refuse("document is not base64")
+ }
+ sig, err := base64.StdEncoding.DecodeString(rec.IdentitySignature)
+ if err != nil {
+ return Evidence{}, refuse("signature is not base64")
+ }
+ var d struct {
+ AccountID string `json:"accountId"`
+ Region string `json:"region"`
+ InstanceID string `json:"instanceId"`
+ }
+ if err := json.Unmarshal(doc, &d); err != nil {
+ return Evidence{}, refuse("document is not JSON")
+ }
+ cert := v.Certs[d.Region]
+ if cert == nil {
+ return Evidence{}, refuse("no signing certificate for region %q", d.Region)
+ }
+ if err := cert.CheckSignature(x509.SHA256WithRSA, doc, sig); err != nil {
+ return Evidence{}, refuse("signature: %v", err)
+ }
+ switch {
+ case d.InstanceID == "":
+ return Evidence{}, refuse("document names no instance")
+ case v.Account == "" || d.AccountID != v.Account:
+ return Evidence{}, refuse("account %q is not the deployment's", d.AccountID)
+ case d.Region != e.AWSRegion:
+ return Evidence{}, refuse("the box is in %q, the placement is in %q", d.Region, e.AWSRegion)
+ case e.TofuInstanceID != "" && e.TofuInstanceID != d.InstanceID:
+ return Evidence{}, refuse("tofu records instance %q for this region, the record %q", e.TofuInstanceID, d.InstanceID)
+ }
+ for _, en := range e.Roster {
+ if en.Box == e.Box && en.Cloud.InstanceID != d.InstanceID {
+ return Evidence{}, refuse("%s is enrolled as instance %q", e.Box, en.Cloud.InstanceID)
+ }
+ if en.Box != e.Box && en.Cloud.InstanceID == d.InstanceID {
+ return Evidence{}, refuse("instance %s is enrolled as %s", d.InstanceID, en.Box)
+ }
+ }
+ sum := sha256.Sum256(raw)
+ return Evidence{
+ Box: e.Box,
+ Cloud: Cloud{Provider: "aws", Account: d.AccountID, Region: d.Region, InstanceID: d.InstanceID},
+ RecordSHA256: hex.EncodeToString(sum[:]),
+ }, nil
+}
+```
+
+- [ ] **Step 5: Run the tests**
+
+Run: `go test ./internal/boxid/ -count=1 -v`
+Expected: PASS, including `TestEnrollRefusesAnInstanceAlreadyEnrolledUnderAnotherReference` and every subtest of `TestEnrollRefusals`.
+
+- [ ] **Step 6: Gate and commit**
+
+```bash
+go build ./... && go vet ./internal/boxid/ && gofmt -l internal/boxid
+go test ./internal/boxid/ ./internal/termfence/ ./internal/console/ -run 'Enroll|Publish|Load|Embedded|Eligible|ChooseBox|Fence|Lookalike|Event|Legacy|Unverified' -count=1
+GOOS=darwin go build ./... && GOOS=windows go build ./...
+git add internal/boxid
+git commit -m "boxid: a box record is enrollment evidence, and one instance never holds two references
+
+Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
+```
+
+Expected: `gofmt -l` prints nothing; every package `ok`.
+
+---
+
+### Task 2: §19's corrections to choice and the warm copy
+
+Spec §19: `eligibleBoxes` takes its identities from the roster, not from loading records; `chooseBox`'s last tie-break becomes claim P5-D2's greatest SHA-256 of `host + "\x00" + reference` (H12); `WarmCopyTrusted` and the activator's warm path are withdrawn (H13), and the activator's one-box comment points at claim R6.
+
+**Files:**
+- Modify (replace whole file): `internal/console/boxchoice.go`
+- Test (replace whole file): `internal/console/boxchoice_test.go`
+- Modify: `internal/activator/installed.go`, `internal/activator/installed_test.go`, `internal/activator/activator.go:1164-1168`
+
+**Interfaces:**
+- Consumes: `boxid.Cloud` (Task 1).
+- Produces:
+ ```go
+ // package console
+ type rosterBox struct{ Placement string; Cloud boxid.Cloud }
+ func eligibleBoxes(snap edgesSnapshot, roster map[string]rosterBox, placement, awsRegion, homeName, expectedFP string, lost map[string]bool) []boxCandidate
+ func chooseBox(host string, cands []boxCandidate, running map[string]boxLoad, app string) (boxCandidate, bool)
+ ```
+ Neither has a caller outside its tests yet: wiring them into create is L8. Removed from `activator`: `Installation`, `WarmCopyTrusted`.
+
+- [ ] **Step 1: Write the failing tests**
+
+Replace `internal/console/boxchoice_test.go` with:
+
+```go
+package console
+
+import (
+ "crypto/sha256"
+ "encoding/hex"
+ "encoding/json"
+ "testing"
+
+ "github.com/carlosframework/platform/internal/boxid"
+)
+
+// haBox is an eligible box's edges file in placement "uk".
+func haBox(id string, mut func(m map[string]any)) string {
+ m := map[string]any{
+ "v": 1, "box": id, "box_id": id, "eip": "1.2.3.4", "region": "eu-west-2", "placement": "uk",
+ "hosts": map[string]any{}, "hibernation_configured": true, "serves_shared_pool": true,
+ "accepts_inbound": true, "secrets_key_fp": testKeyFP, "capabilities": []string{"region-ha-1"},
+ "headroom": map[string]any{"mem_free_mib": 1000, "disk_free_mib": 1000, "awake": 1},
+ }
+ if mut != nil {
+ mut(m)
+ }
+ b, _ := json.Marshal(m)
+ return string(b)
+}
+
+func ids(cs []boxCandidate) []string {
+ out := make([]string, len(cs))
+ for i, c := range cs {
+ out[i] = c.BoxID
+ }
+ return out
+}
+
+// rosterFor enrolls each box in placement "uk" as an instance in eu-west-2.
+func rosterFor(boxes ...string) map[string]rosterBox {
+ out := map[string]rosterBox{}
+ for _, b := range boxes {
+ out[b] = rosterBox{Placement: "uk", Cloud: boxid.Cloud{Provider: "aws", Account: "1", Region: "eu-west-2", InstanceID: "i-" + b}}
+ }
+ return out
+}
+
+func TestEligibleBoxesAppliesThePredicateOneBoxAtATime(t *testing.T) {
+ del := func(k string) func(map[string]any) { return func(m map[string]any) { delete(m, k) } }
+ set := func(k string, v any) func(map[string]any) { return func(m map[string]any) { m[k] = v } }
+ cases := []struct {
+ name string
+ mut func(map[string]any)
+ want bool
+ }{
+ {"all present", nil, true},
+ {"no dial address", set("eip", ""), false},
+ {"hibernation unconfigured", set("hibernation_configured", false), false},
+ {"hibernation unknown", del("hibernation_configured"), false},
+ {"refuses the shared pool", set("serves_shared_pool", false), false},
+ {"refuses inbound", set("accepts_inbound", false), false},
+ {"wrong secrets key", set("secrets_key_fp", "sha256:other"), false},
+ {"no capability", del("capabilities"), false},
+ {"draining", set("draining", true), false},
+ {"another placement", set("placement", "us"), false},
+ {"box_id names no roster entry", set("box_id", "i-0zzz"), false},
+ }
+ for _, tc := range cases {
+ t.Run(tc.name, func(t *testing.T) {
+ snap := views(t, haBox("i-0aaa", tc.mut))
+ got := eligibleBoxes(snap, rosterFor("i-0aaa"), "uk", "eu-west-2", "ie", testKeyFP, nil)
+ if (len(got) == 1) != tc.want {
+ t.Fatalf("eligible = %v, want %v", ids(got), tc.want)
+ }
+ })
+ }
+}
+
+func TestEligibleBoxesNeedsARosterEntryFreshnessAndNotLost(t *testing.T) {
+ snap := views(t, haBox("i-0aaa", nil), haBox("i-0bbb", nil), haBox("i-0ccc", nil))
+ snap = markStale(snap, "i-0bbb")
+ got := eligibleBoxes(snap, rosterFor("i-0aaa", "i-0bbb"), "uk", "eu-west-2", "ie", testKeyFP, nil)
+ if len(got) != 1 || got[0].BoxID != "i-0aaa" {
+ t.Fatalf("got %v, want only i-0aaa (i-0bbb is stale, i-0ccc has no roster entry)", ids(got))
+ }
+ got = eligibleBoxes(snap, rosterFor("i-0aaa"), "uk", "eu-west-2", "ie", testKeyFP, map[string]bool{"i-0aaa": true})
+ if len(got) != 0 {
+ t.Fatalf("a lost box is eligible: %v", ids(got))
+ }
+}
+
+// TestEligibilityNeverCombinesBoxes is nearest-region's rule for boxes: one
+// box holding the key and another accepting inbound do not add up to one.
+func TestEligibleBoxesNeverCombinesTwoHalfBoxes(t *testing.T) {
+ snap := views(t,
+ haBox("i-0aaa", func(m map[string]any) { m["secrets_key_fp"] = "" }),
+ haBox("i-0bbb", func(m map[string]any) { m["accepts_inbound"] = false }))
+ if got := eligibleBoxes(snap, rosterFor("i-0aaa", "i-0bbb"), "uk", "eu-west-2", "ie", testKeyFP, nil); len(got) != 0 {
+ t.Fatalf("got %v, want none", ids(got))
+ }
+}
+
+func TestEligibleBoxesAnIncompleteSnapshotOffersNothing(t *testing.T) {
+ snap := views(t, haBox("i-0aaa", nil))
+ snap.Complete = false
+ if got := eligibleBoxes(snap, rosterFor("i-0aaa"), "uk", "eu-west-2", "ie", testKeyFP, nil); len(got) != 0 {
+ t.Fatalf("got %v from an incomplete snapshot", ids(got))
+ }
+}
+
+func TestEligibleBoxesHomePlacementMatchesTheEmptyCode(t *testing.T) {
+ home := haBox("i-0aaa", func(m map[string]any) { m["placement"] = "" })
+ named := haBox("i-0bbb", func(m map[string]any) { m["placement"] = "ie" })
+ snap := views(t, home, named)
+ roster := rosterFor("i-0aaa", "i-0bbb")
+ for _, b := range []string{"i-0aaa", "i-0bbb"} {
+ rb := roster[b]
+ rb.Placement = "ie"
+ roster[b] = rb
+ }
+ got := eligibleBoxes(snap, roster, "", "eu-west-2", "ie", testKeyFP, nil)
+ if len(got) != 2 {
+ t.Fatalf("got %v, want both home boxes", ids(got))
+ }
+}
+
+func TestChooseBox(t *testing.T) {
+ a, b := boxCandidate{BoxID: "i-0aaa"}, boxCandidate{BoxID: "i-0bbb"}
+ load := func(hosts int, apps ...string) boxLoad {
+ l := boxLoad{Hosts: hosts, Apps: map[string]int{}}
+ for _, x := range apps {
+ l.Apps[x]++
+ }
+ return l
+ }
+ const host = "shop.bab.oncarlos.com"
+ if _, ok := chooseBox(host, nil, nil, "bab/jam"); ok {
+ t.Fatal("chose from nothing")
+ }
+ got, _ := chooseBox(host, []boxCandidate{a, b}, map[string]boxLoad{"i-0aaa": load(5), "i-0bbb": load(2)}, "bab/jam")
+ if got.BoxID != "i-0bbb" {
+ t.Fatalf("fewest hosts: got %s", got.BoxID)
+ }
+ got, _ = chooseBox(host, []boxCandidate{a, b}, map[string]boxLoad{"i-0aaa": load(5), "i-0bbb": load(2, "bab/jam")}, "bab/jam")
+ if got.BoxID != "i-0aaa" {
+ t.Fatalf("spread beats load: got %s", got.BoxID)
+ }
+ got, ok := chooseBox(host, []boxCandidate{a, b}, map[string]boxLoad{"i-0aaa": load(5, "bab/jam"), "i-0bbb": load(2, "bab/jam", "bab/jam")}, "bab/jam")
+ if !ok || got.BoxID != "i-0bbb" {
+ t.Fatalf("every box runs the app: fewest hosts overall, never a refusal; got %s ok=%v", got.BoxID, ok)
+ }
+}
+
+// H12, claim P5-D2: an equal load breaks by the greatest SHA-256 of
+// host + "\x00" + reference, so the choice differs host by host instead
+// of always landing on the lowest name, and does not depend on the order
+// the candidates arrive in.
+func TestChooseBoxBreaksATieByTheHostsHash(t *testing.T) {
+ cands := []boxCandidate{{BoxID: "annie-welding"}, {BoxID: "bert-sawing"}, {BoxID: "cleo-mixing"}}
+ want := func(host string) string {
+ best, bestSum := "", ""
+ for _, c := range cands {
+ sum := sha256.Sum256([]byte(host + "\x00" + c.BoxID))
+ if h := hex.EncodeToString(sum[:]); h > bestSum {
+ best, bestSum = c.BoxID, h
+ }
+ }
+ return best
+ }
+ seen := map[string]bool{}
+ for _, host := range []string{"a.bab.oncarlos.com", "b.bab.oncarlos.com", "c.bab.oncarlos.com", "d.bab.oncarlos.com", "e.bab.oncarlos.com", "f.bab.oncarlos.com"} {
+ forward, _ := chooseBox(host, cands, nil, "bab/jam")
+ backward, _ := chooseBox(host, []boxCandidate{cands[2], cands[1], cands[0]}, nil, "bab/jam")
+ if forward.BoxID != want(host) || backward.BoxID != want(host) {
+ t.Fatalf("%s: chose %s and %s, want %s", host, forward.BoxID, backward.BoxID, want(host))
+ }
+ seen[forward.BoxID] = true
+ }
+ if len(seen) < 2 {
+ t.Fatalf("six hosts all chose %v: the tie-break is not per host", seen)
+ }
+}
+
+// A box enrolled in another AWS region than the placement's is not in that
+// placement, even when its own file says it is.
+func TestEligibleBoxesRefusesABoxEnrolledInAnotherRegion(t *testing.T) {
+ snap := views(t, haBox("i-0aaa", nil), haBox("i-0bbb", nil))
+ roster := rosterFor("i-0aaa", "i-0bbb")
+ roster["i-0bbb"] = rosterBox{Placement: "uk", Cloud: boxid.Cloud{Provider: "aws", Account: "1", Region: "us-east-1", InstanceID: "i-0bbb"}}
+ got := eligibleBoxes(snap, roster, "uk", "eu-west-2", "ie", testKeyFP, nil)
+ if len(got) != 1 || got[0].BoxID != "i-0aaa" {
+ t.Fatalf("got %v, want only i-0aaa", ids(got))
+ }
+ // And a file that advertises another AWS region than the one expected.
+ snap = views(t, haBox("i-0aaa", func(m map[string]any) { m["region"] = "us-east-1" }))
+ if got := eligibleBoxes(snap, rosterFor("i-0aaa"), "uk", "eu-west-2", "ie", testKeyFP, nil); len(got) != 0 {
+ t.Fatalf("got %v, want none", ids(got))
+ }
+ if got := eligibleBoxes(views(t, haBox("i-0aaa", nil)), rosterFor("i-0aaa"), "uk", "", "ie", testKeyFP, nil); len(got) != 0 {
+ t.Fatalf("got %v with no expected region", ids(got))
+ }
+}
+
+// The roster's enrolled placement is the enrollment's word and must agree
+// with the file's; an incomplete cloud is no identity at all.
+func TestEligibleBoxesNeedsTheRosterToAgree(t *testing.T) {
+ snap := views(t, haBox("i-0aaa", nil), haBox("i-0bbb", nil))
+ roster := rosterFor("i-0aaa", "i-0bbb")
+ roster["i-0aaa"] = rosterBox{Placement: "us", Cloud: roster["i-0aaa"].Cloud}
+ roster["i-0bbb"] = rosterBox{Placement: "uk", Cloud: boxid.Cloud{Provider: "aws", Region: "eu-west-2", InstanceID: "i-0bbb"}}
+ if got := eligibleBoxes(snap, roster, "uk", "eu-west-2", "ie", testKeyFP, nil); len(got) != 0 {
+ t.Fatalf("got %v, want none", ids(got))
+ }
+}
+```
+
+- [ ] **Step 2: Run them to see them fail**
+
+Run: `go test ./internal/console/ -run 'EligibleBoxes|ChooseBox' -count=1`
+Expected: FAIL to build, `undefined: rosterBox` and `too many arguments in call to chooseBox`.
+
+- [ ] **Step 3: Write the choice**
+
+Replace `internal/console/boxchoice.go` with:
+
+```go
+package console
+
+import (
+ "crypto/sha256"
+ "encoding/hex"
+ "slices"
+ "sort"
+
+ "github.com/carlosframework/platform/internal/boxid"
+)
+
+// boxCandidate is one box that may be named as a host's executor (region HA
+// spec 1 §2, "eligible box").
+type boxCandidate struct {
+ BoxID string
+ Region string
+ MemFreeMiB, DiskFreeMiB int64
+ Awake int
+}
+
+// rosterBox is what the roster (control/host-actors.json, claim D7) says
+// about one active box: the placement it is enrolled in and the cloud
+// identity enrollment proved from its boxid record (spec §5.1, H11). The
+// caller passes active entries only.
+type rosterBox struct {
+ Placement string
+ Cloud boxid.Cloud
+}
+
+// eligibleBoxes is regionCandidates' predicate applied to one box at a time,
+// plus what a multi-box region adds: a roster entry for the file's box_id,
+// enrolled in this placement, whose proven cloud is in awsRegion (the AWS
+// region the roster records for this placement), the HA capability, not
+// draining, not lost. Two boxes that each satisfy half of it never add up
+// to one. The edges file is evidence of liveness and configuration only:
+// identity and region come from the roster (spec §5.1).
+//
+// An incomplete snapshot offers nothing: an unread file could be the one
+// box that would change the answer.
+func eligibleBoxes(snap edgesSnapshot, roster map[string]rosterBox, placement, awsRegion, homeName, expectedFP string, lost map[string]bool) []boxCandidate {
+ if !snap.complete() || expectedFP == "" || awsRegion == "" {
+ return nil
+ }
+ home := func(code string) string {
+ if homeName != "" && code == homeName {
+ return ""
+ }
+ return code
+ }
+ var out []boxCandidate
+ for _, f := range snap.Files {
+ want := home(placement)
+ rb, ok := roster[f.BoxID]
+ switch {
+ case home(f.Placement) != want:
+ case !f.Fresh, f.EIP == "", f.BoxID == "":
+ case !ok || !rb.Cloud.Complete():
+ case home(rb.Placement) != want:
+ // The file's placement is the box's own word; the roster's is
+ // the enrollment's. Both must agree.
+ case rb.Cloud.Region != awsRegion, f.Region != awsRegion:
+ // The AWS region was proven at enrollment: a box enrolled in
+ // another region is not in this placement, whatever its file
+ // says.
+ case !isTrue(f.HibernationConfigured), !isTrue(f.ServesSharedPool), !isTrue(f.AcceptsInbound):
+ case f.SecretsKeyFP != expectedFP:
+ case !slices.Contains(f.Capabilities, "region-ha-1"):
+ case f.Draining, lost[f.BoxID]:
+ default:
+ c := boxCandidate{BoxID: f.BoxID, Region: rb.Cloud.Region}
+ if f.Headroom != nil {
+ c.MemFreeMiB, c.DiskFreeMiB, c.Awake = f.Headroom.MemFreeMiB, f.Headroom.DiskFreeMiB, f.Headroom.Awake
+ }
+ out = append(out, c)
+ }
+ }
+ sort.Slice(out, func(i, j int) bool { return out[i].BoxID < out[j].BoxID })
+ return out
+}
+
+// boxLoad is what claims already place on a box.
+type boxLoad struct {
+ Hosts int
+ Apps map[string]int // "acct/app" -> hosts of that app
+}
+
+// chooseBox is spec 1 §5.2's choose() until spec 2 (H12): prefer a box
+// that runs no instance of this app (soft spread: it never refuses), then
+// the fewest hosts overall, then the greatest hexadecimal SHA-256 of
+// host + "\x00" + box reference (claim P5-D2), then the smallest reference
+// if two hashes are equal.
+func chooseBox(host string, cands []boxCandidate, running map[string]boxLoad, app string) (boxCandidate, bool) {
+ if len(cands) == 0 {
+ return boxCandidate{}, false
+ }
+ tie := func(ref string) string {
+ sum := sha256.Sum256([]byte(host + "\x00" + ref))
+ return hex.EncodeToString(sum[:])
+ }
+ best := cands[0]
+ for _, c := range cands[1:] {
+ cl, bl := running[c.BoxID], running[best.BoxID]
+ cRuns, bRuns := cl.Apps[app] > 0, bl.Apps[app] > 0
+ switch {
+ case cRuns != bRuns:
+ if !cRuns {
+ best = c
+ }
+ case cl.Hosts != bl.Hosts:
+ if cl.Hosts < bl.Hosts {
+ best = c
+ }
+ case tie(c.BoxID) != tie(best.BoxID):
+ if tie(c.BoxID) > tie(best.BoxID) {
+ best = c
+ }
+ case c.BoxID < best.BoxID:
+ best = c
+ }
+ }
+ return best, true
+}
+```
+
+- [ ] **Step 4: Remove the warm-copy check**
+
+In `internal/activator/installed.go`, replace the `InstallMarker` doc comment:
+
+```go
+// InstallMarker is a box's own record that it finished installing one
+// generation of a host: the database restored and synced, the files
+// applied, the wake state imported (region HA spec 1 §5.5). It is written
+// last, so its presence vouches for all of them.
+//
+// It is evidence, not authority. Authority is the claim naming this box,
+// which the box holds as its active stamp. A box that crashed mid-install
+// has authority and no marker, and must restore.
+```
+
+with:
+
+```go
+// InstallMarker is a box's own note that it finished a local install of
+// one generation of a host. Nothing reads it to trust a local database:
+// claim ruling R6 has every admitted writer purge and restore from the
+// certified remote (region HA spec 1 H13, §5.5 withdrawn). It is kept for
+// relocation's local install journal (§5.8), which plan 1b does not build.
+```
+
+Delete the `Installation` type (the block starting `// Installation is what the claim says is current for a host.`) and the whole of `WarmCopyTrusted` with its doc comment, from `// WarmCopyTrusted is invariant HA3` to the end of the file. In `internal/activator/installed_test.go`, delete `TestWarmCopyTrusted`, from `func TestWarmCopyTrusted(t *testing.T) {` to the end of the file.
+
+In `internal/activator/activator.go`, in `doWake`, replace:
+
+```go
+ // S3 state written by the true holder). When it's still on disk it is the
+ // warm cache: on one box, with the lease held, the local copy can only be
+ // what we ourselves checkpointed at the last sleep.
+```
+
+with:
+
+```go
+ // S3 state written by the true holder). When it's still on disk it is the
+ // warm cache: on one box, with the lease held, the local copy can only be
+ // what we ourselves checkpointed at the last sleep. That holds only while
+ // a region has one box. Claim ruling R6 ends it (region HA spec 1 H13):
+ // once wakes are admitted by hostclaim's AdmitWriter, every admitted
+ // writer purges its local copy and restores from the certified remote,
+ // and no marker is ever read to skip that.
+```
+
+The behaviour of `doWake` does not change: the warm cache stays correct while every region has one box, and replacing it is the claim's `AdmitWriter` wiring, not this plan's.
+
+- [ ] **Step 5: Run the tests**
+
+Run:
+```bash
+go test ./internal/console/ -run 'EligibleBoxes|ChooseBox|CandidateLocation' -count=1 -v
+go test ./internal/activator/ -run 'InstallMarker|DamagedMarker' -count=1
+```
+Expected: PASS, including `TestChooseBoxBreaksATieByTheHostsHash` and `TestEligibleBoxesNeedsTheRosterToAgree`.
+
+- [ ] **Step 6: Gate and commit**
+
+```bash
+go build ./... && go vet ./internal/console/ ./internal/activator/ && gofmt -l internal/console internal/activator
+GOOS=darwin go build ./... && GOOS=windows go build ./...
+git add internal/console/boxchoice.go internal/console/boxchoice_test.go internal/activator/installed.go internal/activator/installed_test.go internal/activator/activator.go
+git commit -m "console: eligible boxes come from the roster's proven cloud, and a tie breaks by the host's hash; activator: no local copy is trusted
+
+Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
+```
+
+---
+
+### Task 3: The agent publishes its identity record (`internal/host/boxrecord.go`)
+
+Spec §19: "Item 6 becomes publishing the enrollment evidence at start." Plan 1a's "After the build" item 13: the record is written under the box's label. With `CARLOS_REGION_HA` on, the agent reads IMDS's identity document and signature once and publishes them create-only. The record grants nothing; enrollment reads it later.
+
+**Files:**
+- Create: `internal/host/boxrecord.go`
+- Test: `internal/host/boxrecord_test.go`
+- Modify: `internal/host/edgesfile.go` (`edgesFileCapabilities`), `internal/host/agent.go:524`
+
+**Interfaces:**
+- Consumes: `boxid.Record`, `boxid.ValidBox`, `boxid.Load`, `boxid.Publish` (Task 1); `openDeployStore`, `metricsBoxLabel`, `readMachineID`, `shouldSkipPublisherElection` (existing).
+- Produces:
+ ```go
+ func regionHAEnabled(getenv func(string) string) bool // used by Tasks 6 and 7
+ func readIdentityRecord(client *http.Client, base, box string, now time.Time) (boxid.Record, error)
+ func publishBoxRecord(ctx context.Context, s release.Store, box string, read func() (boxid.Record, error)) (done bool, err error)
+ func boxRecordLoop()
+ ```
+
+**Two things IMDS does that matter.** The identity document is signed byte for byte, so it is stored exactly as served: `imdsLeaf` trims, and is not used. The signature may arrive wrapped across lines, which Go's base64 decoder refuses, so whitespace is removed from it.
+
+- [ ] **Step 1: Write the failing tests**
+
+Create `internal/host/boxrecord_test.go`:
+
+```go
+package host
+
+import (
+ "context"
+ "encoding/base64"
+ "errors"
+ "io"
+ "net/http"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/carlosframework/platform/internal/boxid"
+ "github.com/carlosframework/platform/internal/release"
+)
+
+// fakeIMDS answers IMDSv2 in-process: no socket, so it runs in any sandbox.
+type fakeIMDS struct {
+ doc, sig string
+ status int // for the document and signature; 0 = 200
+ calls []string
+}
+
+func (f *fakeIMDS) RoundTrip(r *http.Request) (*http.Response, error) {
+ f.calls = append(f.calls, r.Method+" "+r.URL.Path)
+ answer := func(code int, body string) (*http.Response, error) {
+ return &http.Response{StatusCode: code, Body: io.NopCloser(strings.NewReader(body)), Header: http.Header{}, Request: r}, nil
+ }
+ if r.Method == http.MethodPut && r.URL.Path == "/latest/api/token" {
+ return answer(200, "tok\n")
+ }
+ if r.Header.Get("X-aws-ec2-metadata-token") != "tok" {
+ return answer(401, "")
+ }
+ code := f.status
+ if code == 0 {
+ code = 200
+ }
+ switch r.URL.Path {
+ case imdsIdentityDocument:
+ return answer(code, f.doc)
+ case imdsIdentitySignature:
+ return answer(code, f.sig)
+ }
+ return answer(404, "")
+}
+
+func imdsClient(f *fakeIMDS) *http.Client { return &http.Client{Transport: f} }
+
+var recordNow = time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
+
+// The document is signed byte for byte, so its trailing newline must
+// survive; the signature arrives wrapped and must not.
+func TestReadIdentityRecordKeepsTheDocumentsExactBytes(t *testing.T) {
+ doc := "{\n \"accountId\" : \"111122223333\",\n \"instanceId\" : \"i-0abc\"\n}\n"
+ sig := base64.StdEncoding.EncodeToString([]byte(strings.Repeat("s", 256)))
+ wrapped := sig[:64] + "\n" + sig[64:] + "\n"
+ f := &fakeIMDS{doc: doc, sig: wrapped}
+ rec, err := readIdentityRecord(imdsClient(f), "http://imds.test", "annie-welding", recordNow)
+ if err != nil {
+ t.Fatal(err)
+ }
+ got, _ := base64.StdEncoding.DecodeString(rec.IdentityDocument)
+ if string(got) != doc {
+ t.Fatalf("document = %q, want the bytes IMDS served, %q", got, doc)
+ }
+ if rec.IdentitySignature != sig || rec.V != 1 || rec.Box != "annie-welding" || rec.WrittenAt != "2026-10-06T09:00:00Z" {
+ t.Fatalf("record = %+v", rec)
+ }
+}
+
+func TestReadIdentityRecordRefusesWhatIsNotAnAnswer(t *testing.T) {
+ for name, f := range map[string]*fakeIMDS{
+ "document missing": {doc: "{}", sig: "c2lnCg==", status: 404},
+ "signature not base64": {doc: "{}", sig: "not base64!"},
+ "signature empty": {doc: "{}", sig: "\n"},
+ } {
+ t.Run(name, func(t *testing.T) {
+ if rec, err := readIdentityRecord(imdsClient(f), "http://imds.test", "annie-welding", recordNow); err == nil {
+ t.Fatalf("record = %+v, want an error", rec)
+ }
+ })
+ }
+}
+
+func TestPublishBoxRecordWritesOnceAndNeverRewrites(t *testing.T) {
+ ctx, st := context.Background(), release.NewDirStore(t.TempDir())
+ reads := 0
+ read := func() (boxid.Record, error) {
+ reads++
+ return boxid.Record{V: 1, Box: "annie-welding", IdentityDocument: "ZG9j", IdentitySignature: "c2ln", WrittenAt: "2026-10-06T09:00:00Z"}, nil
+ }
+ if done, err := publishBoxRecord(ctx, st, "annie-welding", read); !done || err != nil {
+ t.Fatalf("first publish: done=%v err=%v", done, err)
+ }
+ if done, err := publishBoxRecord(ctx, st, "annie-welding", read); !done || err != nil {
+ t.Fatalf("second publish: done=%v err=%v", done, err)
+ }
+ if reads != 1 {
+ t.Fatalf("IMDS read %d times; an existing record must not be re-read or rewritten", reads)
+ }
+}
+
+func TestPublishBoxRecordRetriesAReadFailureAndStopsOnABadLabel(t *testing.T) {
+ ctx, st := context.Background(), release.NewDirStore(t.TempDir())
+ failing := func() (boxid.Record, error) { return boxid.Record{}, errors.New("IMDS down") }
+ if done, err := publishBoxRecord(ctx, st, "annie-welding", failing); done || err == nil {
+ t.Fatalf("done=%v err=%v: a read failure is retried", done, err)
+ }
+ if done, err := publishBoxRecord(ctx, st, "Annie.Welding", failing); !done || err == nil {
+ t.Fatalf("done=%v err=%v: a reference that is not a label is not retried", done, err)
+ }
+}
+
+// A store that refuses the read with something other than not-found must
+// not be taken as "no record yet": that would publish over an outage.
+type unreadableBoxStore struct{ release.Store }
+
+func (unreadableBoxStore) Get(context.Context, string) ([]byte, string, error) {
+ return nil, "", errors.New("AccessDenied")
+}
+
+func TestPublishBoxRecordDoesNotPublishWhenItCannotRead(t *testing.T) {
+ ctx := context.Background()
+ st := unreadableBoxStore{release.NewDirStore(t.TempDir())}
+ read := func() (boxid.Record, error) {
+ t.Fatal("IMDS read although the store could not say whether a record exists")
+ return boxid.Record{}, nil
+ }
+ if done, err := publishBoxRecord(ctx, st, "annie-welding", read); done || err == nil {
+ t.Fatalf("done=%v err=%v", done, err)
+ }
+}
+
+func TestRegionHAEnabled(t *testing.T) {
+ for v, want := range map[string]bool{"": false, "0": false, " 0 ": false, "1": true, "yes": true} {
+ if got := regionHAEnabled(func(string) string { return v }); got != want {
+ t.Errorf("CARLOS_REGION_HA=%q: %v, want %v", v, got, want)
+ }
+ }
+}
+```
+
+- [ ] **Step 2: Run them to see them fail**
+
+Run: `go test ./internal/host/ -run 'IdentityRecord|BoxRecord|RegionHAEnabled' -count=1`
+Expected: FAIL to build, `undefined: readIdentityRecord`.
+
+- [ ] **Step 3: Write the publisher**
+
+Create `internal/host/boxrecord.go`:
+
+```go
+package host
+
+import (
+ "context"
+ "encoding/base64"
+ "errors"
+ "fmt"
+ "io"
+ "log"
+ "net/http"
+ "os"
+ "strings"
+ "time"
+
+ "github.com/carlosframework/platform/internal/boxid"
+ "github.com/carlosframework/platform/internal/release"
+)
+
+// regionHAEnabled is CARLOS_REGION_HA: on for any value but "" and "0".
+// Every region HA behaviour on a box is behind it (spec 1 §8 step 2).
+func regionHAEnabled(getenv func(string) string) bool {
+ switch strings.TrimSpace(getenv("CARLOS_REGION_HA")) {
+ case "", "0":
+ return false
+ }
+ return true
+}
+
+const (
+ imdsBase = "http://169.254.169.254"
+ imdsIdentityDocument = "/latest/dynamic/instance-identity/document"
+ imdsIdentitySignature = "/latest/dynamic/instance-identity/signature"
+ boxRecordRetry = time.Minute
+)
+
+// imdsGetRaw reads one IMDSv2 path and returns its body untouched. The
+// identity document is signed byte for byte, so unlike imdsLeaf nothing is
+// trimmed.
+func imdsGetRaw(client *http.Client, base, token, path string) ([]byte, error) {
+ req, err := http.NewRequest(http.MethodGet, base+path, nil)
+ if err != nil {
+ return nil, err
+ }
+ req.Header.Set("X-aws-ec2-metadata-token", token)
+ resp, err := client.Do(req)
+ if err != nil {
+ return nil, err
+ }
+ defer resp.Body.Close()
+ body, err := io.ReadAll(io.LimitReader(resp.Body, 64<<10))
+ if err != nil {
+ return nil, err
+ }
+ if resp.StatusCode != http.StatusOK {
+ return nil, fmt.Errorf("IMDS %s answered %d", path, resp.StatusCode)
+ }
+ return body, nil
+}
+
+// readIdentityRecord builds this box's record (spec §5.1) from IMDS: the
+// identity document exactly as served, and its base64 RSA-SHA256
+// signature with any line breaks removed (base64 ignores them; Go's
+// decoder does not).
+func readIdentityRecord(client *http.Client, base, box string, now time.Time) (boxid.Record, error) {
+ req, err := http.NewRequest(http.MethodPut, base+"/latest/api/token", nil)
+ if err != nil {
+ return boxid.Record{}, err
+ }
+ req.Header.Set("X-aws-ec2-metadata-token-ttl-seconds", "60")
+ resp, err := client.Do(req)
+ if err != nil {
+ return boxid.Record{}, fmt.Errorf("IMDS token: %w", err)
+ }
+ tok, _ := io.ReadAll(io.LimitReader(resp.Body, 4<<10))
+ resp.Body.Close()
+ if resp.StatusCode != http.StatusOK || len(strings.TrimSpace(string(tok))) == 0 {
+ return boxid.Record{}, fmt.Errorf("IMDS token answered %d", resp.StatusCode)
+ }
+ token := strings.TrimSpace(string(tok))
+ doc, err := imdsGetRaw(client, base, token, imdsIdentityDocument)
+ if err != nil {
+ return boxid.Record{}, err
+ }
+ sig, err := imdsGetRaw(client, base, token, imdsIdentitySignature)
+ if err != nil {
+ return boxid.Record{}, err
+ }
+ cleaned := strings.Join(strings.Fields(string(sig)), "")
+ if _, err := base64.StdEncoding.DecodeString(cleaned); err != nil || cleaned == "" {
+ return boxid.Record{}, errors.New("IMDS signature is not base64")
+ }
+ return boxid.Record{V: 1, Box: box, IdentityDocument: base64.StdEncoding.EncodeToString(doc),
+ IdentitySignature: cleaned, WrittenAt: now.UTC().Format(time.RFC3339)}, nil
+}
+
+// publishBoxRecord makes one attempt to publish this box's enrollment
+// evidence. done is true once there is nothing left to retry: the record
+// is in place, or a different one already holds this reference, which
+// only an operator can sort out and which is logged loudly.
+func publishBoxRecord(ctx context.Context, s release.Store, box string, read func() (boxid.Record, error)) (done bool, err error) {
+ if !boxid.ValidBox(box) {
+ return true, fmt.Errorf("box reference %q is not a label; no enrollment evidence is published", box)
+ }
+ if _, err := boxid.Load(ctx, s, box); err == nil {
+ return true, nil // create-only: an existing record is never rewritten
+ } else if !errors.Is(err, os.ErrNotExist) {
+ return false, err
+ }
+ rec, err := read()
+ if err != nil {
+ return false, err
+ }
+ err = boxid.Publish(ctx, s, rec)
+ if errors.Is(err, release.ErrCASConflict) {
+ return true, fmt.Errorf("another record already holds %s: %w", box, err)
+ }
+ return err == nil, err
+}
+
+// boxRecordLoop publishes this box's enrollment evidence once, at start,
+// when CARLOS_REGION_HA is on, retrying each minute until it is in place.
+// The record grants nothing: enrollment (an operator, or for Auto Scaling
+// boxes the console with a membership confirmation) reads it later.
+// Platform-box job: skipped on fleet boxes.
+func boxRecordLoop() {
+ if !regionHAEnabled(os.Getenv) || shouldSkipPublisherElection(os.Getenv) {
+ return
+ }
+ ctx := context.Background()
+ store, err := openDeployStore(ctx)
+ if err != nil {
+ log.Printf("agent: box record not published: %v", err)
+ return
+ }
+ box := metricsBoxLabel(os.Getenv, readMachineID())
+ client := &http.Client{Timeout: 2 * time.Second}
+ read := func() (boxid.Record, error) { return readIdentityRecord(client, imdsBase, box, time.Now()) }
+ for attempt := 1; ; attempt++ {
+ done, err := publishBoxRecord(ctx, store, box, read)
+ if done {
+ if err != nil {
+ log.Printf("agent: box record for %s: %v (region HA cannot enroll this box until an operator resolves it)", box, err)
+ }
+ return
+ }
+ if attempt == 1 || attempt%60 == 0 {
+ log.Printf("agent: box record for %s not published yet (#%d): %v", box, attempt, err)
+ }
+ time.Sleep(boxRecordRetry)
+ }
+}
+```
+
+- [ ] **Step 4: Read the flag in one place**
+
+In `internal/host/edgesfile.go`, replace the body of `edgesFileCapabilities`:
+
+```go
+func edgesFileCapabilities(getenv func(string) string) []string {
+ switch strings.TrimSpace(getenv("CARLOS_REGION_HA")) {
+ case "", "0":
+ return nil
+ }
+ return []string{capRegionHA1}
+}
+```
+
+with:
+
+```go
+func edgesFileCapabilities(getenv func(string) string) []string {
+ if !regionHAEnabled(getenv) {
+ return nil
+ }
+ return []string{capRegionHA1}
+}
+```
+
+- [ ] **Step 5: Start the loop**
+
+In `internal/host/agent.go`, in `RunAgent`, replace:
+
+```go
+ go edgesFileLoop(reg)
+```
+
+with:
+
+```go
+ go edgesFileLoop(reg)
+ // Region HA enrollment evidence (spec 1 §5.1): once, at start, and only
+ // with CARLOS_REGION_HA on. See boxrecord.go.
+ go boxRecordLoop()
+```
+
+- [ ] **Step 6: Run the tests**
+
+Run: `go test ./internal/host/ -run 'IdentityRecord|BoxRecord|RegionHAEnabled|EdgesFileCapability|Headroom' -count=1 -v`
+Expected: PASS.
+
+- [ ] **Step 7: Gate and commit**
+
+```bash
+go build ./... && go vet ./internal/host/ && gofmt -l internal/host
+GOOS=darwin go build ./... && GOOS=windows go build ./...
+git add internal/host/boxrecord.go internal/host/boxrecord_test.go internal/host/edgesfile.go internal/host/agent.go
+git commit -m "agent: with region HA on, a box publishes its AWS-signed identity once, under its label
+
+Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
+```
+
+---
+
+### Task 4: The fence binds to the roster entry (`internal/termfence`)
+
+Spec §19: "`Verify` binds to a `boxid.Identity`; it must bind to the roster entry's `cloud` and produce §5.9's record. It takes no volume input (H19) … `FromDescribe` and `FromEvent` stay as the two terminated reads; a `shutting-down` observation is never accepted." §6 adds that the fence refuses any observation not made in the box's own region. This task also gives the describe a seam: one call, pinned to the box's region, with `InvalidInstanceID.NotFound` read as absence.
+
+Writing the record to `control/box-fences/<enrollment>.json`, and citing it in a claim CAS, are L2 to L4 and are not built.
+
+**Files:**
+- Modify (replace whole file): `internal/termfence/fence.go`
+- Test (replace whole file): `internal/termfence/fence_test.go`
+- Modify: `internal/boxid/verify.go` (remove `Identity`)
+
+**Interfaces:**
+- Consumes: `boxid.Cloud` (Task 1).
+- Produces:
+ ```go
+ package termfence
+ const ViaDescribe, ViaEvent = "describe", "event"
+ var ErrNotEvidence error
+ type Observation struct{ Via string; Found bool; InstanceID, Account, Region, State, EventID, RequestID string }
+ type Box struct{ Ref, Enrollment string; Cloud boxid.Cloud }
+ type WrittenBy struct{ Box, Enrollment string } // json: box, enrollment
+ type Record struct{ V int; Box, Enrollment string; Cloud boxid.Cloud; State, ObservedAt, Via, RequestID, EventID string; WrittenBy WrittenBy } // §5.9
+ type EC2 interface{ DescribeInstances(ctx, *ec2.DescribeInstancesInput, ...func(*ec2.Options)) (*ec2.DescribeInstancesOutput, error) }
+ func FromDescribe(out *ec2.DescribeInstancesOutput, instanceID, region string) Observation
+ func FromEvent(body []byte) (Observation, error)
+ func DescribeInstance(ctx context.Context, api EC2, c boxid.Cloud) (*ec2.DescribeInstancesOutput, error) // used by Task 5's callers too
+ func Verify(box, worker Box, o Observation, now time.Time) (Record, error)
+ ```
+ Removed: `Fence`, and `Verify`'s `(string, boxid.Identity, …)` form.
+
+- [ ] **Step 1: Write the failing tests**
+
+Replace `internal/termfence/fence_test.go` with:
+
+```go
+package termfence
+
+import (
+ "context"
+ "errors"
+ "testing"
+ "time"
+
+ "github.com/aws/aws-sdk-go-v2/aws"
+ awsmiddleware "github.com/aws/aws-sdk-go-v2/aws/middleware"
+ "github.com/aws/aws-sdk-go-v2/service/ec2"
+ "github.com/aws/aws-sdk-go-v2/service/ec2/types"
+ "github.com/aws/smithy-go"
+ smithymiddleware "github.com/aws/smithy-go/middleware"
+
+ "github.com/carlosframework/platform/internal/boxid"
+)
+
+var (
+ box = Box{Ref: "annie-welding", Enrollment: "enr-annie-1",
+ Cloud: boxid.Cloud{Provider: "aws", Account: "111122223333", Region: "eu-west-1", InstanceID: "i-0abc"}}
+ worker = Box{Ref: "bert-sawing", Enrollment: "enr-bert-1",
+ Cloud: boxid.Cloud{Provider: "aws", Account: "111122223333", Region: "eu-west-1", InstanceID: "i-0bbb"}}
+ now = time.Date(2026, 10, 6, 12, 0, 0, 0, time.UTC)
+)
+
+func describe(owner, instance string, state types.InstanceStateName) *ec2.DescribeInstancesOutput {
+ return &ec2.DescribeInstancesOutput{Reservations: []types.Reservation{{
+ OwnerId: aws.String(owner),
+ Instances: []types.Instance{{InstanceId: aws.String(instance), State: &types.InstanceState{Name: state}}},
+ }}}
+}
+
+func withRequestID(out *ec2.DescribeInstancesOutput, id string) *ec2.DescribeInstancesOutput {
+ var md smithymiddleware.Metadata
+ awsmiddleware.SetRequestIDMetadata(&md, id)
+ out.ResultMetadata = md
+ return out
+}
+
+func TestATerminatedInstanceOfThisBoxIsAFence(t *testing.T) {
+ out := withRequestID(describe("111122223333", "i-0abc", types.InstanceStateNameTerminated), "req-1")
+ rec, err := Verify(box, worker, FromDescribe(out, "i-0abc", "eu-west-1"), now)
+ if err != nil {
+ t.Fatal(err)
+ }
+ want := Record{V: 1, Box: "annie-welding", Enrollment: "enr-annie-1", Cloud: box.Cloud, State: "terminated",
+ ObservedAt: "2026-10-06T12:00:00Z", Via: "describe", RequestID: "req-1",
+ WrittenBy: WrittenBy{Box: "bert-sawing", Enrollment: "enr-bert-1"}}
+ if rec != want {
+ t.Fatalf("record = %+v\nwant %+v", rec, want)
+ }
+}
+
+// Every lookalike is refused.
+func TestLookalikesAreNotEvidence(t *testing.T) {
+ cases := map[string]Observation{
+ "stopped": FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameStopped), "i-0abc", "eu-west-1"),
+ "stopping": FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameStopping), "i-0abc", "eu-west-1"),
+ "shutting-down": FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameShuttingDown), "i-0abc", "eu-west-1"),
+ "running": FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameRunning), "i-0abc", "eu-west-1"),
+ "another account": FromDescribe(describe("999999999999", "i-0abc", types.InstanceStateNameTerminated), "i-0abc", "eu-west-1"),
+ "another region": FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameTerminated), "i-0abc", "us-east-1"),
+ "another instance": FromDescribe(describe("111122223333", "i-0zzz", types.InstanceStateNameTerminated), "i-0zzz", "eu-west-1"),
+ "not in the reply": FromDescribe(describe("111122223333", "i-0zzz", types.InstanceStateNameTerminated), "i-0abc", "eu-west-1"),
+ "empty reply": FromDescribe(&ec2.DescribeInstancesOutput{}, "i-0abc", "eu-west-1"),
+ "nil reply": FromDescribe(nil, "i-0abc", "eu-west-1"),
+ "no state": FromDescribe(&ec2.DescribeInstancesOutput{Reservations: []types.Reservation{{OwnerId: aws.String("111122223333"), Instances: []types.Instance{{InstanceId: aws.String("i-0abc")}}}}}, "i-0abc", "eu-west-1"),
+ "unknown source": {Via: "cloudtrail", Found: true, InstanceID: "i-0abc", Account: "111122223333", Region: "eu-west-1", State: "terminated"},
+ "zero observation": {},
+ }
+ for name, o := range cases {
+ t.Run(name, func(t *testing.T) {
+ if rec, err := Verify(box, worker, o, now); !errors.Is(err, ErrNotEvidence) {
+ t.Fatalf("record = %+v, err = %v; want ErrNotEvidence", rec, err)
+ }
+ })
+ }
+}
+
+// Review Focus 5: the fence is a regional worker's own read in the box's
+// own region. A worker enrolled elsewhere is refused even when what it
+// read is a perfect terminated observation.
+func TestAWorkerInAnotherRegionNeverFences(t *testing.T) {
+ far := worker
+ far.Cloud.Region = "us-east-1"
+ o := FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameTerminated), "i-0abc", "eu-west-1")
+ if _, err := Verify(box, far, o, now); !errors.Is(err, ErrNotEvidence) {
+ t.Fatalf("err = %v, want ErrNotEvidence", err)
+ }
+}
+
+func TestAnIncompleteRosterEntryIsNeverFenced(t *testing.T) {
+ // A found, terminated observation whose every coordinate is empty, so
+ // it MATCHES an empty cloud field for field. Only the guard on the
+ // entry itself can refuse it.
+ o := Observation{Via: ViaDescribe, Found: true, State: "terminated"}
+ for name, b := range map[string]Box{
+ "empty cloud": {Ref: "annie-welding", Enrollment: "enr-annie-1"},
+ "no enrollment": {Ref: "annie-welding", Cloud: box.Cloud},
+ "no reference": {Enrollment: "enr-annie-1", Cloud: box.Cloud},
+ "not aws": {Ref: "annie-welding", Enrollment: "enr-annie-1", Cloud: boxid.Cloud{Provider: "gcp", Account: "111122223333", Region: "eu-west-1", InstanceID: "i-0abc"}},
+ } {
+ t.Run(name, func(t *testing.T) {
+ full := Observation{Via: ViaDescribe, Found: true, InstanceID: b.Cloud.InstanceID, Account: b.Cloud.Account, Region: b.Cloud.Region, State: "terminated"}
+ for _, obs := range []Observation{o, full} {
+ if _, err := Verify(b, worker, obs, now); !errors.Is(err, ErrNotEvidence) {
+ t.Fatalf("err = %v, want ErrNotEvidence", err)
+ }
+ }
+ })
+ }
+ if _, err := Verify(box, Box{Ref: "bert-sawing"}, FromDescribe(describe("111122223333", "i-0abc", types.InstanceStateNameTerminated), "i-0abc", "eu-west-1"), now); !errors.Is(err, ErrNotEvidence) {
+ t.Fatalf("a worker with no proven identity: err = %v", err)
+ }
+}
+
+const terminatedEvent = `{"version":"0","id":"ev-123","detail-type":"EC2 Instance State-change Notification","source":"aws.ec2","account":"111122223333","time":"2026-10-06T11:59:00Z","region":"eu-west-1","resources":["arn:aws:ec2:eu-west-1:111122223333:instance/i-0abc"],"detail":{"instance-id":"i-0abc","state":"terminated"}}`
+
+func TestATerminatedEventIsAFence(t *testing.T) {
+ o, err := FromEvent([]byte(terminatedEvent))
+ if err != nil {
+ t.Fatal(err)
+ }
+ rec, err := Verify(box, worker, o, now)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if rec.Via != "event" || rec.EventID != "ev-123" || rec.State != "terminated" || rec.RequestID != "" {
+ t.Fatalf("record = %+v", rec)
+ }
+}
+
+func TestEventsThatAreNotEC2StateChangesAreRefused(t *testing.T) {
+ for name, body := range map[string]string{
+ "not JSON": `terminated`,
+ "another source": `{"id":"e","detail-type":"EC2 Instance State-change Notification","source":"my.app","account":"111122223333","region":"eu-west-1","detail":{"instance-id":"i-0abc","state":"terminated"}}`,
+ "another detail-type": `{"id":"e","detail-type":"EC2 Spot Instance Interruption Warning","source":"aws.ec2","account":"111122223333","region":"eu-west-1","detail":{"instance-id":"i-0abc","state":"terminated"}}`,
+ "no event id": `{"detail-type":"EC2 Instance State-change Notification","source":"aws.ec2","account":"111122223333","region":"eu-west-1","detail":{"instance-id":"i-0abc","state":"terminated"}}`,
+ } {
+ t.Run(name, func(t *testing.T) {
+ if _, err := FromEvent([]byte(body)); err == nil {
+ t.Fatal("accepted")
+ }
+ })
+ }
+}
+
+func TestAShuttingDownEventIsNotEvidence(t *testing.T) {
+ body := `{"id":"e","detail-type":"EC2 Instance State-change Notification","source":"aws.ec2","account":"111122223333","region":"eu-west-1","detail":{"instance-id":"i-0abc","state":"shutting-down"}}`
+ o, err := FromEvent([]byte(body))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if _, err := Verify(box, worker, o, now); !errors.Is(err, ErrNotEvidence) {
+ t.Fatalf("err = %v, want ErrNotEvidence", err)
+ }
+}
+
+// fakeEC2 answers DescribeInstances and records the region each call was
+// forced into.
+type fakeEC2 struct {
+ out *ec2.DescribeInstancesOutput
+ err error
+ regions []string
+ ids [][]string
+}
+
+func (f *fakeEC2) DescribeInstances(_ context.Context, in *ec2.DescribeInstancesInput, optFns ...func(*ec2.Options)) (*ec2.DescribeInstancesOutput, error) {
+ o := ec2.Options{Region: "client-default"}
+ for _, fn := range optFns {
+ fn(&o)
+ }
+ f.regions = append(f.regions, o.Region)
+ f.ids = append(f.ids, in.InstanceIds)
+ return f.out, f.err
+}
+
+func TestDescribeInstanceAsksInTheBoxsOwnRegion(t *testing.T) {
+ f := &fakeEC2{out: describe("111122223333", "i-0abc", types.InstanceStateNameTerminated)}
+ out, err := DescribeInstance(context.Background(), f, box.Cloud)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(f.regions) != 1 || f.regions[0] != "eu-west-1" || len(f.ids[0]) != 1 || f.ids[0][0] != "i-0abc" {
+ t.Fatalf("called in %v for %v", f.regions, f.ids)
+ }
+ if _, err := Verify(box, worker, FromDescribe(out, "i-0abc", "eu-west-1"), now); err != nil {
+ t.Fatal(err)
+ }
+}
+
+func TestDescribeInstanceNotFoundIsAbsenceAndOtherErrorsAreErrors(t *testing.T) {
+ notFound := &fakeEC2{err: &smithy.GenericAPIError{Code: "InvalidInstanceID.NotFound", Message: "gone"}}
+ out, err := DescribeInstance(context.Background(), notFound, box.Cloud)
+ if err != nil {
+ t.Fatalf("NotFound: err = %v", err)
+ }
+ if _, err := Verify(box, worker, FromDescribe(out, "i-0abc", "eu-west-1"), now); !errors.Is(err, ErrNotEvidence) {
+ t.Fatalf("NotFound fenced the box: err = %v", err)
+ }
+ denied := &fakeEC2{err: &smithy.GenericAPIError{Code: "UnauthorizedOperation", Message: "no"}}
+ if _, err := DescribeInstance(context.Background(), denied, box.Cloud); err == nil {
+ t.Fatal("an access error read as an answer")
+ }
+ if _, err := DescribeInstance(context.Background(), &fakeEC2{}, boxid.Cloud{InstanceID: "i-0abc"}); err == nil {
+ t.Fatal("described an incomplete cloud")
+ }
+}
+```
+
+- [ ] **Step 2: Run them to see them fail**
+
+Run: `go test ./internal/termfence/ -count=1`
+Expected: FAIL to build, `undefined: Box` and `undefined: DescribeInstance`.
+
+- [ ] **Step 3: Write the fence**
+
+Replace `internal/termfence/fence.go` with:
+
+```go
+// Package termfence decides whether an observation proves that a box's
+// machine has been terminated (region HA spec 1 §5.9, invariants HA7 and
+// H19).
+//
+// Only the state "terminated" counts, only for the instance, account and
+// region the box's roster entry records under cloud, and only when the
+// worker that read it is in that same region. Silence is never evidence,
+// and no disk condition applies: volume records decide only whether the
+// dead box's local chores are discharged (internal/boxvolumes).
+//
+// This package reads and decides. Writing the fence record and citing it
+// in a claim CAS wait for the claim's acceptance of L2 to L5.
+package termfence
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "time"
+
+ "github.com/aws/aws-sdk-go-v2/aws"
+ awsmiddleware "github.com/aws/aws-sdk-go-v2/aws/middleware"
+ "github.com/aws/aws-sdk-go-v2/service/ec2"
+ "github.com/aws/smithy-go"
+
+ "github.com/carlosframework/platform/internal/boxid"
+)
+
+const (
+ ViaDescribe = "describe"
+ ViaEvent = "event"
+
+ stateTerminated = "terminated"
+ eventSource = "aws.ec2"
+ eventDetailType = "EC2 Instance State-change Notification"
+)
+
+// ErrNotEvidence wraps every reason an observation does not fence a box.
+var ErrNotEvidence = errors.New("termfence: not evidence of termination")
+
+// Observation is what one source said about one instance. Found is false
+// when the source said nothing about it: an empty reply, another instance.
+type Observation struct {
+ Via string
+ Found bool
+ InstanceID string
+ Account string
+ Region string
+ State string
+ EventID string
+ RequestID string
+}
+
+// Box is a roster entry as the fence needs it: the reference, the
+// enrollment, and the cloud identity enrollment proved (boxid.Enroll).
+type Box struct {
+ Ref string
+ Enrollment string
+ Cloud boxid.Cloud
+}
+
+func (b Box) complete() bool { return b.Ref != "" && b.Enrollment != "" && b.Cloud.Complete() }
+
+// WrittenBy names the regional worker that made the read.
+type WrittenBy struct {
+ Box string `json:"box"`
+ Enrollment string `json:"enrollment"`
+}
+
+// Record is §5.9's fence record. Its key (control/box-fences/<enrollment>.json)
+// and its writer belong to the claim's L4 and are not built here.
+type Record struct {
+ V int `json:"v"`
+ Box string `json:"box"`
+ Enrollment string `json:"enrollment"`
+ Cloud boxid.Cloud `json:"cloud"`
+ State string `json:"state"`
+ ObservedAt string `json:"observed_at"`
+ Via string `json:"via"`
+ RequestID string `json:"request_id,omitempty"`
+ EventID string `json:"event_id,omitempty"`
+ WrittenBy WrittenBy `json:"written_by"`
+}
+
+// FromDescribe reads one instance out of a DescribeInstances reply. region
+// is the region the call was made in: the reply does not carry it. A reply
+// that does not mention instanceID yields Found false, which Verify
+// refuses: AWS forgets a terminated instance within about an hour, so
+// absence proves nothing.
+func FromDescribe(out *ec2.DescribeInstancesOutput, instanceID, region string) Observation {
+ o := Observation{Via: ViaDescribe, Region: region}
+ if out == nil || instanceID == "" {
+ return o
+ }
+ if id, ok := awsmiddleware.GetRequestIDMetadata(out.ResultMetadata); ok {
+ o.RequestID = id
+ }
+ for _, r := range out.Reservations {
+ for _, inst := range r.Instances {
+ if aws.ToString(inst.InstanceId) != instanceID {
+ continue
+ }
+ o.Found, o.InstanceID, o.Account = true, instanceID, aws.ToString(r.OwnerId)
+ if inst.State != nil {
+ o.State = string(inst.State.Name)
+ }
+ return o
+ }
+ }
+ return o
+}
+
+// FromEvent reads an EventBridge "EC2 Instance State-change Notification".
+// It refuses anything else: the queue's policy admits only the rule that
+// forwards these, and this is the second check.
+func FromEvent(body []byte) (Observation, error) {
+ var ev struct {
+ ID string `json:"id"`
+ DetailType string `json:"detail-type"`
+ Source string `json:"source"`
+ Account string `json:"account"`
+ Region string `json:"region"`
+ Detail struct {
+ InstanceID string `json:"instance-id"`
+ State string `json:"state"`
+ } `json:"detail"`
+ }
+ if err := json.Unmarshal(body, &ev); err != nil {
+ return Observation{}, fmt.Errorf("termfence: event is not JSON: %w", err)
+ }
+ switch {
+ case ev.Source != eventSource:
+ return Observation{}, fmt.Errorf("termfence: event source %q is not %s", ev.Source, eventSource)
+ case ev.DetailType != eventDetailType:
+ return Observation{}, fmt.Errorf("termfence: event detail-type %q is not a state change", ev.DetailType)
+ case ev.ID == "":
+ return Observation{}, errors.New("termfence: event has no id")
+ }
+ return Observation{Via: ViaEvent, Found: ev.Detail.InstanceID != "", InstanceID: ev.Detail.InstanceID,
+ Account: ev.Account, Region: ev.Region, State: ev.Detail.State, EventID: ev.ID}, nil
+}
+
+// EC2 is the one call this package makes. *ec2.Client satisfies it; tests
+// use a fake. It is a read: the box role holds no EC2 write right.
+type EC2 interface {
+ DescribeInstances(ctx context.Context, in *ec2.DescribeInstancesInput, optFns ...func(*ec2.Options)) (*ec2.DescribeInstancesOutput, error)
+}
+
+// DescribeInstance asks EC2 about c.InstanceID alone, in c.Region whatever
+// region the client was built for. InvalidInstanceID.NotFound comes back
+// as an empty reply, which FromDescribe turns into Found false: absence,
+// never termination. Every other error is returned.
+func DescribeInstance(ctx context.Context, api EC2, c boxid.Cloud) (*ec2.DescribeInstancesOutput, error) {
+ if !c.Complete() {
+ return nil, errors.New("termfence: describe of an incomplete cloud identity")
+ }
+ region := c.Region
+ out, err := api.DescribeInstances(ctx, &ec2.DescribeInstancesInput{InstanceIds: []string{c.InstanceID}},
+ func(o *ec2.Options) { o.Region = region })
+ var apiErr smithy.APIError
+ if errors.As(err, &apiErr) && apiErr.ErrorCode() == "InvalidInstanceID.NotFound" {
+ return &ec2.DescribeInstancesOutput{}, nil
+ }
+ return out, err
+}
+
+func refuse(format string, a ...any) error {
+ return fmt.Errorf("%w: %s", ErrNotEvidence, fmt.Sprintf(format, a...))
+}
+
+// Verify returns box's fence record when o proves that the instance its
+// roster entry records is terminated, read by worker, a regional worker of
+// the same AWS region (spec §6: the fence refuses any observation not made
+// in the box's own region). Whether worker is active in the roster is the
+// caller's check, made before it reads.
+func Verify(box, worker Box, o Observation, now time.Time) (Record, error) {
+ c := box.Cloud
+ switch {
+ case !box.complete():
+ return Record{}, refuse("the box has no proven cloud identity")
+ case !worker.complete():
+ return Record{}, refuse("the worker has no proven cloud identity")
+ case worker.Cloud.Region != c.Region:
+ return Record{}, refuse("worker %s is in %q, the box in %q", worker.Ref, worker.Cloud.Region, c.Region)
+ case o.Via != ViaDescribe && o.Via != ViaEvent:
+ return Record{}, refuse("source %q is not one this binary accepts", o.Via)
+ case !o.Found:
+ return Record{}, refuse("the source said nothing about %s; absence is not termination", c.InstanceID)
+ case o.InstanceID != c.InstanceID:
+ return Record{}, refuse("observed %s, the box is %s", o.InstanceID, c.InstanceID)
+ case o.Account != c.Account:
+ return Record{}, refuse("observed in account %q, the box is in %q", o.Account, c.Account)
+ case o.Region != c.Region:
+ return Record{}, refuse("observed in region %q, the box is in %q", o.Region, c.Region)
+ case o.State != stateTerminated:
+ return Record{}, refuse("state is %q; only %q counts", o.State, stateTerminated)
+ }
+ return Record{V: 1, Box: box.Ref, Enrollment: box.Enrollment, Cloud: c, State: o.State,
+ ObservedAt: now.UTC().Format(time.RFC3339), Via: o.Via, RequestID: o.RequestID, EventID: o.EventID,
+ WrittenBy: WrittenBy{Box: worker.Ref, Enrollment: worker.Enrollment}}, nil
+}
+```
+
+- [ ] **Step 4: Remove `boxid.Identity`**
+
+Nothing uses it now. In `internal/boxid/verify.go`, delete:
+
+```go
+// Identity is what an identity document proves. termfence binds to it
+// until it moves to Cloud; nothing else uses it.
+type Identity struct{ AccountID, Region, InstanceID string }
+
+```
+
+Then check: `grep -rn "boxid.Identity" --include=*.go .` prints nothing.
+
+- [ ] **Step 5: Run the tests**
+
+Run: `go test ./internal/termfence/ ./internal/boxid/ -count=1 -v`
+Expected: PASS, including `TestAWorkerInAnotherRegionNeverFences` and `TestDescribeInstanceNotFoundIsAbsenceAndOtherErrorsAreErrors`.
+
+- [ ] **Step 6: Gate and commit**
+
+```bash
+go build ./... && go vet ./internal/termfence/ ./internal/boxid/ && gofmt -l internal/termfence internal/boxid
+GOOS=darwin go build ./... && GOOS=windows go build ./...
+git add internal/termfence internal/boxid/verify.go
+git commit -m "termfence: a fence binds to the roster entry's cloud and is read by a worker in the box's own region
+
+Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
+```
+
+---
+
+### Task 5: Volume records and the discharge check (`internal/boxvolumes`)
+
+Spec §5.9 under H19: regional workers record each enrolled box's block device mappings while it runs, create-only, on change and at least hourly; a removal or any `false` flag rules out automatic discharge for good. A separate function decides discharge from the records. This task builds the record, the decision to write one, the writer, the reader and the discharge check. The loop that describes every enrolled box in the region needs the roster's enrollments (L7), and the discharge CAS is L3: both wait.
+
+**Files:**
+- Create: `internal/boxvolumes/volumes.go`
+- Test: `internal/boxvolumes/volumes_test.go`
+
+**Interfaces:**
+- Consumes: `boxid.Cloud` (Task 1); `release.Store`. Callers get the describe reply from `termfence.DescribeInstance` (Task 4).
+- Produces:
+ ```go
+ package boxvolumes
+ const Prefix = "control/box-volumes/"
+ const RefreshEvery, EnrollmentWindow = time.Hour, time.Hour
+ type Device struct{ Device, VolumeID string; DeleteOnTermination bool; Status string }
+ type WrittenBy struct{ Box, Enrollment string }
+ type Record struct{ V int; Box, Enrollment string; Cloud boxid.Cloud; InstanceState string; BlockDevices []Device; RemovedVolumes []string; ObservedAt, RequestID string; WrittenBy WrittenBy } // §5.9
+ type Entry struct{ Box, Enrollment string; Cloud boxid.Cloud }
+ type Observation struct{ State string; Devices []Device; RequestID string }
+ var ErrNotRecordable error
+ func ValidEnrollment(e string) bool
+ func Key(enrollment string, t time.Time) string
+ func Observe(out *ec2.DescribeInstancesOutput, c boxid.Cloud) (Observation, error)
+ func Next(prev *Record, e Entry, writer WrittenBy, o Observation, now time.Time) (Record, bool)
+ func Write(ctx context.Context, s release.Store, rec Record) (string, error)
+ func List(ctx context.Context, s release.Store, enrollment string) ([]Record, error)
+ type Verdict struct{ Discharge bool; Reason string }
+ func Discharge(records []Record, e Entry, enrolledAt time.Time) Verdict
+ ```
+
+**Three choices the spec leaves to the plan.**
+- A `terminated` read is never recorded. Its mappings are empty (the dev test of 2026-10-06), and a record of it would list every volume as removed.
+- "The first record taken at enrollment" is checked as: the earliest record for this enrollment must be observed within `EnrollmentWindow` (one hour, the refresh period) of the roster's enrollment time. A later first record cannot speak for a volume attached and removed before it.
+- `Discharge` recomputes removals from consecutive records as well as reading `removed_volumes`, so a record one worker failed to write cannot hide a removal.
+
+Keys use a fixed-width UTC time with `-` in place of `:`, so they sort in observation order.
+
+- [ ] **Step 1: Write the failing tests**
+
+Create `internal/boxvolumes/volumes_test.go`:
+
+```go
+package boxvolumes
+
+import (
+ "context"
+ "errors"
+ "testing"
+ "time"
+
+ "github.com/aws/aws-sdk-go-v2/aws"
+ awsmiddleware "github.com/aws/aws-sdk-go-v2/aws/middleware"
+ "github.com/aws/aws-sdk-go-v2/service/ec2"
+ "github.com/aws/aws-sdk-go-v2/service/ec2/types"
+ smithymiddleware "github.com/aws/smithy-go/middleware"
+
+ "github.com/carlosframework/platform/internal/boxid"
+ "github.com/carlosframework/platform/internal/release"
+)
+
+var (
+ cloud = boxid.Cloud{Provider: "aws", Account: "111122223333", Region: "eu-west-1", InstanceID: "i-0abc"}
+ entry = Entry{Box: "annie-welding", Enrollment: "enr-annie-1", Cloud: cloud}
+ writer = WrittenBy{Box: "bert-sawing", Enrollment: "enr-bert-1"}
+ t0 = time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
+)
+
+type vol struct {
+ dev, id string
+ dot bool
+}
+
+func reply(owner string, state types.InstanceStateName, vols ...vol) *ec2.DescribeInstancesOutput {
+ var maps []types.InstanceBlockDeviceMapping
+ for _, v := range vols {
+ maps = append(maps, types.InstanceBlockDeviceMapping{DeviceName: aws.String(v.dev),
+ Ebs: &types.EbsInstanceBlockDevice{VolumeId: aws.String(v.id), DeleteOnTermination: aws.Bool(v.dot), Status: types.AttachmentStatusAttached}})
+ }
+ var md smithymiddleware.Metadata
+ awsmiddleware.SetRequestIDMetadata(&md, "req-1")
+ return &ec2.DescribeInstancesOutput{ResultMetadata: md, Reservations: []types.Reservation{{OwnerId: aws.String(owner),
+ Instances: []types.Instance{{InstanceId: aws.String("i-0abc"), State: &types.InstanceState{Name: state}, BlockDeviceMappings: maps}}}}}
+}
+
+var root, data, scratch = vol{"/dev/xvda", "vol-root", true}, vol{"/dev/xvdb", "vol-data", true}, vol{"/dev/xvdc", "vol-keep", false}
+
+func observe(t *testing.T, out *ec2.DescribeInstancesOutput) Observation {
+ t.Helper()
+ o, err := Observe(out, cloud)
+ if err != nil {
+ t.Fatal(err)
+ }
+ return o
+}
+
+func TestObserveReadsEBSMappingsInVolumeOrder(t *testing.T) {
+ out := reply("111122223333", types.InstanceStateNameRunning, root, data)
+ out.Reservations[0].Instances[0].BlockDeviceMappings = append(out.Reservations[0].Instances[0].BlockDeviceMappings,
+ types.InstanceBlockDeviceMapping{DeviceName: aws.String("/dev/sdz")}) // instance store: no Ebs
+ o := observe(t, out)
+ want := []Device{{"/dev/xvdb", "vol-data", true, "attached"}, {"/dev/xvda", "vol-root", true, "attached"}}
+ if o.State != "running" || o.RequestID != "req-1" || len(o.Devices) != 2 || o.Devices[0] != want[0] || o.Devices[1] != want[1] {
+ t.Fatalf("observation = %+v", o)
+ }
+}
+
+// Review Focus 3: a terminated instance's reply has no mappings. Recorded,
+// it would claim every volume was removed; it must not be recordable.
+func TestObserveRefusesWhatIsNotTheInstancesMapping(t *testing.T) {
+ for name, out := range map[string]*ec2.DescribeInstancesOutput{
+ "terminated": reply("111122223333", types.InstanceStateNameTerminated),
+ "another owner": reply("999999999999", types.InstanceStateNameRunning, root),
+ "another instance": {Reservations: []types.Reservation{{OwnerId: aws.String("111122223333"), Instances: []types.Instance{{InstanceId: aws.String("i-0zzz")}}}}},
+ "empty reply": {},
+ "nil reply": nil,
+ } {
+ t.Run(name, func(t *testing.T) {
+ if o, err := Observe(out, cloud); !errors.Is(err, ErrNotRecordable) {
+ t.Fatalf("observation = %+v, err = %v; want ErrNotRecordable", o, err)
+ }
+ })
+ }
+ // A shutting-down read still lists the mappings and may be recorded.
+ if o := observe(t, reply("111122223333", types.InstanceStateNameShuttingDown, root)); len(o.Devices) != 1 {
+ t.Fatalf("shutting-down: %+v", o)
+ }
+}
+
+func TestNextWritesOnChangeAndAtLeastHourly(t *testing.T) {
+ first, ok := Next(nil, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameRunning, root)), t0)
+ if !ok || first.ObservedAt != "2026-10-06T09:00:00Z" || len(first.RemovedVolumes) != 0 || first.RemovedVolumes == nil {
+ t.Fatalf("first = %+v ok=%v", first, ok)
+ }
+ same := observe(t, reply("111122223333", types.InstanceStateNameRunning, root))
+ if _, ok := Next(&first, entry, writer, same, t0.Add(59*time.Minute)); ok {
+ t.Fatal("wrote an unchanged mapping inside the hour")
+ }
+ if _, ok := Next(&first, entry, writer, same, t0.Add(time.Hour)); !ok {
+ t.Fatal("no record after an hour")
+ }
+ grown, ok := Next(&first, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameRunning, root, scratch)), t0.Add(time.Minute))
+ if !ok || len(grown.BlockDevices) != 2 {
+ t.Fatalf("an attached volume was not recorded: %+v", grown)
+ }
+ shrunk, ok := Next(&grown, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameRunning, root)), t0.Add(2*time.Minute))
+ if !ok || len(shrunk.RemovedVolumes) != 1 || shrunk.RemovedVolumes[0] != "vol-keep" {
+ t.Fatalf("a detached volume is not in removed_volumes: %+v", shrunk)
+ }
+ if _, ok := Next(&first, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameShuttingDown, root)), t0.Add(time.Minute)); !ok {
+ t.Fatal("a state change was not recorded")
+ }
+}
+
+func TestWriteIsCreateOnlyAndListReturnsObservationOrder(t *testing.T) {
+ ctx, st := context.Background(), release.NewDirStore(t.TempDir())
+ a, _ := Next(nil, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameRunning, root)), t0)
+ b, _ := Next(&a, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameRunning, root, data)), t0.Add(10*time.Minute))
+ for _, r := range []Record{b, a} {
+ if _, err := Write(ctx, st, r); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if _, err := Write(ctx, st, a); err != nil {
+ t.Fatalf("the same record twice: %v", err)
+ }
+ other := a
+ other.RequestID = "req-other"
+ if _, err := Write(ctx, st, other); !errors.Is(err, release.ErrCASConflict) {
+ t.Fatalf("a different record under the same key: err = %v", err)
+ }
+ got, err := List(ctx, st, entry.Enrollment)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(got) != 2 || got[0].ObservedAt != a.ObservedAt || got[1].ObservedAt != b.ObservedAt {
+ t.Fatalf("list = %+v", got)
+ }
+ if _, err := Write(ctx, st, Record{Enrollment: "../edges", ObservedAt: a.ObservedAt}); err == nil {
+ t.Fatal("wrote outside the prefix")
+ }
+}
+
+func TestListFailsOnARecordItCannotRead(t *testing.T) {
+ ctx, st := context.Background(), release.NewDirStore(t.TempDir())
+ if _, err := st.Put(ctx, Key(entry.Enrollment, t0), []byte(`{"v":1,"surprise":true}`), ""); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := List(ctx, st, entry.Enrollment); err == nil {
+ t.Fatal("an unreadable record read as a gap")
+ }
+}
+
+func records(t *testing.T, steps ...[]vol) []Record {
+ t.Helper()
+ var out []Record
+ var prev *Record
+ for i, vs := range steps {
+ rec, _ := Next(prev, entry, writer, observe(t, reply("111122223333", types.InstanceStateNameRunning, vs...)), t0.Add(time.Duration(i)*10*time.Minute))
+ out = append(out, rec)
+ prev = &out[len(out)-1]
+ }
+ return out
+}
+
+func TestDischarge(t *testing.T) {
+ enrolled := t0.Add(-5 * time.Minute)
+ if v := Discharge(records(t, []vol{root}, []vol{root, data}, []vol{root, data}), entry, enrolled); !v.Discharge {
+ t.Fatalf("all true, nothing removed: %+v", v)
+ }
+ cases := map[string]struct {
+ recs []Record
+ at time.Time
+ }{
+ "no record at all": {nil, enrolled},
+ "a false flag": {records(t, []vol{root}, []vol{root, scratch}), enrolled},
+ "a removed volume": {records(t, []vol{root, data}, []vol{root}), enrolled},
+ "a replaced root volume": {records(t, []vol{root}, []vol{{"/dev/xvda", "vol-root2", true}}), enrolled},
+ "first record long after": {records(t, []vol{root}), t0.Add(-2 * time.Hour)},
+ "only another enrollment's": {func() []Record { r := records(t, []vol{root}); r[0].Enrollment = "enr-other"; return r }(), enrolled},
+ "only another instance's": {func() []Record { r := records(t, []vol{root}); r[0].Cloud.InstanceID = "i-0zzz"; return r }(), enrolled},
+ }
+ for name, tc := range cases {
+ t.Run(name, func(t *testing.T) {
+ if v := Discharge(tc.recs, entry, tc.at); v.Discharge || v.Reason == "" {
+ t.Fatalf("verdict = %+v, want the chores left open with a reason", v)
+ }
+ })
+ }
+}
+
+// A removal is caught from the sequence even when the record that should
+// have listed it was never written (two workers, one missed write).
+func TestDischargeSeesARemovalNoRecordListed(t *testing.T) {
+ recs := records(t, []vol{root, data}, []vol{root})
+ recs[1].RemovedVolumes = []string{}
+ if v := Discharge(recs, entry, t0); v.Discharge {
+ t.Fatal("discharged across a silent removal")
+ }
+}
+```
+
+- [ ] **Step 2: Run them to see them fail**
+
+Run: `go test ./internal/boxvolumes/ -count=1`
+Expected: FAIL, `no non-test Go files` or `undefined: Observe`.
+
+- [ ] **Step 3: Write the package**
+
+Create `internal/boxvolumes/volumes.go`:
+
+```go
+// Package boxvolumes records a platform box's block device mappings while
+// it runs (region HA spec 1 §5.9, ruling H19), and decides from those
+// records whether a terminated box's park-local chores may be discharged
+// without retirement (claim D5's destroyed-local exception, L3).
+//
+// Nothing here is a termination fence, and nothing about execution,
+// admission, relocation or recovery reads it (spec §8, B4-1). A record that
+// is wrong or missing can leave a disk unretired for an operator; it can
+// never admit a writer.
+//
+// This package builds, writes and reads records. The loop that describes
+// every enrolled box in the region needs the roster's enrollments, and the
+// discharge CAS needs L3: both wait for the claim.
+package boxvolumes
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "slices"
+ "sort"
+ "strings"
+ "time"
+
+ "github.com/aws/aws-sdk-go-v2/aws"
+ awsmiddleware "github.com/aws/aws-sdk-go-v2/aws/middleware"
+ "github.com/aws/aws-sdk-go-v2/service/ec2"
+
+ "github.com/carlosframework/platform/internal/boxid"
+ "github.com/carlosframework/platform/internal/release"
+)
+
+const (
+ Prefix = "control/box-volumes/"
+
+ // RefreshEvery is the longest a running box goes without a new record
+ // when nothing changes (spec §5.9: "at least once an hour").
+ RefreshEvery = time.Hour
+ // EnrollmentWindow is how soon after enrollment the first record must
+ // be taken for it to stand for the launch-time mappings.
+ EnrollmentWindow = time.Hour
+
+ keyTime = "2006-01-02T15-04-05.000000000Z"
+)
+
+// Device is one EBS volume in the instance's block device mappings.
+type Device struct {
+ Device string `json:"device"`
+ VolumeID string `json:"volume_id"`
+ DeleteOnTermination bool `json:"delete_on_termination"`
+ Status string `json:"status"`
+}
+
+// WrittenBy names the regional worker that made the read.
+type WrittenBy struct {
+ Box string `json:"box"`
+ Enrollment string `json:"enrollment"`
+}
+
+// Record is §5.9's volume record,
+// control/box-volumes/<enrollment>/<observed_at>.json.
+type Record struct {
+ V int `json:"v"`
+ Box string `json:"box"`
+ Enrollment string `json:"enrollment"`
+ Cloud boxid.Cloud `json:"cloud"`
+ InstanceState string `json:"instance_state"`
+ BlockDevices []Device `json:"block_devices"`
+ RemovedVolumes []string `json:"removed_volumes"`
+ ObservedAt string `json:"observed_at"`
+ RequestID string `json:"request_id"`
+ WrittenBy WrittenBy `json:"written_by"`
+}
+
+// Entry is the roster entry a record describes.
+type Entry struct {
+ Box string
+ Enrollment string
+ Cloud boxid.Cloud
+}
+
+// Observation is one DescribeInstances read of one enrolled box.
+type Observation struct {
+ State string
+ Devices []Device
+ RequestID string
+}
+
+var ErrNotRecordable = errors.New("boxvolumes: not a recordable observation")
+
+// ValidEnrollment reports whether e can be a key segment: 1 to 128 of
+// [A-Za-z0-9._-], never "." or "..".
+func ValidEnrollment(e string) bool {
+ if e == "" || len(e) > 128 || e == "." || e == ".." {
+ return false
+ }
+ for _, r := range e {
+ ok := r >= 'a' && r <= 'z' || r >= 'A' && r <= 'Z' || r >= '0' && r <= '9' || r == '.' || r == '_' || r == '-'
+ if !ok {
+ return false
+ }
+ }
+ return true
+}
+
+// Key is where a record observed at t lives. The time is fixed-width UTC,
+// so keys sort in observation order.
+func Key(enrollment string, t time.Time) string {
+ return Prefix + enrollment + "/" + t.UTC().Format(keyTime) + ".json"
+}
+
+// recordable is every state in which EC2 still reports the instance's
+// mappings. "terminated" is not one: its BlockDeviceMappings is empty (the
+// dev test of 2026-10-06), which would read as every volume removed.
+var recordable = map[string]bool{"pending": true, "running": true, "shutting-down": true, "stopping": true, "stopped": true}
+
+// Observe reads the entry's instance out of a DescribeInstances reply (see
+// termfence.DescribeInstance for the call). It refuses a reply that does
+// not name the instance, names it under another owner, or reports a state
+// whose mappings are not the instance's.
+func Observe(out *ec2.DescribeInstancesOutput, c boxid.Cloud) (Observation, error) {
+ if out == nil || !c.Complete() {
+ return Observation{}, fmt.Errorf("%w: no reply, or no proven cloud", ErrNotRecordable)
+ }
+ var o Observation
+ if id, ok := awsmiddleware.GetRequestIDMetadata(out.ResultMetadata); ok {
+ o.RequestID = id
+ }
+ for _, r := range out.Reservations {
+ for _, inst := range r.Instances {
+ if aws.ToString(inst.InstanceId) != c.InstanceID {
+ continue
+ }
+ if aws.ToString(r.OwnerId) != c.Account {
+ return Observation{}, fmt.Errorf("%w: %s is owned by %q, the roster says %q", ErrNotRecordable, c.InstanceID, aws.ToString(r.OwnerId), c.Account)
+ }
+ if inst.State != nil {
+ o.State = string(inst.State.Name)
+ }
+ if !recordable[o.State] {
+ return Observation{}, fmt.Errorf("%w: state %q", ErrNotRecordable, o.State)
+ }
+ for _, m := range inst.BlockDeviceMappings {
+ if m.Ebs == nil {
+ continue // instance store dies with the instance
+ }
+ o.Devices = append(o.Devices, Device{Device: aws.ToString(m.DeviceName), VolumeID: aws.ToString(m.Ebs.VolumeId),
+ DeleteOnTermination: aws.ToBool(m.Ebs.DeleteOnTermination), Status: string(m.Ebs.Status)})
+ }
+ sort.Slice(o.Devices, func(i, j int) bool { return o.Devices[i].VolumeID < o.Devices[j].VolumeID })
+ return o, nil
+ }
+ }
+ return Observation{}, fmt.Errorf("%w: the reply does not name %s", ErrNotRecordable, c.InstanceID)
+}
+
+func volumeIDs(ds []Device) []string {
+ out := make([]string, 0, len(ds))
+ for _, d := range ds {
+ out = append(out, d.VolumeID)
+ }
+ return out
+}
+
+func removed(prev, cur []Device) []string {
+ have := volumeIDs(cur)
+ var out []string
+ for _, id := range volumeIDs(prev) {
+ if !slices.Contains(have, id) {
+ out = append(out, id)
+ }
+ }
+ sort.Strings(out)
+ return out
+}
+
+// Next decides whether o, made by writer at now, is a new record after
+// prev (nil when the enrollment has none). It is one when there is no
+// record yet, when the mapping or the instance state differs from prev,
+// and otherwise once RefreshEvery has passed.
+func Next(prev *Record, e Entry, writer WrittenBy, o Observation, now time.Time) (Record, bool) {
+ rec := Record{V: 1, Box: e.Box, Enrollment: e.Enrollment, Cloud: e.Cloud, InstanceState: o.State,
+ BlockDevices: o.Devices, RemovedVolumes: []string{}, ObservedAt: now.UTC().Format(time.RFC3339Nano),
+ RequestID: o.RequestID, WrittenBy: writer}
+ if rec.BlockDevices == nil {
+ rec.BlockDevices = []Device{}
+ }
+ if prev == nil {
+ return rec, true
+ }
+ if r := removed(prev.BlockDevices, o.Devices); len(r) > 0 {
+ rec.RemovedVolumes = r
+ }
+ if !slices.Equal(prev.BlockDevices, rec.BlockDevices) || prev.InstanceState != o.State {
+ return rec, true
+ }
+ at, err := time.Parse(time.RFC3339Nano, prev.ObservedAt)
+ return rec, err != nil || now.Sub(at) >= RefreshEvery
+}
+
+// Write stores rec create-only at its key. Records are never replaced or
+// deleted (spec §6); the same bytes written twice are one record.
+func Write(ctx context.Context, s release.Store, rec Record) (string, error) {
+ if !ValidEnrollment(rec.Enrollment) {
+ return "", fmt.Errorf("boxvolumes: %q is not an enrollment", rec.Enrollment)
+ }
+ at, err := time.Parse(time.RFC3339Nano, rec.ObservedAt)
+ if err != nil {
+ return "", fmt.Errorf("boxvolumes: observed_at: %w", err)
+ }
+ body, err := json.Marshal(rec)
+ if err != nil {
+ return "", err
+ }
+ key := Key(rec.Enrollment, at)
+ _, err = s.Put(ctx, key, body, "")
+ if errors.Is(err, release.ErrCASConflict) {
+ cur, _, gerr := s.Get(ctx, key)
+ if gerr == nil && bytes.Equal(cur, body) {
+ return key, nil
+ }
+ return "", fmt.Errorf("boxvolumes: %s already holds another record: %w", key, err)
+ }
+ return key, err
+}
+
+func decode(body []byte) (Record, error) {
+ dec := json.NewDecoder(bytes.NewReader(body))
+ dec.DisallowUnknownFields()
+ var rec Record
+ if err := dec.Decode(&rec); err != nil {
+ return Record{}, err
+ }
+ if _, err := dec.Token(); err != io.EOF {
+ return Record{}, errors.New("trailing data")
+ }
+ return rec, nil
+}
+
+// List returns every record stored for an enrollment, in key order. Any
+// record that cannot be read or parsed is an error, never a gap: a gap
+// could hide the record that shows a removal.
+func List(ctx context.Context, s release.Store, enrollment string) ([]Record, error) {
+ if !ValidEnrollment(enrollment) {
+ return nil, fmt.Errorf("boxvolumes: %q is not an enrollment", enrollment)
+ }
+ keys, err := s.List(ctx, Prefix+enrollment+"/")
+ if err != nil {
+ return nil, err
+ }
+ sort.Strings(keys)
+ out := make([]Record, 0, len(keys))
+ for _, k := range keys {
+ if !strings.HasSuffix(k, ".json") {
+ continue
+ }
+ body, _, err := s.Get(ctx, k)
+ if err != nil {
+ return nil, fmt.Errorf("boxvolumes: %s: %w", k, err)
+ }
+ rec, err := decode(body)
+ if err != nil {
+ return nil, fmt.Errorf("boxvolumes: %s: %w", k, err)
+ }
+ out = append(out, rec)
+ }
+ return out, nil
+}
+
+// Verdict is whether a terminated box's park-local chores may be
+// discharged without retirement, and if not, why.
+type Verdict struct {
+ Discharge bool
+ Reason string
+}
+
+// Discharge decides from records (as List returns them) whether every EBS
+// volume e's instance had attached, from the first record taken within
+// EnrollmentWindow of enrolledAt to the last, was DeleteOnTermination, with
+// none ever removed (spec §5.9, H19). Records for another enrollment or
+// another instance are ignored. Anything short of that keeps the chores
+// open for an operator, which costs cleanup and never recovery.
+func Discharge(records []Record, e Entry, enrolledAt time.Time) Verdict {
+ no := func(format string, a ...any) Verdict { return Verdict{Reason: fmt.Sprintf(format, a...)} }
+ if !e.Cloud.Complete() || e.Enrollment == "" {
+ return no("the roster entry has no proven cloud")
+ }
+ type stamped struct {
+ at time.Time
+ rec Record
+ }
+ var mine []stamped
+ for _, r := range records {
+ if r.V != 1 || r.Enrollment != e.Enrollment || r.Cloud != e.Cloud {
+ continue
+ }
+ at, err := time.Parse(time.RFC3339Nano, r.ObservedAt)
+ if err != nil {
+ return no("record with unreadable observed_at %q", r.ObservedAt)
+ }
+ mine = append(mine, stamped{at, r})
+ }
+ if len(mine) == 0 {
+ return no("no volume record for %s", e.Enrollment)
+ }
+ sort.SliceStable(mine, func(i, j int) bool { return mine[i].at.Before(mine[j].at) })
+ if first := mine[0].at; first.After(enrolledAt.Add(EnrollmentWindow)) {
+ return no("the first record (%s) is more than %s after enrollment", first.Format(time.RFC3339), EnrollmentWindow)
+ }
+ for i, s := range mine {
+ for _, d := range s.rec.BlockDevices {
+ if !d.DeleteOnTermination {
+ return no("volume %s was not DeleteOnTermination at %s", d.VolumeID, s.rec.ObservedAt)
+ }
+ }
+ if len(s.rec.RemovedVolumes) > 0 {
+ return no("volumes %v were removed by %s", s.rec.RemovedVolumes, s.rec.ObservedAt)
+ }
+ if i > 0 {
+ if r := removed(mine[i-1].rec.BlockDevices, s.rec.BlockDevices); len(r) > 0 {
+ return no("volumes %v left the mapping by %s", r, s.rec.ObservedAt)
+ }
+ }
+ }
+ return Verdict{Discharge: true}
+}
+```
+
+- [ ] **Step 4: Run the tests**
+
+Run: `go test ./internal/boxvolumes/ -count=1 -v`
+Expected: PASS, including `TestObserveRefusesWhatIsNotTheInstancesMapping/terminated` and `TestDischargeSeesARemovalNoRecordListed`.
+
+- [ ] **Step 5: Gate and commit**
+
+```bash
+go build ./... && go vet ./internal/boxvolumes/ && gofmt -l internal/boxvolumes
+GOOS=darwin go build ./... && GOOS=windows go build ./...
+git add internal/boxvolumes
+git commit -m "boxvolumes: a running box's volumes are recorded create-only, and only an unbroken all-true record discharges its chores
+
+Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
+```
+
+---
+
+### Task 6: The edges file follows the claim's executor (`internal/host/edgesclaims.go`)
+
+Spec §5.4: "The edges-file pass publishes a host as `home` only when this box is its executor **and** its row is `ready`, and adds the installation's stamp." Plan 1a built the edge's side (`foldHomes` believes the higher stamp). This task builds the publisher's side, with `CARLOS_REGION_HA` on.
+
+**Decisions.**
+- **Which rows.** Only relocatable hosts (spec §2): a provisioned `instance` row that the activator runs as its own exec child (`Backing == ""`, no `Unit`, no `Addr`/`Addrs`, a `Socket`). Unit-, addr- and socket-only routes, static sites and services keep their box and are published as before. Sidecars are never published (unchanged).
+- **The decision.** A claim whose `op.exec` is this box's reference: home, stamped with `holder.stamp`. A claim whose `op.exec` names another box, or none, or which has no holder: withheld. No claim: home, unstamped, exactly as before; §8's gate admits a second box only once every host has a claim.
+- **Errors.** A read that fails keeps the last decision (claim C6: an error is never absence); a host never read publishes as before.
+- **Cost.** Each read stands 10 minutes (H9), at most 256 reads a pass, never-read hosts first. About 288 thousand GETs a day per 2,000 hosts, under $0.12.
+- **§18's open question.** A positive stamp is the mark: only relocatable hosts with a claim carry one, so the edge's report of two boxes at one positive stamp already concerns them alone. No separate marker field.
+
+The withholding is applied before owners and aliases are computed, so an alias of a withheld host is withheld with it.
+
+**Files:**
+- Create: `internal/host/edgesclaims.go`
+- Test: `internal/host/edgesclaims_test.go`
+- Modify: `internal/host/edgesfile.go`
+
+**Interfaces:**
+- Consumes: `hostclaim.Read`, `hostclaim.Observed`, `hostclaim.Claim` (main); `regionHAEnabled` (Task 3); `edgeHost.Stamp` (plan 1a).
+- Produces:
+ ```go
+ const edgesClaimTTL = 10 * time.Minute
+ const edgesClaimReadsPerPass = 256
+ type edgesClaims struct{ /* self, read, now, entries */ }
+ func newEdgesClaims(self string, read func(context.Context, string) (hostclaim.Observed, error)) *edgesClaims
+ func relocatableRow(r registry.Route) bool
+ func applyEdgesClaims(ctx context.Context, claims *edgesClaims, routes []registry.Route, hosts map[string]edgeHost)
+ func edgesFileWriteClaims(ctx context.Context, s release.Store, reg *registry.Registry, ident edgesFile, eip string, last *[]byte, refresh bool, claims *edgesClaims) error
+ ```
+ `edgesFileWrite` keeps its signature and calls `edgesFileWriteClaims` with `nil`.
+
+- [ ] **Step 1: Write the failing tests**
+
+Create `internal/host/edgesclaims_test.go`:
+
+```go
+package host
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "testing"
+ "time"
+
+ "github.com/carlosframework/platform/internal/hostclaim"
+ "github.com/carlosframework/platform/internal/registry"
+ "github.com/carlosframework/platform/internal/release"
+)
+
+// seedInstance upserts a ready, provisioned exec instance: a relocatable
+// host (spec 1 §2).
+func seedInstance(t *testing.T, reg *registry.Registry, host string) {
+ t.Helper()
+ if err := reg.Upsert(registry.Route{Host: host, Kind: "instance", Socket: "/run/carlos/" + host + ".sock",
+ Provisioned: true, App: "jam", Account: "bab", Status: "ready"}); err != nil {
+ t.Fatal(err)
+ }
+}
+
+// claimsBook answers claim reads from a map; a missing host is an absent
+// claim, a host in errs fails.
+type claimsBook struct {
+ claims map[string]*hostclaim.Claim
+ errs map[string]bool
+ reads []string
+}
+
+func (b *claimsBook) read(_ context.Context, host string) (hostclaim.Observed, error) {
+ b.reads = append(b.reads, host)
+ if b.errs[host] {
+ return hostclaim.Observed{}, errors.New("SlowDown")
+ }
+ c, ok := b.claims[host]
+ if !ok {
+ return hostclaim.Observed{Host: host, Absent: true}, nil
+ }
+ return hostclaim.Observed{Host: host, Claim: c, ETag: "e"}, nil
+}
+
+func serving(exec string, stamp uint64) *hostclaim.Claim {
+ return &hostclaim.Claim{V: 2, Gen: 7, Holder: &hostclaim.Holder{Stamp: stamp},
+ Op: hostclaim.Op{Kind: hostclaim.KindServing, ID: "op-1", Exec: exec}}
+}
+
+func publishWithClaims(t *testing.T, reg *registry.Registry, claims *edgesClaims) map[string]edgeHost {
+ t.Helper()
+ st := release.NewDirStore(t.TempDir())
+ var last []byte
+ if err := edgesFileWriteClaims(context.Background(), st, reg, edgesFile{Box: "annie", BoxID: "annie"}, "1.2.3.4", &last, false, claims); err != nil {
+ t.Fatal(err)
+ }
+ body, _, err := st.Get(context.Background(), edgesFileKey("annie"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ var f edgesFile
+ if err := json.Unmarshal(body, &f); err != nil {
+ t.Fatal(err)
+ }
+ return f.Hosts
+}
+
+func TestEdgesFileFollowsTheClaimsExecutor(t *testing.T) {
+ reg := edgesTestRegistry(t)
+ for _, h := range []string{"mine.bab.test", "theirs.bab.test", "between.bab.test", "unclaimed.bab.test"} {
+ seedInstance(t, reg, h)
+ }
+ seedRoute(t, reg, "static.example.com") // not relocatable: never read, always home
+ book := &claimsBook{claims: map[string]*hostclaim.Claim{
+ "mine.bab.test": serving("annie", 41),
+ "theirs.bab.test": serving("bert", 42),
+ "between.bab.test": serving("", 43), // op.exec none: nobody serves it
+ }}
+ hosts := publishWithClaims(t, reg, newEdgesClaims("annie", book.read))
+ if h, ok := hosts["mine.bab.test"]; !ok || h.Stamp != 41 || h.Role != "home" {
+ t.Fatalf("executor's host = %+v, %v; want home stamped 41", h, ok)
+ }
+ for _, h := range []string{"theirs.bab.test", "between.bab.test"} {
+ if _, ok := hosts[h]; ok {
+ t.Errorf("%s published by a box its claim does not name", h)
+ }
+ }
+ if h, ok := hosts["unclaimed.bab.test"]; !ok || h.Stamp != 0 {
+ t.Errorf("a host with no claim = %+v, %v; want home, unstamped, as before", h, ok)
+ }
+ if h, ok := hosts["static.example.com"]; !ok || h.Stamp != 0 {
+ t.Errorf("a service route = %+v, %v; want home, unstamped", h, ok)
+ }
+ for _, r := range book.reads {
+ if r == "static.example.com" {
+ t.Error("read a claim for a route that is not relocatable")
+ }
+ }
+}
+
+func TestEdgesFileWithholdsTheAliasOfAWithheldHost(t *testing.T) {
+ reg := edgesTestRegistry(t)
+ seedInstance(t, reg, "theirs.bab.test")
+ if err := reg.UpsertAlias(registry.RouteAlias{Alias: "www.customer.com", Host: "theirs.bab.test", Account: "bab", App: "jam", DNSStatus: "pointing"}); err != nil {
+ t.Fatal(err)
+ }
+ book := &claimsBook{claims: map[string]*hostclaim.Claim{"theirs.bab.test": serving("bert", 42)}}
+ hosts := publishWithClaims(t, reg, newEdgesClaims("annie", book.read))
+ if len(hosts) != 0 {
+ t.Fatalf("hosts = %v, want none", hosts)
+ }
+}
+
+// Review Focus 2: a claim that cannot be read never moves a host. It keeps
+// the last decision, or, never read, publishes as it did before claims.
+func TestEdgesFileKeepsTheLastDecisionWhenAClaimCannotBeRead(t *testing.T) {
+ reg := edgesTestRegistry(t)
+ seedInstance(t, reg, "mine.bab.test")
+ seedInstance(t, reg, "theirs.bab.test")
+ book := &claimsBook{claims: map[string]*hostclaim.Claim{"mine.bab.test": serving("annie", 41), "theirs.bab.test": serving("bert", 42)}}
+ clock := time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
+ claims := newEdgesClaims("annie", book.read)
+ claims.now = func() time.Time { return clock }
+ publishWithClaims(t, reg, claims)
+ clock = clock.Add(edgesClaimTTL)
+ book.errs = map[string]bool{"mine.bab.test": true, "theirs.bab.test": true}
+ hosts := publishWithClaims(t, reg, claims)
+ if h, ok := hosts["mine.bab.test"]; !ok || h.Stamp != 41 {
+ t.Fatalf("mine = %+v, %v: an unreadable claim dropped or unstamped a served host", h, ok)
+ }
+ if _, ok := hosts["theirs.bab.test"]; ok {
+ t.Fatal("an unreadable claim published a host this box was told not to")
+ }
+
+ fresh := newEdgesClaims("annie", book.read)
+ if h, ok := publishWithClaims(t, reg, fresh)["mine.bab.test"]; !ok || h.Stamp != 0 {
+ t.Fatalf("never read = %+v, %v; want home, unstamped", h, ok)
+ }
+}
+
+func TestEdgesClaimsRereadOnlyAfterTheTTL(t *testing.T) {
+ book := &claimsBook{claims: map[string]*hostclaim.Claim{"mine.bab.test": serving("annie", 41)}}
+ clock := time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
+ c := newEdgesClaims("annie", book.read)
+ c.now = func() time.Time { return clock }
+ ctx := context.Background()
+ c.decide(ctx, []string{"mine.bab.test"})
+ clock = clock.Add(edgesClaimTTL - time.Second)
+ c.decide(ctx, []string{"mine.bab.test"})
+ if len(book.reads) != 1 {
+ t.Fatalf("%d reads inside the TTL, want 1", len(book.reads))
+ }
+ book.claims["mine.bab.test"] = serving("bert", 50)
+ clock = clock.Add(time.Second)
+ if d := c.decide(ctx, []string{"mine.bab.test"})["mine.bab.test"]; !d.withhold {
+ t.Fatalf("decision after the TTL = %+v; the move to another box was not seen", d)
+ }
+ c.decide(ctx, nil)
+ if len(c.entries) != 0 {
+ t.Fatalf("a host no longer served kept its entry: %v", c.entries)
+ }
+}
+
+func TestEdgesClaimsBoundReadsPerPassNeverReadFirst(t *testing.T) {
+ book := &claimsBook{claims: map[string]*hostclaim.Claim{}}
+ clock := time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC)
+ c := newEdgesClaims("annie", book.read)
+ c.now = func() time.Time { return clock }
+ var hosts []string
+ for i := 0; i < edgesClaimReadsPerPass+10; i++ {
+ hosts = append(hosts, fmt.Sprintf("h%04d.bab.test", i))
+ }
+ ctx := context.Background()
+ c.decide(ctx, hosts)
+ if len(book.reads) != edgesClaimReadsPerPass {
+ t.Fatalf("first pass read %d claims, want %d", len(book.reads), edgesClaimReadsPerPass)
+ }
+ book.reads = nil
+ c.decide(ctx, hosts)
+ if len(book.reads) != 10 {
+ t.Fatalf("second pass read %d claims, want the 10 never read", len(book.reads))
+ }
+}
+```
+
+- [ ] **Step 2: Run them to see them fail**
+
+Run: `go test ./internal/host/ -run 'EdgesFileFollows|EdgesFileWithholds|EdgesFileKeeps|EdgesClaims' -count=1`
+Expected: FAIL to build, `undefined: edgesClaims`.
+
+- [ ] **Step 3: Write the claim cache**
+
+Create `internal/host/edgesclaims.go`:
+
+```go
+package host
+
+import (
+ "context"
+ "math"
+ "sort"
+ "time"
+
+ "github.com/carlosframework/platform/internal/hostclaim"
+ "github.com/carlosframework/platform/internal/registry"
+)
+
+// Region HA spec 1 §5.4: with CARLOS_REGION_HA on, a relocatable host is
+// published as home only by the box its claim's op.exec names, carrying
+// the claim's holder.stamp so the edge's home map believes the newest
+// installation (HA4). The edge never reads claims; this publisher does, on
+// a slow cache, because the safety gate is the activator's fresh read at
+// every spawn (claim C4), not this file.
+
+const (
+ // edgesClaimTTL is how long one claim read stands: reconcile's sweep
+ // (H9). A held row's entry can be stale this long, which can only send
+ // a request to a box that then refuses to spawn.
+ edgesClaimTTL = 10 * time.Minute
+ // edgesClaimReadsPerPass bounds the GETs one pass makes, so a box with
+ // thousands of hosts spreads its first reads over several passes
+ // instead of stalling one. A host not read yet publishes as it did
+ // before this file existed.
+ edgesClaimReadsPerPass = 256
+)
+
+// edgesDecision is what one claim says about this box and one host.
+type edgesDecision struct {
+ withhold bool // the claim names another executor, or none
+ stamp int64 // holder.stamp when this box is the executor
+}
+
+type edgesClaimEntry struct {
+ decision edgesDecision
+ readAt time.Time
+}
+
+// edgesClaims caches one decision per host. It is used only by the edges
+// file loop's goroutine.
+type edgesClaims struct {
+ self string // this box's reference: the edges file's box_id
+ read func(ctx context.Context, host string) (hostclaim.Observed, error)
+ now func() time.Time
+ entries map[string]edgesClaimEntry
+}
+
+func newEdgesClaims(self string, read func(context.Context, string) (hostclaim.Observed, error)) *edgesClaims {
+ return &edgesClaims{self: self, read: read, now: time.Now, entries: map[string]edgesClaimEntry{}}
+}
+
+// relocatableRow is spec 1 §2's relocatable host as a registry row shows
+// it: a provisioned instance whose process is the activator's own exec
+// child. Unit-backed, addr-backed, static and service routes keep their
+// box and are published as before; sidecars are never published at all.
+func relocatableRow(r registry.Route) bool {
+ return r.Provisioned && r.Kind == "instance" && r.Backing == "" && r.Unit == "" &&
+ r.Addr == "" && r.Addrs == "" && r.Socket != ""
+}
+
+// decideClaim turns one observation into a decision. An absent claim
+// leaves the host as it was before claims existed: a region can only be
+// given a second box once every host has one (spec §8).
+func (c *edgesClaims) decideClaim(obs hostclaim.Observed) edgesDecision {
+ if obs.Absent || obs.Claim == nil {
+ return edgesDecision{}
+ }
+ cl := obs.Claim
+ if cl.Holder == nil || cl.Op.Exec != c.self {
+ return edgesDecision{withhold: true}
+ }
+ stamp := cl.Holder.Stamp
+ if stamp > math.MaxInt64 {
+ stamp = 0
+ }
+ return edgesDecision{stamp: int64(stamp)}
+}
+
+// decide returns a decision for each host it has one for. Hosts whose
+// entry is older than edgesClaimTTL are re-read, oldest first, up to
+// edgesClaimReadsPerPass. A read that fails keeps the previous decision
+// (claim C6: an error is never absence); a host with neither a previous
+// decision nor a successful read has none, and publishes as before.
+func (c *edgesClaims) decide(ctx context.Context, hosts []string) map[string]edgesDecision {
+ now := c.now()
+ want := make(map[string]bool, len(hosts))
+ var due []string
+ for _, h := range hosts {
+ want[h] = true
+ if e, ok := c.entries[h]; !ok || now.Sub(e.readAt) >= edgesClaimTTL {
+ due = append(due, h)
+ }
+ }
+ for h := range c.entries {
+ if !want[h] {
+ delete(c.entries, h)
+ }
+ }
+ sort.Slice(due, func(i, j int) bool {
+ ei, iok := c.entries[due[i]]
+ ej, jok := c.entries[due[j]]
+ if iok != jok {
+ return !iok // never read first
+ }
+ if !ei.readAt.Equal(ej.readAt) {
+ return ei.readAt.Before(ej.readAt)
+ }
+ return due[i] < due[j]
+ })
+ if len(due) > edgesClaimReadsPerPass {
+ due = due[:edgesClaimReadsPerPass]
+ }
+ for _, h := range due {
+ obs, err := c.read(ctx, h)
+ if err != nil {
+ continue
+ }
+ c.entries[h] = edgesClaimEntry{decision: c.decideClaim(obs), readAt: now}
+ }
+ out := make(map[string]edgesDecision, len(c.entries))
+ for h, e := range c.entries {
+ out[h] = e.decision
+ }
+ return out
+}
+
+// applyEdgesClaims withholds or stamps every relocatable row in hosts
+// (edgesFileWrite's served map, before aliases join it, so an alias of a
+// withheld host is withheld with it). claims nil leaves hosts untouched.
+func applyEdgesClaims(ctx context.Context, claims *edgesClaims, routes []registry.Route, hosts map[string]edgeHost) {
+ if claims == nil {
+ return
+ }
+ var reloc []string
+ for _, r := range routes {
+ if _, served := hosts[r.Host]; served && relocatableRow(r) {
+ reloc = append(reloc, r.Host)
+ }
+ }
+ for host, d := range claims.decide(ctx, reloc) {
+ h, ok := hosts[host]
+ if !ok {
+ continue
+ }
+ if d.withhold {
+ delete(hosts, host)
+ continue
+ }
+ h.Stamp = d.stamp
+ hosts[host] = h
+ }
+}
+```
+
+- [ ] **Step 4: Wire it into the publisher**
+
+In `internal/host/edgesfile.go`, add the import (keep the group sorted):
+
+```go
+ "github.com/carlosframework/platform/internal/appconfig"
+ "github.com/carlosframework/platform/internal/hostclaim"
+ "github.com/carlosframework/platform/internal/registry"
+```
+
+Replace the head of `edgesFileWrite`:
+
+```go
+func edgesFileWrite(ctx context.Context, s release.Store, reg *registry.Registry, ident edgesFile, eip string, last *[]byte, refresh bool) error {
+ box := ident.Box
+```
+
+with:
+
+```go
+func edgesFileWrite(ctx context.Context, s release.Store, reg *registry.Registry, ident edgesFile, eip string, last *[]byte, refresh bool) error {
+ return edgesFileWriteClaims(ctx, s, reg, ident, eip, last, refresh, nil)
+}
+
+// edgesFileWriteClaims is edgesFileWrite with the claim reader region HA
+// adds (edgesclaims.go). claims is nil unless CARLOS_REGION_HA is on, and
+// nil publishes exactly what edgesFileWrite always has.
+func edgesFileWriteClaims(ctx context.Context, s release.Store, reg *registry.Registry, ident edgesFile, eip string, last *[]byte, refresh bool, claims *edgesClaims) error {
+ box := ident.Box
+```
+
+After the routes loop, replace:
+
+```go
+ hosts[r.Host] = edgeHost{Role: "home", Platform: r.Platform || r.Provisioned, Replica: r.Replica}
+ }
+```
+
+with:
+
+```go
+ hosts[r.Host] = edgeHost{Role: "home", Platform: r.Platform || r.Provisioned, Replica: r.Replica}
+ }
+ // Region HA (spec 1 §5.4): a relocatable host is home only on the box
+ // its claim names, stamped with the installation. Before owners and
+ // aliases, so an alias of a withheld host is withheld with it.
+ applyEdgesClaims(ctx, claims, routes, hosts)
+```
+
+In `edgesFileLoop`, replace:
+
+```go
+ dataDir := provisionTickConfig(os.Getenv).DataDir
+ log.Printf("agent: edges file loop armed (5min + jitter, and %s after a registry change, %s)", edgesFileWatchInterval, edgesFileKey(box))
+```
+
+with:
+
+```go
+ dataDir := provisionTickConfig(os.Getenv).DataDir
+ // Region HA reads each relocatable host's claim (spec 1 §5.4); off, the
+ // file is exactly what it always was.
+ var claims *edgesClaims
+ if regionHAEnabled(os.Getenv) {
+ claims = newEdgesClaims(box, func(ctx context.Context, host string) (hostclaim.Observed, error) {
+ return hostclaim.Read(ctx, store, host)
+ })
+ }
+ log.Printf("agent: edges file loop armed (5min + jitter, and %s after a registry change, %s)", edgesFileWatchInterval, edgesFileKey(box))
+```
+
+and replace the pass call:
+
+```go
+ err = edgesFileWrite(ctx, store, reg, ident, eip, &last, sched.timed)
+```
+
+with:
+
+```go
+ err = edgesFileWriteClaims(ctx, store, reg, ident, eip, &last, sched.timed, claims)
+```
+
+`box` here is `metricsBoxLabel(…)`, the same value the file publishes as `box_id`, which is the reference claims name (spec §2).
+
+- [ ] **Step 5: Run the tests**
+
+Run: `go test ./internal/host/ -run 'EdgesFile|EdgesClaims' -count=1 -v`
+Expected: PASS: the five new tests and every existing edges-file test.
+
+- [ ] **Step 6: Gate and commit**
+
+```bash
+go build ./... && go vet ./internal/host/ && gofmt -l internal/host
+GOOS=darwin go build ./... && GOOS=windows go build ./...
+git add internal/host/edgesclaims.go internal/host/edgesclaims_test.go internal/host/edgesfile.go
+git commit -m "edges: with region HA on, a relocatable host is home only on the box its claim names, stamped
+
+Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
+```
+
+---
+
+### Task 7: Root refuses an update while the box takes part in a claim operation (`internal/host/systemparticipation.go`)
+
+Spec §5.10, root's side (H14): under root's update lock, before an executable change, write `/opt/carlos/system/guard/update-begun.json` durably, then scan `<DataDir>/.hostexec/` completely. Any record there, in any state, or anything root cannot read, list or parse, removes the marker and refuses the update. Otherwise the update proceeds and the marker stays until it finishes or is recovered.
+
+On main every executable change goes through one path, `systemApplier.install` (an operator's `carlos system rollback` re-points the channel, and the box applies it the same way); the guarded activation and the console's activation reservation are claim D1 and D7 and not on main. Recovery of an update already begun needs no check (spec §5.10: no participant acknowledged meanwhile). The participant's half (write `pending`, then refuse to acknowledge while the marker exists) belongs to claim D6's controller and waits for L6.
+
+Nothing writes `.hostexec/` on main, so with the flag on this changes nothing but a marker file during an update.
+
+**Files:**
+- Create: `internal/host/systemparticipation.go`
+- Test: `internal/host/systemparticipation_test.go`
+- Modify: `internal/host/systemapply.go` (`systemApplier`, `RunSystemApply`, `Run`, `install`)
+
+**Interfaces:**
+- Consumes: `regionHAEnabled` (Task 3); `writeJSONDurable`, `syncDir`, `readSystemJournal`, `errSystemRefused`, `systemApplier.record` (existing).
+- Produces:
+ ```go
+ func systemGuardDir(optDir string) string
+ func updateBegunPath(optDir string) string
+ func hostexecParticipation(dataDir string) error
+ func (a *systemApplier) beginExecutableChange(target string, promotedAt time.Time) error
+ func (a *systemApplier) clearUpdateBegun() error
+ func (a *systemApplier) settleUpdateBegun()
+ // systemApplier gains: regionHA bool
+ ```
+
+- [ ] **Step 1: Write the failing tests**
+
+Create `internal/host/systemparticipation_test.go`:
+
+```go
+package host
+
+import (
+ "context"
+ "os"
+ "path/filepath"
+ "runtime"
+ "strings"
+ "testing"
+)
+
+func hostexec(t *testing.T, f *applyFixture) string {
+ t.Helper()
+ dir := filepath.Join(f.a.dataDir, hostexecDir)
+ if err := os.MkdirAll(dir, 0o755); err != nil {
+ t.Fatal(err)
+ }
+ return dir
+}
+
+func assertNotUpdated(t *testing.T, f *applyFixture, why string) {
+ t.Helper()
+ r := f.result()
+ if r == nil || r.Outcome != systemOutcomeRefused || !strings.Contains(r.Reason, why) {
+ t.Fatalf("result = %+v, want refused naming %q", r, why)
+ }
+ if bin, _ := os.ReadFile(systemBinaryPath(f.a.optDir)); !strings.Contains(string(bin), "carlos v1") {
+ t.Fatal("the binary changed under a refusal")
+ }
+ if j, _ := readSystemJournal(f.a.optDir); j != nil {
+ t.Fatalf("a refusal opened a journal: %+v", j)
+ }
+ if _, err := os.Stat(updateBegunPath(f.a.optDir)); !os.IsNotExist(err) {
+ t.Fatalf("the update-begun marker outlived the refusal: %v", err)
+ }
+ if f.called("systemctl stop") || f.called("systemctl restart") {
+ t.Fatal("a refused update touched a unit")
+ }
+}
+
+// Spec §5.10: any controller record, in any state, refuses the update.
+func TestApplyRefusesWhileThisBoxTakesPartInAClaimOperation(t *testing.T) {
+ for _, state := range []string{"pending", "active", "lost"} {
+ t.Run(state, func(t *testing.T) {
+ f, _ := applyHappy(t)
+ f.a.regionHA = true
+ os.WriteFile(filepath.Join(hostexec(t, f), "shop.bab.test.json"), []byte(`{"state":"`+state+`"}`), 0o600)
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ assertNotUpdated(t, f, "takes part in a running claim operation")
+ })
+ }
+}
+
+// Review Focus 4: what root cannot read is a refusal, never a pass.
+func TestApplyRefusesWhenItCannotTellWhetherThisBoxTakesPart(t *testing.T) {
+ cases := map[string]func(t *testing.T, f *applyFixture){
+ "a temporary file": func(t *testing.T, f *applyFixture) {
+ os.WriteFile(filepath.Join(hostexec(t, f), ".shop.bab.test.json.tmp"), nil, 0o600)
+ },
+ "garbage": func(t *testing.T, f *applyFixture) {
+ os.WriteFile(filepath.Join(hostexec(t, f), "shop.bab.test.json"), []byte("{half"), 0o600)
+ },
+ "a symlink in place of the directory": func(t *testing.T, f *applyFixture) {
+ elsewhere := t.TempDir()
+ if err := os.Symlink(elsewhere, filepath.Join(f.a.dataDir, hostexecDir)); err != nil {
+ t.Skip("no symlinks here:", err)
+ }
+ },
+ "a file in place of the directory": func(t *testing.T, f *applyFixture) {
+ os.WriteFile(filepath.Join(f.a.dataDir, hostexecDir), nil, 0o600)
+ },
+ }
+ for name, setup := range cases {
+ t.Run(name, func(t *testing.T) {
+ f, _ := applyHappy(t)
+ f.a.regionHA = true
+ setup(t, f)
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ assertNotUpdated(t, f, "hostexec")
+ })
+ }
+}
+
+func TestApplyRefusesWhenItCannotListTheControllerDirectory(t *testing.T) {
+ if runtime.GOOS == "windows" || os.Geteuid() == 0 {
+ t.Skip("permission bits do not stop this user")
+ }
+ f, _ := applyHappy(t)
+ f.a.regionHA = true
+ dir := hostexec(t, f)
+ os.Chmod(dir, 0)
+ t.Cleanup(func() { os.Chmod(dir, 0o755) })
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ assertNotUpdated(t, f, "cannot be listed")
+}
+
+// The marker is written before the box changes and is gone once the
+// update is done; an empty controller directory is no participation.
+func TestApplyHoldsTheUpdateBegunMarkerAcrossTheInstall(t *testing.T) {
+ f, req := applyHappy(t)
+ f.a.regionHA = true
+ hostexec(t, f) // present and empty
+ seen := false
+ orig := f.a.ops.systemctl
+ f.a.ops.systemctl = func(args ...string) (string, error) {
+ if args[0] == "restart" && args[len(args)-1] == systemEdgeUnit {
+ _, err := os.Stat(updateBegunPath(f.a.optDir))
+ seen = err == nil
+ }
+ return orig(args...)
+ }
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ if r := f.result(); r == nil || r.Outcome != systemOutcomeApplied || !r.matches("stable", "v2", req.PromotedAt) {
+ t.Fatalf("result = %+v", r)
+ }
+ if !seen {
+ t.Fatal("no update-begun marker while the edge restarted onto the new binary")
+ }
+ if _, err := os.Stat(updateBegunPath(f.a.optDir)); !os.IsNotExist(err) {
+ t.Fatalf("marker left after the update finished: %v", err)
+ }
+}
+
+func TestApplyWithoutRegionHAWritesNoMarkerAndIgnoresControllerRecords(t *testing.T) {
+ f, _ := applyHappy(t)
+ os.WriteFile(filepath.Join(hostexec(t, f), "shop.bab.test.json"), []byte(`{}`), 0o600)
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ if r := f.result(); r == nil || r.Outcome != systemOutcomeApplied {
+ t.Fatalf("result = %+v: the flag is off, so nothing changes", r)
+ }
+ if _, err := os.Stat(systemGuardDir(f.a.optDir)); !os.IsNotExist(err) {
+ t.Fatalf("the guard directory was created with the flag off: %v", err)
+ }
+}
+
+// A crash between the marker and the journal leaves a marker with no
+// update behind it; the next run, --recover included, clears it.
+func TestAStaleUpdateBegunMarkerIsClearedByTheNextRun(t *testing.T) {
+ f := newApplyFixture(t, "stable", "v1")
+ os.MkdirAll(systemGuardDir(f.a.optDir), 0o755)
+ os.WriteFile(updateBegunPath(f.a.optDir), []byte(`{"target":"v2"}`), 0o644)
+ if err := f.a.Run(context.Background(), true); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := os.Stat(updateBegunPath(f.a.optDir)); !os.IsNotExist(err) {
+ t.Fatalf("stale marker kept: %v", err)
+ }
+}
+
+// A rollback that could not finish keeps its journal for the next run, and
+// the marker with it: the update is not over.
+func TestTheMarkerStaysWhileAJournalIsOpen(t *testing.T) {
+ f := newApplyFixture(t, "stable", "v1")
+ f.a.regionHA = true
+ os.MkdirAll(systemGuardDir(f.a.optDir), 0o755)
+ os.WriteFile(updateBegunPath(f.a.optDir), []byte(`{"target":"v2"}`), 0o644)
+ // A journal recovery cannot act on: no carlos.prev to restore.
+ if err := writeJSONDurable(f.a.file("journal.json"), systemJournal{Phase: systemPhaseInstalled, Target: "v2"}, 0o644); err != nil {
+ t.Fatal(err)
+ }
+ _ = f.a.Run(context.Background(), true)
+ if j, _ := readSystemJournal(f.a.optDir); j == nil {
+ t.Fatal("recovery closed the journal; this test needs one it cannot finish")
+ }
+ if _, err := os.Stat(updateBegunPath(f.a.optDir)); err != nil {
+ t.Fatalf("marker cleared while the journal is still open: %v", err)
+ }
+}
+```
+
+- [ ] **Step 2: Run them to see them fail**
+
+Run: `go test ./internal/host/ -run 'TakesPart|CannotTell|CannotList|UpdateBegun|RegionHAWrites|MarkerStays' -count=1`
+Expected: FAIL to build, `f.a.regionHA undefined` and `undefined: hostexecDir`.
+
+- [ ] **Step 3: Write root's check**
+
+Create `internal/host/systemparticipation.go`:
+
+```go
+package host
+
+import (
+ "errors"
+ "fmt"
+ "os"
+ "path/filepath"
+ "sort"
+ "strings"
+ "time"
+)
+
+// Region HA spec 1 §5.10 (H14), root's half. A box that takes part in a
+// running claim operation never takes an executable update. Each side
+// records its own intent durably, then checks the other's, so of two that
+// race at least one sees the other:
+//
+// - root writes its update-begun marker, then scans <DataDir>/.hostexec/
+// and refuses the update while any controller record is there, in any
+// state, or while it cannot tell;
+// - a participant writes its .hostexec record (claim D6), then reads the
+// marker and refuses to acknowledge while one exists. That half waits
+// for the claim's D6 controller and is not built here.
+//
+// The marker stays until the update finishes or is recovered. Dormant
+// unless CARLOS_REGION_HA is on.
+
+const (
+ updateBegunFile = "update-begun.json"
+ hostexecDir = ".hostexec"
+)
+
+// systemGuardDir is /opt/carlos/system/guard, beside root's update state.
+func systemGuardDir(optDir string) string { return filepath.Join(optDir, "system", "guard") }
+
+func updateBegunPath(optDir string) string {
+ return filepath.Join(systemGuardDir(optDir), updateBegunFile)
+}
+
+type updateBegun struct {
+ Target string `json:"target"`
+ PromotedAt time.Time `json:"promoted_at"`
+ At time.Time `json:"at"`
+}
+
+// hostexecParticipation reports why this box may still be a participant
+// in a claim operation, or nil when it certainly is not. The directory
+// absent is the only "not". Anything in it, whatever its name or state,
+// and anything root cannot list or that is not a plain directory, is a
+// refusal (spec §5.10, B1-5). Only names are read: root never opens a
+// record here.
+func hostexecParticipation(dataDir string) error {
+ dir := filepath.Join(dataDir, hostexecDir)
+ fi, err := os.Lstat(dir)
+ if errors.Is(err, os.ErrNotExist) {
+ return nil
+ }
+ if err != nil {
+ return fmt.Errorf("%s cannot be read: %v", dir, err)
+ }
+ if !fi.IsDir() {
+ return fmt.Errorf("%s is not a directory", dir)
+ }
+ entries, err := os.ReadDir(dir)
+ if err != nil {
+ return fmt.Errorf("%s cannot be listed: %v", dir, err)
+ }
+ if len(entries) == 0 {
+ return nil
+ }
+ names := make([]string, 0, len(entries))
+ for _, e := range entries {
+ names = append(names, e.Name())
+ }
+ sort.Strings(names)
+ if len(names) > 3 {
+ names = append(names[:3], fmt.Sprintf("and %d more", len(entries)-3))
+ }
+ return fmt.Errorf("this box takes part in a running claim operation (%s holds %s)", dir, strings.Join(names, ", "))
+}
+
+// beginExecutableChange is §5.10's root side, run under the apply lock
+// just before the first box-changing step: write the marker durably, then
+// look for participation. On a refusal the marker is removed again and the
+// error says why; the caller records it as a refusal, so the console's
+// reservation stays held and the update is retried later.
+func (a *systemApplier) beginExecutableChange(target string, promotedAt time.Time) error {
+ if !a.regionHA {
+ return nil
+ }
+ if err := os.MkdirAll(systemGuardDir(a.optDir), 0o755); err != nil {
+ return err
+ }
+ if err := writeJSONDurable(updateBegunPath(a.optDir), updateBegun{Target: target, PromotedAt: promotedAt, At: a.ops.now().UTC()}, 0o644); err != nil {
+ return err
+ }
+ if reason := hostexecParticipation(a.dataDir); reason != nil {
+ if err := a.clearUpdateBegun(); err != nil {
+ a.ops.logf("system apply: %v", err)
+ }
+ return fmt.Errorf("%w: %v", errSystemRefused, reason)
+ }
+ return nil
+}
+
+// clearUpdateBegun removes the marker durably. A marker that is not there
+// is already cleared.
+func (a *systemApplier) clearUpdateBegun() error {
+ err := os.Remove(updateBegunPath(a.optDir))
+ if errors.Is(err, os.ErrNotExist) {
+ return nil
+ }
+ if err != nil {
+ return fmt.Errorf("remove %s: %w", updateBegunPath(a.optDir), err)
+ }
+ return syncDir(systemGuardDir(a.optDir))
+}
+
+// settleUpdateBegun clears the marker once no update is in progress: no
+// journal means the update finished, rolled back, was recovered, or never
+// got past its first step. A kept journal (a rollback that could not
+// finish) keeps the marker with it. Run with or without CARLOS_REGION_HA,
+// so a marker left behind after the flag is turned off still goes.
+func (a *systemApplier) settleUpdateBegun() {
+ if j, err := readSystemJournal(a.optDir); err != nil || j != nil {
+ return
+ }
+ if err := a.clearUpdateBegun(); err != nil {
+ a.ops.logf("system apply: %v", err)
+ }
+}
+```
+
+- [ ] **Step 4: Wire it into the apply**
+
+In `internal/host/systemapply.go`, add the field at the end of `systemApplier`:
+
+```go
+ platform string // "<goos>-<goarch>"; the caller sets it from runtime
+ gateWait, soak time.Duration
+ // regionHA is CARLOS_REGION_HA: root writes its update-begun marker and
+ // refuses an update while this box takes part in a claim operation
+ // (region HA spec 1 §5.10; systemparticipation.go).
+ regionHA bool
+}
+```
+
+In `RunSystemApply`, set it:
+
+```go
+ platform: runtime.GOOS + "-" + runtime.GOARCH, gateWait: systemGateWait, soak: systemGateSoak,
+ regionHA: regionHAEnabled(os.Getenv),
+ }
+```
+
+In `Run`, right after `defer os.RemoveAll(a.stagingDir())`, add:
+
+```go
+ // Whatever this run does, a finished or recovered update leaves no
+ // update-begun marker behind (spec §5.10).
+ defer a.settleUpdateBegun()
+```
+
+In `install`, immediately before `// PromotedAt rides along for the whole transaction`, add:
+
+```go
+ // Region HA (spec 1 §5.10): the marker, then the participation check,
+ // before the first step that changes the box. A refusal leaves the box
+ // exactly as it was.
+ if err := a.beginExecutableChange(target, req.PromotedAt); err != nil {
+ if !errors.Is(err, errSystemRefused) {
+ err = fmt.Errorf("%w: update-begun marker: %v", errSystemRefused, err)
+ }
+ return a.record(req, err)
+ }
+```
+
+A refusal is recorded in `result.json` like every other refusal, so the agent's next request retries it.
+
+- [ ] **Step 5: Run the tests**
+
+Run: `go test ./internal/host/ -run 'Apply|Recover|Rollback|Prev|Layout|UpdateBegun|MarkerStays|TakesPart|CannotTell|CannotList' -count=1 -v`
+Expected: PASS: the seven new tests and every existing apply and recovery test.
+
+- [ ] **Step 6: Gate and commit**
+
+```bash
+go build ./... && go vet ./internal/host/ && gofmt -l internal/host
+GOOS=darwin go build ./... && GOOS=windows go build ./... && GOOS=linux GOARCH=arm64 go build ./...
+git add internal/host/systemparticipation.go internal/host/systemparticipation_test.go internal/host/systemapply.go
+git commit -m "system: with region HA on, root marks an update begun and refuses it while the box takes part in a claim operation
+
+Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
+```
+
+---
+
+### Task 8: Storage policy, the module notes, the spec note and the branch gate
+
+Spec §6: "The control bucket denies `PutObject` without `s3:if-none-match: *`, and `DeleteObject` and `DeleteObjectVersion`, on `control/boxes/*` (plan 1a's policy) and on `control/box-volumes/*`, `control/box-enrollment/*` and `control/box-fences/*`. Each Deny is scoped to its prefix only." `control/boxes/` is already covered. Task 5's writer uses `control/box-volumes/`; `control/box-enrollment/` and `control/box-fences/` have no writer yet, but a Deny grants nothing, and putting all four in place at once means one apply. The `*.bind.json` statement and the `operator_role_arns` variable stay (§19: "harmless and can go at the next apply"): removing the variable would break every root that sets it.
+
+Nothing here is applied. The flagship's root lives in `carlosframework/carloku-infrastructure` and consumes these modules at a release tag. Two cautions bind this task: a whole-root apply of the flagship restarts every box (user-data drift), and this task changes no `user_data`; a broad Deny on these buckets once broke every lease release, and every Deny here names one evidence prefix and nothing wider.
+
+**Files:**
+- Modify: `infra/modules/deployment-buckets/main.tf`
+- Create: `infra/modules/deployment-buckets/tests/evidence.tftest.hcl`
+- Modify: `infra/modules/region-host/README.md`
+- Modify: `docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md` (append §24)
+
+- [ ] **Step 1: Write the failing module test**
+
+Create `infra/modules/deployment-buckets/tests/evidence.tftest.hcl`:
+
+```hcl
+# Region HA spec 1 §6: every box-evidence prefix is create-only and never
+# deleted, and the Denies name those prefixes and nothing wider (a broad
+# Deny on this bucket once broke every lease release).
+
+mock_provider "aws" {}
+
+override_resource {
+ target = aws_s3_bucket.deployment
+ values = { arn = "arn:aws:s3:::dep" }
+}
+
+variables {
+ deployment_bucket_name = "dep"
+ data_bucket_name = "data"
+}
+
+run "evidence_prefixes_are_create_only_and_never_deleted" {
+ command = plan
+
+ assert {
+ condition = alltrue([
+ for sid in ["BoxRecordsAreCreateOnly", "BoxRecordsAreNeverDeleted"] :
+ toset([for s in jsondecode(aws_s3_bucket_policy.deployment_box_records.policy).Statement : s if s.Sid == sid][0].Resource) == toset([
+ "arn:aws:s3:::dep/control/boxes/*",
+ "arn:aws:s3:::dep/control/box-volumes/*",
+ "arn:aws:s3:::dep/control/box-enrollment/*",
+ "arn:aws:s3:::dep/control/box-fences/*",
+ ])
+ ])
+ error_message = "the create-only and never-deleted Denies must name exactly the four box-evidence prefixes"
+ }
+}
+```
+
+- [ ] **Step 2: Run it to see it fail**
+
+Run (sandbox off; `tofu init` needs the network once):
+```bash
+cd infra/modules/deployment-buckets && tofu init -backend=false -input=false && tofu test; cd -
+```
+Expected: `Failure! 0 passed, 1 failed.`, naming the four prefixes. Afterwards remove `.terraform/` and `.terraform.lock.hcl` from the module directory, as no module here commits them.
+
+- [ ] **Step 3: Extend the policy**
+
+In `infra/modules/deployment-buckets/main.tf`, replace the section comment:
+
+```hcl
+# --- Box records (region HA spec 1 §5.1) -----------------------------------
+# control/boxes/<id>.json binds a box's name to one AWS instance, and
+# control/boxes/<id>.bind.json is an operator's word for a legacy label.
+# Both are written once and never changed. Storage enforces that, so no
+# bug or stolen box credential can rebind a name to another machine.
+#
+# EVERY statement here names control/boxes/ and nothing wider. A broad Deny
+# on this bucket has broken lease releases before.
+```
+
+with:
+
+```hcl
+# --- Box evidence (region HA spec 1 §5.1, §5.9, §6) ------------------------
+# control/boxes/<ref>.json is a box's AWS-signed identity, the evidence its
+# enrollment reads. control/box-volumes/, control/box-enrollment/ and
+# control/box-fences/ are the volume records, membership confirmations
+# and termination fence records regional workers write. Every one is
+# written once and never changed or deleted. Storage enforces that, so no
+# bug or stolen box credential can rewrite the evidence behind a decision.
+# (control/boxes/*.bind.json is no longer written: enrollment replaced the
+# operator bind. Its statement stays harmless until operator_role_arns is
+# retired with it.)
+#
+# EVERY statement here names one of these prefixes and nothing wider. A
+# broad Deny on this bucket has broken lease releases before.
+
+locals {
+ box_evidence_prefixes = ["control/boxes", "control/box-volumes", "control/box-enrollment", "control/box-fences"]
+}
+```
+
+and replace the first two statements of `aws_s3_bucket_policy.deployment_box_records`:
+
+```hcl
+ {
+ # A record or bind may be created, never replaced. The qualifier
+ # leaves multipart preparation alone and still covers its completion.
+ Sid = "BoxRecordsAreCreateOnly"
+ Effect = "Deny"
+ Principal = "*"
+ Action = "s3:PutObject"
+ Resource = "${aws_s3_bucket.deployment.arn}/control/boxes/*"
+ Condition = {
+ Null = { "s3:if-none-match" = "true" }
+ Bool = { "s3:ObjectCreationOperation" = "true" }
+ }
+ },
+ {
+ Sid = "BoxRecordsAreNeverDeleted"
+ Effect = "Deny"
+ Principal = "*"
+ Action = ["s3:DeleteObject", "s3:DeleteObjectVersion"]
+ Resource = "${aws_s3_bucket.deployment.arn}/control/boxes/*"
+ },
+```
+
+with:
+
+```hcl
+ {
+ # Evidence may be created, never replaced. The qualifier leaves
+ # multipart preparation alone and still covers its completion.
+ Sid = "BoxRecordsAreCreateOnly"
+ Effect = "Deny"
+ Principal = "*"
+ Action = "s3:PutObject"
+ Resource = [for p in local.box_evidence_prefixes : "${aws_s3_bucket.deployment.arn}/${p}/*"]
+ Condition = {
+ Null = { "s3:if-none-match" = "true" }
+ Bool = { "s3:ObjectCreationOperation" = "true" }
+ }
+ },
+ {
+ Sid = "BoxRecordsAreNeverDeleted"
+ Effect = "Deny"
+ Principal = "*"
+ Action = ["s3:DeleteObject", "s3:DeleteObjectVersion"]
+ Resource = [for p in local.box_evidence_prefixes : "${aws_s3_bucket.deployment.arn}/${p}/*"]
+ },
+```
+
+The Sids are kept, so a reader of the bucket policy's history sees one statement widened, not two replaced.
+
+- [ ] **Step 4: Run the module test**
+
+Run: `cd infra/modules/deployment-buckets && tofu fmt -check && tofu validate && tofu test; cd -` (sandbox off).
+Expected: `Success! The configuration is valid.` and `Success! 1 passed, 0 failed.` Remove `.terraform/` and `.terraform.lock.hcl` again.
+
+- [ ] **Step 5: Revise the region-host README**
+
+The current text says region HA depends on the root volume being deleted. Under H19 it does not. In `infra/modules/region-host/README.md`, replace:
+
+```markdown
+The root volume is deleted when the instance is terminated. Region HA
+depends on that: a terminated box must leave no disk behind.
+
+Open item for region HA plan 1b: the box role's `ec2:DescribeInstances`
+is unconditioned, because the infrastructure inventory reads every
+region. The HA spec asks for an `ec2:Region` condition on the call the
+placer makes. Decide there whether that is a second, narrower statement.
+```
+
+with (after Paul's copy review, his wording verbatim):
+
+```markdown
+The root volume is deleted when the instance is terminated. The fence
+does not depend on that: a `terminated` read is enough. It decides only
+whether a dead box's local cleanup can be skipped automatically, so keep
+it, and keep Recycle Bin, Data Lifecycle Manager and AWS Backup rules off
+platform box volumes.
+
+The box role's `ec2:DescribeInstances` has no region condition, because
+the infrastructure inventory reads every region, and a second, narrower
+Allow could not take that away. Region HA keeps to the box's own region
+in code instead: every describe it makes for a fence or a volume record
+is pinned to the region the box was enrolled in.
+```
+
+- [ ] **Step 6: Record this plan's decisions in the spec**
+
+Append to `docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md`:
+
+```markdown
+
+## 24. Notes from plan 1b (2026-10-06)
+
+Plan `docs/superpowers/plans/2026-10-06-region-ha-1b-claim-readers.md`
+builds the parts of rev 5.4 that need none of L1 to L20: enrollment
+evidence, §19's corrections, the agent publishing its identity record,
+the fence bound to the roster's `cloud`, volume records and the discharge
+check, the edges file following `op.exec`, and root's half of §5.10. All
+of it is dormant behind `CARLOS_REGION_HA`. It keeps every property above
+and settles five details:
+
+1. **Enrollment refuses one instance under two references (§5.1).** The
+ record's `box` field is not covered by AWS's signature, so box A's
+ record copied to B's key with that field rewritten still verifies.
+ `boxid.Enroll` refuses an instance the roster holds under another
+ reference, as well as a reference the roster holds for another
+ instance, counting entries in every state.
+2. **A positive stamp is the relocatable mark (§18's open question).**
+ Only relocatable hosts with a claim carry a stamp, so the edge's report
+ of two boxes at one positive stamp already concerns them alone. No
+ separate marker.
+3. **The edges file's claim reads (§5.4).** A relocatable host with no
+ claim is published as before, unstamped: §8's gate admits a second box
+ only once every host has a claim. A read that fails keeps the last
+ decision. Each read stands 10 minutes (H9), at most 256 reads a pass.
+4. **"The first record taken at enrollment" (§5.9)** is the earliest
+ volume record for the enrollment, observed within one hour of the
+ roster's enrollment time. A later first record cannot speak for a
+ volume attached and removed before it, so the chores stay open.
+5. **No volume record from a `terminated` read.** Its mappings are empty
+ (§8 step 5's dev test), and recording it would list every volume as
+ removed.
+
+It also removes plan 1a's `Bind`, `BindKey`, `Verifier.Verify` and
+`activator.WarmCopyTrusted`, and makes every box-evidence prefix
+create-only and never deleted in the deployment bucket's policy.
+```
+
+- [ ] **Step 7: Run the whole gate**
+
+Run: `go build ./... && make test`, then `GOOS=darwin go build ./... && GOOS=windows go build ./... && GOOS=linux GOARCH=arm64 go build ./...`.
+Expected: `gofmt -l .` clean and every package `ok`, with the sandbox off (see Global Constraints for the socket tests that fail inside it). `make test` takes up to thirty minutes. If a package unrelated to this plan fails, do not fix it here and do not skip it: run that package alone, confirm it also fails on `origin/main`, and report both results.
+
+- [ ] **Step 8: Commit**
+
+```bash
+git add infra/modules/deployment-buckets infra/modules/region-host/README.md docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md
+git commit -m "infra: box evidence prefixes are create-only and never deleted; spec: plan 1b's decisions
+
+Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
+```
+
+- [ ] **Step 9: Push and read CI**
+
+```bash
+git push origin region-ha-1b
+amadan ci status carlos/platform -branch region-ha-1b
+```
+
+Expected: exit 0 (pass). Exit 2 is pending: wait and read it again. Exit 1 is a failure: read the log, fix it, and do not ask for the branch to be merged. Merging is Paul's call, through `amadan branch merge`, never a push to `main`.
+
+---
+
+## Waits for the claim
+
+Everything spec 1 rev 5.4 asks for that this plan does not build, with the request it waits on (§9). Item numbers are §19's revised list for plan 1b.
+
+| What | Waits for | Why it cannot be built now |
+|---|---|---|
+| Relocation: the `relocating` op kind, its phases, start CAS, replacement table, source-loss conversion, two-replacement bound (§5.8; item 7) | **L1** | a new claim op kind |
+| The release sequence's writers-stopped check (§5.8 step 1) | **L10** | changes `stopevidence`'s contract |
+| Coverage `released` on the first probe admission (§5.8) | **L9** | changes `hostclaim.AdmitWriter`'s admission rules |
+| A relocation start despite a regional `park-generation` chore | **L11** | changes `hostclaim.StartGuard` (CC11) |
+| Building the prepared generation from a pinned set; the local install receipt | **L16**, **L17** (and L1) | shared with moves' landing; changes moves |
+| Reconcile's row cases, held participant rows, former executor rows, the 10-minute sweep as an exception to C4 (§5.3; item 2) | **L12** | changes claim C4 and moves §9.6 |
+| Create names a box: `chooseBox(eligibleBoxes(…))` in the create intent's `selected_executor` (§5.2; item 1) | **L8** (claim P5-D6, the claim's plan 5) | changes create's step 2 and step 4 CASes |
+| A move into a multi-box region naming its target box (§5.2) | **L15** | changes moves |
+| A target terminated mid-move taken off the move by the fence | **L14** | changes moves |
+| Writing fence records to `control/box-fences/`, the `lost` list, the fence CAS (§5.9) | **L2**, **L4** | a lost-box fence with no operator is a claim rule change |
+| Discharging `park-local` chores in the fence CAS from `boxvolumes.Discharge` | **L3** | a claim D5 change |
+| The census excluding a fenced box and a console on it; the `compute` binding (§5.9) | **L5** | a claim D7 change |
+| A stale console report accepted for box-started relocations (H18) | **L20** | a claim D7 change |
+| The participant's half of §5.10: write `pending`, then refuse to acknowledge while root's marker exists | **L6** (and claim D6's controller, not on main) | `.hostexec/` is D6's; nothing writes it yet |
+| Enrollment itself: the console verifying records with `boxid.Enroll` into `control/host-actors.json`, the tofu check, refusals | **L7** (claim D7's roster, not on main) | there is no roster to write |
+| The volume recorder loop: describing every enrolled box in the region on the 10-minute sweep and on `shutting-down` | **L7** | records are keyed by enrollment, which only the roster assigns |
+| `eligibleBoxes`' caller, reading the roster and the region's AWS region from it (§18's second correction) | **L7**, **L8** | no roster |
+| Auto Scaling membership confirmations, and the cross-region confirmer for an emptied region (H16, H17), with the cross-region IAM statement | **L19** (and spec 4's group and the roster's `regions` entry) | the console may accept them only under L19; nothing to confirm yet |
+| The files publisher as a generation-bound writer, the copied manifest at each cold admission, retirement by `park-generation` (§5.6); `files_durability` on routes and the agent starting `hostfiles.Run` (item 5) | **L13** | a second ordinary-replication exception to claim C3 |
+| The multi-box gate's roster half and the HA floor (§8; item 8) | **L7** and claim D1's floor | no roster, no floor on main |
+| The joint model with two boxes per region | **L18** | the model belongs to the claim and moves sessions |
+| Reading the termination queue and the placer loop | spec 3 (§12), then **L2** to **L4** | nothing can be done with an event before a fence can be written |
+
+**Not waiting for the claim, and not in this plan.** Plan 1a's "After the build" items 9 (a streaming put, lifting the 64 MiB limit, and the data bucket's conditional-write policy for `files/`), 11 (one inotify instance per agent) and 12 (a size cap in `StagePlatform`, a rate limit on `Commit`) need no claim change. They matter only once the files publisher is switched on, which waits for L13, so they belong with that plan.
+
+## What the dry run could not check
+
+- `readIdentityRecord` against a real box's IMDS. Plan 1a verified the regional certificates and the signature on the dev box; this plan's reader was tested only against an in-process fake. Before the branch lands, on the dev box with `CARLOS_REGION_HA=1` set for the agent, confirm `control/boxes/<label>.json` appears once and that `boxid.Enroll` accepts its bytes with the dev account and region.
+- `termfence.DescribeInstance` and `boxvolumes.Observe` against EC2. Both were tested with fakes and the SDK's own types. The first live use is spec 3's placer.
+- Root's marker under a real `carlos-system-update.service`. The apply's tests run the whole `Run` with faked `systemctl`; whether `/etc/carlos/host.env` carries `CARLOS_REGION_HA` on a given box is a deployment fact.
+- The tests listed under Global Constraints that open Unix sockets, inside the sandbox.
+
+## Self-review
+
+- **Spec coverage.** §19's corrections: Tasks 1, 2 and 4 (box records as evidence, the bind's tofu check moved into enrollment, `eligibleBoxes` from the roster, `chooseBox`'s tie-break, `WarmCopyTrusted`, `termfence` on `cloud` with no volume input). §5.1 enrollment evidence: Tasks 1 and 3. §5.4: Task 6. §5.9 under H19: Tasks 4 and 5. §5.10 root's half: Task 7. §6 storage: Task 8. §10's unit tests that need no claim: enrollment evidence (Task 1), `eligibleBoxes` and `choose` (Task 2), termination-fence acceptance (Task 4: NotFound, wrong account, wrong region, stopped, an event alone, a describe alone, `shutting-down` offered as evidence, accepted with no volume record), the volume recorder and chore discharge (Task 5), root's refusal on any controller record (Task 7). Everything else is in "Waits for the claim".
+- **Placeholders.** None: every code step is a whole file or an exact replacement.
+- **Type consistency.** `boxid.Cloud` is defined in Task 1 and used by Tasks 2, 4 and 5; `boxid.Identity` is kept through Task 3 and removed in Task 4, its last user; `regionHAEnabled` is defined in Task 3 and used by Tasks 6 and 7; `termfence.DescribeInstance` (Task 4) returns the `*ec2.DescribeInstancesOutput` that `boxvolumes.Observe` (Task 5) reads. The dry run applied the tasks in order to a clean tree and built each.
+- **Review Focus.** Each of the five has a test in its owning task, named above, and each was watched to fail with its guard removed.