amadan

carlos / platform Public

Plan 2a rev 2: Astra round 1 and census ruling region-ha-2a

Answers Astra's seven P2s (H27 in spec rev 15): every create door chooses
before it registers anything and refuses with a 503 and an alert when no
box has room (instance create, domain attach's first route, standalone
sidecar create, the move door's destination); instance create's earlier
region check uses the chooser's eligible set with region HA on; the move
door reads roles only with region HA on; the home map dials private
addresses only with the reader's region HA on; the alert limit is per
console process; the copy gate is real.

Meets census ruling region-ha-2a's conditions: private_ip comes from IMDS
or host.env, validated (IPv4, private, inside the region's network, not
loopback or link-local) and otherwise omitted; Server.boxRoster is
operator-only and read-only; the overlay loads last and every value is
recomputed on the build base. 13 tasks, every code block dry-run and
replayed onto a clean export.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Paul Campbell pushed by claude 5a17317b3b607b0c15e6ee0abcb33d2c374dc33e parent aa5b6b0
1 file changed, +1412 −570
  • docs/superpowers/plans/2026-10-16-region-ha-2a-core.md +1412 −570
diff --git a/docs/superpowers/plans/2026-10-16-region-ha-2a-core.md b/docs/superpowers/plans/2026-10-16-region-ha-2a-core.md
index fb9b0c93c..86857d493 100644
--- a/docs/superpowers/plans/2026-10-16-region-ha-2a-core.md
+++ b/docs/superpowers/plans/2026-10-16-region-ha-2a-core.md
@@ -1,14 +1,33 @@
-# Region HA 2a: The Provider-Neutral Core Implementation Plan
+# Region HA 2a: The Provider-Neutral Core Implementation Plan (rev 2)
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
-**Goal:** Build region HA spec 2's provider-neutral core, dormant behind `CARLOS_REGION_HA`: box and group roles, the placement policy, the create door's refusal when no box has room, the load balancer health answer, private-address forwarding with an address source, and the eligible-capacity metric.
+**Goal:** Build region HA spec 2's provider-neutral core, dormant behind `CARLOS_REGION_HA`: box and group roles, the placement policy, H27's no-room refusal on every door that chooses a box, the load balancer health answer, private-address forwarding with a validated address source, and the eligible-capacity metric.
-**Architecture:** Every new rule sits on spec 1's code without changing what it proves. The placement policy runs over `boxchoice.Eligible`'s identity-filtered candidates inside `hostops.ExecutorChooser`, which creates and relocation targets already share; roles live in the signed roster. The health answer is an in-memory state (`internal/lbhealth`) that the agent's own loops feed and the edge reads on `:80`. Forwarding changes only the address an edge dials. Each AWS part is an adapter in a file named for AWS (`internal/host/awsaddr.go`, `internal/console/awsregionmetric.go`), and a test holds the core to that.
+**Architecture:** Every new rule sits on spec 1's code without changing what it proves. The placement policy runs over `boxchoice.Eligible`'s identity-filtered candidates inside `hostops.ExecutorChooser`, which creates, relocation targets and (with region HA on) move destinations share; roles live in the signed roster. A create door chooses its box before it registers anything, so a refusal leaves nothing behind. The health answer is an in-memory state (`internal/lbhealth`) that the agent's own loops feed and the edge reads on `:80`. Forwarding changes only the address an edge dials. Each AWS part is an adapter in a file named for AWS (`internal/host/awsaddr.go`, `internal/console/awsregionmetric.go`), and a test holds the core to that.
**Tech Stack:** Go 1.26, module `github.com/carlosframework/platform`; `internal/hostactors`, `internal/boxchoice`, `internal/edgesfile`, `internal/hostops`, `internal/host`, `internal/edge`, `internal/activator`, `internal/console`, the `carlos` CLI (repo root); AWS SDK v2 CloudWatch in one adapter.
-**Spec:** `docs/superpowers/specs/2026-10-15-region-ha-two-box-regions-design.md` (rev 14, approved), with Paul's ruling **H27** (2026-10-16), which the spec's rev 15 will record: pending creates are dropped. Spec 1, for context: `docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md`.
+**Spec:** `docs/superpowers/specs/2026-10-15-region-ha-two-box-regions-design.md`, **rev 15** (approved), which records Paul's ruling **H27** (no pending creates; a refused request writes nothing and alerts; choose before registering; the four choosing doors; the alert limit is per console process) and settles §6's certificate readiness as the mirror's first load. Spec 1, for context: `docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md`.
+
+## Rev 2: Astra round 1 and census ruling region-ha-2a
+
+Rev 2 answers Astra's review of rev 1 (`docs/superpowers/plans/CODEX-REVIEW-REGION-HA-2A-1.md`, seven P2s, all accepted, with the coordinator's rulings now in spec rev 15's H27) and platform-1e's census ruling region-ha-2a (approved in principle with conditions, amadan #236).
+
+| # | Finding or condition | Rev 2 | Where |
+|---|---|---|---|
+| A1 | The no-capacity path left a registration behind; the test called `StartCreate` directly | Every create door chooses before `RegisterStart`: `beginClaimCreate` calls `chooseExecutorFirst`, which refuses before anything is registered and otherwise carries the choice to the create (`hostops.WithChosenExecutor`, honoured by `SelectExecutor`). Tested through the HTTP door and a recovery pass: no registration, no claim, nothing written by recovery | Task 11 |
+| A2 | H27 was missing from the other choosing doors | One refusal (`noCapacityError`, answered by `claimResponseRefusal`) on instance create, domain attach's first route (API and browser, through `attachClaimFirstRoute`, unchanged), standalone sidecar create, and the move door's destination. Each has an HTTP or door-level test. Sidecars attached to an instance choose nothing and are unchanged | Tasks 11, 12 |
+| A3 | Instance create's earlier region check refused before the chooser | With region HA on, an app-derived region is judged by the chooser's eligible set (`PlaceableBoxes`); an empty set is the no-room refusal. HTTP tests: every box stale (503 and an alert, not 400), and a misconfigured sibling beside an eligible box (the create goes ahead; 400 with region HA off, as on main) | Task 11 |
+| A4 | Move role filtering ran with region HA off | The move destination is filtered only through `placementChooser`, which exists only with region HA on; no roster is read otherwise. Test: host claims on, region HA off, a legacy destination and an unreadable roster, both main's answer | Task 12 |
+| A5 | `dialAddress` used private addresses with the reader's region HA off | The home map dials private only with the reader's own switch on (`homeMap.privateDial`). Tested both ways | Task 8 |
+| A6 | The alert limit did not survive a restart | H27 (rev 15) makes the limit per console process and accepts one extra alert after a restart. The limiter stays in memory; the wording says so | Task 11 |
+| A7 | Pending copy reached CLI help and docs | The copy gate is real: an id in `regionHACopyBuilt` must be an approved row. No pending row is ever marked approved for checking. Tasks 10 and 11 wait for their rows' approval | Tasks 1, 10, 11 |
+| B1 | `private_ip`: root-configured source; validated (IPv4, private, inside the region's VPC CIDR, not loopback or link-local); omitted and refused otherwise; routing data only; tests | The source is IMDS on AWS (the VPC's CIDR blocks from `network/interfaces/macs/<mac>/vpc-ipv4-cidr-blocks`) or `host.env` (`CARLOS_PRIVATE_IP` with a required `CARLOS_PRIVATE_CIDR`). `edgesfile.PrivateAddress` validates on both sides: the writer omits what fails, and a reader dials a private address only when it passes against the reader's own published network. Tests: a malformed, a metadata-range and an out-of-CIDR address, each omitted and refused | Tasks 7, 8 |
+| B2 | `Server.boxRoster`: operator-only, read-only, the same roster source, no role feeds root's claim authority | `boxRoster` is called only by the two role doors, past `apiSystemRoller`; the move door no longer reads roles through it (it uses the chooser) | Task 10 |
+| B3 | The overlay loads last; values recomputed on the actual build base, after Plan 14 Parts 2 and 3 | Stated in the census section below; the recompute tooling is kept | Census |
+
+Task numbers moved: the move door is its own task (12), and the metric is now Task 13.
## Scope
@@ -17,32 +36,39 @@ Spec 2 is four plans. This is **2a**. The others, and the seams 2a leaves them:
| Plan | Builds | 2a's seam |
|---|---|---|
| 2b | Joining mode, the membership-confirmation worker, the console's enrolment pass | `lbhealth.Default.SetHosting` (Task 9): 2b's mode selection owns it, and joining mode leaves it false. `hostactors.Directory.EnrollAutomatic` already gives an admitted box its group's role in the enrolling write (Task 3) |
-| 2c | The `region-host` module's group mode, the start-up script, SSM | The health path `/.carlos/lb-health` on port 80 (Task 9). The metric `Carlos/Region EligibleBoxes`, dimension `Region` = the placement code, in each region's own CloudWatch (Task 12). 2c grants the platform box's role `cloudwatch:PutMetricData` conditioned on `cloudwatch:namespace = Carlos/Region`, sets `CARLOS_REGION_METRICS=cloudwatch` on the console, and opens 443 from the region's VPC CIDR for private forwarding |
-| 2d | The §8 single-box fixes | The move door's destination already skips legacy and platform boxes (Task 10); 2d's rework of the door (source from the claim, §7's full choice for the destination, its own sentence for "every box here is excluded") replaces Task 10's interim filter |
+| 2c | The `region-host` module's group mode, the start-up script, SSM | The health path `/.carlos/lb-health` on port 80 (Task 9). The metric `Carlos/Region EligibleBoxes`, dimension `Region` = the placement code, in each region's own CloudWatch (Task 13): 2c grants the platform box's role `cloudwatch:PutMetricData` conditioned on `cloudwatch:namespace = Carlos/Region`, and sets `CARLOS_REGION_METRICS=cloudwatch` on the console. The box role's IMDS access must cover `network/interfaces/macs/*` (Task 7), and the security group opens 443 from the region's VPC CIDR |
+| 2d | The §8 single-box fixes | The move door's destination already takes §7's placeable set with region HA on (Task 12); 2d's rework of the door (its multi-box regions, the source from the claim) builds on it |
-**Ruling H27 (Paul, 2026-10-16).** Spec 2's §7 "Unplaced" section (pending creates, the `starting` state, a caller-supplied operation id, reconciliation, a shared reservation, a placement retry pass) is dropped. In its place: when the placement policy finds no eligible box, the create door refuses the create with a 503 and a clear message, and fires the operator alert at most once per region an hour. No executor-less create is written, so the claim is not touched. The eligible-capacity metric stays, published by a small periodic console pass (Task 12).
-
-**Not in 2a:** forwarding of platform routes from a group box to the platform box (§6 "Platform routes"), the region ingress list in the console (§6 "What the console tells customers"), and every row of §8; those are 2d's or 2c's. The sidecar and first-route create doors keep today's refusal when no box is eligible: only the instance create door gets H27's 503 (open question 2).
+**Not in 2a:** forwarding of platform routes from a group box to the platform box (§6 "Platform routes"), the region ingress list in the console (§6), and every row of §8; those are 2d's or 2c's.
## Global Constraints
-- Everything new is dormant with `CARLOS_REGION_HA` off: with it off, every door, file, edge and box behaves exactly as on main. The health answer is not served, the role doors answer 404, the edges file publishes no `private_ip` and keeps its EIP hold, the no-capacity refusal is inert, and the capacity pass does not start.
+- Everything new is dormant with `CARLOS_REGION_HA` off: with it off, every door, file, edge and box behaves exactly as on main. The health answer is not served, the role doors answer 404, the edges file publishes no `private_ip` and keeps its EIP hold, the home map dials public addresses only, the create and move doors choose as on main (no early choice, no placeable filter, no roster read), and the capacity pass does not start.
- Multi-box regions stay AWS-only until #299 (H26). Spec 2's new core files import no `github.com/aws/...` package; AWS parts are adapters in files named for AWS (§25). `region_ha_boundary_test.go` holds both, plus `s3compat.Options()` on every S3 client in the packages 2a touches.
-- Wire shapes are additive: `private_ip`, `mem_total_mib`, `disk_total_mib` (edges file), `group` and `role` (roster box), `role` (roster region) are all `omitempty`, and a reader that predates them is unaffected.
-- The roster's default role is stored as `""`. Role values: box `""`, `legacy`, `platform`; group `""`, `legacy`. A role is never `platform` for a group.
+- Wire shapes are additive: `private_ip`, `private_cidr`, `mem_total_mib`, `disk_total_mib` (edges file), `group` and `role` (roster box), `role` (roster region) are all `omitempty`, and a reader that predates them is unaffected.
+- `private_ip` is routing data only: an edge dials it, and nothing grants anything on it. It comes only from IMDS or root-owned `host.env`, never from a file the agent can write, and it is published and dialled only when `edgesfile.PrivateAddress` passes it.
+- The roster's default role is stored as `""`. Role values: box `""`, `legacy`, `platform`; group `""`, `legacy`. Roles are read by the placement chooser alone and never feed root's claim authority.
- §7's thresholds: free memory at least 15% of total (`boxchoice.MinMemPercent`), free disk at least 10% of the volume (`MinDiskPercent`), an edges file no older than 15 minutes (`MaxEdgesAge`). Choice: spread, then most `mem_free_mib`, then the P5-D2 hash.
+- H27: when no box qualifies, every door that chooses a box (instance create, domain attach's first route, standalone sidecar create, the move door's destination) answers 503 with RHA-C17, chooses before it registers anything, and alerts at most once per region an hour per console process.
- §6's health answer: `GET` or `HEAD` `/.carlos/lb-health`, any host, port 80. 200 only when hosting, forwarding armed (map and key each loaded once, latched), the certificate mirror loaded once (latched), the edge's and the activator's beats within 60 seconds, and a home map refresh attempt completed within 5 minutes.
-- User-facing copy is in `docs/superpowers/plans/2026-10-16-region-ha-2a-copy.md`, gated by `approved_region_ha_copy_test.go`. No em dashes. No production console turns `CARLOS_REGION_HA` on before Paul approves every row.
+- User-facing copy is in `docs/superpowers/plans/2026-10-16-region-ha-2a-copy.md`, gated by `approved_region_ha_copy_test.go`. No pending row reaches code, CLI help or docs. No em dashes.
- Go commands: `export GOFLAGS=-buildvcs=false GOCACHE=$TMPDIR/gocache`. Run tests with `nice -n 19` and a `-run` filter; never the whole `internal/host` suite (CI's `make test` is the full run). Tests that bind unix sockets fail inside the sandbox ("socket: operation not permitted"); that is the sandbox, not the change.
-- Census: every task that changes a non-test Go file under a package directory re-pins that directory's `@coverage` row in `internal/provision/testdata/plan14-root-reads-region-ha-2a.tsv`. Tasks 7 and 10 need platform-1e (below).
+- Census: every task that changes a non-test Go file under a package directory re-pins that directory's `@coverage` row in `internal/provision/testdata/plan14-root-reads-region-ha-2a.tsv`.
+
+## Census items for platform-1e (census ruling region-ha-2a)
-## Census rulings this plan asks platform-1e for
+Approved in principle with conditions (amadan #236). The overlay asks for exactly these:
-- **Task 7** re-digests one authority row, `internal/host/edgesfile.go:edgesFileWriteClaims` (class authority, task F,J1, basis and sites unchanged). No seal moves.
-- **Task 10** adds one accepted-legacy unit, `internal/console/apiboxroles.go:Server.boxRoster` (the console's `Directory.Read` of the signed roster), with basis `accepted-legacy;rev3=J4;basis=root-input;census-ruling-region-ha-2a`. The seal moves from `fc38db00…` to `a57250c3…`.
-- Every other task is envelope-only. No legacy row changes.
+| Task | Unit | Class | Change | Seal |
+|---|---|---|---|---|
+| 7 | `internal/host/edgesfile.go:edgesFileWriteClaims` | authority, F,J1 (kept) | Re-digest, class, task, basis and sites unchanged: with region HA on it reads the address source before every publication | stands |
+| 10 | `internal/console/apiboxroles.go:Server.boxRoster` | accepted-legacy, J4 root-input | New: the role doors' read of the signed roster (`Directory.Read`), operator-only, read-only | moves |
+| 11 | `internal/console/claim_create.go:Server.beginClaimCreate` | accepted-legacy, J4 root-input | Re-freeze: one call (`chooseExecutorFirst`) before `RegisterStart`; its one read site unchanged | moves |
+| 12 | `internal/console/apiinstancemove.go:Server.handleAPIInstanceMoveRequest` | accepted-legacy | Re-freeze: one more refusal before any write (no room in the destination); read sites unchanged, offsets moved | moves |
-The census values below were computed at this branch's base, main `70e57449`. Main has moved since (`a89e35ca` adds overlays `plan14-root-reads-seed.tsv` and `plan14-root-reads-home-part2.tsv`, a new seal, and changes in `internal/host`, `internal/activator` and `internal/console`). On a newer base, keep the overlay last in the load order and recompute every value with the tooling below; the rows to write stay the same.
+`Server.attachClaimFirstRoute` is not changed: the first route's refusal comes through `beginClaimCreate` and `claimResponseRefusal`, which is not a root-read unit. Record-write census (B8) digests change for `Directory.Replace` (Task 3), `registerCLIRoutes` (Task 10), `handleAPIInstanceCreate` (Task 11) and `handleAPIInstanceMoveRequest` (Task 12).
+
+**Build base (condition 3).** `plan14-root-reads-region-ha-2a.tsv` loads last at build time. The values in this plan were computed at this branch's base, main `70e57449`; the build base will be later (after Plan 14 Parts 2 and 3 land), so recompute every value there with the tooling below: each `@coverage` row, each B8 digest, each unit's kinds and digest, each `@legacy` fingerprint, and each seal. The rows to write stay the same.
## Census tooling
@@ -240,11 +266,11 @@ nice -n 19 go test ./internal/provision/ -run 'TestPlan9B8RecordWriteCensus|Test
2. **IMDS answering blank on one pass** must publish no public address rather than an old one, and the box then drops out of `Eligible` until the next pass, never forwarding to an address it no longer holds. Task 7's `TestEdgesFilePublishesOnlyWhatTheSourceReports`.
3. **An edges file stamped by a box whose clock runs ahead** is believed for a minute of skew and no more; a stale or unreadable generation is never trusted. Task 4's `TestPlaceableAppliesEachRule` (the future and unreadable cases).
4. **A bucket outage after a box has armed** must leave its health answer at 200 (no termination for a shared outage), while a refresh stuck inside one attempt turns it 503. Task 2's `TestBucketOutageAfterArmingStaysHealthy` and Task 9's `TestHomeMapFeedsTheHealthState`.
-5. **A client retrying a refused create in a loop** must not page the operator on every retry. Task 11's `TestCreateWithNoEligibleBoxIsRefusedAndAlertedOncePerHour`.
+5. **A client retrying a refused create in a loop** must not page the operator on every retry, and must leave nothing for recovery each time. Task 11's `TestCreateDoorsRefuseWithNoRoomBeforeRegistering` (three refusals, one page, no registration, no claim after a recovery pass).
## Dry run
-Every code block below was written into this branch at `47b40803` (main `70e57449`), built, vetted with `GOOS=linux` and `GOOS=darwin` (and built with `GOOS=windows`), and its tests run as each task's steps say, task by task in order. The census values are those runs'. The code was then removed.
+Every code block below was written into this branch at `aa5b6b07` (main `70e57449`), task by task in order, built, vetted with `GOOS=linux` and `GOOS=darwin` (and built with `GOOS=windows`), and its tests run as each task's steps say; the census values are those runs'. The copy gate was checked without approving anything: with every row pending, `TestRegionHABuiltCopyIsApplied`, `TestRegionHACopyRecordIsComplete` and `TestRegionHACopyHasNoEmDash` pass, and `TestRegionHABuiltCopyIsApproved` fails naming exactly the built ids (RHA-C1 to RHA-C16 after Task 10, RHA-C1 to RHA-C19 after Task 11). That failure is what an implementer sees until Paul approves the rows. The whole `TestClaimConsoleMutationInvariant` ran once on the final tree. The plan text was then replayed onto a clean export, task by task, and every intermediate state built and passed both censuses. The code was then removed.
---
@@ -255,10 +281,10 @@ Every code block below was written into this branch at `47b40803` (main `70e5744
- Create: `approved_region_ha_copy_test.go`
**Interfaces:**
-- Consumes: the moves gate's helpers at the repo root (`parseCopyTable`, `checkMovesInventory`, `checkMovesNoEmDash`, `checkRunApplied`, `reportAll`, `copyRow`).
-- Produces: `regionHACopyWired []string`, which Tasks 10 and 11 extend with the ids whose text they write; the record's ids `RHA-C1` to `RHA-C19`.
+- Consumes: the run and moves gates' helpers at the repo root (`parseCopyTable`, `checkMovesInventory`, `checkMovesNoEmDash`, `checkRunBuilt`, `checkRunApplied`, `builtRows`, `reportAll`, `copyRow`).
+- Produces: `regionHACopyBuilt []string`, which Tasks 10 and 11 extend with the ids whose text they write; the record's ids `RHA-C1` to `RHA-C19`.
-Every user-facing string 2a writes is drafted here first, all `pending`. The gate requires a wired row's text to be in its file, whatever its status: a pending string may be wired because each is reached only with `CARLOS_REGION_HA` on.
+Every user-facing string 2a writes is drafted here first, all `pending`. The gate is real: an id in `regionHACopyBuilt` must be an `approved` row, and its text must be in the file the row names. Nothing marks a pending row approved for checking.
- [ ] **Step 1: Write the record**
@@ -276,21 +302,21 @@ moves record's: one table per heading, columns id, file, text, status.
The rules:
- A row is `pending` until Paul approves it in the copy review (the
- `copy-review` skill). His edits are applied verbatim, here and in the
- file the row names, in one commit, and the row becomes `approved`.
-- A `pending` string may be wired while region HA is dormant. Every
- string here is reached only with `CARLOS_REGION_HA` on (the role doors
- answer 404 without it, and the no-capacity refusal is inert), so no
- production console turns region HA on before every row is `approved`.
-- A task that writes a row's text adds its id to `regionHACopyWired` in
- the gate in the same commit; from then on the gate fails unless the
- text is in the file the row names, whatever its status.
+ `copy-review` skill). His edits are applied to the text here verbatim
+ and the row becomes `approved`.
+- No pending string reaches code, CLI help or docs. The task that writes
+ a row's text adds its id to `regionHACopyBuilt` in the gate in the same
+ commit, and from then on the gate fails unless the row is `approved`
+ and its text is in the file it names. Tasks 10 and 11 therefore wait
+ for their rows' approval.
- In a Go file the text is one string literal, or one `fmt.Sprintf`
- format, equal to the row with `%s` or `%d` where the row has a
- `<placeholder>`. A row whose file cell says "a fragment" is a piece of
- a larger literal (the `carlos system` usage text).
-- No em dashes. A full stop and a new sentence instead. Console refusals
- start lower case, as the console's other CLI refusals do.
+ format, equal to the row with `%s` where the row has a `<placeholder>`.
+ A row whose file cell says "a fragment" is a piece of a larger literal
+ (the `carlos system` usage text).
+- No em dashes. A full stop and a new sentence instead. The role doors'
+ refusals start lower case, as the console's other CLI refusals do; the
+ no-capacity refusal is shared with the browser's domain door, so it is
+ a capitalised sentence, as `claimResponseUnavailable` is.
Placeholders: `<box>` a box's roster reference or label; `<role>` a role
name (default, legacy or platform); `<region>` a placement code or the
@@ -317,13 +343,16 @@ home region's name.
| RHA-C15 | docs/site/cli.md | Box and group roles | pending |
| RHA-C16 | docs/site/cli.md | carlos system box role says whether new instances may be placed on a box. A box with the default role can take them. A legacy or platform box keeps the instances it has and takes no new ones: no new instance, relocation or move lands on it. carlos system group role sets the role of every box a region's group has enrolled, and of every box it enrolls later. Both work only on a console with region HA on. | pending |
-## A create no box can take (Task 11)
+## No box has room (Task 11)
+
+One refusal for every door that chooses a box (H27): instance create,
+domain attach, standalone sidecar create and the move door's destination.
| id | file | text | status |
|---|---|---|---|
-| RHA-C17 | internal/console/nocapacity.go | no box in <region> can take a new instance right now. Try again in a few minutes. | pending |
-| RHA-C18 | internal/console/nocapacity.go (the operator alert's subject) | No room for new instances in <region> | pending |
-| RHA-C19 | internal/console/nocapacity.go (the operator alert's body) | A create in <region> was refused because no box there can take a new instance. Every box is full, stale or excluded by its role. Add a box, or free memory or disk on one, and check the eligible-capacity alarm for <region>. | pending |
+| RHA-C17 | internal/console/nocapacity.go | No box in <region> has room right now. Try again in a few minutes. | pending |
+| RHA-C18 | internal/console/nocapacity.go (the operator alert's subject) | No box in <region> has room | pending |
+| RHA-C19 | internal/console/nocapacity.go (the operator alert's body) | A request in <region> was refused because no box there can take it. Every box is full, stale or excluded by its role. Add a box, or free memory or disk on one, and check the eligible-capacity alarm for <region>. | pending |
````
- [ ] **Step 2: Write the gate**
@@ -335,7 +364,6 @@ home region's name.
package main
import (
- "fmt"
"os"
"regexp"
"strconv"
@@ -344,15 +372,16 @@ import (
// The copy gate of region HA plan 2a (Task 1). The record is a markdown
// table in the moves record's format, read with the moves gate's parser
-// and checked with its helpers:
+// and checked with the run gate's helpers, so "approved" and "applied"
+// mean what they mean there:
//
// - TestRegionHACopyRecordIsComplete: the record names exactly RHA-C1 to
// RHA-C19;
// - TestRegionHACopyHasNoEmDash;
-// - TestRegionHAWiredCopyIsApplied: a wired row's text is in the file it
-// names, whatever its status. A pending row may be wired because every
-// one of these strings is reached only with CARLOS_REGION_HA on, and no
-// production console turns it on before Paul approves the record.
+// - TestRegionHABuiltCopyIsApproved: an id in regionHACopyBuilt is an
+// approved row. No pending string reaches code, CLI help or docs;
+// - TestRegionHABuiltCopyIsApplied: a built row's text is in the file it
+// names.
const regionHACopyRecordPath = "docs/superpowers/plans/2026-10-16-region-ha-2a-copy.md"
var regionHACopyIDPattern = regexp.MustCompile(`^RHA-C[0-9]+$`)
@@ -366,9 +395,10 @@ var regionHACopyIDs = func() []string {
return ids
}()
-// regionHACopyWired is every id whose text the code or docs carry. The
-// task that writes a row's text adds its id here in the same commit.
-var regionHACopyWired = []string{}
+// regionHACopyBuilt is every id whose text the code or docs carry. The
+// task that writes a row's text adds its id here in the same commit, once
+// Paul has approved the row.
+var regionHACopyBuilt = []string{}
func mustParseRegionHARecord(t *testing.T) []copyRow {
t.Helper()
@@ -383,33 +413,6 @@ func mustParseRegionHARecord(t *testing.T) []copyRow {
return rows
}
-// regionHAWiredRows is the wired rows, each marked approved so the moves
-// gate's application check reads them: a wired pending row must be in its
-// file exactly as an approved one would be.
-func regionHAWiredRows(rows []copyRow, wired []string) ([]copyRow, []string) {
- byID := map[string]copyRow{}
- for _, r := range rows {
- byID[r.ID] = r
- }
- var out []copyRow
- var problems []string
- seen := map[string]bool{}
- for _, id := range wired {
- if seen[id] {
- problems = append(problems, fmt.Sprintf("regionHACopyWired names %s twice", id))
- }
- seen[id] = true
- r, ok := byID[id]
- if !ok {
- problems = append(problems, fmt.Sprintf("regionHACopyWired names %s, which the record has no row for", id))
- continue
- }
- r.Status = "approved"
- out = append(out, r)
- }
- return out, problems
-}
-
func TestRegionHACopyRecordIsComplete(t *testing.T) {
reportAll(t, checkMovesInventory(mustParseRegionHARecord(t), regionHACopyIDs))
}
@@ -418,20 +421,23 @@ func TestRegionHACopyHasNoEmDash(t *testing.T) {
reportAll(t, checkMovesNoEmDash(mustParseRegionHARecord(t)))
}
-func TestRegionHAWiredCopyIsApplied(t *testing.T) {
- rows, problems := regionHAWiredRows(mustParseRegionHARecord(t), regionHACopyWired)
- reportAll(t, append(problems, checkRunApplied(rows, ".")...))
+func TestRegionHABuiltCopyIsApproved(t *testing.T) {
+ reportAll(t, checkRunBuilt(mustParseRegionHARecord(t), regionHACopyBuilt))
+}
+
+func TestRegionHABuiltCopyIsApplied(t *testing.T) {
+ reportAll(t, checkRunApplied(builtRows(mustParseRegionHARecord(t), regionHACopyBuilt), "."))
}
````
- [ ] **Step 3: Run it**
-Run: `nice -n 19 go test . -run 'RegionHACopy|RegionHAWired' -count=1`
-Expected: PASS (no row is wired yet).
+Run: `nice -n 19 go test . -run 'RegionHACopy|RegionHABuilt' -count=1`
+Expected: PASS (nothing is built yet).
- [ ] **Step 4: Put the record in front of Paul**
-Run the `copy-review` skill over the record's rows. Apply his edits verbatim to the record in its own commit, and carry them into the code as Tasks 10 and 11 wire each row. Implementation does not wait for the review: the rows are dormant until region HA is turned on.
+Run the `copy-review` skill over the record's rows. Apply his edits verbatim to the record, mark each row he approves `approved`, and commit that on its own. Tasks 10 and 11 wire their rows only once they are approved, with Paul's text: where his text differs from a draft below, use his in the code too. Tasks 2 to 9 do not wait.
- [ ] **Step 5: Commit**
@@ -455,7 +461,7 @@ No package envelope changes: neither file is a non-test Go file.
**Interfaces:**
- Produces (`internal/lbhealth`): `const Path = "/.carlos/lb-health"`; `HeartbeatWindow = 60 * time.Second`; `RefreshWindow = 5 * time.Minute`; the condition names `CondHosting`, `CondForwarding`, `CondCerts`, `CondEdge`, `CondActivator`, `CondRefresh`; `type State`; `var Default = &State{}`; methods (all safe on a nil `*State`) `SetHosting(bool)`, `Hosting() bool`, `MapLoaded()`, `KeyLoaded()`, `CertsLoaded()`, `EdgeBeat(time.Time)`, `RefreshAttempted(time.Time)`, `SetActivator(func() time.Time)`, `Failing(now time.Time) []string`; `func Serve(w http.ResponseWriter, r *http.Request, s *State, now time.Time) bool`.
- Produces (repo root): `regionHACoreFiles`, `regionHAAdapterFiles`, `regionHAPackages`, `regionHAS3Exceptions` in `region_ha_boundary_test.go`, which later tasks extend.
-- Produces: the overlay file every later task's census step edits, and the `seal = ...` line after it, which Task 10 changes.
+- Produces: the overlay file every later task's census step edits, and the `seal = ...` line after it, which Tasks 10, 11 and 12 change.
- [ ] **Step 1: Write the failing tests**
@@ -786,7 +792,7 @@ func f(cfg any) { _ = s3.NewFromConfig(cfg, s3compat.Options()) }
- [ ] **Step 2: Run them to see them fail**
Run: `nice -n 19 go test ./internal/lbhealth/ -count=1`
-Expected: FAIL, the package does not build (`undefined: State`). `go test . -run TestRegionHA` fails the same way on `internal/lbhealth/lbhealth.go` not existing.
+Expected: FAIL, the package does not build (`undefined: State`). `go test . -run 'TestRegionHA(Core|Adapters|S3)'` fails the same way on `internal/lbhealth/lbhealth.go` not existing.
- [ ] **Step 3: Write the package**
@@ -997,7 +1003,7 @@ func Serve(w http.ResponseWriter, r *http.Request, s *State, now time.Time) bool
- [ ] **Step 4: Run the tests**
-Run: `nice -n 19 go test ./internal/lbhealth/ -count=1 && nice -n 19 go test . -run TestRegionHA -count=1`
+Run: `nice -n 19 go test ./internal/lbhealth/ -count=1 && nice -n 19 go test . -run 'TestRegionHA(Core|Adapters|S3)' -count=1`
Expected: PASS. The S3 call-site test passes with `internal/host/claimproduction_marker.go` excepted for #298.
- [ ] **Step 5: Census**
@@ -1609,7 +1615,7 @@ Then set these rows. A `@coverage` row replaces the overlay's row for the same d
@coverage internal/hostactors 6736698380a247764ada687f2476f89382fc32a1183a50c06afe75967d716eab
````
-Run the census checks and `go test . -run TestRegionHA -count=1`. Expected: PASS.
+Run the census checks and `go test . -run 'TestRegionHA(Core|Adapters|S3)' -count=1`. Expected: PASS.
- [ ] **Step 6: Commit**
@@ -2040,7 +2046,7 @@ Then set these rows. A `@coverage` row replaces the overlay's row for the same d
@coverage internal/edgesfile da42c799d62e065817b5cce4f9a3ead1308dbd379615a85a7b14533c16d7e804
````
-Run the census checks and `go test . -run TestRegionHA -count=1`. Expected: PASS.
+Run the census checks and `go test . -run 'TestRegionHA(Core|Adapters|S3)' -count=1`. Expected: PASS.
- [ ] **Step 6: Commit**
@@ -2222,13 +2228,13 @@ git commit -m "region HA 2a: edges files carry total memory and disk"
### Task 6: Creates and relocation targets choose by the policy
**Files:**
-- Modify: `internal/hostops/executor.go` (`Choose`, `Permits`, `Capacity`, `placeable`)
+- Modify: `internal/hostops/executor.go` (`Choose`, `Permits`, `PlaceableBoxes`, `Capacity`, `placeable`)
- Modify: `internal/hostops/executor_test.go`, `internal/hostrelocate/chooser_test.go` (fixtures publish roomy, fresh headroom)
- Modify: the overlay
**Interfaces:**
- Consumes: `boxchoice.Placeable`, `boxchoice.ChooseRoomiest` (Task 4); `hostactors.Box.Role` (Task 3).
-- Produces: `func (x ExecutorChooser) Capacity(ctx context.Context, placement string) (int, error)`, which Task 12 publishes; `Choose` and `Permits` now apply the roles and thresholds. `ExecutorChooser` exists only with `CARLOS_REGION_HA` on (the console binds it only then, and the box only in its region HA relocation runtime), so the policy is dormant with it off.
+- Produces: `func (x ExecutorChooser) PlaceableBoxes(ctx context.Context, placement string) ([]string, error)` (Tasks 11 and 12 read it) and `func (x ExecutorChooser) Capacity(ctx context.Context, placement string) (int, error)` (Task 13 publishes it); `Choose` and `Permits` now apply the roles and thresholds. `ExecutorChooser` exists only with `CARLOS_REGION_HA` on (the console binds it only then, and the box only in its region HA relocation runtime), so the policy is dormant with it off.
`ExecutorChooser.eligible` is an accepted-legacy unit (merge ruling 2) and does not change: the policy runs over its result. Relocation's `RegionChooser` uses `Choose` and `Permits`, so relocation targets follow the same policy.
@@ -2264,7 +2270,7 @@ git commit -m "region HA 2a: edges files carry total memory and disk"
absent := hostclaim.Observed{Host: "shop.bab.oncarlos.com", Absent: true}
first, err := x.Choose(ctx, absent, chooserRecord())
if err != nil {
-@@ -85,10 +90,77 @@
+@@ -85,10 +90,80 @@
if got, _ := x.Choose(ctx, absent, chooserRecord()); got.Box != other {
t.Fatalf("soft spread: got %s, want %s", got.Box, other)
}
@@ -2334,6 +2340,9 @@ git commit -m "region HA 2a: edges files carry total memory and disk"
+ if n, err := x.Capacity(ctx, "ie"); err != nil || n != 1 {
+ t.Fatalf("Capacity = %d, %v; want 1", n, err)
+ }
++ if boxes, err := x.PlaceableBoxes(ctx, "ie"); err != nil || len(boxes) != 1 || boxes[0] != "box-b" {
++ t.Fatalf("PlaceableBoxes = %v, %v; want [box-b]", boxes, err)
++ }
+ })
+ }
+ x, _, _, _ := chooserFixture(t, "box-a", "box-b")
@@ -2382,7 +2391,7 @@ git commit -m "region HA 2a: edges files carry total memory and disk"
- [ ] **Step 2: Run them to see them fail**
Run: `nice -n 19 go test ./internal/hostops/ -run 'Executor|Chooser' -count=1`
-Expected: FAIL to build (`x.Capacity undefined`).
+Expected: FAIL to build (`x.Capacity undefined`, `x.PlaceableBoxes undefined`).
- [ ] **Step 3: Apply the policy**
@@ -2411,7 +2420,7 @@ Expected: FAIL to build (`x.Capacity undefined`).
if !ok {
return hostclaim.ExecutorSelection{}, ErrNoEligibleBox
}
-@@ -69,18 +71,44 @@
+@@ -69,18 +71,56 @@
// in r's placement now (spec 1 §2, §5.8 "The start CAS ... refused ...
// when `to` is not eligible for the host"): the same read Choose makes,
// without choosing. A stale edges file, a missing capability or secrets
@@ -2434,17 +2443,29 @@ Expected: FAIL to build (`x.Capacity undefined`).
return fmt.Errorf("%w: %s", ErrNoEligibleBox, box)
+}
+
-+// Capacity is how many boxes in placement could take a new instance in
-+// the chooser's namespace and pool now: Choose's set, before the choice.
-+// The console publishes it per region as the eligible-capacity metric
-+// (region HA spec 2 §4, A1-11).
-+func (x ExecutorChooser) Capacity(ctx context.Context, placement string) (int, error) {
++// PlaceableBoxes is every box in placement that could take a new
++// instance in the chooser's namespace and pool now, sorted: Choose's set,
++// before the choice. The console's move door takes a move's destination
++// from it (region HA spec 2 §7, H27).
++func (x ExecutorChooser) PlaceableBoxes(ctx context.Context, placement string) ([]string, error) {
+ r := Record{Holder: hostclaim.NewHolder{Region: placement}}
+ o, cands, err := x.eligible(ctx, hostclaim.Observed{Absent: true}, r)
+ if err != nil {
-+ return 0, err
++ return nil, err
+ }
-+ return len(x.placeable(o, cands)), nil
++ boxes := []string{}
++ for _, c := range x.placeable(o, cands) {
++ boxes = append(boxes, c.BoxID)
++ }
++ return boxes, nil
++}
++
++// Capacity is how many boxes PlaceableBoxes names. The console publishes
++// it per region as the eligible-capacity metric (region HA spec 2 §4,
++// A1-11).
++func (x ExecutorChooser) Capacity(ctx context.Context, placement string) (int, error) {
++ boxes, err := x.PlaceableBoxes(ctx, placement)
++ return len(boxes), err
+}
+
+// placeable is region HA spec 2's policy over the eligible set (§7):
@@ -2471,13 +2492,13 @@ Expected: PASS.
Add this comment line after the overlay's last comment line:
````text
-# Task 6: hostops.ExecutorChooser applies spec 2's policy in Choose and Permits (creates and relocation targets) and counts it in Capacity. ExecutorChooser.eligible, the accepted-legacy unit (merge ruling 2), is unchanged: the policy runs over its result. Envelope-only, and the legacy seal stands.
+# Task 6: hostops.ExecutorChooser applies spec 2's policy in Choose and Permits (creates and relocation targets) and names the placeable set in PlaceableBoxes and Capacity. ExecutorChooser.eligible, the accepted-legacy unit (merge ruling 2), is unchanged: the policy runs over its result. Envelope-only, and the legacy seal stands.
````
Then set these rows. A `@coverage` row replaces the overlay's row for the same directory, or is appended when there is none; every other row is appended:
````text
-@coverage internal/hostops cede4be93f73d3e14547b19a811dd7fa5d8978d08c7fcf5a91bad80dd5176118
+@coverage internal/hostops 3dc23aaff7bdae0341a8c9a6fbac076a2f57ca1e4676d83543433ff9ad7d1c79
````
Run the census checks. Expected: PASS.
@@ -2491,26 +2512,72 @@ git commit -m "region HA 2a: creates and relocations choose by the placement pol
---
-### Task 7: The box's own addresses, and `private_ip` in the edges file
+### Task 7: The box's own addresses, validated, and `private_ip` in the edges file
**Files:**
-- Create: `internal/host/boxaddress.go` (core: `BoxAddresses`, `AddressSource`, the `CARLOS_PRIVATE_IP` / `CARLOS_PUBLIC_IP` source)
-- Create: `internal/host/awsaddr.go` (the AWS adapter: IMDSv2 `local-ipv4` and `public-ipv4`)
+- Create: `internal/edgesfile/private.go`, `internal/edgesfile/private_test.go` (`PrivateAddress`, the one validator the writer and the reader share)
+- Create: `internal/host/boxaddress.go` (core: `BoxAddresses`, `AddressSource`, the `host.env` source)
+- Create: `internal/host/awsaddr.go` (the AWS adapter: IMDSv2 `local-ipv4`, `public-ipv4`, `mac` and the VPC's `vpc-ipv4-cidr-blocks`)
- Create: `internal/host/boxaddress_test.go`
- Modify: `internal/host/edgesclaims.go` (`edgesClaims.addrs`)
-- Modify: `internal/host/edgesfile.go` (`edgesFile.PrivateIP`; `edgesFileWriteClaims` reads the source with region HA on)
+- Modify: `internal/host/edgesfile.go` (`edgesFile.PrivateIP`, `PrivateCIDR`; `edgesFileWriteClaims` reads the source with region HA on)
- Modify: `internal/host/edgesclaims_test.go`, `region_ha_boundary_test.go`, the overlay
**Interfaces:**
- Consumes: `imdsToken`, `imdsLeaf` (agent.go, unchanged).
-- Produces: `type BoxAddresses struct{ Private, Public string }`; `type AddressSource interface{ BoxAddresses() BoxAddresses }`; `envAddresses{getenv, provider}`; `imdsAddresses`; `func boxAddressSource(getenv func(string) string) AddressSource`; `edgesFile.PrivateIP` (json `private_ip`, omitempty), which Task 8 reads.
+- Produces: `func edgesfile.PrivateAddress(addr string, cidrs []string) (string, string)`; `type BoxAddresses struct{ Private, PrivateCIDR, Public string }`; `type AddressSource interface{ BoxAddresses() BoxAddresses }`; `envAddresses{getenv, provider}`; `imdsAddresses`; `func boxAddressSource(getenv func(string) string) AddressSource`; `edgesFile.PrivateIP`, `edgesFile.PrivateCIDR` (json `private_ip`, `private_cidr`, omitempty), which Task 8 reads.
+
+Census ruling region-ha-2a's first condition, in full:
+- **Source.** The address comes from the same root-configured source as the rest of the box's entry: IMDS on AWS, or root-owned `host.env` otherwise (`CARLOS_PRIVATE_IP` with `CARLOS_PRIVATE_CIDR`, which is required beside it; `CARLOS_PUBLIC_IP`). Never from a file the agent can write.
+- **Validation before writing** (`PrivateAddress`): a well-formed IPv4 address, in a private range, inside the region's VPC CIDR (from IMDS `network/interfaces/macs/<mac>/vpc-ipv4-cidr-blocks` on AWS, `CARLOS_PRIVATE_CIDR` otherwise), and not loopback, link-local or 169.254.0.0/16. Otherwise it is omitted, and forwarding to the box stays public.
+- **Routing data only.** It is dialled, and nothing grants anything on it.
-With region HA on (the edges claims cache exists exactly then), every publication reads the address source afresh and publishes only what it reports: a blank read publishes no address, instead of holding the last EIP, which may since have moved to the region's load balancer (A1-12). With region HA off the publisher is main's, EIP hold included. `CARLOS_PRIVATE_IP` and `CARLOS_PUBLIC_IP`, when set, win field by field over IMDS; a set value that is not an IPv4 address publishes nothing for its field.
+With region HA on (the edges claims cache exists exactly then), every publication reads the address source afresh and publishes only what it reports: a blank read publishes no address, instead of holding the last EIP, which may since have moved to the region's load balancer (A1-12). With region HA off the publisher is main's, EIP hold included. A host.env setting, when present, is the answer for its field, and one that does not validate publishes nothing rather than fall back to IMDS.
-**Census (flagged):** `edgesFileWriteClaims` is a kept authority row (F,J1). Its sites do not move; its digest does. The overlay re-digests it with its class, task and basis unchanged, and asks platform-1e to confirm. No seal moves.
+**Census (platform-1e):** `edgesFileWriteClaims` is a kept authority row (F,J1). Its sites do not move; its digest does. The overlay re-digests it with its class, task and basis unchanged. No seal moves.
- [ ] **Step 1: Write the failing tests**
+`internal/edgesfile/private_test.go`:
+
+````go
+package edgesfile
+
+import "testing"
+
+// Census ruling region-ha-2a: a private address passes only when it is a
+// well-formed private IPv4 address inside the region network's CIDR, and
+// never loopback or link-local (the metadata range included).
+func TestPrivateAddress(t *testing.T) {
+ vpc := []string{"10.0.0.0/16"}
+ cases := []struct {
+ name, addr string
+ cidrs []string
+ want, cidr string
+ }{
+ {"inside the region network", "10.0.1.5", vpc, "10.0.1.5", "10.0.0.0/16"},
+ {"inside the second block", "172.31.4.9", []string{"10.0.0.0/16", " 172.31.0.0/16 "}, "172.31.4.9", "172.31.0.0/16"},
+ {"malformed", "10.0.1", vpc, "", ""},
+ {"a hostname", "box.internal", vpc, "", ""},
+ {"the metadata range", "169.254.169.254", []string{"169.254.0.0/16"}, "", ""},
+ {"loopback", "127.0.0.1", []string{"127.0.0.0/8"}, "", ""},
+ {"outside the region network", "10.1.1.5", vpc, "", ""},
+ {"a public address", "203.0.113.4", []string{"203.0.113.0/24"}, "", ""},
+ {"IPv6", "fd00::5", []string{"fd00::/8"}, "", ""},
+ {"no network to check it against", "10.0.1.5", nil, "", ""},
+ {"a malformed network", "10.0.1.5", []string{"10.0.0.0/33"}, "", ""},
+ }
+ for _, c := range cases {
+ t.Run(c.name, func(t *testing.T) {
+ got, cidr := PrivateAddress(c.addr, c.cidrs)
+ if got != c.want || cidr != c.cidr {
+ t.Fatalf("PrivateAddress(%q, %v) = %q, %q; want %q, %q", c.addr, c.cidrs, got, cidr, c.want, c.cidr)
+ }
+ })
+ }
+}
+````
+
`internal/host/boxaddress_test.go`:
````go
@@ -2546,42 +2613,55 @@ func envOf(m map[string]string) func(string) string { return func(k string) stri
// The core source works with no provider adapter at all (§25).
func TestEnvAddressesWithoutAProvider(t *testing.T) {
- got := envAddresses{getenv: envOf(map[string]string{"CARLOS_PRIVATE_IP": " 10.0.1.5 ", "CARLOS_PUBLIC_IP": "198.51.100.7"})}.BoxAddresses()
- if got != (BoxAddresses{Private: "10.0.1.5", Public: "198.51.100.7"}) {
+ got := envAddresses{getenv: envOf(map[string]string{"CARLOS_PRIVATE_IP": " 10.0.1.5 ", "CARLOS_PRIVATE_CIDR": "10.0.0.0/16", "CARLOS_PUBLIC_IP": "198.51.100.7"})}.BoxAddresses()
+ if got != (BoxAddresses{Private: "10.0.1.5", PrivateCIDR: "10.0.0.0/16", Public: "198.51.100.7"}) {
t.Fatalf("got %+v", got)
}
if got := (envAddresses{getenv: envOf(nil)}).BoxAddresses(); got != (BoxAddresses{}) {
t.Fatalf("nothing set: %+v", got)
}
- // Not IPv4: published as nothing.
- got = envAddresses{getenv: envOf(map[string]string{"CARLOS_PRIVATE_IP": "fd00::5", "CARLOS_PUBLIC_IP": "box.example.com"})}.BoxAddresses()
+ // CARLOS_PRIVATE_IP needs CARLOS_PRIVATE_CIDR beside it.
+ if got := (envAddresses{getenv: envOf(map[string]string{"CARLOS_PRIVATE_IP": "10.0.1.5"})}).BoxAddresses(); got.Private != "" {
+ t.Fatalf("a private address with no network: %+v", got)
+ }
+ got = envAddresses{getenv: envOf(map[string]string{"CARLOS_PRIVATE_IP": "169.254.169.254", "CARLOS_PRIVATE_CIDR": "169.254.0.0/16", "CARLOS_PUBLIC_IP": "box.example.com"})}.BoxAddresses()
if got != (BoxAddresses{}) {
- t.Fatalf("not IPv4: %+v", got)
+ t.Fatalf("invalid settings: %+v", got)
}
}
func TestEnvAddressesOverrideTheProviderFieldByField(t *testing.T) {
- provider := &countingAddresses{answers: []BoxAddresses{{Private: "10.0.9.9", Public: "203.0.113.9"}}}
+ provider := &countingAddresses{answers: []BoxAddresses{{Private: "10.0.9.9", PrivateCIDR: "10.0.0.0/16", Public: "203.0.113.9"}}}
got := envAddresses{getenv: envOf(map[string]string{"CARLOS_PUBLIC_IP": "198.51.100.7"}), provider: provider}.BoxAddresses()
- if got != (BoxAddresses{Private: "10.0.9.9", Public: "198.51.100.7"}) {
+ if got != (BoxAddresses{Private: "10.0.9.9", PrivateCIDR: "10.0.0.0/16", Public: "198.51.100.7"}) {
t.Fatalf("got %+v", got)
}
// Both set: the provider is not asked.
provider.reads = 0
- envAddresses{getenv: envOf(map[string]string{"CARLOS_PRIVATE_IP": "10.0.1.5", "CARLOS_PUBLIC_IP": "198.51.100.7"}), provider: provider}.BoxAddresses()
+ envAddresses{getenv: envOf(map[string]string{"CARLOS_PRIVATE_IP": "10.0.1.5", "CARLOS_PRIVATE_CIDR": "10.0.0.0/16", "CARLOS_PUBLIC_IP": "198.51.100.7"}), provider: provider}.BoxAddresses()
if provider.reads != 0 {
- t.Fatalf("provider asked %d times with both overrides set", provider.reads)
+ t.Fatalf("provider asked %d times with both settings present", provider.reads)
}
- // A bad override publishes nothing for its field; the provider's
+ // A bad private setting publishes no private address; the provider's
// answer does not replace it.
- got = envAddresses{getenv: envOf(map[string]string{"CARLOS_PRIVATE_IP": "not-an-ip"}), provider: provider}.BoxAddresses()
- if got != (BoxAddresses{Private: "", Public: "203.0.113.9"}) {
- t.Fatalf("bad override: %+v", got)
+ got = envAddresses{getenv: envOf(map[string]string{"CARLOS_PRIVATE_IP": "not-an-ip", "CARLOS_PRIVATE_CIDR": "10.0.0.0/16"}), provider: provider}.BoxAddresses()
+ if got != (BoxAddresses{Public: "203.0.113.9"}) {
+ t.Fatalf("bad setting: %+v", got)
+ }
+ // The provider's answer is validated too.
+ bad := &countingAddresses{answers: []BoxAddresses{{Private: "10.9.9.9", PrivateCIDR: "10.0.0.0/16"}}}
+ if got := (envAddresses{getenv: envOf(nil), provider: bad}).BoxAddresses(); got.Private != "" {
+ t.Fatalf("an out-of-network provider address: %+v", got)
}
}
-func TestIMDSAddressesReadsBothLeaves(t *testing.T) {
- leaves := map[string]string{"/latest/meta-data/local-ipv4": "10.0.1.5", "/latest/meta-data/public-ipv4": "203.0.113.4"}
+func TestIMDSAddressesReadsTheLeavesAndTheVPCNetwork(t *testing.T) {
+ leaves := map[string]string{
+ "/latest/meta-data/local-ipv4": "10.0.1.5",
+ "/latest/meta-data/public-ipv4": "203.0.113.4",
+ "/latest/meta-data/mac": "0e:12:34:56:78:9a",
+ "/latest/meta-data/network/interfaces/macs/0e:12:34:56:78:9a/vpc-ipv4-cidr-blocks": "172.31.0.0/16\n10.0.0.0/16",
+ }
s := imdsAddresses{token: func() string { return "tok" }, leaf: func(token, path string) (string, int) {
if token != "tok" {
t.Fatalf("token %q", token)
@@ -2592,12 +2672,23 @@ func TestIMDSAddressesReadsBothLeaves(t *testing.T) {
}
return v, http.StatusOK
}}
- if got := s.BoxAddresses(); got != (BoxAddresses{Private: "10.0.1.5", Public: "203.0.113.4"}) {
+ if got := s.BoxAddresses(); got != (BoxAddresses{Private: "10.0.1.5", PrivateCIDR: "10.0.0.0/16", Public: "203.0.113.4"}) {
t.Fatalf("got %+v", got)
}
+ // An address outside the VPC's blocks is not published.
+ leaves["/latest/meta-data/local-ipv4"] = "10.9.1.5"
+ if got := s.BoxAddresses(); got.Private != "" || got.PrivateCIDR != "" {
+ t.Fatalf("out of network: %+v", got)
+ }
+ // No VPC network answer: no private address.
+ leaves["/latest/meta-data/local-ipv4"] = "10.0.1.5"
+ delete(leaves, "/latest/meta-data/network/interfaces/macs/0e:12:34:56:78:9a/vpc-ipv4-cidr-blocks")
+ if got := s.BoxAddresses(); got.Private != "" {
+ t.Fatalf("no network: %+v", got)
+ }
// No public address (a box without one, or the EIP gone): blank.
delete(leaves, "/latest/meta-data/public-ipv4")
- if got := s.BoxAddresses(); got != (BoxAddresses{Private: "10.0.1.5"}) {
+ if got := s.BoxAddresses(); got.Public != "" {
t.Fatalf("no public leaf: %+v", got)
}
noToken := imdsAddresses{token: func() string { return "" }, leaf: s.leaf}
@@ -2628,8 +2719,8 @@ func TestEdgesFilePublishesOnlyWhatTheSourceReports(t *testing.T) {
seedRoute(t, reg, "app.example.com")
claims := newEdgesClaims("annie", (&claimsBook{}).read)
src := &countingAddresses{answers: []BoxAddresses{
- {Private: "10.0.1.5", Public: "203.0.113.9"},
- {Private: "10.0.1.5"},
+ {Private: "10.0.1.5", PrivateCIDR: "10.0.0.0/16", Public: "203.0.113.9"},
+ {Private: "10.0.1.5", PrivateCIDR: "10.0.0.0/16"},
}}
claims.addrs = src
var last []byte
@@ -2638,8 +2729,8 @@ func TestEdgesFilePublishesOnlyWhatTheSourceReports(t *testing.T) {
if err := edgesFileWriteClaims(context.Background(), s, reg, ident, "198.51.100.1", &last, false, claims); err != nil {
t.Fatal(err)
}
- if f := readEdgesFile(t, s, "annie"); f.EIP != "203.0.113.9" || f.PrivateIP != "10.0.1.5" {
- t.Fatalf("first publication: eip %q private %q", f.EIP, f.PrivateIP)
+ if f := readEdgesFile(t, s, "annie"); f.EIP != "203.0.113.9" || f.PrivateIP != "10.0.1.5" || f.PrivateCIDR != "10.0.0.0/16" {
+ t.Fatalf("first publication: %+v", f)
}
if err := edgesFileWriteClaims(context.Background(), s, reg, ident, "", &last, false, claims); err != nil {
t.Fatal(err)
@@ -2665,8 +2756,8 @@ func TestEdgesFileWithRegionHAOffKeepsTheEIPHold(t *testing.T) {
if err := edgesFileWriteClaims(context.Background(), s, reg, ident, "", &last, true, nil); err != nil {
t.Fatal(err)
}
- if f := readEdgesFile(t, s, "annie"); f.EIP != "203.0.113.9" || f.PrivateIP != "" {
- t.Fatalf("eip %q private %q", f.EIP, f.PrivateIP)
+ if f := readEdgesFile(t, s, "annie"); f.EIP != "203.0.113.9" || f.PrivateIP != "" || f.PrivateCIDR != "" {
+ t.Fatalf("%+v", f)
}
}
````
@@ -2692,10 +2783,49 @@ The existing claims tests get a fixed address source, never IMDS:
- [ ] **Step 2: Run them to see them fail**
-Run: `nice -n 19 go test ./internal/host/ -run 'Address|IMDS|EdgesFile' -count=1`
-Expected: FAIL to build (`undefined: BoxAddresses`, `claims.addrs undefined`).
+Run: `nice -n 19 go test ./internal/edgesfile/ -run PrivateAddress -count=1; nice -n 19 go test ./internal/host/ -run 'Address|IMDS|EdgesFile' -count=1`
+Expected: FAIL to build (`undefined: PrivateAddress`, `undefined: BoxAddresses`, `claims.addrs undefined`).
+
+- [ ] **Step 3: Write the validator, the source and the publisher's change**
-- [ ] **Step 3: Write the source and the publisher's change**
+`internal/edgesfile/private.go`:
+
+````go
+package edgesfile
+
+import (
+ "net"
+ "strings"
+)
+
+// PrivateAddress validates a box's private address against its region
+// network's CIDR blocks (region HA spec 2 §6, census ruling region-ha-2a):
+// a well-formed IPv4 address, in a private range, neither loopback nor
+// link-local (169.254.0.0/16, the metadata range, included), and inside
+// one of cidrs. It returns the address and the block that holds it, both
+// canonical, or "", "". The writer publishes only what it passes, and a
+// reader dials only what it passes against its own region's network.
+func PrivateAddress(addr string, cidrs []string) (string, string) {
+ ip := net.ParseIP(strings.TrimSpace(addr))
+ if ip == nil || ip.To4() == nil {
+ return "", ""
+ }
+ ip = ip.To4()
+ if !ip.IsPrivate() || ip.IsLoopback() || ip.IsLinkLocalUnicast() || ip.IsUnspecified() {
+ return "", ""
+ }
+ for _, c := range cidrs {
+ _, block, err := net.ParseCIDR(strings.TrimSpace(c))
+ if err != nil || block.IP.To4() == nil {
+ continue
+ }
+ if block.Contains(ip) {
+ return ip.String(), block.String()
+ }
+ }
+ return "", ""
+}
+````
`internal/host/boxaddress.go`:
@@ -2705,56 +2835,66 @@ package host
import (
"net"
"strings"
+
+ "github.com/carlosframework/platform/internal/edgesfile"
)
// BoxAddresses are the addresses this box publishes in its edges file for
-// other edges to dial (region HA spec 2 §6): Private for an edge in the
-// same region, Public for one in another. "" is an address the source
-// could not report, and nothing is published for it.
+// other edges to dial (region HA spec 2 §6). Public is for an edge in
+// another region. Private, with PrivateCIDR (the region network it lies
+// in), is for an edge in the same region. They are routing data only: an
+// edge dials them, and nothing grants anything on them. "" is an address
+// the source could not report, or one that failed validation, and nothing
+// is published for it.
type BoxAddresses struct {
- Private string
- Public string
+ Private string
+ PrivateCIDR string
+ Public string
}
// AddressSource reads this box's addresses afresh on every call. The edges
// file publisher asks it before every publication and publishes only what
// it reports, so an address the box no longer holds is never republished.
+// Every source is root-configured: the cloud's metadata service, or
+// host.env. None reads a file the agent can write.
type AddressSource interface {
BoxAddresses() BoxAddresses
}
-// envAddresses is the provider-neutral source (§25): CARLOS_PRIVATE_IP and
-// CARLOS_PUBLIC_IP from host.env. Each, when set, is the answer for its
-// own field, and a value that is not an IPv4 address publishes nothing for
-// it: an operator's setting is never silently replaced. A field left unset
-// comes from provider, the cloud's adapter, when there is one.
+// envAddresses is the provider-neutral source (§25), from host.env:
+// CARLOS_PRIVATE_IP with CARLOS_PRIVATE_CIDR, and CARLOS_PUBLIC_IP. A field
+// whose setting is present is the answer for that field, and a value that
+// does not validate publishes nothing: an operator's setting is never
+// silently replaced. CARLOS_PRIVATE_IP without CARLOS_PRIVATE_CIDR is
+// invalid (census ruling region-ha-2a). A field left unset comes from
+// provider, the cloud's adapter, when there is one.
type envAddresses struct {
getenv func(string) string
provider AddressSource
}
func (s envAddresses) BoxAddresses() BoxAddresses {
- private, privateSet := s.field("CARLOS_PRIVATE_IP")
- public, publicSet := s.field("CARLOS_PUBLIC_IP")
+ ip := strings.TrimSpace(s.getenv("CARLOS_PRIVATE_IP"))
+ cidr := strings.TrimSpace(s.getenv("CARLOS_PRIVATE_CIDR"))
+ public := strings.TrimSpace(s.getenv("CARLOS_PUBLIC_IP"))
+ privateSet, publicSet := ip != "" || cidr != "", public != ""
+ out := BoxAddresses{}
+ if privateSet {
+ out.Private, out.PrivateCIDR = edgesfile.PrivateAddress(ip, []string{cidr})
+ }
+ if publicSet {
+ out.Public = ipv4Only(public)
+ }
if (!privateSet || !publicSet) && s.provider != nil {
p := s.provider.BoxAddresses()
if !privateSet {
- private = ipv4Only(p.Private)
+ out.Private, out.PrivateCIDR = edgesfile.PrivateAddress(p.Private, []string{p.PrivateCIDR})
}
if !publicSet {
- public = ipv4Only(p.Public)
+ out.Public = ipv4Only(p.Public)
}
}
- return BoxAddresses{Private: private, Public: public}
-}
-
-// field reads one override: its IPv4 address, and whether it was set.
-func (s envAddresses) field(key string) (string, bool) {
- v := strings.TrimSpace(s.getenv(key))
- if v == "" {
- return "", false
- }
- return ipv4Only(v), true
+ return out
}
// ipv4Only is s when it is an IPv4 address, in its canonical form, and ""
@@ -2775,14 +2915,18 @@ package host
import (
"net/http"
+ "strings"
"time"
+
+ "github.com/carlosframework/platform/internal/edgesfile"
)
// imdsAddresses is the AWS adapter of AddressSource (region HA spec 2
-// §25): IMDSv2's local-ipv4 and public-ipv4, read on every call. IMDS
-// reports only addresses this instance holds, so once the region's EIP has
-// moved to the load balancer no box reports it (A1-12). A leaf that does
-// not answer 200 is blank.
+// §25): IMDSv2's local-ipv4, public-ipv4, and the VPC's CIDR blocks for
+// the primary interface (network/interfaces/macs/<mac>/vpc-ipv4-cidr-blocks),
+// read on every call. IMDS reports only addresses this instance holds, so
+// once the region's EIP has moved to the load balancer no box reports it
+// (A1-12). A leaf that does not answer 200 is blank.
type imdsAddresses struct {
token func() string
leaf func(token, path string) (string, int)
@@ -2808,11 +2952,19 @@ func (s imdsAddresses) BoxAddresses() BoxAddresses {
}
return v
}
- return BoxAddresses{Private: read("/latest/meta-data/local-ipv4"), Public: read("/latest/meta-data/public-ipv4")}
+ out := BoxAddresses{Public: read("/latest/meta-data/public-ipv4")}
+ ip := read("/latest/meta-data/local-ipv4")
+ mac := read("/latest/meta-data/mac")
+ if ip == "" || mac == "" || strings.ContainsAny(mac, "/\\ ") {
+ return out
+ }
+ blocks := strings.Fields(read("/latest/meta-data/network/interfaces/macs/" + mac + "/vpc-ipv4-cidr-blocks"))
+ out.Private, out.PrivateCIDR = edgesfile.PrivateAddress(ip, blocks)
+ return out
}
// boxAddressSource is what the edges file publisher reads with region HA
-// on: host.env's overrides, then IMDS on EC2.
+// on: host.env's settings, then IMDS on EC2.
func boxAddressSource(getenv func(string) string) AddressSource {
return envAddresses{getenv: getenv, provider: newIMDSAddresses()}
}
@@ -2851,40 +3003,42 @@ func boxAddressSource(getenv func(string) string) AddressSource {
````diff
--- a/internal/host/edgesfile.go
+++ b/internal/host/edgesfile.go
-@@ -205,6 +205,10 @@
+@@ -205,6 +205,13 @@
Box string `json:"box"`
EIP string `json:"eip"`
Region string `json:"region,omitempty"`
+ // PrivateIP is the address an edge in this box's region dials (region
-+ // HA spec 2 §6). Additive and omitempty: a reader that does not know
-+ // it, or finds it absent, dials EIP, as before.
-+ PrivateIP string `json:"private_ip,omitempty"`
++ // HA spec 2 §6), and PrivateCIDR the region network that holds it,
++ // both validated by privateAddress before they are written. Routing
++ // data only. Additive and omitempty: a reader that does not know them,
++ // or finds them absent, dials EIP, as before.
++ PrivateIP string `json:"private_ip,omitempty"`
++ PrivateCIDR string `json:"private_cidr,omitempty"`
// Placement is this box's CARLOS placement code — CARLOS_REGION verbatim
// ("ie", "uk"), the OTHER region vocabulary (see Region above, issue
// #329). Additive and omitempty like Region: a pre-placement body parses
-@@ -591,7 +595,15 @@
+@@ -591,7 +598,15 @@
hosts[key] = edgeHost{Role: role, Wildcard: true, Alias: alias, Account: acct, App: app, Stamp: stamp}
}
- if eip == "" && *last != nil {
-+ var privateIP string
++ var private BoxAddresses
+ if claims != nil && claims.addrs != nil {
+ // Region HA spec 2 §6 (A1-12): publish only what the box's address
+ // source reports now. A blank read publishes no address, never the
+ // last one: that may be the region's EIP, which the load balancer
+ // has since taken.
-+ a := claims.addrs.BoxAddresses()
-+ eip, privateIP = a.Public, a.Private
++ private = claims.addrs.BoxAddresses()
++ eip = private.Public
+ } else if eip == "" && *last != nil {
var prior edgesFile
if err := json.Unmarshal(*last, &prior); err == nil {
eip = prior.EIP
-@@ -599,7 +611,7 @@
- }
+@@ -600,6 +615,7 @@
f := ident
-- f.V, f.EIP, f.Hosts, f.Generation = 1, eip, hosts, ""
-+ f.V, f.EIP, f.PrivateIP, f.Hosts, f.Generation = 1, eip, privateIP, hosts, ""
+ f.V, f.EIP, f.Hosts, f.Generation = 1, eip, hosts, ""
++ f.PrivateIP, f.PrivateCIDR = private.Private, private.PrivateCIDR
gateCopy := f
gateCopy.Headroom = nil // a changing reading is not, by itself, a reason to write
gate, err := json.Marshal(gateCopy)
@@ -2892,19 +3046,20 @@ func boxAddressSource(getenv func(string) string) AddressSource {
- [ ] **Step 4: Run the tests**
-Run: `nice -n 19 go test ./internal/host/ -run 'Address|IMDS|EdgesFile|Edges|Headroom' -count=1`
-Expected: PASS.
+Run: `nice -n 19 go test ./internal/edgesfile/ -count=1 && nice -n 19 go test ./internal/host/ -run 'Address|IMDS|EdgesFile|Edges|Headroom' -count=1`
+Expected: PASS: a malformed, a metadata-range, a loopback, a public, an IPv6 and an out-of-network address are each omitted.
- [ ] **Step 5: Boundary list and census**
````diff
--- a/region_ha_boundary_test.go
+++ b/region_ha_boundary_test.go
-@@ -22,11 +22,14 @@
+@@ -22,11 +22,15 @@
"internal/lbhealth/lbhealth.go",
"internal/hostactors/roles.go",
"internal/boxchoice/policy.go",
+ "internal/host/boxaddress.go",
++ "internal/edgesfile/private.go",
}
// regionHAAdapterFiles are spec 2's AWS adapters. Each is named for AWS so
@@ -2921,39 +3076,41 @@ Expected: PASS.
Add this comment line after the overlay's last comment line:
````text
-# Task 7: the box's own addresses (internal/host boxaddress.go, the provider-neutral source over CARLOS_PRIVATE_IP and CARLOS_PUBLIC_IP; awsaddr.go, the IMDS adapter) and the edges file's private_ip. edgesFileWriteClaims (authority, F,J1) is re-digested with its class, task, basis and sites unchanged: with region HA on it reads the address source before every publication and no longer holds a blank EIP. It adds no registry, /data, release or claim read, and nothing runs as root that did not before; the legacy seal stands. Re-digesting an authority row is flagged for platform-1e's confirmation.
+# Task 7: the box's own addresses (internal/host boxaddress.go, the provider-neutral source over CARLOS_PRIVATE_IP with CARLOS_PRIVATE_CIDR, and CARLOS_PUBLIC_IP, all from root-owned host.env; awsaddr.go, the IMDS adapter, which reads the VPC's CIDR blocks too) and the edges file's private_ip and private_cidr, validated before they are written by edgesfile.PrivateAddress (census ruling region-ha-2a, condition 1). edgesFileWriteClaims (authority, F,J1) is re-digested with its class, task, basis and sites unchanged: with region HA on it reads the address source before every publication and no longer holds a blank EIP. It adds no registry, /data, release or claim read, and nothing runs as root that did not before; the legacy seal stands.
````
Then set these rows. A `@coverage` row replaces the overlay's row for the same directory, or is appended when there is none; every other row is appended:
````text
-@coverage internal/host 5d9945ec79672e86958261f28dc398f16f1ca51511deb8b48a271a200733ba3a
-internal/host/edgesfile.go edgesFileWriteClaims authority F,J1 root-route;kept=F;scope=protected-decision-only registry:Account@10835,registry:Alias@5297,registry:Alias@5332,registry:Alias@5754,registry:Alias@5911,registry:Alias@5981,registry:Alias@6013,registry:Alias@6482,registry:Alias@8772,registry:App@10847,registry:DNSStatus@5429,registry:DNSStatus@5920,registry:Host@10480,registry:Host@2878,registry:Host@5389,registry:Host@6234,registry:Host@6567,registry:Host@8819,registry:Host@8866,registry:Host@8916,registry:Host@8951,registry:Host@9000,registry:Platform@2921,registry:Provisioned@2935,registry:Replica@2959,registry:Status@877,whole:Route@3212,whole:Route@323,whole:Route@3505,whole:Route@351,whole:RouteAlias@5282,whole:RouteAlias@8725,whole:RoutePath@3520,whole:WildcardDomain@8905 53f9240b5f842da209a8499f821c2c06cabae4c26a1ac5c1a45baa38e1a25d1f
+@coverage internal/edgesfile f62c3973bb48fa36f561d26f496c0ba9a14fe882fe5843867e9f7a9eaf0abbdf
+@coverage internal/host 02e98726846daefb57c11e2731344f4f7ae7eb4d4a6ec32388290bf1369091a0
+internal/host/edgesfile.go edgesFileWriteClaims authority F,J1 root-route;kept=F;scope=protected-decision-only registry:Account@10835,registry:Alias@5297,registry:Alias@5332,registry:Alias@5754,registry:Alias@5911,registry:Alias@5981,registry:Alias@6013,registry:Alias@6482,registry:Alias@8772,registry:App@10847,registry:DNSStatus@5429,registry:DNSStatus@5920,registry:Host@10480,registry:Host@2878,registry:Host@5389,registry:Host@6234,registry:Host@6567,registry:Host@8819,registry:Host@8866,registry:Host@8916,registry:Host@8951,registry:Host@9000,registry:Platform@2921,registry:Provisioned@2935,registry:Replica@2959,registry:Status@877,whole:Route@3212,whole:Route@323,whole:Route@3505,whole:Route@351,whole:RouteAlias@5282,whole:RouteAlias@8725,whole:RoutePath@3520,whole:WildcardDomain@8905 48bad9a5f49c5fca53fcf7bd6a241576fd2cd332974114e463b2cb3832e3598d
````
-The appended row is `edgesFileWriteClaims`'s, re-digested: its kinds are main's row's, unchanged (check with the scratch test). Run the census checks and `go test . -run TestRegionHA -count=1`. Expected: PASS.
+The appended row is `edgesFileWriteClaims`'s, re-digested: its kinds are main's row's, unchanged (check with the scratch test). Run the census checks and `go test . -run 'TestRegionHA(Core|Adapters|S3)' -count=1`. Expected: PASS.
- [ ] **Step 6: Commit**
```bash
-git add internal/host/boxaddress.go internal/host/awsaddr.go internal/host/boxaddress_test.go internal/host/edgesclaims.go internal/host/edgesclaims_test.go internal/host/edgesfile.go region_ha_boundary_test.go internal/provision/testdata/plan14-root-reads-region-ha-2a.tsv
-git commit -m "region HA 2a: publish the box's private address and only what its source reports"
+git add internal/edgesfile/private.go internal/edgesfile/private_test.go internal/host/boxaddress.go internal/host/awsaddr.go internal/host/boxaddress_test.go internal/host/edgesclaims.go internal/host/edgesclaims_test.go internal/host/edgesfile.go region_ha_boundary_test.go internal/provision/testdata/plan14-root-reads-region-ha-2a.tsv
+git commit -m "region HA 2a: publish the box's validated private address and only what its source reports"
```
---
-### Task 8: Forwarding dials the private address within a region
+### Task 8: Forwarding dials the private address within a region, with the reader's region HA on
**Files:**
-- Modify: `internal/edge/homemap.go` (`edgesFileView` reads `private_ip`, `region`, `placement`; `dialAddress`; `foldHomes`)
+- Modify: `internal/edge/homemap.go` (`edgesFileView` reads `private_ip`, `private_cidr`, `region`, `placement`; `dialAddress`; `regionHAFromEnv`; `homeMap.privateDial`; `foldHomes`)
+- Modify: `internal/edge/homemap_test.go` (`foldHomes`'s new argument)
- Create: `internal/edge/homemap_private_test.go`
- Modify: the overlay
**Interfaces:**
-- Consumes: `private_ip` from Task 7's writer.
-- Produces: `func dialAddress(f, self edgesFileView, selfKnown bool) string`. `homeEntry.eip` is now the address to dial, so `Home`, `ExactHome` and `Owner` answer it, and the backhaul transport, keyed by that address, dials it.
+- Consumes: `edgesfile.PrivateAddress` and `private_ip`/`private_cidr` from Task 7.
+- Produces: `func dialAddress(f, self edgesFileView, selfKnown, privateDial bool) string`; `func regionHAFromEnv(getenv func(string) string) bool` (Task 9 reuses it); `foldHomes(files, self, privateDial)`. `homeEntry.eip` is now the address to dial, so `Home`, `ExactHome` and `Owner` answer it, and the backhaul transport, keyed by that address, dials it.
-The hop is unchanged: one hop, today's credential and mesh key, the TLS name the host's. Only the address dialled changes: a box in this box's own AWS region and placement, publishing `private_ip`, is dialled there; every other box at its EIP, as before. An edge that cannot find its own edges file knows no region of its own and dials public addresses only.
+The hop is unchanged: one hop, today's credential and mesh key, the TLS name the host's. Only the address dialled changes, and only with **this edge's own** `CARLOS_REGION_HA` on (Astra finding 5): a box in this box's own AWS region and placement, publishing a `private_ip` that passes `PrivateAddress` against the network **this box's own file** publishes, is dialled there. Every other box, and every box with this edge's switch off, is dialled at its EIP, as before; so is every box when this edge cannot find its own file.
- [ ] **Step 1: Write the failing tests**
@@ -2962,108 +3119,154 @@ The hop is unchanged: one hop, today's credential and mesh key, the TLS name the
````go
package edge
-import (
- "net/http"
- "testing"
-)
+import "testing"
-// Region HA spec 2 §6: within a region an edge dials the home box at its
-// private address; between regions, at the home box's own public address.
-// A reader of a file with no private_ip dials the public address, as
-// before, and so does an edge that cannot find its own file.
-func TestHomeMapDialsPrivateWithinTheRegionAndPublicBetween(t *testing.T) {
- s := meshTestStore(t)
- putJSON(t, s, "control/edges/annie.json", `{"v":1,"box":"annie","eip":"1.1.1.1","private_ip":"10.0.1.1","region":"eu-west-1","placement":"ie","hosts":{}}`)
- putJSON(t, s, "control/edges/betty.json", `{"v":1,"box":"betty","eip":"2.2.2.2","private_ip":"10.0.1.2","region":"eu-west-1","placement":"ie","hosts":{"b.test":{"role":"home"}}}`)
- putJSON(t, s, "control/edges/carla.json", `{"v":1,"box":"carla","eip":"3.3.3.3","private_ip":"10.1.1.3","region":"eu-west-2","placement":"uk","hosts":{"c.test":{"role":"home"}}}`)
- putJSON(t, s, "control/edges/dora.json", `{"v":1,"box":"dora","eip":"4.4.4.4","region":"eu-west-1","placement":"ie","hosts":{"d.test":{"role":"home"}}}`)
- putJSON(t, s, "control/edges/edna.json", `{"v":1,"box":"edna","eip":"5.5.5.5","private_ip":"10.0.1.5","region":"eu-west-1","placement":"fleet/shop","hosts":{"e.test":{"role":"home"}}}`)
- putJSON(t, s, "control/edges/fern.json", `{"v":1,"box":"fern","eip":"","private_ip":"10.0.1.6","region":"eu-west-1","placement":"ie","hosts":{"f.test":{"role":"home"}}}`)
- m := newHomeMap(s, "annie")
+// privateFleet publishes this edge's own file (annie, in ie, network
+// 10.0.0.0/16) and one file per case.
+func privateFleet(t *testing.T, m *homeMap) {
+ t.Helper()
+ putJSON(t, m.s, "control/edges/annie.json", `{"v":1,"box":"annie","eip":"1.1.1.1","private_ip":"10.0.1.1","private_cidr":"10.0.0.0/16","region":"eu-west-1","placement":"ie","hosts":{}}`)
+ putJSON(t, m.s, "control/edges/betty.json", `{"v":1,"box":"betty","eip":"2.2.2.2","private_ip":"10.0.1.2","private_cidr":"10.0.0.0/16","region":"eu-west-1","placement":"ie","hosts":{"b.test":{"role":"home"}}}`)
+ putJSON(t, m.s, "control/edges/carla.json", `{"v":1,"box":"carla","eip":"3.3.3.3","private_ip":"10.1.1.3","private_cidr":"10.1.0.0/16","region":"eu-west-2","placement":"uk","hosts":{"c.test":{"role":"home"}}}`)
+ putJSON(t, m.s, "control/edges/dora.json", `{"v":1,"box":"dora","eip":"4.4.4.4","region":"eu-west-1","placement":"ie","hosts":{"d.test":{"role":"home"}}}`)
+ putJSON(t, m.s, "control/edges/edna.json", `{"v":1,"box":"edna","eip":"5.5.5.5","private_ip":"10.0.1.5","region":"eu-west-1","placement":"fleet/shop","hosts":{"e.test":{"role":"home"}}}`)
+ putJSON(t, m.s, "control/edges/fern.json", `{"v":1,"box":"fern","eip":"","private_ip":"10.0.1.6","region":"eu-west-1","placement":"ie","hosts":{"f.test":{"role":"home"}}}`)
+ // Addresses a reader refuses: each falls back to the public address.
+ putJSON(t, m.s, "control/edges/gail.json", `{"v":1,"box":"gail","eip":"7.7.7.7","private_ip":"10.0.1","region":"eu-west-1","placement":"ie","hosts":{"g.test":{"role":"home"}}}`)
+ putJSON(t, m.s, "control/edges/hana.json", `{"v":1,"box":"hana","eip":"8.8.8.8","private_ip":"169.254.169.254","private_cidr":"169.254.0.0/16","region":"eu-west-1","placement":"ie","hosts":{"h.test":{"role":"home"}}}`)
+ putJSON(t, m.s, "control/edges/iris.json", `{"v":1,"box":"iris","eip":"9.9.9.9","private_ip":"10.9.1.9","private_cidr":"10.9.0.0/16","region":"eu-west-1","placement":"ie","hosts":{"i.test":{"role":"home"}}}`)
refreshHomeMap(t, m)
+}
+
+// Region HA spec 2 §6: with this edge's region HA on, a box in the same
+// region is dialled at its private address and a box in another region at
+// its own public address. An address that fails validation against this
+// edge's own network is refused, and the dial is public.
+func TestHomeMapDialsPrivateWithinTheRegionAndPublicBetween(t *testing.T) {
+ m := newHomeMap(meshTestStore(t), "annie")
+ m.privateDial = true
+ privateFleet(t, m)
for host, want := range map[string]string{
"b.test": "10.0.1.2", // same region and placement: private
"c.test": "3.3.3.3", // another region: its own public address
"d.test": "4.4.4.4", // a writer with no private_ip: public, as before
"e.test": "5.5.5.5", // same AWS region, another placement: public
"f.test": "10.0.1.6", // no public address, but private within the region
+ "g.test": "7.7.7.7", // a malformed private address: refused
+ "h.test": "8.8.8.8", // the metadata range: refused
+ "i.test": "9.9.9.9", // outside this region's network: refused
} {
if addr, _, ok := m.Home(host); !ok || addr != want {
t.Errorf("%s dials %q (ok=%v), want %q", host, addr, ok, want)
}
}
- // An edge whose own file is not in the map knows no region of its own:
- // every dial is public, and a box with no public address is no target.
- m = newHomeMap(s, "ghost")
- refreshHomeMap(t, m)
- if addr, _, _ := m.Home("b.test"); addr != "2.2.2.2" {
+ // An edge whose own file is not in the map knows no region or network
+ // of its own: every dial is public, and a box with no public address
+ // is no target.
+ ghost := newHomeMap(m.s, "ghost")
+ ghost.privateDial = true
+ refreshHomeMap(t, ghost)
+ if addr, _, _ := ghost.Home("b.test"); addr != "2.2.2.2" {
t.Errorf("without its own file: b.test dials %q, want 2.2.2.2", addr)
}
- if _, _, ok := m.Home("f.test"); ok {
+ if _, _, ok := ghost.Home("f.test"); ok {
t.Error("without its own file, a box with no public address became a target")
}
}
-// The whole hop, dialled privately: the home box's public address is
-// unroutable (TEST-NET-1), so only a dial to its private address reaches
-// it. It is still one hop.
-func TestBackhaulDialsThePrivateAddressWithinTheRegion(t *testing.T) {
- e, m, _ := hopEdge(t, "annie", "local.test")
- rec, private := fakeHomeEdge(t, nil)
- putJSON(t, m.s, "control/edges/annie.json", `{"v":1,"box":"annie","eip":"192.0.2.10","region":"eu-west-1","placement":"ie","hosts":{"local.test":{"role":"home"}}}`)
- putJSON(t, m.s, "control/edges/betty.json", `{"v":1,"box":"betty","eip":"192.0.2.11","private_ip":"`+private+`","region":"eu-west-1","placement":"ie","hosts":{"example.com":{"role":"home"}}}`)
- refreshHomeMap(t, m)
-
- resp := hopRequest(t, e, "example.com", nil)
- if resp.StatusCode != http.StatusOK {
- t.Fatalf("status = %d, want 200 through the private address", resp.StatusCode)
+// With this edge's region HA off, a peer publishing private_ip changes
+// nothing: every dial is public, as on main, during a mixed roll or a
+// rollback.
+func TestHomeMapWithRegionHAOffDialsPublicOnly(t *testing.T) {
+ m := newHomeMap(meshTestStore(t), "annie")
+ privateFleet(t, m)
+ for host, want := range map[string]string{"b.test": "2.2.2.2", "c.test": "3.3.3.3", "d.test": "4.4.4.4"} {
+ if addr, _, ok := m.Home(host); !ok || addr != want {
+ t.Errorf("%s dials %q (ok=%v), want %q", host, addr, ok, want)
+ }
+ }
+ if _, _, ok := m.Home("f.test"); ok {
+ t.Error("a box with only a private address became a target with region HA off")
}
- if body := readBody(t, resp); body != homeEdgeBody {
- t.Fatalf("body = %q", body)
+}
+
+func TestEdgeHomeMapFollowsRegionHA(t *testing.T) {
+ e := &Edge{staticStore: meshTestStore(t), box: "annie"}
+ t.Setenv("CARLOS_REGION_HA", "")
+ if e.newHomeMap().privateDial {
+ t.Fatal("region HA unset dials private")
}
- if seen := rec.get(); seen.n != 1 || seen.hop != "1" || seen.sni != "example.com" {
- t.Fatalf("home edge saw %+v, want one verified hop for example.com", seen)
+ t.Setenv("CARLOS_REGION_HA", "1")
+ if !e.newHomeMap().privateDial {
+ t.Fatal("region HA on does not dial private")
}
}
````
- [ ] **Step 2: Run them to see them fail**
-Run: `nice -n 19 go test ./internal/edge/ -run 'DialsPrivate|PrivateAddress' -count=1`
-Expected: FAIL: `b.test dials "2.2.2.2" (ok=true), want "10.0.1.2"`, and the private backhaul answers 502 (its dial to the unroutable EIP times out).
+Run: `nice -n 19 go test ./internal/edge/ -run 'DialsPrivate|PublicOnly|FollowsRegionHA' -count=1`
+Expected: FAIL to build (`m.privateDial undefined`).
- [ ] **Step 3: Choose the dial address**
````diff
--- a/internal/edge/homemap.go
+++ b/internal/edge/homemap.go
-@@ -132,6 +132,25 @@
+@@ -80,10 +80,12 @@
+ "os"
+ "slices"
+ "sort"
++ "strconv"
+ "strings"
+ "sync"
+ "time"
+
++ "github.com/carlosframework/platform/internal/edgesfile"
+ "github.com/carlosframework/platform/internal/release"
+ )
+
+@@ -132,6 +134,39 @@
Box string `json:"box"`
EIP string `json:"eip"`
Hosts map[string]edgesHostView `json:"hosts"`
-+ // PrivateIP, Region and Placement choose the dial address (region HA
-+ // spec 2 §6, dialAddress). All three are absent from a writer that
++ // PrivateIP, PrivateCIDR, Region and Placement choose the dial address
++ // (region HA spec 2 §6, dialAddress). All are absent from a writer that
+ // predates them, and the dial is then EIP, as before.
-+ PrivateIP string `json:"private_ip"`
-+ Region string `json:"region"`
-+ Placement string `json:"placement"`
++ PrivateIP string `json:"private_ip"`
++ PrivateCIDR string `json:"private_cidr"`
++ Region string `json:"region"`
++ Placement string `json:"placement"`
+}
+
-+// dialAddress is where this edge dials f's box (region HA spec 2 §6): its
-+// private address when it publishes one and both boxes publish the same
-+// AWS region and placement, so the dial stays inside the region's
-+// network; otherwise its public address, as before. self is this box's
-+// own file, which selfKnown says was found. "" is nothing to dial.
-+func dialAddress(f, self edgesFileView, selfKnown bool) string {
-+ if selfKnown && f.PrivateIP != "" && f.Region != "" && f.Placement != "" &&
++// dialAddress is where this edge dials f's box (region HA spec 2 §6). With
++// privateDial (this edge's own region HA switch) on, a box that publishes
++// the same AWS region and placement as this one is dialled at its private
++// address, so the dial stays inside the region's network, but only when
++// that address passes edgesfile.PrivateAddress against the network this
++// box's own file publishes (census ruling region-ha-2a). Every other box,
++// and every box with privateDial off, is dialled at its public address,
++// as before. self is this box's own file, which selfKnown says was found.
++// "" is nothing to dial.
++func dialAddress(f, self edgesFileView, selfKnown, privateDial bool) string {
++ if privateDial && selfKnown && f.Region != "" && f.Placement != "" &&
+ f.Region == self.Region && f.Placement == self.Placement {
-+ return f.PrivateIP
++ if addr, _ := edgesfile.PrivateAddress(f.PrivateIP, []string{self.PrivateCIDR}); addr != "" {
++ return addr
++ }
+ }
+ return f.EIP
++}
++
++// regionHAFromEnv is CARLOS_REGION_HA as strconv.ParseBool reads it, the
++// reading every region HA switch makes.
++func regionHAFromEnv(getenv func(string) string) bool {
++ on, err := strconv.ParseBool(strings.TrimSpace(getenv("CARLOS_REGION_HA")))
++ return err == nil && on
}
// edgesHostView is the per-host object. An object rather than a bare string is
-@@ -161,6 +180,8 @@
+@@ -161,6 +196,8 @@
// homeEntry is one folded answer: where to dial, and which box that is.
type homeEntry struct {
@@ -3072,7 +3275,38 @@ Expected: FAIL: `b.test dials "2.2.2.2" (ok=true), want "10.0.1.2"`, and the pri
eip string
box string
// platform carries the publisher's class mark through the fold.
-@@ -532,9 +553,11 @@
+@@ -238,6 +275,10 @@
+ keyMu sync.RWMutex
+ keyCurrent []byte
+ keyNext []byte
++
++ // privateDial is this edge's region HA switch: only with it on does
++ // the fold dial a same-region box's private address (dialAddress).
++ privateDial bool
+ }
+
+ func newHomeMap(s release.Store, self string) *homeMap {
+@@ -253,7 +294,9 @@
+ if e.staticStore == nil {
+ return nil
+ }
+- return newHomeMap(e.staticStore, e.box)
++ m := newHomeMap(e.staticStore, e.box)
++ m.privateDial = regionHAFromEnv(os.Getenv)
++ return m
+ }
+
+ // Refresh rebuilds the host fold and re-reads the mesh keys. The two halves
+@@ -330,7 +373,7 @@
+ if failed > 0 {
+ m.logPartialPoll(failed, tried, lastErr)
+ }
+- hosts, tied := foldHomes(next, m.self)
++ hosts, tied := foldHomes(next, m.self, m.privateDial)
+ if len(tied) > 0 {
+ // One claim revision names one executor, so two boxes publishing
+ // one host at the same stamp is a fault upstream. The fold stays
+@@ -532,9 +575,11 @@
// target would have the edge dial itself. Excluded by the file's own box
// field rather than by its object name, because the field is what the
// publisher stamps and what self is derived to match.
@@ -3087,7 +3321,14 @@ Expected: FAIL: `b.test dials "2.2.2.2" (ok=true), want "10.0.1.2"`, and the pri
// - A ROLE THIS BUILD DOES NOT KNOW. Only "home" routes. The role field
// exists so read-local roles can be added later; a reader that treated
// every role as a home would send traffic to a box that had declared it
-@@ -552,22 +575,30 @@
+@@ -546,28 +591,36 @@
+ // split-and-wrong: the whole fleet aiming at one box is a condition that heals
+ // on the next publish, where half the fleet aiming at each is a condition that
+ // looks fine from any single box.
+-func foldHomes(files map[string]edgesFileView, self string) (map[string]homeEntry, []string) {
++func foldHomes(files map[string]edgesFileView, self string, privateDial bool) (map[string]homeEntry, []string) {
+ keys := make([]string, 0, len(files))
+ for k := range files {
keys = append(keys, k)
}
slices.Sort(keys)
@@ -3104,7 +3345,7 @@ Expected: FAIL: `b.test dials "2.2.2.2" (ok=true), want "10.0.1.2"`, and the pri
if f.Box == "" || f.Box == self {
continue
}
-+ addr := dialAddress(f, own, ownKnown)
++ addr := dialAddress(f, own, ownKnown, privateDial)
for host, h := range f.Hosts {
if host == "" {
continue
@@ -3120,7 +3361,7 @@ Expected: FAIL: `b.test dials "2.2.2.2" (ok=true), want "10.0.1.2"`, and the pri
if !h.Wildcard || !strings.HasPrefix(host, "*.") {
continue
}
-@@ -583,7 +614,7 @@
+@@ -583,7 +636,7 @@
default:
continue
}
@@ -3131,9 +3372,41 @@ Expected: FAIL: `b.test dials "2.2.2.2" (ok=true), want "10.0.1.2"`, and the pri
if !dup {
````
+````diff
+--- a/internal/edge/homemap_test.go
++++ b/internal/edge/homemap_test.go
+@@ -1715,7 +1715,7 @@
+ "control/edges/betty.json": file("betty", "2.2.2.2", map[string]edgesHostView{
+ "tied.test": {Role: "home", Stamp: 7}, "fine.test": {Role: "home", Stamp: 9}, "plain.test": {Role: "home"}}),
+ }
+- hosts, tied := foldHomes(files, "carol")
++ hosts, tied := foldHomes(files, "carol", false)
+ if len(tied) != 1 || tied[0] != "tied.test" {
+ t.Fatalf("tied = %v, want [tied.test]: unstamped duplicates are normal (static sites run on every box) and are not reported", tied)
+ }
+@@ -1735,7 +1735,7 @@
+ "control/edges/betty.json": file("betty", "2.2.2.2", map[string]edgesHostView{"h.test": {Role: "home", Stamp: 7}}),
+ "control/edges/carol.json": file("carol", "3.3.3.3", map[string]edgesHostView{"h.test": {Role: "home", Stamp: 7}}),
+ }
+- hosts, tied := foldHomes(files, "dave")
++ hosts, tied := foldHomes(files, "dave", false)
+ if len(tied) != 1 || tied[0] != "h.test" {
+ t.Fatalf("tied = %v, want [h.test]", tied)
+ }
+@@ -1751,7 +1751,7 @@
+ "control/edges/annie.json": file("annie", "1.1.1.1", map[string]edgesHostView{"h.test": {Role: "home", Stamp: 7}}),
+ "control/edges/betty.json": file("betty", "2.2.2.2", map[string]edgesHostView{"h.test": {Role: "home", Stamp: 7}}),
+ }
+- hosts, tied := foldHomes(files, "annie")
++ hosts, tied := foldHomes(files, "annie", false)
+ if len(tied) != 1 || tied[0] != "h.test" {
+ t.Fatalf("tied = %v, want [h.test]", tied)
+ }
+````
+
- [ ] **Step 4: Run the tests**
-Run: `nice -n 19 go test ./internal/edge/ -run 'HomeMap|Backhaul|Hop|KnownHost|DialsPrivate|PrivateAddress' -count=1`
+Run: `nice -n 19 go test ./internal/edge/ -run 'HomeMap|Backhaul|Hop|KnownHost|DialsPrivate|PublicOnly|FollowsRegionHA' -count=1`
Expected: PASS.
- [ ] **Step 5: Census**
@@ -3141,13 +3414,13 @@ Expected: PASS.
Add this comment line after the overlay's last comment line:
````text
-# Task 8: forwarding dials the home box's private address within its region and its public address between regions (internal/edge homemap.go: dialAddress, edgesFileView's private_ip, region and placement). The home map is not a root-read unit. Envelope-only, and the legacy seal stands.
+# Task 8: with the reader's own region HA on, forwarding dials a same-region box's private address when it passes edgesfile.PrivateAddress against the reader's own published network, and every other box at its public address (internal/edge homemap.go: dialAddress, regionHAFromEnv, edgesFileView's private_ip, private_cidr, region and placement). The home map is not a root-read unit. Envelope-only, and the legacy seal stands.
````
Then set these rows. A `@coverage` row replaces the overlay's row for the same directory, or is appended when there is none; every other row is appended:
````text
-@coverage internal/edge 11a6e94dbe8025c05d75509e56701e90092a225b191aee0a45dbd7307f660e31
+@coverage internal/edge 2c474d685471295daf20f061e595becab78175c5754dbef0d2187f0f8519fbe3
````
Run the census checks. Expected: PASS.
@@ -3155,7 +3428,7 @@ Run the census checks. Expected: PASS.
- [ ] **Step 6: Commit**
```bash
-git add internal/edge/homemap.go internal/edge/homemap_private_test.go internal/provision/testdata/plan14-root-reads-region-ha-2a.tsv
+git add internal/edge/homemap.go internal/edge/homemap_test.go internal/edge/homemap_private_test.go internal/provision/testdata/plan14-root-reads-region-ha-2a.tsv
git commit -m "region HA 2a: dial a box's private address within its region"
```
@@ -3175,7 +3448,7 @@ git commit -m "region HA 2a: dial a box's private address within its region"
- Modify: `region_ha_boundary_test.go`, the overlay
**Interfaces:**
-- Consumes: Task 2's `lbhealth.State` and `lbhealth.Serve`.
+- Consumes: Task 2's `lbhealth.State` and `lbhealth.Serve`; Task 8's `regionHAFromEnv`.
- Produces: `func lbHealthFromEnv(getenv func(string) string, act Activator) *lbhealth.State` (nil with region HA off); `func (e *Edge) lbHealthAnswer(w http.ResponseWriter, r *http.Request) bool`; `func (a *activator.Service) LastSweep() time.Time`; the hosting hook `lbhealth.Default.SetHosting(true)` in `ClaimWiring.startRegional`. **Seam for 2b:** 2b's mode selection takes `SetHosting` over; joining mode never sets it.
The answer is served on port 80 only (the `:80` handler, `httpFallbackHandler`), ahead of every host-based branch, and reads only in-memory state. Each condition is fed by the loop it describes:
@@ -3184,7 +3457,7 @@ The answer is served on port 80 only (the `:80` handler, `httpFallbackHandler`),
|---|---|
| hosting | `ClaimWiring.startRegional`: the claim agent only starts with its host-actor enrollment and a matching roster row |
| forwarding | `homeMap.Refresh`: the first successful map load and the first successful mesh key load, latched |
-| certificates | `certMirror.Refresh`: the first refresh that lands `control/certs/` in the local cache the handshake reads, latched |
+| certificates | `certMirror.Refresh`: the first refresh that lands `control/certs/` in the local cache the handshake reads, latched (spec rev 15: certificates are per host, so there is no single default certificate) |
| edge | `Edge.Watch`: every completed registry poll (every 2 seconds) |
| activator | `activator.Service.LastSweep`: every finished sweep (at most 30 seconds apart) |
| refresh | `homeMap.Refresh`: every completed attempt, failed or not |
@@ -3505,7 +3778,7 @@ func TestClaimWiringStartMarksTheBoxHosting(t *testing.T) {
- [ ] **Step 2: Run them to see them fail**
Run: `nice -n 19 go test ./internal/edge/ -run 'LBHealth|FeedsTheHealth|NoMeshKeyIsNotArmed|CertMirrorLatches|RegistryWatchBeats' -count=1`
-Expected: FAIL to build (`undefined: lbHealthFromEnv`, `e.lbHealth undefined`). The activator and host tests fail to build the same way (`a.LastSweep undefined`; `lbhealth` not set by `startRegional`, so `TestClaimWiringStartMarksTheBoxHosting` fails).
+Expected: FAIL to build (`undefined: lbHealthFromEnv`, `e.lbHealth undefined`). The activator test fails to build on `a.LastSweep undefined`; the host test builds and fails (`a running claim agent is not hosting`).
- [ ] **Step 3: Write the answer and its feeds**
@@ -3516,8 +3789,6 @@ package edge
import (
"net/http"
- "strconv"
- "strings"
"time"
"github.com/carlosframework/platform/internal/lbhealth"
@@ -3535,8 +3806,7 @@ import (
// activator's sweep is the hosting loop's beat; an edge with no activator
// (carlos edge) has none, and its answer is never 200.
func lbHealthFromEnv(getenv func(string) string, act Activator) *lbhealth.State {
- on, err := strconv.ParseBool(strings.TrimSpace(getenv("CARLOS_REGION_HA")))
- if err != nil || !on {
+ if !regionHAFromEnv(getenv) {
return nil
}
s := lbhealth.Default
@@ -3616,18 +3886,18 @@ func (e *Edge) lbHealthAnswer(w http.ResponseWriter, r *http.Request) bool {
````diff
--- a/internal/edge/homemap.go
+++ b/internal/edge/homemap.go
-@@ -84,6 +84,7 @@
- "sync"
+@@ -86,6 +86,7 @@
"time"
+ "github.com/carlosframework/platform/internal/edgesfile"
+ "github.com/carlosframework/platform/internal/lbhealth"
"github.com/carlosframework/platform/internal/release"
)
-@@ -259,6 +260,11 @@
- keyMu sync.RWMutex
- keyCurrent []byte
- keyNext []byte
+@@ -279,6 +280,11 @@
+ // privateDial is this edge's region HA switch: only with it on does
+ // the fold dial a same-region box's private address (dialAddress).
+ privateDial bool
+
+ // health is the load balancer health state (region HA spec 2 §6),
+ // nil without region HA: Refresh latches the first loaded map and key
@@ -3636,17 +3906,15 @@ func (e *Edge) lbHealthAnswer(w http.ResponseWriter, r *http.Request) bool {
}
func newHomeMap(s release.Store, self string) *homeMap {
-@@ -274,17 +280,32 @@
- if e.staticStore == nil {
- return nil
+@@ -296,6 +302,7 @@
}
-- return newHomeMap(e.staticStore, e.box)
-+ m := newHomeMap(e.staticStore, e.box)
+ m := newHomeMap(e.staticStore, e.box)
+ m.privateDial = regionHAFromEnv(os.Getenv)
+ m.health = e.lbHealth
-+ return m
+ return m
}
- // Refresh rebuilds the host fold and re-reads the mesh keys. The two halves
+@@ -303,10 +310,23 @@
// fail INDEPENDENTLY — a bucket that answered the edges list but not the key
// object should still land the fold — so both run and their errors join.
// Whatever fails leaves its own last-good in place; the caller logs.
@@ -3787,7 +4055,7 @@ func (e *Edge) lbHealthAnswer(w http.ResponseWriter, r *http.Request) bool {
- [ ] **Step 4: Run the tests**
```bash
-nice -n 19 go test ./internal/edge/ -run 'LBHealth|FeedsTheHealth|NoMeshKeyIsNotArmed|CertMirrorLatches|RegistryWatchBeats|HomeMap|Backhaul|Hop|HTTPFallback' -count=1
+nice -n 19 go test ./internal/edge/ -run 'LBHealth|FeedsTheHealth|NoMeshKeyIsNotArmed|CertMirrorLatches|RegistryWatchBeats|HomeMap|Backhaul|Hop|HTTPFallback|FollowsRegionHA|PublicOnly' -count=1
nice -n 19 go test ./internal/activator/ -run LastSweep -count=1
nice -n 19 go test ./internal/host/ -run ClaimWiring -count=1
```
@@ -3798,10 +4066,10 @@ Expected: PASS.
````diff
--- a/region_ha_boundary_test.go
+++ b/region_ha_boundary_test.go
-@@ -23,6 +23,7 @@
- "internal/hostactors/roles.go",
+@@ -24,6 +24,7 @@
"internal/boxchoice/policy.go",
"internal/host/boxaddress.go",
+ "internal/edgesfile/private.go",
+ "internal/edge/lbhealth.go",
}
@@ -3817,12 +4085,12 @@ Add this comment line after the overlay's last comment line:
Then set these rows. A `@coverage` row replaces the overlay's row for the same directory, or is appended when there is none; every other row is appended:
````text
-@coverage internal/host 5a5341c760fb8c16bf79ca6fc5cf82e9174157f5fe05d78a74532e237fe40c85
-@coverage internal/edge f05b773955396fd0e54bce32b8a41c095130503005f43871b4bd96c255d3b87d
+@coverage internal/host 87d13f40c52d39585cd9579eafea61b772577c6d3bf23e9e3951af778711e19a
+@coverage internal/edge 8e47251c54e3c5ee9cb6e8b9b1ad3ae87da528436c6a870cadb07ca3a8ac3146
@coverage internal/activator c2eb11024375a706d70652dee68d446202a264927f819deab6321b652e82c2bb
````
-Run the census checks and `go test . -run TestRegionHA -count=1`. Expected: PASS.
+Run the census checks and `go test . -run 'TestRegionHA(Core|Adapters|S3)' -count=1`. Expected: PASS.
- [ ] **Step 6: Commit**
@@ -3833,12 +4101,11 @@ git commit -m "region HA 2a: the load balancer health answer"
---
-### Task 10: The role doors, `carlos system box role` and `group role`, and the move destination
+### Task 10: The role doors and `carlos system box role` and `group role`
**Files:**
- Create: `internal/console/apiboxroles.go`, `internal/console/apiboxroles_test.go`
- Modify: `internal/console/authcli.go` (two routes, with host claims on)
-- Modify: `internal/console/boxselect.go` (`selectMoveBoxes` skips excluded destinations)
- Modify: `internal/console/svcroutes_test.go`, `internal/console/claim_invariant_fixtures_test.go`, `internal/console/claim_invariant_probes_nonapp_test.go`, `internal/console/testdata/claim-routes.txt`
- Create: `system_roles.go`, `system_roles_test.go`
- Modify: `system.go` (the `box` and `group` verbs and their usage)
@@ -3847,12 +4114,14 @@ git commit -m "region HA 2a: the load balancer health answer"
- Modify: `internal/provision/plan9_b8_census_test.go`, `internal/provision/plan14_root_reads_test.go` (the seal), the overlay
**Interfaces:**
-- Consumes: Task 3's `ParseBoxRole`, `ParseGroupRole`, `RoleName`, `SetBoxRole`, `SetGroupRole`; Task 4's `boxchoice.ExcludeRoles`; `apiSystemRoller` (ops account owners and `CARLOS_SYSTEM_OPERATORS`).
-- Produces: `POST /api/cli/system/boxes/{box}/role` and `POST /api/cli/system/groups/{region}/role`, body `{"role": "<name>"}`, answering `{"box", "role"}` and `{"region", "group", "role", "boxes"}`; `func (s *Server) boxRoster(ctx) (hostactors.Directory, release.Signer, hostactors.Observed, error)`; `func (s *Server) boxRoles(ctx) (map[string]string, error)`; CLI `setBoxRole`, `setGroupRole`.
+- Consumes: Task 3's `ParseBoxRole`, `ParseGroupRole`, `RoleName`, `SetBoxRole`, `SetGroupRole`; `apiSystemRoller` (ops account owners and `CARLOS_SYSTEM_OPERATORS`).
+- Produces: `POST /api/cli/system/boxes/{box}/role` and `POST /api/cli/system/groups/{region}/role`, body `{"role": "<name>"}`, answering `{"box", "role"}` and `{"region", "group", "role", "boxes"}`; `func (s *Server) boxRoster(ctx) (hostactors.Directory, release.Signer, hostactors.Observed, error)`; CLI `setBoxRole`, `setGroupRole`, `runSystemBoxVerb`, `runSystemGroupVerb`.
+
+**Wait for the copy.** This task writes RHA-C1 to RHA-C16 into console replies, CLI help and `docs/site/cli.md`. Start it only once Paul has approved those rows (Task 1, Step 4), and use his text where it differs from the drafts below. Until then `TestRegionHABuiltCopyIsApproved` fails, by design; do not mark a row approved to get past it.
-The doors are registered only with host claims on (the roster exists only then), closed to every service credential, and answer 404 unless `CARLOS_REGION_HA` is on too. A box is named by its roster reference or its label. With host claims on, the move door's destination skips legacy and platform boxes (the source is never filtered, so an instance on such a box can still move out, A10-3); an unreadable roster refuses the move. The move door then says "no box in `<to>` publishes an edges file yet" when the only eligible boxes are excluded: 2d's rework of the door gives that case its own sentence (open question 3).
+The doors are registered only with host claims on (the roster exists only then), closed to every service credential, and answer 404 unless `CARLOS_REGION_HA` is on too. A box is named by its roster reference or its label.
-**Census (flagged):** `Server.boxRoster` reads the signed roster (`Directory.Read`, kind `file:Read`), a new root-read unit. It is asked of platform-1e as census ruling region-ha-2a, accepted-legacy with main's basis for a console root input (J4). The seal moves.
+**Census ruling region-ha-2a, condition 2.** `Server.boxRoster` reads the signed roster (`Directory.Read`, kind `file:Read`) through the same source and verification as every other roster read. It is operator-only (it is reached only past `apiSystemRoller`, from the two role doors), it is read-only (the doors write through Task 3's `SetBoxRole` and `SetGroupRole`, which verify and sign as `Directory.Replace` always has), and the roles it carries are read by the placement chooser alone and never feed root's claim authority. The move door does not call it (Task 12 reads roles through the chooser). It is a new accepted-legacy unit (J4 root input), and the seal moves.
- [ ] **Step 1: Write the failing tests**
@@ -4014,35 +4283,6 @@ func TestRoleDoorsAreRegisteredWithHostClaims(t *testing.T) {
t.Fatalf("region HA off = %d, want 404", status)
}
}
-
-// Region HA spec 2 §7: a legacy or platform box is never a move's
-// destination, and an instance on one can still move out.
-func TestMoveDestinationSkipsExcludedRoles(t *testing.T) {
- const host, app = "acme.go.tito.io", "bab/titogo"
- fx := newConsoleFixture(t)
- armMoveFleet(t, fx, ieBox("ie-box"), ukBox("uk-box"))
- ie, uk := moveRosterBox(ieBox("ie-box")), moveRosterBox(ukBox("uk-box"))
- ie.Role, uk.Role = hostactors.RolePlatform, hostactors.RoleLegacy
- putMoveRoster(t, fx, hostactors.Roster{Boxes: map[string]hostactors.Box{"ie-box": ie, "uk-box": uk}})
- // Claims off: no roster roles apply, and main's selection stands.
- got, err := fx.srv.selectMoveBoxes(context.Background(), host, "uk", "ie", app)
- if err != nil || got.Target != "uk-box" {
- t.Fatalf("claims off: %+v %v", got, err)
- }
- fx.srv.cfg.HostClaim = true
- got, err = fx.srv.selectMoveBoxes(context.Background(), host, "uk", "ie", app)
- if err != nil {
- t.Fatal(err)
- }
- if got.Target != "" || got.Source != "ie-box" {
- t.Fatalf("selection %+v: want no target (uk-box is legacy) and the platform source kept", got)
- }
- // A roster that cannot be read refuses the move rather than guess.
- fx.srv.cfg.Releases = failingGet{Store: fx.rel, key: hostactors.Key}
- if _, err := fx.srv.selectMoveBoxes(context.Background(), host, "uk", "ie", app); err == nil {
- t.Fatal("an unreadable roster chose a destination")
- }
-}
````
`system_roles_test.go`:
@@ -4225,10 +4465,10 @@ The route censuses learn the two doors:
- [ ] **Step 2: Run them to see them fail**
-Run: `nice -n 19 go test ./internal/console/ -run 'BoxRoleDoor|GroupRoleDoor|RoleWriteErrors|RoleDoorsAreRegistered|MoveDestinationSkips' -count=1`
+Run: `nice -n 19 go test ./internal/console/ -run 'BoxRoleDoor|GroupRoleDoor|RoleWriteErrors|RoleDoorsAreRegistered' -count=1`
Expected: FAIL to build (`undefined: handleAPISystemBoxRole`, `undefined: boxRoster`). `go test . -run 'SystemVerbKnowsBoxAndGroup|SetBoxRole|SetGroupRole'` fails to build on `undefined: setBoxRole`.
-- [ ] **Step 3: Write the doors, the move filter and the CLI**
+- [ ] **Step 3: Write the doors and the CLI**
`internal/console/apiboxroles.go`:
@@ -4271,7 +4511,13 @@ func noGroupIn(region string) string { return fmt.Sprintf("%s has no group", reg
func retiredBoxRole(box string) string { return fmt.Sprintf("%s is retired, so it takes no role", box) }
-// boxRoster reads the signed roster with the root signer that writes it.
+// boxRoster reads the signed roster with the root signer that writes it,
+// for the role doors alone, which have already let only an operator in
+// (roleOperator). It is a read, of the one roster every reader reads
+// (control/host-actors.json through hostactors.Directory). Census ruling
+// region-ha-2a's conditions hold it to that: operator-only, read-only, the
+// same roster, and no role it reads feeds root's claim authority (roles
+// are read by the placement chooser alone).
func (s *Server) boxRoster(ctx context.Context) (hostactors.Directory, release.Signer, hostactors.Observed, error) {
if s.cfg.Releases == nil || s.cfg.Bucket == "" || len(s.cfg.Key) == 0 {
return hostactors.Directory{}, release.Signer{}, hostactors.Observed{}, errors.New("console: no deployment bucket or root key")
@@ -4403,20 +4649,6 @@ func (s *Server) handleAPISystemGroupRole(w http.ResponseWriter, r *http.Request
}
writeJSON(w, http.StatusOK, map[string]any{"region": region, "group": g.GroupName, "role": hostactors.RoleName(role), "boxes": boxes})
}
-
-// boxRoles is every enrolled box's role, for a chooser that reads no
-// roster of its own (the move door's destination, selectMoveBoxes).
-func (s *Server) boxRoles(ctx context.Context) (map[string]string, error) {
- _, _, o, err := s.boxRoster(ctx)
- if err != nil {
- return nil, err
- }
- roles := make(map[string]string, len(o.Roster.Boxes))
- for ref, b := range o.Roster.Boxes {
- roles[ref] = b.Role
- }
- return roles, nil
-}
````
````diff
@@ -4435,47 +4667,6 @@ func (s *Server) boxRoles(ctx context.Context) (map[string]string, error) {
// on, box by box. Same posture as the door above — an empty role set, the
````
-````diff
---- a/internal/console/boxselect.go
-+++ b/internal/console/boxselect.go
-@@ -7,6 +7,7 @@
- "log"
- "os"
-
-+ "github.com/carlosframework/platform/internal/boxchoice"
- "github.com/carlosframework/platform/internal/boxid"
- "github.com/carlosframework/platform/internal/moves"
- )
-@@ -89,6 +90,17 @@
- if err != nil {
- return moveSelection{}, err
- }
-+ // Region HA spec 2 §7: a legacy or platform box is never a move's
-+ // destination. The roles are the signed roster's, which exists only
-+ // with host claims on; a roster that cannot be read refuses the move.
-+ // The source is never filtered: an instance on such a box can still
-+ // move out (A10-3).
-+ var roles map[string]string
-+ if s.cfg.HostClaim {
-+ if roles, err = s.boxRoles(ctx); err != nil {
-+ return moveSelection{}, err
-+ }
-+ }
- sel := moveSelection{FloorOK: true, ConsoleBelowFloor: floor != nil && moves.ProtocolVersion < floor.Min}
- seen := map[string]bool{}
- for _, f := range snap.Files {
-@@ -113,6 +125,9 @@
- fp := s.expectedSecretsFP()
- for _, code := range []string{to, from} {
- cands := eligibleBoxes(snap, roster, code, aws[code], s.cfg.HomeRegion, fp, nil)
-+ if code == to {
-+ cands = boxchoice.ExcludeRoles(cands, roles)
-+ }
- if len(cands) > 1 {
- sel.Many, sel.ManyRegion = true, s.homeRegionOut(code)
- return sel, nil
-````
-
`system_roles.go`:
````go
@@ -4672,7 +4863,7 @@ func runSystemGroupVerb(rest []string) {
- [ ] **Step 4: Run the tests**
```bash
-nice -n 19 go test ./internal/console/ -run 'BoxRoleDoor|GroupRoleDoor|RoleWriteErrors|RoleDoorsAreRegistered|MoveDestinationSkips|SelectMoveBoxes|EveryCLIDoor|ClaimConsoleRouteCensus|NoRoleReaches' -count=1
+nice -n 19 go test ./internal/console/ -run 'BoxRoleDoor|GroupRoleDoor|RoleWriteErrors|RoleDoorsAreRegistered|EveryCLIDoor|ClaimConsoleRouteCensus|NoRoleReaches' -count=1
nice -n 19 go test ./internal/console/ -run 'TestClaimConsoleMutationInvariant$/probes/route_POST_/api/cli/system/' -count=1 -v | grep -E -- '--- (PASS|FAIL).*role'
nice -n 19 go test . -run 'SystemVerb|SetBoxRole|SetGroupRole' -count=1
nice -n 19 go test ./internal/docsite/ -count=1
@@ -4684,23 +4875,26 @@ Expected: PASS, and both role probes PASS. The second command runs only the prob
````diff
--- a/approved_region_ha_copy_test.go
+++ b/approved_region_ha_copy_test.go
-@@ -35,7 +35,7 @@
-
- // regionHACopyWired is every id whose text the code or docs carry. The
- // task that writes a row's text adds its id here in the same commit.
--var regionHACopyWired = []string{}
-+var regionHACopyWired = []string{"RHA-C1", "RHA-C2", "RHA-C3", "RHA-C4", "RHA-C5", "RHA-C6", "RHA-C7", "RHA-C8", "RHA-C9", "RHA-C10", "RHA-C11", "RHA-C12", "RHA-C13", "RHA-C14", "RHA-C15", "RHA-C16"}
+@@ -36,7 +36,7 @@
+ // regionHACopyBuilt is every id whose text the code or docs carry. The
+ // task that writes a row's text adds its id here in the same commit, once
+ // Paul has approved the row.
+-var regionHACopyBuilt = []string{}
++var regionHACopyBuilt = []string{"RHA-C1", "RHA-C2", "RHA-C3", "RHA-C4", "RHA-C5", "RHA-C6", "RHA-C7", "RHA-C8", "RHA-C9", "RHA-C10", "RHA-C11", "RHA-C12", "RHA-C13", "RHA-C14", "RHA-C15", "RHA-C16"}
func mustParseRegionHARecord(t *testing.T) []copyRow {
t.Helper()
````
+Run: `nice -n 19 go test . -run 'RegionHACopy|RegionHABuilt' -count=1`
+Expected: PASS once RHA-C1 to RHA-C16 are approved. While any is pending, `TestRegionHABuiltCopyIsApproved` fails naming it, and the task is not done.
+
````diff
--- a/region_ha_boundary_test.go
+++ b/region_ha_boundary_test.go
-@@ -24,6 +24,8 @@
- "internal/boxchoice/policy.go",
+@@ -25,6 +25,8 @@
"internal/host/boxaddress.go",
+ "internal/edgesfile/private.go",
"internal/edge/lbhealth.go",
+ "internal/console/apiboxroles.go",
+ "system_roles.go",
@@ -4725,18 +4919,18 @@ Expected: PASS, and both role probes PASS. The second command runs only the prob
{"internal/console/browserretry.go", "Server.handleInstanceRetry", "key-site", "lit:instances", "4d85d392f3660373f3c282c6391f58d8e25b6df738257ccc067b816e24dfe7ff"},
````
-Ask platform-1e for census ruling region-ha-2a with the `Server.boxRoster` rows below, and write them once it is given (with the ruling's own basis text if it differs; then recompute the fingerprint with `legacyrow.py` and the seal):
+The `Server.boxRoster` rows are census ruling region-ha-2a's (platform-1e). If the ruling's own basis text differs from the row's, use it, then recompute the fingerprint with `legacyrow.py` and the seal.
Add this comment line after the overlay's last comment line:
````text
-# Task 10: the operator's box and group role doors (internal/console apiboxroles.go, registered with host claims on in registerCLIRoutes), the move door's destination skipping legacy and platform boxes (selectMoveBoxes through boxRoles), and the CLI (system_roles.go). Census ruling region-ha-2a (platform-1e) is asked for one accepted-legacy unit, Server.boxRoster: the console's read of the signed roster (Directory.Read), the read every console roster reader makes, with the console's root signer. Its basis is main's for a console root input (J4). The seal moves.
+# Task 10: the operator's box and group role doors (internal/console apiboxroles.go, registered with host claims on in registerCLIRoutes) and the CLI (system_roles.go). Census ruling region-ha-2a (platform-1e, approved with conditions) covers one accepted-legacy unit, Server.boxRoster: the role doors' read of the signed roster (Directory.Read), operator-only (it is reached only past apiSystemRoller), read-only, the same roster every reader reads, and no role it reads feeds root's claim authority. Its basis is main's for a console root input (J4). The seal moves.
````
Then set these rows. A `@coverage` row replaces the overlay's row for the same directory, or is appended when there is none; every other row is appended:
````text
-@coverage internal/console 021626360903282245dfba6d99f505572a4388c4433c39e1aa3880ebb245e368
+@coverage internal/console bffe4b4c505d98b077d2e5e7906c32615ed62efb369da70eb18b4cc7084a1882
@coverage . 011b3548e5615fa1bb046e5a6d0b0e4345206ac7774c571a8320cfd124ba9af3
@legacy b02b534b6617f57381ba12e9d9a2bf1bd63f737dc68005de926ff18b7db93dd3 internal/console/apiboxroles.go Server.boxRoster legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;census-ruling-region-ha-2a file:Read@484 9be9925de14dbbd7d86123bf9faf1535c7b116575cadcb8ba983a665af0513f4
internal/console/apiboxroles.go Server.boxRoster legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;census-ruling-region-ha-2a file:Read@484 9be9925de14dbbd7d86123bf9faf1535c7b116575cadcb8ba983a665af0513f4
@@ -4756,31 +4950,45 @@ internal/console/apiboxroles.go Server.boxRoster legacy accepted-legacy accepted
if len(errs) > 0 {
````
-Run the census checks and `go test . -run 'TestRegionHA|RegionHACopy|RegionHAWired' -count=1`. Expected: PASS.
+Run the census checks and `go test . -run 'TestRegionHA(Core|Adapters|S3)' -count=1`. Expected: PASS.
- [ ] **Step 6: Commit**
```bash
-git add internal/console system.go system_roles.go system_roles_test.go docs/site/cli.md approved_region_ha_copy_test.go region_ha_boundary_test.go internal/provision
+git add internal/console/apiboxroles.go internal/console/apiboxroles_test.go internal/console/authcli.go internal/console/svcroutes_test.go internal/console/claim_invariant_fixtures_test.go internal/console/claim_invariant_probes_nonapp_test.go internal/console/testdata/claim-routes.txt system.go system_roles.go system_roles_test.go docs/site/cli.md approved_region_ha_copy_test.go region_ha_boundary_test.go internal/provision
git commit -m "region HA 2a: box and group role doors and the carlos system verbs"
```
---
-### Task 11: A create no box can take is refused (H27)
+### Task 11: The create doors choose first, and refuse when no box has room (H27)
**Files:**
+- Create: `internal/hostops/chosen.go` (`WithChosenExecutor`, `chosenFor`)
+- Modify: `internal/hostops/backfill.go` (`ProductionAuthority.SelectExecutor` honours the door's choice)
+- Modify: `internal/hostops/executor_test.go`
- Create: `internal/console/nocapacity.go`, `internal/console/nocapacity_test.go`
-- Modify: `internal/console/apiinstances.go` (`handleAPIInstanceCreate` refuses before `writeClaimResponseError`)
+- Modify: `internal/console/claim_create.go` (`beginClaimCreate` calls `chooseExecutorFirst` before `RegisterStart`)
+- Modify: `internal/console/claim_response.go` (`claimResponseRefusal` answers `noCapacityError` with a 503)
+- Modify: `internal/console/apiinstances.go` (`handleAPIInstanceCreate`'s app-derived region check uses the chooser with region HA on)
- Modify: `internal/console/server.go` (`Server.noCapacity`)
-- Modify: `internal/hostops/executor_test.go` (the claim is never written)
-- Modify: `approved_region_ha_copy_test.go` (wire RHA-C17 to RHA-C19), `region_ha_boundary_test.go`, `internal/provision/plan9_b8_census_test.go`, the overlay
+- Modify: `approved_region_ha_copy_test.go` (wire RHA-C17 to RHA-C19), `region_ha_boundary_test.go`, `internal/provision/plan9_b8_census_test.go`, `internal/provision/plan14_root_reads_test.go` (the seal), the overlay
**Interfaces:**
-- Consumes: `hostops.ErrNoEligibleBox` (from Task 6's `Choose`, through `ProductionAuthority.StartCreate`); `alerting.Page`; `Server.homeRegionOut`.
-- Produces: `func (s *Server) refuseNoCapacity(ctx context.Context, w http.ResponseWriter, err error, region string) bool`; `type noCapacityAlerts` (`due(region string, now time.Time) bool`); `var pageNoCapacity = alerting.Page` (the test seam).
+- Consumes: Task 6's `ExecutorChooser.PlaceableBoxes` and `Choose` (through `ProductionAuthority.SelectExecutor`); `hostops.ErrNoEligibleBox`; `alerting.Page`; `Server.homeRegionOut`.
+- Produces: `func hostops.WithChosenExecutor(ctx, host string, sel hostclaim.ExecutorSelection) context.Context`; `type noCapacityError struct{ Region string }` (`errors.Is(err, hostops.ErrNoEligibleBox)`); `type noCapacityAlerts` (`due(region string, now time.Time) bool`); `func (s *Server) noRoom(ctx, region string) error`; `func (s *Server) placementChooser(pool string) *hostops.ExecutorChooser` (Tasks 12 and 13 use it); `func (s *Server) chooseExecutorFirst(ctx, before hostclaim.Observed, build func(time.Time, string) (hostops.Record, error)) (context.Context, error)`; `var pageNoCapacity = alerting.Page` (the test seam).
+
+**Wait for the copy.** RHA-C17 to RHA-C19 are the refusal and the alert. Start once Paul has approved them, and use his text.
+
+H27 (spec rev 15), as Astra's findings 1 to 3 and 6 ask:
+- **Choose before registering (finding 1).** Every create through the claim starts in `beginClaimCreate`. With region HA on, `chooseExecutorFirst` builds the record the create would write, asks `SelectExecutor` for its executor, and returns before `RegisterStart` when there is none. Otherwise it puts the choice in the context, and `SelectExecutor` returns that choice to `StartCreate` (claim Plan 5's checks still run on it), so the create names exactly the box the door checked. A refused request leaves no registration, no claim and nothing for a recovery pass.
+- **Every choosing door (finding 2).** One error, `noCapacityError`, answered in one place, `claimResponseRefusal`: instance create, domain attach's first route (the API's `handleAPIDomainAttach` and the browser's `handleAppDomainAttach`, both through `attachClaimFirstRoute`, which is unchanged), and standalone sidecar create. The move door's destination is Task 12. A sidecar attached to an instance chooses nothing (it lives on its instance's box) and is unchanged.
+- **The earlier region check (finding 3).** `handleAPIInstanceCreate` derives an app's region from `regionCandidates` before the claim. With region HA on, that region is judged by the chooser's eligible set instead: no placeable box is the no-room refusal, not a 400, and a misconfigured box beside an eligible one no longer blocks the create.
+- **The alert limit (finding 6).** At most one alert per region an hour, remembered in the console process; a restart may send one more, which H27 accepts.
-`ProductionAuthority.StartCreate` chooses its executor before any claim write, so a create the chooser refuses writes nothing to the claim: `TestStartCreateWithNoEligibleBoxWritesNothing` pins it against the store. The door answers 503 with `Retry-After: 60` and sentence RHA-C17, and pages at `warning`, at most once per region an hour (in memory: a console restart may page once more). Inert with region HA off.
+With region HA off nothing here runs: `chooseExecutorFirst` returns at once and the create is main's.
+
+**Census (platform-1e):** `Server.beginClaimCreate` is an accepted-legacy unit (J4 root input). It gains one call before `RegisterStart`; its one read site is unchanged. The overlay re-freezes it with the ruling's tag, and the seal moves. `Server.attachClaimFirstRoute` does not change. The record-write census digest of `handleAPIInstanceCreate` changes.
- [ ] **Step 1: Write the failing tests**
@@ -4795,128 +5003,374 @@ import (
"errors"
"fmt"
"net/http"
- "net/http/httptest"
+ "strings"
"testing"
"time"
+ "github.com/carlosframework/platform/internal/hostclaim"
"github.com/carlosframework/platform/internal/hostops"
+ "github.com/carlosframework/platform/internal/provision"
+ "github.com/carlosframework/platform/internal/status"
)
-// Region HA spec 2 §7 (H27): a create no box can take is refused with a
-// 503 to retry, and the operator hears of it once per region an hour.
-func TestCreateWithNoEligibleBoxIsRefusedAndAlertedOncePerHour(t *testing.T) {
- pages := make(chan string, 8)
+func TestNoCapacityAlertsRepeatAfterAnHour(t *testing.T) {
+ var a noCapacityAlerts
+ t0 := time.Date(2026, 10, 16, 12, 0, 0, 0, time.UTC)
+ if !a.due("uk", t0) || a.due("uk", t0.Add(59*time.Minute)) || !a.due("ie", t0) || !a.due("uk", t0.Add(time.Hour)) {
+ t.Fatal("once per region an hour")
+ }
+}
+
+func TestClaimRefusalAnswersNoRoom(t *testing.T) {
+ ref := claimResponseRefusal(fmt.Errorf("begin: %w", &noCapacityError{Region: "uk"}))
+ if ref.Status != http.StatusServiceUnavailable || ref.Msg != noCapacityRefusal("uk") {
+ t.Fatalf("refusal %+v", ref)
+ }
+ var err error = &noCapacityError{Region: "uk"}
+ if !errors.Is(err, hostops.ErrNoEligibleBox) {
+ t.Fatal("a no-room refusal is not ErrNoEligibleBox")
+ }
+}
+
+// pagesTo replaces the alert sender for the test and returns what it sent.
+func pagesTo(t *testing.T) chan string {
+ t.Helper()
+ pages := make(chan string, 16)
old := pageNoCapacity
pageNoCapacity = func(_ context.Context, level, subject, _ string) error {
pages <- level + " " + subject
return nil
}
t.Cleanup(func() { pageNoCapacity = old })
- s := &Server{cfg: Config{RegionHA: true, HomeRegion: "ie"}}
- refuse := func(err error, region string) (*httptest.ResponseRecorder, bool) {
- w := httptest.NewRecorder()
- return w, s.refuseNoCapacity(context.Background(), w, err, region)
- }
- if _, ok := refuse(errors.New("SlowDown"), "uk"); ok {
- t.Fatal("another error was refused as no capacity")
- }
- wrapped := fmt.Errorf("%w: box-a", hostops.ErrNoEligibleBox)
- w, ok := refuse(wrapped, "uk")
- if !ok || w.Code != http.StatusServiceUnavailable || w.Header().Get("Retry-After") != "60" {
- t.Fatalf("refusal: %v %d %v", ok, w.Code, w.Header())
- }
- var out map[string]string
- if err := json.Unmarshal(w.Body.Bytes(), &out); err != nil || out["error"] != noCapacityRefusal("uk") {
- t.Fatalf("body %s", w.Body.String())
- }
- if got := <-pages; got != "warning "+mustSubject("uk") {
- t.Fatalf("page %q", got)
- }
- // A retry loop: refused every time, paged once.
- for range 5 {
- if _, ok := refuse(hostops.ErrNoEligibleBox, "uk"); !ok {
- t.Fatal("a retry was not refused")
+ return pages
+}
+
+func expectPages(t *testing.T, pages chan string, want ...string) {
+ t.Helper()
+ for _, w := range want {
+ select {
+ case got := <-pages:
+ if got != w {
+ t.Fatalf("page %q, want %q", got, w)
+ }
+ case <-time.After(5 * time.Second):
+ t.Fatalf("no page %q", w)
}
}
- // The home region by its name, and its own alert.
- if w, _ := refuse(hostops.ErrNoEligibleBox, ""); !jsonHas(w, noCapacityRefusal("ie")) {
- t.Fatalf("home refusal %s", w.Body.String())
- }
- if got := <-pages; got != "warning "+mustSubject("ie") {
- t.Fatalf("home page %q", got)
- }
select {
case got := <-pages:
- t.Fatalf("paged again within the hour: %q", got)
+ t.Fatalf("an extra page: %q", got)
case <-time.After(100 * time.Millisecond):
}
- s.cfg.RegionHA = false
- if _, ok := refuse(hostops.ErrNoEligibleBox, "uk"); ok {
- t.Fatal("region HA off: the door's main answer must stand")
- }
-}
-
-func TestNoCapacityAlertsRepeatAfterAnHour(t *testing.T) {
- var a noCapacityAlerts
- t0 := time.Date(2026, 10, 16, 12, 0, 0, 0, time.UTC)
- if !a.due("uk", t0) || a.due("uk", t0.Add(59*time.Minute)) || !a.due("ie", t0) || !a.due("uk", t0.Add(time.Hour)) {
- t.Fatal("once per region an hour")
- }
}
-func mustSubject(region string) string {
+func noRoomSubject(region string) string {
subject, _ := noCapacityAlert(region)
- return subject
+ return "warning " + subject
}
-func jsonHas(w *httptest.ResponseRecorder, msg string) bool {
- var out map[string]string
- return json.Unmarshal(w.Body.Bytes(), &out) == nil && out["error"] == msg
+// haClaimCase is a host-claim console with region HA on: its create doors
+// choose through the region HA chooser over the edges files the test
+// publishes (publishHABox). home names the console's home region ("" for
+// none): a first route is placed in the home.
+func haClaimCase(t *testing.T, home string) *claimCase {
+ t.Helper()
+ cx := newClaimCaseWith(t, func(c *Config) { c.HomeRegion = home })
+ cx.srv.cfg.RegionHA = true
+ a := cx.srv.cfg.ClaimAuthority
+ a.Executors = cx.srv.executorChooser(a.Directory)
+ return cx
}
-````
-````diff
---- a/internal/hostops/executor_test.go
-+++ b/internal/hostops/executor_test.go
-@@ -488,3 +488,30 @@
- t.Fatalf("unbound: %v", err)
- }
- }
-+
-+// Region HA spec 2 §7 (H27): with every box ineligible a create is
-+// refused before anything is written. The executor is chosen first, so no
-+// claim without one ever exists.
-+func TestStartCreateWithNoEligibleBoxWritesNothing(t *testing.T) {
-+ ctx := context.Background()
-+ x, s, snap, _ := chooserFixture(t, "box-a", "box-b")
-+ for i := range snap.Boxes {
-+ snap.Boxes[i].Headroom.MemFreeMiB = 100 // under 15% of 8000
-+ }
-+ r := chooserRecord()
-+ a := ProductionAuthority{Store: s, Directory: x.Directory, Namespace: "platform", Now: x.Now, Executors: &x}
-+ before, err := s.List(ctx, "")
-+ if err != nil {
-+ t.Fatal(err)
+// publishHABox publishes a roster box's edges file and status snapshot: placeable
+// in ie unless mut changes it, and stale when stale is set.
+func publishHABox(t *testing.T, cx *claimCase, box string, stale bool, mut func(map[string]any)) {
+ t.Helper()
+ ctx := context.Background()
+ m := map[string]any{
+ "v": 1, "box": box, "box_id": box, "eip": "203.0.113.7", "region": "eu-west-1", "placement": "ie",
+ "hibernation_configured": true, "serves_shared_pool": true, "accepts_inbound": true,
+ "secrets_key_fp": cx.srv.expectedSecretsFP(), "capabilities": []string{"region-ha-1"},
+ "headroom": map[string]any{"mem_free_mib": 4000, "mem_total_mib": 8000, "disk_free_mib": 50000, "disk_total_mib": 100000, "awake": 0},
+ "generation": time.Now().UTC().Format(time.RFC3339), "hosts": map[string]any{},
+ }
+ if mut != nil {
+ mut(m)
+ }
+ body, err := json.Marshal(m)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if _, err := cx.rel.Put(ctx, "control/edges/"+box+".json", body, ""); err != nil {
+ t.Fatal(err)
+ }
+ at := time.Now()
+ if stale {
+ at = at.Add(-time.Hour)
+ }
+ if err := status.WriteBox(ctx, cx.rel, status.Box{Label: box, GeneratedAt: at}); err != nil {
+ t.Fatal(err)
+ }
+}
+
+// registrations counts account/app's registered operations.
+func registrations(t *testing.T, cx *claimCase, account, app string) int {
+ t.Helper()
+ a, err := hostops.ReadCanonicalAdmission(context.Background(), cx.rel, account, app)
+ if err != nil {
+ t.Fatal(err)
+ }
+ return len(a.Value.Registrations)
+}
+
+// refusedBeforeRegistering checks what H27 promises of a refused create:
+// nothing registered, no claim, and nothing recovery writes afterwards.
+func refusedBeforeRegistering(t *testing.T, cx *claimCase, host string, registered int) {
+ t.Helper()
+ ctx := context.Background()
+ if n := registrations(t, cx, "bab", "titogo"); n != registered {
+ t.Fatalf("registrations %d after the refusal, want %d: the door registered before choosing", n, registered)
+ }
+ if obs, err := hostclaim.Read(ctx, cx.rel, host); err != nil || !obs.Absent {
+ t.Fatalf("claim after the refusal: %+v %v", obs, err)
+ }
+ if err := cx.srv.claimRecoveryPass(ctx); err != nil {
+ t.Fatalf("recovery: %v", err)
+ }
+ if obs, err := hostclaim.Read(ctx, cx.rel, host); err != nil || !obs.Absent {
+ t.Fatalf("claim after recovery: %+v %v", obs, err)
+ }
+ if n := registrations(t, cx, "bab", "titogo"); n != registered {
+ t.Fatalf("registrations %d after recovery, want %d", n, registered)
+ }
+}
+
+// Region HA spec 2 §7 (H27): every create door that chooses a box refuses
+// with a 503 when none is eligible, before it registers anything, and the
+// operator hears of it once per region an hour. Every box here is stale.
+func TestCreateDoorsRefuseWithNoRoomBeforeRegistering(t *testing.T) {
+ t.Run("instance create", func(t *testing.T) {
+ pages := pagesTo(t)
+ cx := haClaimCase(t, "")
+ cx.instanceWorld(t)
+ publishHABox(t, cx, "box-ie-tito", true, nil)
+ before := registrations(t, cx, "bab", "titogo")
+ for range 3 { // a retry loop: refused every time, paged once
+ r := cx.apiReply(t, http.MethodPost, "/api/cli/apps/bab/titogo/instances", `{"host":"acme.go.tito.io","region":"ie"}`)
+ if r.Status != http.StatusServiceUnavailable || !strings.Contains(r.Body, noCapacityRefusal("ie")) {
+ t.Fatalf("create = %d %s", r.Status, r.Body)
+ }
+ }
+ refusedBeforeRegistering(t, cx, "acme.go.tito.io", before)
+ expectPages(t, pages, noRoomSubject("ie"))
+ })
+ t.Run("standalone sidecar create", func(t *testing.T) {
+ pages := pagesTo(t)
+ cx := haClaimCase(t, "")
+ cx.sidecarWorld(t)
+ before := registrations(t, cx, "bab", "titogo")
+ r := cx.apiReply(t, http.MethodPost, "/api/cli/apps/bab/titogo/sidecars", `{"name":"bot","region":"ie"}`)
+ if r.Status != http.StatusServiceUnavailable || !strings.Contains(r.Body, noCapacityRefusal("ie")) {
+ t.Fatalf("sidecar create = %d %s", r.Status, r.Body)
+ }
+ refusedBeforeRegistering(t, cx, provision.SidecarHost("bot", "titogo", "bab"), before)
+ expectPages(t, pages, noRoomSubject("ie"))
+ })
+ t.Run("domain attach, a first route", func(t *testing.T) {
+ pages := pagesTo(t)
+ cx := haClaimCase(t, "ie")
+ shipVersion(t, cx.consoleFixture, "titogo", "v1")
+ seedChannel(t, cx.consoleFixture, "titogo", "edge", "v1")
+ before := registrations(t, cx, "bab", "titogo")
+ r := cx.apiReply(t, http.MethodPost, "/api/cli/apps/bab/titogo/domains", `{"host":"first.titogo-claim.example","kind":"static"}`)
+ if r.Status != http.StatusServiceUnavailable || !strings.Contains(r.Body, noCapacityRefusal("ie")) {
+ t.Fatalf("attach = %d %s", r.Status, r.Body)
+ }
+ refusedBeforeRegistering(t, cx, "first.titogo-claim.example", before)
+ expectPages(t, pages, noRoomSubject("ie"))
+ })
+ t.Run("browser domain attach, a first route", func(t *testing.T) {
+ pages := pagesTo(t)
+ cx := haClaimCase(t, "ie")
+ shipVersion(t, cx.consoleFixture, "titogo", "v1")
+ seedChannel(t, cx.consoleFixture, "titogo", "edge", "v1")
+ before := registrations(t, cx, "bab", "titogo")
+ r := cx.formReply(t, "/a/bab/apps/titogo/domains/attach", "host", "first.titogo-claim.example", "kind", "static")
+ if r.Status != http.StatusServiceUnavailable || !strings.Contains(r.Body, noCapacityRefusal("ie")) {
+ t.Fatalf("browser attach = %d %s", r.Status, r.Body)
+ }
+ refusedBeforeRegistering(t, cx, "first.titogo-claim.example", before)
+ expectPages(t, pages, noRoomSubject("ie"))
+ })
+}
+
+// appDerivedWorld is titogo with no pool and no region allow-list, whose
+// app region is ie: a create that names no region takes the app's.
+func appDerivedWorld(t *testing.T, cx *claimCase) {
+ t.Helper()
+ m := titoManifest()
+ m.Regions = nil
+ m.Pool = ""
+ enableInstances(t, cx.consoleFixture, "titogo", m)
+ cx.srv.appRegionOf = func(int64, string) (string, string, error) { return "ie", "app", nil }
+}
+
+// Region HA spec 2 §7 (H27, Astra 2a round 1 finding 3): with region HA
+// on, an app-derived region is judged by the chooser's eligible set. Every
+// box stale is the no-room refusal, alert included, never the region
+// refusal's 400.
+func TestAppRegionWithEveryBoxStaleIsNoRoom(t *testing.T) {
+ pages := pagesTo(t)
+ cx := haClaimCase(t, "")
+ appDerivedWorld(t, cx)
+ publishHABox(t, cx, "box-ie-default", true, nil)
+ before := registrations(t, cx, "bab", "titogo")
+ r := cx.apiReply(t, http.MethodPost, "/api/cli/apps/bab/titogo/instances", `{"host":"acme.go.tito.io"}`)
+ if r.Status != http.StatusServiceUnavailable || !strings.Contains(r.Body, noCapacityRefusal("ie")) {
+ t.Fatalf("create = %d %s", r.Status, r.Body)
+ }
+ refusedBeforeRegistering(t, cx, "acme.go.tito.io", before)
+ expectPages(t, pages, noRoomSubject("ie"))
+}
+
+// ... and a misconfigured sibling (a box in ie holding another secrets
+// key, outside the roster) no longer refuses a region whose eligible set
+// holds another box: the create goes ahead on that box. With region HA off
+// the sibling still refuses, as on main.
+func TestAppRegionWithAMisconfiguredSiblingStillCreates(t *testing.T) {
+ cx := haClaimCase(t, "")
+ appDerivedWorld(t, cx)
+ publishHABox(t, cx, "box-ie-default", false, nil)
+ publishHABox(t, cx, "box-ie-stray", false, func(m map[string]any) { m["secrets_key_fp"] = "sha256:another-key" })
+ off := cx.srv.placementChooser("")
+ cx.srv.cfg.RegionHA = false
+ r := cx.apiReply(t, http.MethodPost, "/api/cli/apps/bab/titogo/instances", `{"host":"other.go.tito.io"}`)
+ if r.Status != http.StatusBadRequest {
+ t.Fatalf("region HA off: create = %d %s, want main's region refusal", r.Status, r.Body)
+ }
+ cx.srv.cfg.RegionHA = true
+ if off == nil {
+ t.Fatal("no chooser with region HA on")
+ }
+ r = cx.apiReply(t, http.MethodPost, "/api/cli/apps/bab/titogo/instances", `{"host":"acme.go.tito.io"}`)
+ if r.Status != http.StatusCreated && r.Status != http.StatusAccepted {
+ t.Fatalf("create = %d %s", r.Status, r.Body)
+ }
+ obs, err := hostclaim.Read(context.Background(), cx.rel, "acme.go.tito.io")
+ if err != nil || obs.Absent {
+ t.Fatalf("claim: %+v %v", obs, err)
+ }
+ if exec, chosen := obs.Claim.Op.Exec, obs.Claim.Op.Intent["selected_executor"]; exec != "box-ie-default" && chosen != "box-ie-default" {
+ t.Fatalf("executor %q / %q, want box-ie-default", exec, chosen)
+ }
+}
+````
+
+````diff
+--- a/internal/hostops/executor_test.go
++++ b/internal/hostops/executor_test.go
+@@ -491,3 +491,31 @@
+ t.Fatalf("unbound: %v", err)
+ }
+ }
++
++// Region HA spec 2 §7 (H27): a door that chose before registering holds
++// the create to its choice, for that host only; Plan 5's scope still
++// binds it (TestProductionCreateSelectsThroughTheChooser).
++func TestSelectExecutorHonoursTheDoorsChoice(t *testing.T) {
++ ctx := context.Background()
++ x, _, snap, _ := chooserFixture(t, "box-a", "box-b")
++ a := ProductionAuthority{Namespace: "platform", Executors: &x}
++ r := chooserRecord()
++ absent := hostclaim.Observed{Host: r.Holder.Host, Absent: true}
++ chosen := hostclaim.ExecutorSelection{Box: "box-b", Enrollment: strings.Repeat("b", 32)}
++ // Every box is now ineligible, yet the door's earlier choice stands.
++ for i := range snap.Boxes {
++ snap.Boxes[i].Draining = true
+ }
-+ if _, err := a.StartCreate(ctx, hostclaim.Observed{Host: r.Holder.Host, Absent: true}, r, "op-1", "paul@keymail.dev"); !errors.Is(err, ErrNoEligibleBox) {
-+ t.Fatalf("StartCreate = %v, want ErrNoEligibleBox", err)
++ sel, ok, err := a.SelectExecutor(WithChosenExecutor(ctx, r.Holder.Host, chosen), absent, r)
++ if err != nil || !ok || sel != chosen {
++ t.Fatalf("with the door's choice: %+v %v %v", sel, ok, err)
+ }
-+ if obs, err := hostclaim.Read(ctx, s, r.Holder.Host); err != nil || !obs.Absent {
-+ t.Fatalf("claim after the refusal: %+v %v", obs, err)
++ // Another host's choice is not this create's.
++ if _, _, err := a.SelectExecutor(WithChosenExecutor(ctx, "other.bab.oncarlos.com", chosen), absent, r); !errors.Is(err, ErrNoEligibleBox) {
++ t.Fatalf("another host's choice was used: %v", err)
+ }
-+ after, err := s.List(ctx, "")
-+ if err != nil || len(after) != len(before) {
-+ t.Fatalf("objects before %v, after %v (%v)", before, after, err)
++ // No chooser (region HA off): no selection, whatever the context says.
++ if _, ok, err := (ProductionAuthority{}).SelectExecutor(WithChosenExecutor(ctx, r.Holder.Host, chosen), absent, r); ok || err != nil {
++ t.Fatalf("region HA off selected: %v %v", ok, err)
+ }
+}
````
- [ ] **Step 2: Run them to see them fail**
-Run: `nice -n 19 go test ./internal/console/ -run 'NoEligibleBox|NoCapacity' -count=1`
-Expected: FAIL to build (`undefined: pageNoCapacity`, `s.refuseNoCapacity undefined`). The hostops test passes already: it pins what Task 6 left true.
+Run: `nice -n 19 go test ./internal/console/ -run 'NoCapacity|ClaimRefusalAnswersNoRoom|CreateDoorsRefuse|AppRegionWith' -count=1`
+Expected: FAIL to build (`undefined: pageNoCapacity`, `undefined: noCapacityError`). The hostops test fails to build on `undefined: WithChosenExecutor`.
+
+- [ ] **Step 3: Write the early choice and the refusal**
+
+`internal/hostops/chosen.go`:
+
+````go
+package hostops
+
+import (
+ "context"
+
+ "github.com/carlosframework/platform/internal/hostclaim"
+)
+
+// A door that chooses its executor before it registers anything (region HA
+// spec 2 §7, H27) carries the choice to the create through the context, so
+// the create names exactly the box the door checked, and a refusal leaves
+// nothing registered, nothing written to the claim and nothing for
+// recovery to settle. selectExecutor still checks the choice against
+// claim Plan 5's scope (namespace, region, pool, active, unfenced).
+
+type chosenExecutorKey struct{}
+
+type chosenExecutor struct {
+ host string
+ sel hostclaim.ExecutorSelection
+}
+
+// WithChosenExecutor records sel as host's executor for the create that
+// ctx carries.
+func WithChosenExecutor(ctx context.Context, host string, sel hostclaim.ExecutorSelection) context.Context {
+ return context.WithValue(ctx, chosenExecutorKey{}, chosenExecutor{host: host, sel: sel})
+}
+
+// chosenFor is the executor ctx carries for host, if any.
+func chosenFor(ctx context.Context, host string) (hostclaim.ExecutorSelection, bool) {
+ c, ok := ctx.Value(chosenExecutorKey{}).(chosenExecutor)
+ if !ok || c.host == "" || c.host != host || c.sel.Box == "" || c.sel.Enrollment == "" {
+ return hostclaim.ExecutorSelection{}, false
+ }
+ return c.sel, true
+}
+````
-- [ ] **Step 3: Write the refusal**
+````diff
+--- a/internal/hostops/backfill.go
++++ b/internal/hostops/backfill.go
+@@ -107,9 +107,15 @@
+ // before the create CAS, when a chooser is configured; otherwise no
+ // selection, and the create is today's. The chooser chooses within claim
+ // Plan 5's scope: this authority's namespace and the record's pool.
++//
++// A door that already chose for this host (WithChosenExecutor) is held to
++// that choice, so the create names the box the door checked (H27).
+ func (a ProductionAuthority) SelectExecutor(ctx context.Context, obs hostclaim.Observed, r Record) (hostclaim.ExecutorSelection, bool, error) {
+ if a.Executors == nil {
+ return hostclaim.ExecutorSelection{}, false, nil
++ }
++ if sel, ok := chosenFor(ctx, r.Holder.Host); ok {
++ return sel, true, nil
+ }
+ var shape struct {
+ Pool string `json:"pool"`
+````
`internal/console/nocapacity.go`:
@@ -4928,39 +5382,52 @@ import (
"errors"
"fmt"
"log"
- "net/http"
"sync"
"time"
"github.com/carlosframework/platform/internal/alerting"
+ "github.com/carlosframework/platform/internal/hostclaim"
"github.com/carlosframework/platform/internal/hostops"
)
-// A create no box can take (region HA spec 2 §7, ruling H27). When the
-// placement policy finds no eligible box, the create door refuses the
-// create with a 503 the client may retry, and the claim is never written:
-// ProductionAuthority.StartCreate chooses its executor before any claim
-// write, so no create without an executor exists. An operator alert fires
-// too, at most once per region an hour, so a client retrying in a loop
-// does not flood the operator.
+// No box has room (region HA spec 2 §7, ruling H27). Every door that
+// chooses a box (instance create, domain attach's first route, standalone
+// sidecar create, and the move door's destination) refuses with a 503 when
+// the placement policy finds no eligible box, and the operator hears of it
+// at most once per region an hour per console process: a restart may send
+// one extra alert, which H27 accepts rather than persist the limit. A
+// create door chooses before it registers anything (chooseExecutorFirst),
+// so a refusal leaves nothing registered, nothing written to the claim and
+// nothing for recovery to settle.
// noCapacityAlertEvery is how often one region's alert may repeat.
const noCapacityAlertEvery = time.Hour
func noCapacityRefusal(region string) string {
- return fmt.Sprintf("no box in %s can take a new instance right now. Try again in a few minutes.", region)
+ return fmt.Sprintf("No box in %s has room right now. Try again in a few minutes.", region)
}
func noCapacityAlert(region string) (subject, body string) {
- return fmt.Sprintf("No room for new instances in %s", region),
- fmt.Sprintf("A create in %s was refused because no box there can take a new instance. Every box is full, stale or excluded by its role. Add a box, or free memory or disk on one, and check the eligible-capacity alarm for %s.", region, region)
+ return fmt.Sprintf("No box in %s has room", region),
+ fmt.Sprintf("A request in %s was refused because no box there can take it. Every box is full, stale or excluded by its role. Add a box, or free memory or disk on one, and check the eligible-capacity alarm for %s.", region, region)
}
+// noCapacityError is a request refused because no box in Region (its
+// display name) can take it. It is hostops.ErrNoEligibleBox, and
+// claimResponseRefusal answers it with noCapacityRefusal.
+type noCapacityError struct{ Region string }
+
+func (e *noCapacityError) Error() string {
+ return "no eligible box in " + e.Region + ": " + hostops.ErrNoEligibleBox.Error()
+}
+
+func (e *noCapacityError) Is(target error) bool { return target == hostops.ErrNoEligibleBox }
+
// pageNoCapacity sends the alert; tests replace it.
var pageNoCapacity = alerting.Page
-// noCapacityAlerts remembers when each region's alert was last sent. The
-// zero value is ready.
+// noCapacityAlerts remembers when each region's alert was last sent, in
+// this console process. The zero value is ready.
type noCapacityAlerts struct {
mu sync.Mutex
last map[string]time.Time
@@ -4981,15 +5448,12 @@ func (a *noCapacityAlerts) due(region string, now time.Time) bool {
return true
}
-// refuseNoCapacity answers a create whose placement found no eligible
-// box, and reports whether it did. region is the record's placement code
-// ("" for the home). Inert with region HA off.
-func (s *Server) refuseNoCapacity(ctx context.Context, w http.ResponseWriter, err error, region string) bool {
- if !s.cfg.RegionHA || !errors.Is(err, hostops.ErrNoEligibleBox) {
- return false
- }
+// noRoom is the refusal for region (record vocabulary, "" for the home):
+// it logs, sends the operator alert when one is due, and returns the
+// error the door answers with.
+func (s *Server) noRoom(ctx context.Context, region string) error {
name := s.homeRegionOut(region)
- log.Printf("console: create refused: no eligible box in %s: %v", name, err)
+ log.Printf("console: refused: no eligible box in %s", name)
if s.noCapacity.due(name, time.Now()) {
subject, body := noCapacityAlert(name)
go func() {
@@ -5000,49 +5464,133 @@ func (s *Server) refuseNoCapacity(ctx context.Context, w http.ResponseWriter, er
}
}()
}
- w.Header().Set("Retry-After", "60")
- jsonError(w, http.StatusServiceUnavailable, noCapacityRefusal(name))
- return true
+ return &noCapacityError{Region: name}
+}
+
+// placementChooser is the create door's chooser scoped as
+// ProductionAuthority.SelectExecutor scopes it, for pool ("" the shared
+// pool), or nil with region HA off or no claim runtime.
+func (s *Server) placementChooser(pool string) *hostops.ExecutorChooser {
+ a := s.cfg.ClaimAuthority
+ if !s.cfg.RegionHA || a == nil || a.Executors == nil {
+ return nil
+ }
+ x := *a.Executors
+ x.Namespace, x.Pool = a.Namespace, pool
+ return &x
+}
+
+// chooseExecutorFirst is beginClaimCreate's first step with region HA on:
+// it chooses the executor for the record build would make, before anything
+// is registered, and carries the choice to the create
+// (hostops.WithChosenExecutor). No eligible box is noRoom's refusal. With
+// region HA off it returns ctx unchanged, and the create is main's.
+func (s *Server) chooseExecutorFirst(ctx context.Context, before hostclaim.Observed, build func(time.Time, string) (hostops.Record, error)) (context.Context, error) {
+ a := s.cfg.ClaimAuthority
+ if !s.cfg.RegionHA || a == nil || a.Executors == nil {
+ return ctx, nil
+ }
+ probe, err := build(time.Now().UTC(), "")
+ if err != nil {
+ return ctx, err
+ }
+ sel, chosen, err := a.SelectExecutor(ctx, before, probe)
+ if errors.Is(err, hostops.ErrNoEligibleBox) {
+ return ctx, s.noRoom(ctx, probe.Holder.RecordRegion)
+ }
+ if err != nil || !chosen {
+ return ctx, err
+ }
+ return hostops.WithChosenExecutor(ctx, probe.Holder.Host, sel), nil
}
````
````diff
--- a/internal/console/server.go
+++ b/internal/console/server.go
-@@ -837,6 +837,11 @@
+@@ -837,6 +837,10 @@
type Server struct {
claimLossPageMu sync.Mutex
claimLossPageDone chan struct{}
+
+ // noCapacity limits the no-capacity alert to once per region an hour
-+ // (region HA spec 2 §7, nocapacity.go).
++ // in this process (region HA spec 2 §7, H27, nocapacity.go).
+ noCapacity noCapacityAlerts
-+
// Enabled claim runtime (ConfigureClaimRuntime). Unset flag-off.
claimRuntimeConfigured bool
claimOwner hostops.RegistrationOwner
````
+````diff
+--- a/internal/console/claim_create.go
++++ b/internal/console/claim_create.go
+@@ -37,6 +37,10 @@
+ }
+ before, err := hostclaim.Read(ctx, s.cfg.Releases, host)
+ if err != nil {
++ return nil, err
++ }
++ // Region HA (spec 2 §7, H27): choose before registering anything.
++ if ctx, err = s.chooseExecutorFirst(ctx, before, build); err != nil {
+ return nil, err
+ }
+ appID, err := s.store.ClaimAppID(acct.ID, app)
+````
+
+````diff
+--- a/internal/console/claim_response.go
++++ b/internal/console/claim_response.go
+@@ -15,6 +15,11 @@
+
+ func claimResponseRefusal(err error) *doorRefusal {
+ log.Printf("console: claim request refused: %v", err)
++ // No box has room (region HA spec 2 §7, H27): every create door's 503.
++ var full *noCapacityError
++ if errors.As(err, &full) {
++ return &doorRefusal{http.StatusServiceUnavailable, noCapacityRefusal(full.Region)}
++ }
+ if errors.Is(err, hostops.ErrPending) || errors.Is(err, hostclaim.ErrConflict) || errors.Is(err, hostclaim.ErrWrongState) || errors.Is(err, release.ErrCASConflict) {
+ return &doorRefusal{http.StatusConflict, claimRetireBusyMsg}
+ }
+````
+
````diff
--- a/internal/console/apiinstances.go
+++ b/internal/console/apiinstances.go
-@@ -1588,6 +1588,11 @@
- return prepared, err
- })
- if createErr != nil {
-+ // No box can take it (region HA spec 2 §7, H27): a 503 to retry,
-+ // and nothing was written to the claim.
-+ if s.refuseNoCapacity(r.Context(), w, createErr, rec.Region) {
+@@ -1259,6 +1259,25 @@
+ if c, ok := findCandidate(cands, region); ok {
+ reason = c.unavailableReason()
+ }
++ // Region HA (spec 2 §7, H27): the region is judged by the eligible
++ // set the create door's chooser will choose from, never by every
++ // sibling's own answer, and an empty set is the door's no-room
++ // refusal, alert included.
++ if x := s.placementChooser(""); x != nil {
++ switch boxes, err := x.PlaceableBoxes(ctx, s.cfg.claimPlacementRegion(region)); {
++ case errors.Is(err, hostops.ErrContract):
++ reason = "absent"
++ case err != nil:
++ log.Printf("console: api instances create %s/%s: region %s placeable boxes: %v", acct.Sqid, app, region, err)
++ jsonError(w, http.StatusServiceUnavailable, msgAppRegionUnconfirmed(s.homeRegionOut(region)))
++ return
++ case len(boxes) == 0:
++ writeClaimResponseError(w, s.noRoom(ctx, region))
+ return
++ default:
++ reason = ""
+ }
- writeClaimResponseError(w, createErr)
- return
- }
++ }
+ // Each list names only what an explicit request for this same
+ // instance would pass (codex plan-R4 2): instanceChoices.
+ if reason != "" {
````
- [ ] **Step 4: Run the tests**
-Run: `nice -n 19 go test ./internal/console/ -run 'NoEligibleBox|NoCapacity' -count=1 && nice -n 19 go test ./internal/hostops/ -run 'NoEligibleBoxWritesNothing|Executor' -count=1`
+```bash
+nice -n 19 go test ./internal/console/ -run 'NoCapacity|ClaimRefusalAnswersNoRoom|CreateDoorsRefuse|AppRegionWith|ClaimCreate|InstanceCreate|DomainAttach|Sidecar' -count=1
+nice -n 19 go test ./internal/hostops/ -run 'Executor|Chooser|SelectExecutor|StartCreate|ProductionCreate' -count=1
+```
Expected: PASS.
- [ ] **Step 5: Copy gate, boundary list and census**
@@ -5050,25 +5598,29 @@ Expected: PASS.
````diff
--- a/approved_region_ha_copy_test.go
+++ b/approved_region_ha_copy_test.go
-@@ -35,7 +35,7 @@
-
- // regionHACopyWired is every id whose text the code or docs carry. The
- // task that writes a row's text adds its id here in the same commit.
--var regionHACopyWired = []string{"RHA-C1", "RHA-C2", "RHA-C3", "RHA-C4", "RHA-C5", "RHA-C6", "RHA-C7", "RHA-C8", "RHA-C9", "RHA-C10", "RHA-C11", "RHA-C12", "RHA-C13", "RHA-C14", "RHA-C15", "RHA-C16"}
-+var regionHACopyWired = []string{"RHA-C1", "RHA-C2", "RHA-C3", "RHA-C4", "RHA-C5", "RHA-C6", "RHA-C7", "RHA-C8", "RHA-C9", "RHA-C10", "RHA-C11", "RHA-C12", "RHA-C13", "RHA-C14", "RHA-C15", "RHA-C16", "RHA-C17", "RHA-C18", "RHA-C19"}
+@@ -36,7 +36,7 @@
+ // regionHACopyBuilt is every id whose text the code or docs carry. The
+ // task that writes a row's text adds its id here in the same commit, once
+ // Paul has approved the row.
+-var regionHACopyBuilt = []string{"RHA-C1", "RHA-C2", "RHA-C3", "RHA-C4", "RHA-C5", "RHA-C6", "RHA-C7", "RHA-C8", "RHA-C9", "RHA-C10", "RHA-C11", "RHA-C12", "RHA-C13", "RHA-C14", "RHA-C15", "RHA-C16"}
++var regionHACopyBuilt = []string{"RHA-C1", "RHA-C2", "RHA-C3", "RHA-C4", "RHA-C5", "RHA-C6", "RHA-C7", "RHA-C8", "RHA-C9", "RHA-C10", "RHA-C11", "RHA-C12", "RHA-C13", "RHA-C14", "RHA-C15", "RHA-C16", "RHA-C17", "RHA-C18", "RHA-C19"}
func mustParseRegionHARecord(t *testing.T) []copyRow {
t.Helper()
````
+Run: `nice -n 19 go test . -run 'RegionHACopy|RegionHABuilt' -count=1`
+Expected: PASS once RHA-C17 to RHA-C19 are approved.
+
````diff
--- a/region_ha_boundary_test.go
+++ b/region_ha_boundary_test.go
-@@ -26,6 +26,7 @@
+@@ -27,6 +27,8 @@
"internal/edge/lbhealth.go",
"internal/console/apiboxroles.go",
"system_roles.go",
+ "internal/console/nocapacity.go",
++ "internal/hostops/chosen.go",
}
// regionHAAdapterFiles are spec 2's AWS adapters. Each is named for AWS so
@@ -5084,7 +5636,7 @@ Expected: PASS.
// Re-reviewed 2026-10-12 (merge of main's home-region work): main's body (claimPlacementRegion) with stage B's one-line EvidenceComplete call; no new write.
// Re-reviewed at the merge of carlos-run into seeded-instances (2026-10-13): main's door with the two seed hand-offs: once every ordinary check has passed, a create that names `from` or `import` leaves for createFrom or createImport, which write their own record (beginFrom, beginImport, below) and refuse a claim console first (seedRegionReady); the ordinary create asks bindEnvironmentCreate before its write. The door's own CreateRecord and its claim create are as main wrote them.
- {"internal/console/apiinstances.go", "Server.handleAPIInstanceCreate", "record-writer", "key:CreateRecord", "6763eab45774b9e624a9b820ce1585ef3910355bc3ce589b82298e1d12469a2e"},
-+ {"internal/console/apiinstances.go", "Server.handleAPIInstanceCreate", "record-writer", "key:CreateRecord", "1f2bda930d791d6be625eccb9cb0b68c5ddf1e225e6580d787771187f3766c60"},
++ {"internal/console/apiinstances.go", "Server.handleAPIInstanceCreate", "record-writer", "key:CreateRecord", "385e85091d7ee5c89e3218bc55cd15dd4cac753df71b6ebbad5a67c15a3c14de"},
// Re-reviewed at the merge of carlos-run into seeded-instances (2026-10-13): main's door under the environment-binding lock (lockEnvironmentUpdate); the UpdateRecord call is unchanged, and UpdateRecord refuses any change to the record's seed.
{"internal/console/apiinstances.go", "Server.handleAPIInstanceUpdate", "record-writer", "key:UpdateRecord", "43a8bead675a4aa33777c6c68431882d55d79a9c1d27e1a54c8f103dda087de0"},
{"internal/console/apiscan.go", "Server.writeScanRecord", "nonrecord", "sel:Put", "c3c092f4b7be9bf6bf09d0fe1e2c9fe52892ee97e57f6c7ab964b85fe36fb5c6"},
@@ -5093,27 +5645,328 @@ Expected: PASS.
Add this comment line after the overlay's last comment line:
````text
-# Task 11: the create door refuses a create no box can take with a 503 and an hourly operator alert (internal/console nocapacity.go, called from handleAPIInstanceCreate; region HA spec 2 §7 as ruled by H27). Neither unit is a root-read unit. Envelope-only, and the seal stands.
+# Task 11: H27 on the create doors (internal/console nocapacity.go; hostops chosen.go): beginClaimCreate chooses the executor before it registers anything (chooseExecutorFirst) and carries the choice to the create (hostops.WithChosenExecutor, honoured by ProductionAuthority.SelectExecutor), claimResponseRefusal answers no room with a 503 on instance create, domain attach's first route (attachClaimFirstRoute, unchanged) and standalone sidecar create, and handleAPIInstanceCreate judges an app-derived region by the chooser's eligible set. Census ruling region-ha-2a is asked to re-freeze one accepted-legacy unit, Server.beginClaimCreate: one call added before RegisterStart, its one root-read site (the claim read, file:Read@470) unchanged. The seal moves.
+````
+
+Then set these rows. A `@coverage` row replaces the overlay's row for the same directory, or is appended when there is none; every other row is appended:
+
+````text
+@coverage internal/hostops 7c8e88d80099105a0cf3da88141ec6b054bbaaac756a6d441ec035bcd89db1ab
+@coverage internal/console 6e39bd88c50d06c7c1a67098d2aa0ca00efe03e35c3aaf082d4e57df965bede6
+@legacy 1d712288cb56c8e2391ca40dc4c9bd2cfb02d1cd8455c64726807464106c1439 internal/console/claim_create.go Server.beginClaimCreate legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;census-ruling-region-ha-2a file:Read@470 97ac0e0ead81eb2c72de7534a19f31724ecae42f074db6388d90c05a5f2747f7
+internal/console/claim_create.go Server.beginClaimCreate legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;census-ruling-region-ha-2a file:Read@470 97ac0e0ead81eb2c72de7534a19f31724ecae42f074db6388d90c05a5f2747f7
+````
+
+````diff
+--- a/internal/provision/plan14_root_reads_test.go
++++ b/internal/provision/plan14_root_reads_test.go
+@@ -769,7 +769,7 @@
+ if err = loadRootReadInventory(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-region-ha-2a.tsv"), &inv, true); err != nil {
+ t.Fatal(err)
+ }
+- seal = "a57250c350c20e4116e5c87b559cfda71f3e0c65d59a2858d83c5f78bea4e9b5"
++ seal = "2a0c03ae8d2c0ba3712151a98e1533e6be8512271dc98968bc5435b014bd6bab"
+ errs := append(compareRootReads(got, inv), checkRootReadLegacy(inv, seal)...)
+ errs = append(errs, checkRootReadDeclarations(got, inv)...)
+ if len(errs) > 0 {
+````
+
+Run the census checks and `go test . -run 'TestRegionHA(Core|Adapters|S3)' -count=1`. Expected: PASS.
+
+- [ ] **Step 6: Commit**
+
+```bash
+git add internal/hostops/chosen.go internal/hostops/backfill.go internal/hostops/executor_test.go internal/console/nocapacity.go internal/console/nocapacity_test.go internal/console/server.go internal/console/claim_create.go internal/console/claim_response.go internal/console/apiinstances.go approved_region_ha_copy_test.go region_ha_boundary_test.go internal/provision
+git commit -m "region HA 2a: create doors choose first and refuse when no box has room"
+```
+
+---
+
+### Task 12: The move door's destination is the placeable set
+
+**Files:**
+- Modify: `internal/console/boxselect.go` (`placeableMoveTargets`; `selectMoveBoxes` filters the destination)
+- Modify: `internal/console/apiinstancemove.go` (`handleAPIInstanceMoveRequest` answers the no-room refusal)
+- Create: `internal/console/boxselect_ha_test.go`
+- Modify: `internal/provision/plan9_b8_census_test.go`, `internal/provision/plan14_root_reads_test.go` (the seal), the overlay
+
+**Interfaces:**
+- Consumes: Task 11's `placementChooser`, `noRoom`, `noCapacityError`, `writeClaimResponseError`; Task 6's `PlaceableBoxes`.
+- Produces: `func (s *Server) placeableMoveTargets(ctx, to string, cands []boxCandidate) ([]boxCandidate, error)`.
+
+The move door chooses a box in the destination, so it is H27's fourth door. With region HA on, the destination's candidates are `eligibleBoxes`' (main's identity and floor checks) narrowed to the chooser's placeable set (roles, headroom, fresh edges files); an empty result is the no-room refusal, answered and alerted as Task 11's. The source is never filtered, so an instance on an excluded or full box can still move out (A10-3).
+
+With region HA off (Astra finding 4), `placementChooser` is nil, so the destination is main's and no roster is read: host claims on and region HA off behave exactly as main, an unreadable roster included.
+
+**Census (platform-1e):** `Server.handleAPIInstanceMoveRequest` is an accepted-legacy unit. It gains one refusal before any write; its read sites are unchanged and their offsets move. The overlay re-freezes it with the ruling's basis, and the seal moves. Its record-write census digest changes too.
+
+- [ ] **Step 1: Write the failing tests**
+
+`internal/console/boxselect_ha_test.go`:
+
+````go
+package console
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "testing"
+ "time"
+
+ "github.com/carlosframework/platform/internal/hostactors"
+ "github.com/carlosframework/platform/internal/hostops"
+)
+
+// haMoveFleet is armMoveFleet's ie and uk boxes, each with fresh, roomy
+// headroom, a signed roster naming both regions, and, with ha, a region
+// HA chooser bound as the create door binds it. ukRole is uk-box's role.
+func haMoveFleet(t *testing.T, ha bool, ukRole string) *consoleFixture {
+ t.Helper()
+ ctx := context.Background()
+ fx := newConsoleFixture(t)
+ armMoveFleet(t, fx, ieBox("ie-box"), ukBox("uk-box"))
+ for _, box := range []string{"ie-box", "uk-box"} {
+ key := edgesFilePrefix + box + ".json"
+ raw, etag, err := fx.rel.Get(ctx, key)
+ if err != nil {
+ t.Fatal(err)
+ }
+ var m map[string]any
+ if err := json.Unmarshal(raw, &m); err != nil {
+ t.Fatal(err)
+ }
+ m["headroom"] = map[string]any{"mem_free_mib": 4000, "mem_total_mib": 8000, "disk_free_mib": 50000, "disk_total_mib": 100000, "awake": 0}
+ m["generation"] = time.Now().UTC().Format(time.RFC3339)
+ body, _ := json.Marshal(m)
+ if _, err := fx.rel.Put(ctx, key, body, etag); err != nil {
+ t.Fatal(err)
+ }
+ }
+ ie, uk := moveRosterBox(ieBox("ie-box")), moveRosterBox(ukBox("uk-box"))
+ uk.Role = ukRole
+ putMoveRoster(t, fx, hostactors.Roster{Boxes: map[string]hostactors.Box{"ie-box": ie, "uk-box": uk},
+ Regions: map[string]hostactors.Region{
+ "ie": {Account: moveTestAccount, AWSRegion: "eu-west-1"},
+ "uk": {Account: moveTestAccount, AWSRegion: "eu-west-2"},
+ }})
+ fx.srv.cfg.HostClaim = true
+ fx.srv.cfg.RegionHA = ha
+ if ha {
+ d := hostactors.Directory{Store: fx.rel, PublicKey: fx.pub, Bucket: moveTestBucket}
+ fx.srv.cfg.ClaimAuthority = &hostops.ProductionAuthority{Namespace: "platform", Directory: d, Executors: fx.srv.executorChooser(d)}
+ }
+ return fx
+}
+
+// Region HA spec 2 §7 (H27): a move's destination is a door that chooses
+// a box. With region HA on, only a box the chooser would place on is a
+// destination, and none is the no-room refusal with its alert.
+func TestMoveDestinationIsTheChoosersPlaceableSet(t *testing.T) {
+ const host, app = "acme.go.tito.io", "bab/titogo"
+ t.Run("placeable", func(t *testing.T) {
+ fx := haMoveFleet(t, true, hostactors.RoleDefault)
+ got, err := fx.srv.selectMoveBoxes(context.Background(), host, "uk", "ie", app)
+ if err != nil || got.Target != "uk-box" || got.Source != "ie-box" {
+ t.Fatalf("selection %+v %v", got, err)
+ }
+ })
+ t.Run("the only destination is legacy", func(t *testing.T) {
+ pages := pagesTo(t)
+ fx := haMoveFleet(t, true, hostactors.RoleLegacy)
+ _, err := fx.srv.selectMoveBoxes(context.Background(), host, "uk", "ie", app)
+ var full *noCapacityError
+ if !errors.As(err, &full) || full.Region != "uk" {
+ t.Fatalf("err = %v, want no room in uk", err)
+ }
+ if ref := claimResponseRefusal(err); ref.Status != 503 || ref.Msg != noCapacityRefusal("uk") {
+ t.Fatalf("refusal %+v", ref)
+ }
+ expectPages(t, pages, noRoomSubject("uk"))
+ })
+ t.Run("a platform source still moves out", func(t *testing.T) {
+ fx := haMoveFleet(t, true, hostactors.RoleDefault)
+ ctx := context.Background()
+ o, err := hostactors.Directory{Store: fx.rel, PublicKey: fx.pub, Bucket: moveTestBucket}.Read(ctx)
+ if err != nil {
+ t.Fatal(err)
+ }
+ r := hostactors.Clone(o.Roster)
+ b := r.Boxes["ie-box"]
+ b.Role = hostactors.RolePlatform
+ r.Boxes["ie-box"] = b
+ putMoveRoster(t, fx, r)
+ got, err := fx.srv.selectMoveBoxes(ctx, host, "uk", "ie", app)
+ if err != nil || got.Source != "ie-box" || got.Target != "uk-box" {
+ t.Fatalf("selection %+v %v", got, err)
+ }
+ })
+}
+
+// Astra 2a round 1 finding 4: with host claims on and region HA off, the
+// move door is main's. A persisted role excludes nothing, and the door
+// reads no roster, so an unreadable one cannot refuse the move.
+func TestMoveDestinationWithRegionHAOffIsMains(t *testing.T) {
+ const host, app = "acme.go.tito.io", "bab/titogo"
+ fx := haMoveFleet(t, false, hostactors.RoleLegacy)
+ got, err := fx.srv.selectMoveBoxes(context.Background(), host, "uk", "ie", app)
+ if err != nil || got.Target != "uk-box" {
+ t.Fatalf("selection %+v %v", got, err)
+ }
+ fx.srv.cfg.Releases = failingGet{Store: fx.rel, key: hostactors.Key}
+ if got, err := fx.srv.selectMoveBoxes(context.Background(), host, "uk", "ie", app); err != nil || got.Target != "uk-box" {
+ t.Fatalf("with the roster unreadable: %+v %v", got, err)
+ }
+}
+````
+
+- [ ] **Step 2: Run them to see them fail**
+
+Run: `nice -n 19 go test ./internal/console/ -run 'MoveDestination' -count=1`
+Expected: FAIL: `TestMoveDestinationIsTheChoosersPlaceableSet` reports `err = <nil>, want no room in uk` (with region HA on, a destination whose only box is legacy is still chosen). `TestMoveDestinationWithRegionHAOffIsMains` passes already: it pins main's behaviour.
+
+- [ ] **Step 3: Filter the destination and answer the refusal**
+
+````diff
+--- a/internal/console/boxselect.go
++++ b/internal/console/boxselect.go
+@@ -8,6 +8,7 @@
+ "os"
+
+ "github.com/carlosframework/platform/internal/boxid"
++ "github.com/carlosframework/platform/internal/hostops"
+ "github.com/carlosframework/platform/internal/moves"
+ )
+
+@@ -113,6 +114,11 @@
+ fp := s.expectedSecretsFP()
+ for _, code := range []string{to, from} {
+ cands := eligibleBoxes(snap, roster, code, aws[code], s.cfg.HomeRegion, fp, nil)
++ if code == to {
++ if cands, err = s.placeableMoveTargets(ctx, to, cands); err != nil {
++ return moveSelection{}, err
++ }
++ }
+ if len(cands) > 1 {
+ sel.Many, sel.ManyRegion = true, s.homeRegionOut(code)
+ return sel, nil
+@@ -162,3 +168,35 @@
+ }
+ return out, nil
+ }
++
++// placeableMoveTargets is a move's destination under region HA (spec 2
++// §7, H27): the destination is a door that chooses a box, so only boxes
++// the create door's chooser would place on (roles, headroom, fresh edges
++// files) are candidates, and none is the no-room refusal. The source is
++// never filtered, so an instance on an excluded box can still move out
++// (A10-3). With region HA off, or no claim runtime, cands are main's and
++// no roster is read.
++func (s *Server) placeableMoveTargets(ctx context.Context, to string, cands []boxCandidate) ([]boxCandidate, error) {
++ x := s.placementChooser("")
++ if x == nil {
++ return cands, nil
++ }
++ boxes, err := x.PlaceableBoxes(ctx, s.cfg.claimPlacementRegion(to))
++ if err != nil && !errors.Is(err, hostops.ErrContract) {
++ return nil, err
++ }
++ placeable := map[string]bool{}
++ for _, b := range boxes {
++ placeable[b] = true
++ }
++ var out []boxCandidate
++ for _, c := range cands {
++ if placeable[c.BoxID] {
++ out = append(out, c)
++ }
++ }
++ if len(out) == 0 {
++ return nil, s.noRoom(ctx, to)
++ }
++ return out, nil
++}
+````
+
+````diff
+--- a/internal/console/apiinstancemove.go
++++ b/internal/console/apiinstancemove.go
+@@ -602,6 +602,12 @@
+ }
+ from := s.homeRegionOut(rec.Region)
+ sel, err := s.selectMoveBoxes(ctx, host, to, rec.Region, acct.Sqid+"/"+app)
++ // No box in the destination has room (region HA spec 2 §7, H27).
++ var full *noCapacityError
++ if errors.As(err, &full) {
++ writeClaimResponseError(w, err)
++ return
++ }
+ if errors.Is(err, errEdgesIncomplete) {
+ jsonError(w, http.StatusConflict, "the fleet's edges files could not all be read. try again")
+ return
+````
+
+- [ ] **Step 4: Run the tests**
+
+Run: `nice -n 19 go test ./internal/console/ -run 'MoveDestination|SelectMoveBoxes|InstanceMove|MoveDoor' -count=1`
+Expected: PASS.
+
+- [ ] **Step 5: Census**
+
+````diff
+--- a/internal/provision/plan9_b8_census_test.go
++++ b/internal/provision/plan9_b8_census_test.go
+@@ -655,7 +655,7 @@
+ {"internal/console/apidoorkeys.go", "Server.publishDoorKeysLocked", "foreign-recordkey", "key:RecordKey,sel:Put", "0fc498cd4af50570a0b7ed3ff71994213cab40040f6701a303bdf85d94f958da"},
+ {"internal/console/apiinstancemove.go", "Server.moveTransitionSaying", "record-writer", "key:SetRecordMove", "d1f7162eecdc234607c708a83e9633cdf77cc6931bac6754fb51042dd8409377"},
+ // Re-reviewed at the merge of carlos-run into seeded-instances (2026-10-13): main's door with the seed rule (refuseSeedMove) after its gate and before the transition; the transition is unchanged and refuses the same seed again under its CAS.
+- {"internal/console/apiinstancemove.go", "Server.handleAPIInstanceMoveRequest", "record-writer", "key:SetRecordMove,lit:instances", "d4350f8b160076e4e69a0e6fdb5da9a042f4445a7c768dee49be0f6281fc20f3"},
++ {"internal/console/apiinstancemove.go", "Server.handleAPIInstanceMoveRequest", "record-writer", "key:SetRecordMove,lit:instances", "d9b96ed9652f5879ada1700f94473347d652dcfae1e024e35256770952384783"},
+ {"internal/console/apiinstancemove.go", "Server.registerMovesPhase1Routes", "key-site", "lit:instances", "5af589d288ba22856f631481c40cc001aac342ea74cee6ee0464ec77d0320bbc"},
+ {"internal/console/apiinstances.go", "Server.instanceManifest", "key-site", "lit:instances", "e6160ffe84abda9163527ccb33e2ffff2109b35908adb2f3a754534325658fdf"},
+ {"internal/console/apiinstances.go", "Server.handleAPIInstanceList", "key-site", "lit:instances", "558d0b3e7b0f06faa8e5067ff6f8c131eb899f13d08b71218dfb2454c3e319e1"},
+````
+
+Add this comment line after the overlay's last comment line:
+
+````text
+# Task 12: H27 on the move door's destination (internal/console boxselect.go placeableMoveTargets, through hostops.ExecutorChooser.PlaceableBoxes and only with region HA on). Census ruling region-ha-2a is asked to re-freeze one accepted-legacy unit, Server.handleAPIInstanceMoveRequest: one more refusal before any write, no box in the destination has room; its read sites unchanged, offsets moved. The seal moves.
````
Then set these rows. A `@coverage` row replaces the overlay's row for the same directory, or is appended when there is none; every other row is appended:
````text
-@coverage internal/console c6e6f902978fd3edc87cb8bff14d87dd8ddc3f734ba08aebb32c003af379fb9c
+@coverage internal/console 90eea3fdd16b35a1f43058b1082480206f6e0815f41c8863f3b3952578eaac1a
+@legacy 831306bc42c758ff8d82c4eb1e1860b35ebadd24ca2f8e1a45177ec7bb70019b internal/console/apiinstancemove.go Server.handleAPIInstanceMoveRequest legacy accepted-legacy accepted-legacy;rev3=census-ruling-region-ha-2a;basis=the console (cmd/console serve under console.service, user carlos-console): main's body with one more refusal before any write: no box in the destination has room (H27); read sites unchanged, offsets moved file:Exec@7187,file:Exec@7275,file:Exec@8266,file:Exec@8443,file:Exec@8594,file:Exec@9594,file:Read@5673,file:Read@7985 d9b96ed9652f5879ada1700f94473347d652dcfae1e024e35256770952384783
+internal/console/apiinstancemove.go Server.handleAPIInstanceMoveRequest legacy accepted-legacy accepted-legacy;rev3=census-ruling-region-ha-2a;basis=the console (cmd/console serve under console.service, user carlos-console): main's body with one more refusal before any write: no box in the destination has room (H27); read sites unchanged, offsets moved file:Exec@7187,file:Exec@7275,file:Exec@8266,file:Exec@8443,file:Exec@8594,file:Exec@9594,file:Read@5673,file:Read@7985 d9b96ed9652f5879ada1700f94473347d652dcfae1e024e35256770952384783
````
-Run the census checks and `go test . -run 'TestRegionHA|RegionHACopy|RegionHAWired' -count=1`. Expected: PASS.
+````diff
+--- a/internal/provision/plan14_root_reads_test.go
++++ b/internal/provision/plan14_root_reads_test.go
+@@ -769,7 +769,7 @@
+ if err = loadRootReadInventory(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-region-ha-2a.tsv"), &inv, true); err != nil {
+ t.Fatal(err)
+ }
+- seal = "2a0c03ae8d2c0ba3712151a98e1533e6be8512271dc98968bc5435b014bd6bab"
++ seal = "a7efc5beddc7793ca2647df7812ebd48120fb148f79649ddcca110131d393da4"
+ errs := append(compareRootReads(got, inv), checkRootReadLegacy(inv, seal)...)
+ errs = append(errs, checkRootReadDeclarations(got, inv)...)
+ if len(errs) > 0 {
+````
+
+Run the census checks. Expected: PASS.
- [ ] **Step 6: Commit**
```bash
-git add internal/console/nocapacity.go internal/console/nocapacity_test.go internal/console/server.go internal/console/apiinstances.go internal/hostops/executor_test.go approved_region_ha_copy_test.go region_ha_boundary_test.go internal/provision
-git commit -m "region HA 2a: refuse a create no box can take, and alert once an hour"
+git add internal/console/boxselect.go internal/console/boxselect_ha_test.go internal/console/apiinstancemove.go internal/provision
+git commit -m "region HA 2a: the move door's destination is the placeable set"
```
---
-### Task 12: The eligible-capacity metric
+### Task 13: The eligible-capacity metric
**Files:**
- Create: `internal/console/regioncapacity.go` (core: the count and the pass)
@@ -5123,10 +5976,10 @@ git commit -m "region HA 2a: refuse a create no box can take, and alert once an
- Modify: `region_ha_boundary_test.go`, the overlay
**Interfaces:**
-- Consumes: Task 6's `ExecutorChooser.Capacity`; `Server.liveEdges`; `rootaws.LoadDefaultConfig`.
-- Produces: `type regionCapacity struct{ Placement, AWSRegion string; Eligible int }`; `type capacityPublisher interface{ PublishCapacity(ctx, []regionCapacity) error }`; `func regionCapacities(ctx, snap boxchoice.Snapshot, capacity func(context.Context, string) (int, error)) ([]regionCapacity, error)`; `(s *Server) capacityChooser`, `publishCapacity`, `runCapacityPass`; the adapter's `capacityData`, `cloudWatchCapacity`, `capacityPublisherFromEnv(ctx, getenv)`. **For 2c:** the metric is `Carlos/Region` / `EligibleBoxes`, dimension `Region` = the placement code, unit Count, every minute, in the region's own CloudWatch; the console needs `CARLOS_REGION_METRICS=cloudwatch` and the platform box role's `cloudwatch:PutMetricData` on that namespace.
+- Consumes: Task 6's `ExecutorChooser.Capacity`; Task 11's `placementChooser`; `Server.liveEdges`; `rootaws.LoadDefaultConfig`.
+- Produces: `type regionCapacity struct{ Placement, AWSRegion string; Eligible int }`; `type capacityPublisher interface{ PublishCapacity(ctx, []regionCapacity) error }`; `func regionCapacities(ctx, snap boxchoice.Snapshot, capacity func(context.Context, string) (int, error)) ([]regionCapacity, error)`; `(s *Server) publishCapacity`, `runCapacityPass`; the adapter's `capacityData`, `cloudWatchCapacity`, `capacityPublisherFromEnv(ctx, getenv)`. **For 2c:** the metric is `Carlos/Region` / `EligibleBoxes`, dimension `Region` = the placement code, unit Count, every minute, in the region's own CloudWatch; the console needs `CARLOS_REGION_METRICS=cloudwatch` and the platform box role's `cloudwatch:PutMetricData` on that namespace.
-The pass runs beside claim recovery, so only on a host-claim console, and only with region HA on and a publisher armed. Regions are the placements the boxes' edges files name, so a region whose every box has stopped publishing gets no data point, which the alarm reads as breaching. A count that fails stops the whole pass rather than publish a partial set.
+The pass runs beside claim recovery, so only on a host-claim console, and only with region HA on and a publisher armed. It counts through the same chooser the create doors use (`placementChooser("")`), so the alarm and the refusal agree. Regions are the placements the boxes' edges files name, so a region whose every box has stopped publishing gets no data point, which the alarm reads as breaching. A count that fails stops the whole pass rather than publish a partial set.
- [ ] **Step 1: Write the failing tests**
@@ -5185,17 +6038,21 @@ func TestRegionCapacitiesCountsEachPlacement(t *testing.T) {
}
}
-func TestCapacityChooserIsScopedLikeCreate(t *testing.T) {
- s := &Server{}
- if s.capacityChooser() != nil {
+func TestPlacementChooserIsScopedLikeCreate(t *testing.T) {
+ s := &Server{cfg: Config{RegionHA: true}}
+ if s.placementChooser("") != nil {
t.Fatal("a chooser with no claim runtime")
}
x := &hostops.ExecutorChooser{Namespace: "stale", Pool: "acme"}
s.cfg.ClaimAuthority = &hostops.ProductionAuthority{Namespace: "platform", Executors: x}
- got := s.capacityChooser()
+ got := s.placementChooser("")
if got == nil || got.Namespace != "platform" || got.Pool != "" || x.Namespace != "stale" {
t.Fatalf("chooser %+v, original %+v", got, x)
}
+ s.cfg.RegionHA = false
+ if s.placementChooser("") != nil {
+ t.Fatal("a chooser with region HA off")
+ }
}
func TestCloudWatchCapacityPublishesEachRegionInItsOwnRegion(t *testing.T) {
@@ -5226,7 +6083,7 @@ func TestCloudWatchCapacityPublishesEachRegionInItsOwnRegion(t *testing.T) {
- [ ] **Step 2: Run them to see them fail**
-Run: `nice -n 19 go test ./internal/console/ -run 'RegionCapacities|CapacityChooser|CloudWatchCapacity' -count=1`
+Run: `nice -n 19 go test ./internal/console/ -run 'RegionCapacities|PlacementChooserIsScoped|CloudWatchCapacity' -count=1`
Expected: FAIL to build (`undefined: regionCapacities`, `undefined: cloudWatchCapacity`).
- [ ] **Step 3: Write the pass and the adapter**
@@ -5301,22 +6158,9 @@ func regionCapacities(ctx context.Context, snap boxchoice.Snapshot, capacity fun
return out, nil
}
-// capacityChooser is the create door's chooser scoped as SelectExecutor
-// scopes it for the shared pool, or nil without one (region HA off, or no
-// claim runtime).
-func (s *Server) capacityChooser() *hostops.ExecutorChooser {
- a := s.cfg.ClaimAuthority
- if a == nil || a.Executors == nil {
- return nil
- }
- x := *a.Executors
- x.Namespace, x.Pool = a.Namespace, ""
- return &x
-}
-
// publishCapacity counts and publishes once.
func (s *Server) publishCapacity(ctx context.Context, pub capacityPublisher) error {
- x := s.capacityChooser()
+ x := s.placementChooser("")
if x == nil {
return hostops.ErrContract
}
@@ -5468,7 +6312,7 @@ func capacityPublisherFromEnv(ctx context.Context, getenv func(string) string) c
- [ ] **Step 4: Run the tests**
-Run: `nice -n 19 go test ./internal/console/ -run 'RegionCapacities|CapacityChooser|CloudWatchCapacity' -count=1`
+Run: `nice -n 19 go test ./internal/console/ -run 'RegionCapacities|PlacementChooserIsScoped|CloudWatchCapacity|ClaimRecovery' -count=1`
Expected: PASS.
- [ ] **Step 5: Boundary list and census**
@@ -5476,10 +6320,10 @@ Expected: PASS.
````diff
--- a/region_ha_boundary_test.go
+++ b/region_ha_boundary_test.go
-@@ -27,12 +27,14 @@
- "internal/console/apiboxroles.go",
+@@ -29,12 +29,14 @@
"system_roles.go",
"internal/console/nocapacity.go",
+ "internal/hostops/chosen.go",
+ "internal/console/regioncapacity.go",
}
@@ -5496,16 +6340,16 @@ Expected: PASS.
Add this comment line after the overlay's last comment line:
````text
-# Task 12: the eligible-capacity metric (internal/console regioncapacity.go, counted through hostops.ExecutorChooser.Capacity; awsregionmetric.go, the CloudWatch adapter), started beside claim recovery (StartClaimRecoveryPass). No root-read unit changed or was added. Envelope-only, and the seal stands.
+# Task 13: the eligible-capacity metric (internal/console regioncapacity.go, counted through hostops.ExecutorChooser.Capacity; awsregionmetric.go, the CloudWatch adapter), started beside claim recovery (StartClaimRecoveryPass). No root-read unit changed or was added. Envelope-only, and the seal stands.
````
Then set these rows. A `@coverage` row replaces the overlay's row for the same directory, or is appended when there is none; every other row is appended:
````text
-@coverage internal/console 415c03036eab35637459b27ccf0078e96d5ad57f5c10689df3d79074765a8221
+@coverage internal/console cddef46a81289d748b6b50a1f2bf9e2aa2c95144bdd1cea4095d3fb476058cbd
````
-Run the census checks and `go test . -run TestRegionHA -count=1`. Expected: PASS: `regioncapacity.go` imports no AWS package, and `awsregionmetric.go` is named for AWS.
+Run the census checks and `go test . -run 'TestRegionHA(Core|Adapters|S3)' -count=1`. Expected: PASS: `regioncapacity.go` imports no AWS package, and `awsregionmetric.go` is named for AWS.
- [ ] **Step 6: Commit**
@@ -5521,13 +6365,11 @@ git commit -m "region HA 2a: publish each region's eligible capacity"
- [ ] `gofmt -l .` is empty, and `go build ./...` passes with `GOOS` linux, darwin and windows.
- [ ] `GOOS=linux go vet ./...` and `GOOS=darwin go vet ./...` pass.
- [ ] Every task's targeted tests pass again, then once: `nice -n 19 go test ./internal/console/ -run 'TestClaimConsoleMutationInvariant$' -count=1 -timeout 90m` and the census checks.
+- [ ] `go test . -run 'RegionHACopy|RegionHABuilt' -count=1` passes with every built row approved by Paul. A pending row means the branch is not ready.
- [ ] Push the branch and read `amadan ci status`: CI's `make test` is the full run, `internal/host` included.
-- [ ] The rollout checklist for region HA gains: the copy record approved before any production console turns `CARLOS_REGION_HA` on; consoles before boxes (the console is the roster's only writer, and an older console's roster write would drop the roles); 2c's IAM grant and `CARLOS_REGION_METRICS` before the alarm exists.
+- [ ] The rollout checklist for region HA gains: consoles before boxes (the console is the roster's only writer, and an older console's roster write would drop the roles); 2c's IAM grant, the IMDS path `network/interfaces/macs/*` and `CARLOS_REGION_METRICS` before the alarm exists; on a box outside AWS, `CARLOS_PRIVATE_IP` with `CARLOS_PRIVATE_CIDR` in `host.env`, or no private address is published.
## Open questions
-1. **"The default certificate available" (§6).** The edge has no single default certificate: every certificate is per host. Task 9 reads the condition as "the certificate mirror has landed `control/certs/` in the local cache once" (latched like forwarding). A deployment with no published certificate passes it. If a specific certificate is meant (the apps-domain wildcard?), the condition needs naming.
-2. **H27's refusal covers the instance create door only.** The sidecar door and the first-route path also create through the claim; with no eligible box they keep today's 409-class claim refusal, without the alert. Should they share the 503 and the alert?
-3. **The move door's destination (brief item 1).** Task 10 filters legacy and platform boxes out of the destination, which leaves the door's existing "no box in `<to>` publishes an edges file yet" when only excluded boxes remain. A sentence of its own means changing `handleAPIInstanceMoveRequest`, an accepted-legacy unit that 2d rewrites anyway. Is the interim wording acceptable until 2d?
-4. **Hosting before 2b.** Task 9 marks a box hosting when the claim agent's wiring starts, which proves enrollment and a matching roster row but does not itself check that the floor guard is armed for that enrollment. 2b's mode selection makes it exact. Is that acceptable for 2a, given no load balancer reads the answer until 2c?
-5. **Census rulings.** Task 7's authority re-digest and Task 10's accepted-legacy `Server.boxRoster` need platform-1e (census ruling region-ha-2a). On a base newer than `70e57449` every census value here must be recomputed (see Census tooling).
+1. **Hosting before 2b.** Task 9 marks a box hosting when the claim agent's wiring starts, which proves enrolment and a matching roster row but does not itself check that the floor guard is armed for that enrolment. 2b's mode selection makes it exact. Is that acceptable for 2a, given no load balancer reads the answer until 2c?
+2. **The census values (condition 3).** Every value here was computed at main `70e57449`. Plan 14 Parts 2 and 3 will move the base; the implementer recomputes every value there with the tooling above, and platform-1e's ruling covers the rows, not these hashes.