| index 23977dc9c..18854292f 100644 |
| --- a/docs/superpowers/specs/2026-10-15-region-ha-two-box-regions-design.md |
| +++ b/docs/superpowers/specs/2026-10-15-region-ha-two-box-regions-design.md |
| @@ -1,6 +1,8 @@ |
| # Region HA, spec 2: two-box regions |
| |
| -Design, 2026-10-15, **rev 8**. DRAFT. Nothing here is built. |
| +Design, 2026-10-15, **rev 9**. DRAFT. Nothing here is built. |
| + |
| +**Rev 9** answers Astra's round 8 (`CODEX-FINDINGS-REGION-HA-SPEC2-8.md`, cited **A8-n**). Rounds 6 to 8 each found another way an instance could reach a draining box. So the settlement barrier is now one general rule over the claim, not a list of operation kinds. Never-woken instances get one platform wake before relocation. §21 maps both. |
| |
| **Rev 8** answers Astra's round 7 (`CODEX-FINDINGS-REGION-HA-SPEC2-7.md`, cited **A7-n**): spec 2 builds the durable `draining` control that S1 left to spec 4, and the emptiness check covers in-flight creates. §20 maps both. |
| |
| @@ -296,13 +298,20 @@ The choice is made once and recorded in the create intent, as now (H12). Recover |
| - **What it shows:** the box mirrors the roster value into its edges file's `draining` field for display only. Readers that decide anything read the roster. |
| - **The acknowledgement** that drain has taken effect is the settlement barrier below. Nothing needs to wait for a box to republish. |
| |
| -**The settlement barrier** (A7-2) runs after drain is set, before relocation begins, and again before the cut-over and before termination. It covers every way an instance can come to land on the box: |
| -1. **Creates in flight in the console.** The console tracks every create request between choosing an executor and writing the claim. The barrier waits until every request that began before the drain was set has finished, one way or the other. |
| -2. **Claim creates not yet complete.** An immediate create records its chosen box in `intent.selected_executor` (`internal/hostclaim/create.go:115`) and only sets `op.exec` when it completes. Recovery keeps that selection without checking `draining` (`internal/hostops/production_create.go:257`). So the barrier scans the claim for incomplete creates whose `selected_executor` is the drained box, and waits for each to complete or be abandoned. A completed one then joins the migration. |
| -3. **Pending creates in `starting`** that chose the box (§7): reconciled to completion first, likewise. |
| -4. **Completed hosts:** the claim names no host whose executor is the box. |
| +**The settlement barrier** (A7-2, A8-1) runs after drain is set, before relocation begins, and again before the cut-over and before termination. Rather than listing the ways an instance can come to land on a box, it uses one rule. **A box is settled when nothing anywhere still names it as a destination:** |
| +1. **No claim record names it, in any role.** Every claim record, of every operation kind and in every phase, is scanned. None may name the box as: |
| + - its executor (`op.exec`); |
| + - its selected executor (`intent.selected_executor`, `internal/hostclaim/create.go:115`); |
| + - a relocation target (`intent.target_box`, `internal/hostrelocate/worker.go:577`); |
| + - a move target (`intent.target`, `internal/hostclaim/move.go:58`); |
| + - any other box field that a later claim revision adds. The scan is written against the claim's box-reference fields, not a list of operation kinds, and a test fails if a new box field is added without the scan reading it. |
| +2. **No request chose it before the drain and is still in flight.** That means: |
| + - the console's create and move doors, and its pending creates in `starting` (§7). The console tracks every request between choosing a box and writing the claim, and the barrier waits until every request that began before the drain has finished; |
| + - box-side starters, such as box-started relocations (S1 H18). They choose with `boxchoice`, which reads the roster's `draining` (above). The barrier waits one roster refresh interval after the drain, plus margin, so every box has seen it. |
| + |
| +An operation found in step 1 isn't cancelled. It's allowed to finish, and whatever it lands on the box then joins the migration. |
| |
| -The box is empty only when all four are clear. |
| +**Never-woken instances** (A8-2). Relocation refuses a host whose holder has never had a session (`Holder.Session == 0`, `internal/hostrelocate/worker.go:329`, kept by `worker_test.go:482`). A create that completed but never received traffic is such a host. So migration first **wakes each never-woken instance once on the old box**: a platform-initiated wake through the activator's normal admission, with no customer request. It then has a session, and relocation proceeds as for any other instance. The wake runs the customer's app once, as a scheduled wake would. If a wake fails, the instance stays and the emptiness check doesn't pass, and the operator sees which one. |
| |
| **Rebalancing:** none automatic. An operator moves an instance between boxes in a region with spec 1's relocation. Automatic rebalancing is spec 4's. |
| |
| @@ -346,7 +355,8 @@ Each is closed by a code change, or by showing it's safe under claim mode, befor |
| - The old fixed box keeps the region's EIP and all public traffic. It forwards to the group boxes over their private addresses as instances move to them. |
| 2. **Migrate.** |
| - **First, drain the old box** (A6-1, A7-1): `carlos system box drain <ref>`. The drain is persistent, and the box is ineligible for every new placement, pending creates included (§7). |
| - - **Run the settlement barrier** (§7, A7-2): in-flight console creates, incomplete claim creates that selected the box, and `starting` pending creates. Each one that lands on the box joins the migration. |
| + - **Run the settlement barrier** (§7): no claim record names the box in any role, and no request that chose it before the drain is in flight. Operations already heading to it finish, and whatever they land joins the migration. |
| + - **Wake never-woken instances once** (§7, A8-2), so each has a session to relocate. |
| - **Then relocate** every instance on the old box to §7's choice, one at a time, with spec 1's relocation. Each relocated instance is reached the usual ways: through the old box, which still holds the region's EIP, privately, and from other regions at the group box's own public address. When this step ends, the old box hosts no customer instance. |
| - **Check it's empty:** the full settlement barrier (§7) is clear. It's repeated immediately before step 3 and again before the box is terminated. |
| 3. **Cut over the EIP.** The old box is empty by now, so moving its address strands nothing. |
| @@ -412,6 +422,10 @@ Each is closed by a code change, or by showing it's safe under claim mode, befor |
| - the full §9 sequence, with migration before the cut-over, the cut-over window measured, the zone-B rehearsal, and the rollback sequence run once; |
| - a create racing the start of migration, through each path separately: an immediate create that selected the box before the drain and completes after it; one that stalls and completes through recovery; a console request between choosing and the claim write; and a `starting` pending create. Each is found by the settlement barrier and joins the migration, and the box is never reported empty while one is outstanding; |
| - drain and undrain surviving a console restart and a box restart; |
| + - a relocation and a move that chose the box before the drain, each stalled and then resumed: both found by the claim scan, and each landed instance migrated; |
| + - a box-started relocation choosing just before the drain, covered by the roster-refresh wait; |
| + - a never-woken instance on the box: woken once, then relocated. A failed wake leaves the box not empty, and says why; |
| + - the claim scan's coverage test: adding a box-reference field to the claim without the scan reading it fails the test; |
| - IE's role order: `draining`, rollback clearing it and relocating back, and `platform` only at acceptance; |
| - one group box terminated by hand, with replacement and re-enrolment observed; |
| - a box made unhealthy, observed being terminated. |
| @@ -507,3 +521,10 @@ All three were answered in Astra's round 1 and folded in: |
| |---|---| |
| | A7-1 P2: the draining control is still spec 4's | Accepted: spec 2 builds it, as a roster field written by an operator-only console API and CLI. It's persistent, and placement reads it directly (§7) | |
| | A7-2 P2: settlement misses immediate creates | Accepted: a settlement barrier over in-flight console creates, incomplete claim creates by `selected_executor`, `starting` pending creates and completed hosts. It's run before relocation, before the cut-over and before termination (§7, §9) | |
| + |
| +## 21. Astra round 8 dispositions (rev 9) |
| + |
| +| Finding | Disposition | |
| +|---|---| |
| +| A8-1 P2: the barrier misses moves and relocations heading to the box | Accepted, generalised: one rule over every claim box-reference field, enforced by a coverage test, plus in-flight requests from the console doors and box-side starters (§7) | |
| +| A8-2 P2: relocation refuses never-woken instances | Accepted: each is woken once on the old box before relocation, and a failure blocks the emptiness check visibly (§7, §9) | |
amadan