amadan
design: host-scoped atomic ownership (amadan #232), rev 1
The lifecycle claim's holder owns every record-backed host and its CAS is the create door's atomic step; the global hostname claim owns aliases; the two families compose by write-then-verify; minted hosts are owned by name. Backfill, lazy adoption, privacy, cost, tests and rollout.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+584
−0
docs/superpowers/specs/2026-10-06-host-ownership-design.md+584 −0
CI passed — run details
diff --git a/docs/superpowers/specs/2026-10-06-host-ownership-design.md b/docs/superpowers/specs/2026-10-06-host-ownership-design.md| new file mode 100644 |
| index 000000000..ca5a4e4ac |
| --- /dev/null |
| +++ b/docs/superpowers/specs/2026-10-06-host-ownership-design.md |
| @@ -0,0 +1,584 @@ |
| +# Host-scoped atomic ownership |
| + |
| +Design, 2026-10-06, **rev 1**. Closes amadan #232. Design only; nothing |
| +here is built. |
| + |
| +Related specs: `2026-10-02-host-lifecycle-claim-design.md` rev 6.14 on |
| +`design-replica-ownership` (the lifecycle claim), with its library at |
| +`internal/hostclaim` on `feat/hostclaim-library`; `2026-08-12-domain-attach-design.md` |
| +(the global hostname claim, #151, #82). |
| + |
| +## 0. Summary |
| + |
| +A hostname on the platform is held today by one of three things, and no |
| +single object says which: |
| + |
| +- an instance record at `apps/<acct>/<app>/instances/<host>.json` |
| + (provision.RecordKey), keyed per account and app, so two apps can both |
| + write a record for one host; |
| +- a global hostname claim at `domains/<host>.json` (domains.ClaimKey), |
| + keyed by host alone, create-only, written by the domain attach doors; |
| +- a registry route a box minted from an app's name (its platform apex and |
| + a static app's canary hosts), with no record and no claim. |
| + |
| +The instance create door checks the first two before it writes |
| +(`hostHolder`, `hostClaimHolder` in internal/console/apex.go, 04cd2d78), |
| +then writes a record whose conditional create protects only its own key. |
| +The check and the write are two steps, so two concurrent creates of one |
| +host from two apps can both get 201. The same shape exists between an |
| +alias attach and an instance create. |
| + |
| +The design: |
| + |
| +1. **Record-backed hosts** (instances, replicas, static records, sidecars) |
| + are owned by the **lifecycle claim's holder** at |
| + `control/hosts/<host>.json`. The create door's claim CAS is the atomic |
| + gate. There is no second ownership object for these hosts, and this |
| + spec adds no field to that claim. |
| +2. **Aliases** (hosts with a claim and no record) are owned by the |
| + **global hostname claim** at `domains/<host>.json`, as today. |
| +3. The two families meet at one rule: **write your own object, then |
| + re-read the other family's, and back off if it is held.** With S3's |
| + strong read-after-write consistency this guarantees at least one of |
| + two racing writers sees the other, so two writers can never both |
| + finish. A writer that backs off leaves nothing behind and answers 409. |
| +4. **Minted hosts** (apex `<app>.<sqid>.<domain>`, static canary |
| + `<name>.<app>.<sqid>.<domain>`) are owned by the app whose name they |
| + carry. That is decided from the name, with the console's unique |
| + (account, app) row as the atomic gate, and the alias door refuses |
| + those names unless the attaching app is the name's app. |
| +5. Legacy hosts get a claim from a backfill verb, and the door adopts |
| + lazily until a completion marker says the backfill is done. |
| + |
| +## 1. The problem, in the code |
| + |
| +`handleAPIInstanceCreate` (apiinstances.go:772-792) asks `hostHolder`, |
| +which reads this box's registry and then one `GetRecord` per other app |
| +of the account (live and trashed, SQL-listed), and `hostClaimHolder`, |
| +which reads `domains/<host>.json`. Then `provision.CreateRecord` |
| +(provision.go:820) does `Put(key, body, "")`, create-only on |
| +`apps/<acct>/<app>/instances/<host>.json`. |
| + |
| +Consequences: |
| + |
| +- Two creates for one host from apps A and B both pass the checks and |
| + both write, because their keys differ. The provisioning pass keeps the |
| + record it lists first and logs `duplicate host` for the other |
| + (provision/reconcile.go:819). The loser looks created and serves |
| + nothing, or a different box serves it. |
| +- An alias attach (`domains.AttachClaim`, apidomains.go:444 and |
| + domainattach.go:153) and an instance create for the same host each |
| + check the other's object before writing their own. The same window. |
| +- The record scan costs one sequential GET per other app, inside the |
| + handler's 15 s budget; it sees only SQL-listed apps and only this |
| + console box's registry; direct-bucket records of apps not in SQL are |
| + outside it. |
| + |
| +## 2. Writer inventory |
| + |
| +Every path that puts a hostname on the platform or takes one off, with |
| +what it writes today and what gates it. |
| + |
| +| Writer | Hosts | Writes today | Gate today | Gap | |
| +|---|---|---|---|---| |
| +| Instance create (`handleAPIInstanceCreate`) | `<label>.<m.Domain>`; platform `<label>.<sqid>.<apps>` or a custom manifest domain | record, create-only | sibling-label check, `hostHolder`, `hostClaimHolder`, post-write label re-check | check then write; scan cost; local registry only | |
| +| Instance delete (`handleAPIInstanceDelete`, :1675) | same | `DeleteRecord` | app gate | leaves no release of anything host-scoped (nothing to release yet) | |
| +| Replica create (same door, `Role: replica`, `CreateRecordReplica`) | one label + the app's domain | record | same as create | same as create | |
| +| Sidecar create and delete (apisidecars.go:285, :347) | `<name>.<app>.<sqid>.sidecar.internal` | sidecar record at `sidecars/<name>.json` | name validity | none: the host embeds account and app, so no cross-app collision exists; never routable | |
| +| Canary (`carlos canary`, canary.go:231) | `<canary>-<app>.<sqid>.<apps>` | the instance create door | the door's | the door's; flat label makes cross-app collisions easy (#207) | |
| +| Static canary mint (domains/reconcile.go:328) | `<name>.<app>.<sqid>.<apps>` | registry route on each box, no record, no claim | `byHost` (local routes) | an alias claimed at that name is shadowed silently | |
| +| Platform apex mint (domains/reconcile.go:266) | `<app>.<sqid>.<apps>` | registry route, no record, no claim | `byHost`; `hasApex` per (account, app) | defers to whatever route got there first, forever | |
| +| App create, rename, move (server.go:3667, appsettings.go:802, :1305) | the apex | console SQL row | `apexHolder` (the same scan) | check then write against records; scan cost | |
| +| Domain attach, alias branch (apidomains.go:444; domainattach.go:153) | any hostname | `domains/<host>.json`, create-only | `deploymentRouteTaken` (fleet fold + claim), `domainHostPolicy` | does not read records or the lifecycle claim | |
| +| Domain attach, first-route branch (domainattach.go:183-192) | any hostname | claim (Route "") then static record | the same, claim-then-record order | the record half has the instance door's race against another app's record | |
| +| Platform attach (`--platform`, `attachDomainAs`) | names in the deployment's zones | the same two branches | `platformOperator` | the same | |
| +| Domain detach (apidomains.go:853; domainattach.go:304) | | `DeleteRecord` when Route "" then `DetachClaim` | owner match | none new | |
| +| Domains pass (`ReconcileAliases`, aliases.go) | | `route_aliases`, `wildcard_domains`, `route_paths` | routes win; `deploymentRoutes` | derived, not a writer of ownership; the final backstop | |
| +| Route aliases (`route_aliases` table) | | derived from claims only | | not a writer | |
| +| Moves (apiinstancemove.go, `SetRecordMove`) | unchanged host | record fields | lifecycle op (moves spec) | never changes who holds a host | |
| +| Placement (`carlos apps place`, `SetRecordRegion`) and pools | unchanged host | record Region | | never changes who holds a host | |
| +| Account migrate (`carlos accounts migrate`, accounts.go:312) | unchanged host | copies `apps/<src>/<app>/` to `apps/<dst>/<app>/`, create-only; stamps routes' account; deletes nothing | | two records per host until the operator deletes the source; `domains/` claims still name the source account; see §4.8 | |
| +| App purge (`release.PurgeApp`, trash.go:182) | all of an app's hosts | deletes every key under the app prefix | `PurgeReady` | leaves the app's `domains/` claims in place (an existing orphan source) | |
| +| Mail domains (`email/domain-claims/<domain>.json`) | email sending domains | its own create-only claim | its own owner gate | a different namespace: a domain can both serve a site and send mail. Out of scope | |
| + |
| +Two things stand out. Every record-backed writer already funnels through |
| +`provision.CreateRecord` and `DeleteRecord` (or `PurgeApp`), and the |
| +lifecycle claim spec already wraps exactly those two with a per-host CAS. |
| +Every alias writer already funnels through `domains.AttachClaim` and |
| +`DetachClaim`, which are already create-only by host. |
| + |
| +## 3. The objects |
| + |
| +### 3.1 The global hostname claim, `domains/<host>.json` |
| + |
| +domains.Claim: `{schema, host, account, app, route, created_at, |
| +created_by, wildcard, catchall, paths}`. `AttachClaim` reads, then |
| +`Put(key, body, "")` when absent; a lost create re-reads once and judges |
| +the winner; another owner gets `ErrHostUnavailable`, which never names |
| +the holder. Same owner converges `Route`. `DetachClaim` is owner-gated |
| +and quiet on mismatch. Every box folds `domains/` on its fast-converge |
| +leg through `ClaimCache` (one `ListMod` per pass, GETs only for changed |
| +objects), and `carlos domains list` and app Settings list it too. |
| + |
| +It is the right owner for aliases. It is the wrong owner for instance |
| +hosts, for three concrete reasons: |
| + |
| +- the fleet folds this prefix every few seconds on every box, and |
| + `ListObjectsV2` pages at 1000 keys; putting every instance host under |
| + it multiplies that listing by the instance count, forever; |
| +- the detach door's first-route branch deletes the record when |
| + `Route == ""` (apidomains.go:853). An instance ownership claim with no |
| + route would make `carlos domains detach <instance host>` an instance |
| + delete, unless every reader of the prefix learns a new `kind`; |
| +- the lifecycle claim already is a per-host, create-only, holder-naming |
| + object for record-backed hosts, with crash recovery designed and |
| + reviewed. A second one would be the duplicate the brief rules out. |
| + |
| +### 3.2 The lifecycle claim, `control/hosts/<host>.json` |
| + |
| +hostclaim.Claim: `{v, host, gen, holder{family, key, account, app, name, |
| +incarnation, created_at, location, ...}, op{kind, id, ...}, touched, |
| +inventory, obligations, lost, retired_incarnations, quiet_after, |
| +cancelled_creates}`. `hostclaim.Update` writes the whole object with |
| +`If-Match` on the ETag read, or `If-None-Match` when it read absence. |
| +The object is never deleted (C5a); free is a written state. |
| +`BeginCreate` refuses unless `c.Free()`. `Key` accepts any valid host |
| +shape, custom domains included. |
| + |
| +Spec §5.1 is the create protocol: read the claim (ETag or absent), |
| +CAS to `{holder: new, op: creating}`, `CreateRecord` create-only, CAS to |
| +`serving`. §5.1 step 5 is the recovery pass for a `creating` op older |
| +than five minutes: record present with this incarnation, complete it; |
| +record authoritatively absent, cancel it (free the claim, retire the |
| +incarnation, list the key under `cancelled_creates`). §5.2 is delete: |
| +CAS `deleting`, `DeleteIfMatch` the record, park, free. |
| + |
| +That CAS is host-scoped and atomic. Two creates of one host from two |
| +apps both read absent, both `If-None-Match`, and exactly one succeeds. |
| +**The lifecycle claim's holder is the ownership object for every |
| +record-backed host.** This spec adds nothing to it; it specifies how the |
| +console's doors use it and how aliases and minted hosts compose with it. |
| + |
| +### 3.3 The record and the registry |
| + |
| +The record stays the provisioning authority: boxes mint routes from |
| +records, and (under the lifecycle spec's C4) admit them only when the |
| +claim's holder names the record's incarnation. The registry's routes and |
| +`route_aliases` stay derived. Nothing here changes the box side. |
| + |
| +## 4. The design |
| + |
| +### 4.1 The rule |
| + |
| +Each hostname has exactly one **ownership authority** by family: |
| + |
| +| Family | Authority | Atomic step | |
| +|---|---|---| |
| +| record-backed (instance, replica, static record, sidecar) | lifecycle holder | `hostclaim.Update(BeginCreate)` from absent or free | |
| +| alias | `domains/<host>.json` | `AttachClaim`'s create-only `Put` | |
| +| minted (apex, static canary host) | the app named in the host | the console's unique (account, app) row, trash included | |
| + |
| +A host can be in only one family at a time. The cross-family rule: |
| + |
| +> **Write your own authority first. Then re-read every other family's |
| +> authority for the host. If any is held, undo your own write and answer |
| +> 409. Only then write anything else (the record, the alias config, the |
| +> SQL row's dependants).** |
| + |
| +Why it is enough: S3 gives strong read-after-write consistency for |
| +`PutObject` and `GetObject` on a key. Take two writers W1 (family X) and |
| +W2 (family Y) racing for host H. W1's write of X happens at t1 and W2's |
| +write of Y at t2. If t1 < t2, W2's re-read of X is after t2 > t1 and sees |
| +W1's object. Symmetrically for t2 < t1. So at least one of them backs |
| +off. Both may back off; that is two 409s and no residue, and either |
| +retry then wins. A deterministic tie-break was considered and rejected: |
| +the side that must yield cannot tell whether the other side also saw it, |
| +so any rule that lets one side keep going is unsound in some |
| +interleaving. |
| + |
| +The pre-write check stays as a cheap filter (it answers the common, |
| +sequential case without a write), but it is never what the design relies |
| +on. |
| + |
| +### 4.2 Instance create |
| + |
| +Replaces `hostHolder` and `hostClaimHolder` in `handleAPIInstanceCreate`. |
| +Replicas, static records and canary instances go through the same door. |
| + |
| +1. Shape, apex and sibling-label checks, unchanged (name-derived, §4.6). |
| +2. **Pre-check, cheap.** `GET domains/<host>`. A claim for another owner, |
| + or this app's own alias at that host, is 409 (an app cannot hold a host |
| + as both an alias and an instance; detach first). One `registry.Get` on |
| + this box, kept as today for operator-added routes with no record. The |
| + fleet-wide fold `edgesfile.DeploymentRoutes` replaces "this box only": |
| + a host a live box publishes as a real route that no record and no |
| + claim explains is 409 as an operator repair. The fold is best-effort |
| + and permissive on read failure, exactly as the attach door uses it. |
| +3. **Claim.** `hostclaim.Read`, then `Update(BeginCreate(...))`. The |
| + lifecycle spec's own create step 2. `ErrConflict` (lost the |
| + `If-None-Match` or `If-Match`) re-reads once: a holder that is not |
| + this app is 409; a `creating` op by anyone is 409 (never wait inside a |
| + handler); free means retry the CAS once. Other errors fail closed. |
| +4. **Verify.** `GET domains/<host>` again. An alias claim (`Route != ""`) |
| + for anyone: `Update(CancelCreate)`, 409. A first-route claim |
| + (`Route == ""`) is not a reason to back off: that writer has to pass |
| + through this same lifecycle CAS and will lose it (§4.5). |
| +5. `CreateRecord`, create-only. Then `Update(CompleteCreate)`. These are |
| + lifecycle steps 3 and 4 and need no change. |
| +6. The post-write sibling-label re-check stays (apiinstances.go:1180), |
| + and its undo becomes `CancelCreate` plus `DeleteIfMatch` on the |
| + record, so the host does not stay held by a withdrawn create. |
| + |
| +The scan of other apps' records is gone. The only thing that can say a |
| +host is held is the host's own claim. |
| + |
| +### 4.3 Instance delete |
| + |
| +Lifecycle §5.2 unchanged: CAS `deleting`, `DeleteIfMatch` the record, |
| +park, free. Ownership is released when the claim reaches free. A host |
| +that is `deleting` or `parking` is held for the create door (step 3 |
| +sees a non-free claim) and answers 409 with a sentence that says the |
| +address is being released, not who held it. |
| + |
| +### 4.4 Alias attach and detach |
| + |
| +In `handleAPIDomainAttach`'s alias branch and `attachDomainAs`'s alias |
| +branch, around the existing `AttachClaim`: |
| + |
| +1. Policy and `deploymentRouteTaken`, unchanged and in the same order |
| + (the ordering rules from the Task 5 review stand: nothing goes between |
| + `deploymentRouteTaken` and the write). |
| +2. **Pre-check, cheap.** `hostclaim.Read`. A holder, or any op other than |
| + none, is 409 with `hostUnavailableMsg`. Absent and free pass. |
| +3. `AttachClaim`. `created == false` (a re-attach or a no-op) ends here: |
| + ownership was decided on an earlier call. This keeps the steady-state |
| + cost of titogo's reload re-attaches at one GET, as today. |
| +4. **Verify.** `hostclaim.Read` again. A holder or a `creating` op: |
| + `DetachClaim` (own, just created), 409 with the same bytes. |
| +5. Audit and serial, unchanged. |
| + |
| +Detach is unchanged: an alias has no lifecycle claim to release. |
| + |
| +### 4.5 First-route attach |
| + |
| +The first-route branch writes a claim with `Route == ""` and then a |
| +static record. The record makes the host record-backed, so the lifecycle |
| +holder is its authority and the `domains/` claim is the attach surface's |
| +marker (detach reads it to know it owns the record). |
| + |
| +Order: `AttachClaim` (create-only, as today), then the create protocol of |
| +§4.2 steps 3 to 5 for the static record, with the pre-check of step 2 |
| +reduced to the registry and fold reads. If the lifecycle CAS loses, |
| +`DetachClaim` the claim this call created and answer 409. The instance |
| +create's verify step 4 ignores `Route == ""` claims for this reason: the |
| +first-route writer cannot finish without winning the lifecycle CAS. |
| + |
| +The existing idempotent branch (a record already at the key that |
| +`attachWrote` recognises) becomes: the claim's holder is this app's |
| +record, so the attach is already complete. |
| + |
| +Detach's first-route branch becomes lifecycle delete (§4.3) followed by |
| +`DetachClaim`, the mirror of attach. |
| + |
| +### 4.6 Minted hosts: apex and static canary |
| + |
| +The apex `<app>.<sqid>.<apps>` and a static app's canary host |
| +`<name>.<app>.<sqid>.<apps>` are minted by boxes from the app's name and |
| +channel pointers. They have no record and no claim, and should not get |
| +one: the name is the ownership. |
| + |
| +- **Atomic gate:** the console's app row. `claimApp` and the rename and |
| + move doors already refuse a name that is live or trashed in the |
| + account. That row is the apex's `If-None-Match`. |
| +- **Cross-family rule applied:** `claimApp` writes the row, then re-reads |
| + the lifecycle claim of the apex host (one GET, replacing `apexHolder`'s |
| + scan). Held by another app: delete the row, refuse. The instance |
| + create door already does the other half: write, then re-read |
| + `AppNameState` for the label, withdraw on a hit (§4.2 step 6). Rename |
| + and move do the same for the new apex. |
| +- **Static canary space.** The alias doors refuse any host of the form |
| + `<x>.<app>.<sqid>.<apps>` unless the attaching app is `app` in account |
| + `sqid`. Decided from the name alone, so it is not an oracle. On a |
| + deployment with `ReservePlatformDomains` the whole apps domain is |
| + already `hostReserved` except the delegated alias accounts; this rule |
| + matters when reservation is off. Instance hosts cannot be in this |
| + space (`ValidHost` is one label under `m.Domain`). |
| +- The domains pass's "routes always win" and `deploymentRoutes` checks |
| + stay as the backstop; they should no longer fire for anything this |
| + design admits. |
| + |
| +### 4.7 Replicas, sidecars, fleets, pools, moves |
| + |
| +- A replica is a record with its own host, created by the same door: |
| + §4.2 applies unchanged. Its lifecycle is the moves spec's. |
| +- Sidecars: the lifecycle claim covers them (family `sidecar`) for |
| + lifecycle. For ownership nothing is needed: the host embeds account and |
| + app, the attach door refuses the suffix, and the edge never routes it. |
| +- Fleets, pools and placement change `Region` on an existing record and |
| + never the host or the holder. |
| +- Moves change `op` and `holder.location`, never `holder.account` or |
| + `holder.app`. A host under a move is held; the create door sees a |
| + non-free claim and refuses. |
| + |
| +### 4.8 Account migrate |
| + |
| +`carlos accounts migrate --app A --to <dst>` copies records to the |
| +destination prefix and deletes nothing, so a host has two records until |
| +the operator removes the source. The lifecycle holder names the exact |
| +record key and the incarnation hashes the account, so the holder must |
| +move or the destination record is an orphan by C1 and is never admitted. |
| + |
| +This needs a holder transfer: one CAS from `serving` that rewrites |
| +`holder.key`, `holder.account` and `holder.incarnation` to the |
| +destination record, from a verb that has proved the destination record |
| +exists with the same body. That is a lifecycle transition and belongs to |
| +the lifecycle spec, not here. Until it exists, migrate must refuse an app |
| +whose hosts carry a claim, with the sentence naming this spec. |
| + |
| +Also recorded: migrate does not touch `domains/` claims, so an app's |
| +aliases keep naming the source account and the alias pass refuses them |
| +after the move (`claim says <src>`). A `--rebind-claims` step for the |
| +verb is a follow-up. |
| + |
| +## 5. Crash recovery |
| + |
| +Each crash window, what it leaves, and who clears it. No step below |
| +deletes anything it did not create, and nothing frees a host that has a |
| +record with the holder's incarnation. |
| + |
| +| Crash after | Left behind | Cleared by | |
| +|---|---|---| |
| +| §4.2 step 3 (claim `creating`, no record) | an orphaned claim | the lifecycle recovery pass (§5.1 step 5): record absent, `CancelCreate`. The host is held for at most the five-minute age plus the pass interval. It never steals: a record present with this incarnation completes instead | |
| +| §4.2 step 5 record PUT, before `CompleteCreate` | claim `creating`, record present | the same pass completes the create | |
| +| §4.2 step 4 back-off, before `CancelCreate` | claim `creating`, an alias claim present | the pass cancels; the alias was the earlier writer and stands | |
| +| §4.4 step 3 (alias claim written, verify not run) | an alias claim that may collide with a lifecycle holder | the alias pass refuses it (routes win), so nothing is served wrongly. A console sweep lists `domains/` claims with `Route != ""` whose host has a lifecycle holder, logs them as conflicts, and leaves removal to detach or an operator: deleting a tenant's claim automatically is not this design's to do | |
| +| §4.5 first-route, after `AttachClaim`, before the lifecycle CAS | a `Route == ""` claim with no record | inert today (nothing materialises from a routeless claim), visible to its owner, cleared by detach or a retry. Unchanged | |
| +| delete (§4.3) at any step | the lifecycle spec's recovery owners | unchanged | |
| +| app purge | records gone, `domains/` claims of the app left | the same console sweep lists claims whose account and app are purged and removes them. Removal is safe here: the owner no longer exists | |
| + |
| +Old-console deletes during the roll (a record gone with a `serving` |
| +claim) are the lifecycle spec's §10 case: the host is held until |
| +`carlos ops host resolve`. For the create door that is a 409 naming an |
| +operator repair, which is visible and safe. |
| + |
| +## 6. Composition with the lifecycle claim |
| + |
| +Ownership and lifecycle are **one object for record-backed hosts** and |
| +**two for aliases**, and the two never both own a host. Concretely: |
| + |
| +- This spec asks nothing new of `internal/hostclaim`: `Read`, `Update`, |
| + `BeginCreate`, `CancelCreate`, `CompleteCreate`, and the recovery pass |
| + are used as specified. The ownership question "who holds H" is |
| + `Read(H).Claim.Holder` and nothing else. |
| +- It asks one thing of the lifecycle spec: the holder transfer for |
| + account migrate (§4.8), as a new transition or a ruling that migrate |
| + is refused for claimed hosts. |
| +- It adds console-side protocol around the lifecycle CAS: the pre-check |
| + and verify reads of `domains/`, and the alias doors' reads of |
| + `control/hosts/`. These are reads, and the lifecycle claim does not |
| + know aliases exist. |
| +- The lifecycle floor (§10 of that spec) gates this design too, since |
| + the create door's atomic step is the lifecycle create. Below the floor |
| + the door keeps today's checks. |
| + |
| +The alternative, an `alias` holder family in the lifecycle claim so that |
| +one object owns every routable name, was considered (§12). It is |
| +simpler to reason about but changes a spec that twelve joint rounds and |
| +a simulation have settled, and it puts every custom domain into the |
| +recovery pass's listing. |
| + |
| +## 7. Privacy |
| + |
| +- A refusal caused by another **account** never names it. The lifecycle |
| + holder's account differing from the caller's produces the same |
| + `hostUnavailableMsg` bytes and status as the `domains/` door's |
| + `ErrHostUnavailable`, on every door, including the create door's |
| + back-off in §4.2 step 4. Today's `hostHeldSentence` can say "routed for |
| + account X"; that sentence is retired for cross-account holds and kept |
| + only when the route names the caller's own account. |
| +- Within the caller's account the refusal may name the holding app and |
| + whether it is trashed, as today: a member can already list the |
| + account's apps. |
| +- Ordering stays neutral: the new reads sit where `deploymentRouteTaken` |
| + sits, below every gate that could refuse for another reason, so a |
| + prober cannot tell a held host from a free one by which refusal comes |
| + first. |
| +- The claim objects live in the deployment bucket and are read by |
| + consoles and boxes only. No door returns a claim body to a member. |
| +- The console sweep's log lines name hosts and owners; they go to the |
| + operator log, not to any tenant's activity log. |
| + |
| +## 8. Migration |
| + |
| +### 8.1 Backfill |
| + |
| +A console-side verb, `carlos ops hosts backfill [--dry-run]`: |
| + |
| +1. List the bucket, not SQL: every `apps/*/*/instances/*.json` and |
| + `apps/*/*/sidecars/*.json` (the reconciler's own listing). This covers |
| + direct-bucket records of apps SQL does not list. |
| +2. For each host with no lifecycle claim, `claim.Adopt` with the current |
| + record as holder, as the lifecycle spec's §9 defines (one adoption |
| + routine, discovery obligations included). Create-only; a claim that |
| + appeared since is left alone. |
| +3. A host with **two or more records** (an existing collision) is not |
| + adopted. It is reported with both keys, and an operator picks the |
| + winner by deleting the loser's record, then re-runs. The door refuses |
| + creates at that host meanwhile (§8.2), which is strictly safer than |
| + today. |
| +4. On a clean pass, write `control/hosts-backfill.json` with the count |
| + and time. Re-running is idempotent. |
| + |
| +Aliases need no backfill: `domains/` already is their authority. |
| + |
| +### 8.2 Lazy adoption at the door |
| + |
| +Until the marker exists, a create that reads an absent claim cannot |
| +treat absence as free: a legacy record under another app may hold the |
| +host. So the door keeps today's `hostHolder` scan **only on the absent |
| +path**, and on a hit it adopts the claim for the record it found and |
| +refuses. On a miss it proceeds with `BeginCreate`. Once the marker is |
| +present the scan is removed and absent means free. A host with a claim |
| +never scans, so the cost falls as the backfill proceeds and goes to zero |
| +at the marker. |
| + |
| +### 8.3 Mixed-version consoles |
| + |
| +An old console creates without a claim. Its record is a legacy host and |
| +is adopted by the first new actor that touches it (§8.2, or reconcile |
| +under the lifecycle spec). Two consoles of different versions racing for |
| +one host remain racy until every console is new; that is the lifecycle |
| +floor's job and the roll is consoles first. An old console's delete |
| +leaves a `serving` claim with no record; §5 covers it. |
| + |
| +### 8.4 Cross-account |
| + |
| +- Backfill adopts whatever record exists, including records under |
| + accounts the console does not know. It reports those but never |
| + rewrites or deletes them. |
| +- The door's refusals follow §7. Lazy adoption on the absent path reads |
| + only the caller's own account's records, as the scan does today, so a |
| + foreign record is not found by it; the foreign host's claim comes from |
| + the backfill or from its own account's next touch, and until then the |
| + box-side `duplicate host` log is what it is today. |
| + |
| +## 9. Cost |
| + |
| +Per instance create, today: one registry read, two SQL reads, N |
| +sequential `GetRecord` calls (one per other app of the account, live and |
| +trashed), one claim GET, one record PUT. For an account with 50 apps that |
| +is 50 GETs and roughly a second of the 15 s budget. |
| + |
| +After: one `domains/` GET, one fleet fold read (`edgesfile.DeploymentRoutes`, |
| +the same read the attach door already pays), one claim GET, one claim |
| +PUT, one `domains/` GET, one record PUT, one claim PUT. Five bucket calls |
| +plus the lifecycle spec's two, independent of account size. The lifecycle CAS pair is a |
| +cost the lifecycle spec already commits to; this design adds two GETs |
| +to it and removes N. |
| + |
| +Per alias attach on the create path: two claim GETs more than today. On |
| +the re-attach path (titogo's reload loop): unchanged, one GET, because |
| +§4.4 stops after `created == false`. |
| + |
| +Boxes: nothing new. `control/hosts/` is folded only by the console's |
| +recovery pass, on the cadence the lifecycle spec sets. |
| + |
| +## 10. Tests |
| + |
| +All against a dirstore with real CAS, in `internal/console`, foreground. |
| + |
| +1. **Concurrent create.** Two apps, one host, 2 and then 16 goroutines |
| + per app. Exactly one 201; every other response is 409 with the |
| + within-account sentence; exactly one record in the bucket; the claim's |
| + holder names it; `op.kind == serving`. Run at seeds 1..200. |
| +2. **Alias versus instance, both orders and concurrent.** Attach then |
| + create, create then attach, and both in parallel with an injected |
| + delay at every point between steps 2 and 5 of §4.2 and 2 and 4 of |
| + §4.4. Invariant after both return: never both 2xx; the bucket holds |
| + either an alias claim and a free or absent lifecycle claim, or a |
| + record with a matching holder and no alias claim, or neither. |
| +3. **First-route versus instance** under the same harness; the |
| + `Route == ""` claim never causes a create to back off on its own. |
| +4. **Crash between claim and record.** Inject a failure after |
| + `BeginCreate`. Run the recovery pass with the clock advanced past five |
| + minutes: claim free, incarnation retired, `cancelled_creates` lists the |
| + key; a new create of the host by another app succeeds. The negative: |
| + with the record present, the pass completes and never frees. |
| +5. **Crash after alias claim, before verify** with a lifecycle holder |
| + present: the sweep reports it; the alias pass does not materialise it; |
| + detach clears it. |
| +6. **Backfill.** Seed records with no claims across three apps and two |
| + accounts, including one host with two records. The verb adopts the |
| + singletons, refuses the pair and names both keys, writes no marker; |
| + after the loser is deleted it adopts and writes the marker. The door |
| + refuses creates at the collided host throughout. |
| +7. **Lazy adoption.** No marker, a legacy record under app A, a create |
| + for the host under app B: 409, and the claim now names A's record. |
| + With the marker, the same create reads absent and succeeds (the |
| + fixture has no legacy record then). |
| +8. **Privacy.** A cross-account hold on every door answers the same |
| + status and body as a foreign `domains/` claim, byte for byte, and the |
| + body contains no sqid. A table test over the holder kinds. |
| +9. **Apex.** `claimApp` racing an instance create at the label: the |
| + write-then-verify rule yields at most one winner, in both orders. |
| +10. **Alias refusal in canary space** with reservation off. |
| +11. **Migrate refusal** for an app with a claimed host. |
| + |
| +The joint simulation (`internal/sim/hostlife`) should gain the alias |
| +actor and the two-family rule as an invariant, if the moves session |
| +agrees; this spec does not depend on it. |
| + |
| +## 11. Rollout |
| + |
| +1. Land `internal/hostclaim` on main (`feat/hostclaim-library`). |
| +2. Ship the backfill verb and the console sweep (§5) with `--dry-run`. |
| + Run dry on flagship, read the collision list, repair, run live, confirm |
| + the marker. |
| +3. Ship the doors: create, delete, attach, first-route, detach, claimApp, |
| + rename, move. Behind the lifecycle floor; below it the doors keep |
| + today's checks. The absent-path scan stays until the marker (§8.2). |
| +4. Roll consoles, then boxes, in the lifecycle spec's order. Publish the |
| + floor. |
| +5. Remove the absent-path scan and `hostHolder`'s record loop once every |
| + deployment has the marker. `apexHolder` and `ReportApexSquats` read the |
| + claim instead of scanning. |
| +6. Follow-ups filed, not built here: migrate's holder transfer and claim |
| + rebind (§4.8); the purge sweep's claim removal (§5). |
| + |
| +## 12. Alternatives considered |
| + |
| +- **`domains/<host>.json` as the ownership object for everything.** |
| + Rejected for the three reasons in §3.1: fleet-wide listing cost, the |
| + detach door's record delete on `Route == ""`, and duplicating the |
| + lifecycle claim's create protocol. |
| +- **A new `hosts/<host>.json` ownership-only object.** Rejected: it would |
| + duplicate the lifecycle holder for every record-backed host and need |
| + its own crash recovery. |
| +- **An `alias` holder family in the lifecycle claim.** One object for |
| + every routable name, no two-family rule. Rejected for now: it changes a |
| + settled spec and its validation (`holderKeyMatches`), gives every custom |
| + domain a lifecycle object the recovery pass lists, and gives titogo's |
| + reload loop a second GET per host unless the same `created == false` |
| + short-circuit is added. It remains the cleanest end state if the |
| + two-family rule proves hard to keep as doors are added, and §13 asks. |
| +- **Deterministic tie-break instead of both backing off.** Rejected as |
| + unsound (§4.1). |
| +- **Lazy adoption only, no backfill.** Rejected: the scan cost would |
| + persist indefinitely and the existing collisions would never be found. |
| + |
| +## 13. Open questions for Paul |
| + |
| +1. Two families with the write-then-verify rule (this spec), or one |
| + object with an `alias` family in the lifecycle claim (§12)? The second |
| + needs the lifecycle spec reopened. |
| +2. Account migrate: add a holder-transfer transition to the lifecycle |
| + spec, or refuse migrate for claimed hosts until moves are done? |
| +3. The console sweep: should it remove conflicting alias claims |
| + automatically after a grace period, or only report them (this spec)? |
| +4. Backfill collisions on flagship: operator picks the winner by hand |
| + (this spec), or the verb keeps the record the provisioning pass |
| + currently keeps (first in listing order)? |
| +5. Is the `<x>.<app>.<sqid>.<apps>` alias refusal acceptable on |
| + deployments with reservation off, or should those names stay open? |
| +6. Should an app's own alias at a host it now wants as an instance be |
| + converted in place rather than refused? |