carlos / platform Public

design: host ownership rev 4, one ownership record per hostname (Paul's ruling)

domains/<host>.json owns every family (instance, sidecar, alias, apex)
with immutable acquisition ids and pending/committed/releasing states;
the lifecycle claim keeps execution, parking and retirement, bound by
op.intent.ownership and the holder incarnation; release follows
lifecycle retirement and is recoverable; detach dispatches by family;
per-app owned/ index for complete purge enumeration; app-row
incarnation fencing; derived alias index for fleet folds; freeze with a
write fence, reconciliation and quarantine for migration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
claude agent, operated by paul@keymail.dev b2cdc501f87a4deeea8f43cc0e4b331f8957476e parent 5c4c31f
1 file changed, +725 −869
  • docs/superpowers/specs/2026-10-06-host-ownership-design.md +725 −869

CI passed — run details

diff --git a/docs/superpowers/specs/2026-10-06-host-ownership-design.md b/docs/superpowers/specs/2026-10-06-host-ownership-design.md
index f51f22cbd..26690dc09 100644
--- a/docs/superpowers/specs/2026-10-06-host-ownership-design.md
+++ b/docs/superpowers/specs/2026-10-06-host-ownership-design.md
@@ -1,926 +1,782 @@
# Host-scoped atomic ownership
-Design, 2026-10-06, **rev 3**. Closes amadan #232. Design only; nothing
+Design, 2026-10-06, **rev 4**. Closes amadan #232. Design only; nothing
here is built.
-Review history. Rev 1 (b70d4d30) and rev 2 (519d25e9) each went to Astra
-for a read-only review against the code; the findings are cited `A-n`
-(round 1) and `B-n` (round 2) below. Rev 2 answered A-1 to A-15. Rev 3
-answers B-1 to B-15 and the parts of round 1 that round 2 judged partial:
-the alias acquisition carries an immutable id and kind for the life of
-the claim, takeover is a CAS before verification, the sweep is delete-only
-(B-1, B-7, B-11, B-12); creates are frozen between the floor and the
-marker so no unclaimed record can be written after the floor (B-2); the
-adoption rule and a quarantine guard are rollout prerequisites and the
-backfill validates every host across all three authorities (B-3, B-4);
-alias finalizers verify the app row (B-5); the first-route marker is
-bound to the record's incarnation (B-6); console materialization detaches
-claims before it releases a purged name (B-8); app-name operations get an
-op id and reserve both names (B-9); the own-app apex exception is explicit
-and direct-bucket apps block completion (B-10); migrate quiesces the app
-through admission (B-13); the response mapping is ordered and complete
-(B-14); readers and costs are listed (B-15). Rev 3 has not been reviewed.
-
-Related specs: `2026-10-02-host-lifecycle-claim-design.md` rev 6.14 on
-`design-replica-ownership` (the lifecycle claim), with its library at
-`internal/hostclaim` on `feat/hostclaim-library`; `2026-08-12-domain-attach-design.md`
+**Ruling (Paul, 2026-10-06): one ownership record per hostname.**
+`domains/<host>.json` is the single authoritative ownership object for
+every family. Astra's round 3 (`~/.claude/tmp/hostown-r3.md` on DOCS-T1,
+§3 and §4) recommended it. Rev 4 is a rewrite around that model; revs 1
+to 3 (b70d4d30, 519d25e9, 5c4c31f1) carried a two-family design whose
+cross-family verification this ruling removes. Their review findings are
+cited `A-n`, `B-n`, `C-n` where rev 4 still answers them.
+
+Related specs: `2026-10-02-host-lifecycle-claim-design.md` **rev 6.16**
+(1915bac2) on `design-replica-ownership`, library at `internal/hostclaim`
+on `feat/hostclaim-library` (47426cc1); `2026-08-12-domain-attach-design.md`
(the global hostname claim, #151, #82).
## 0. Summary
-A hostname on the platform is held today by one of three things, and no
-single object says which:
-
-- an instance record at `apps/<acct>/<app>/instances/<host>.json`
- (provision.RecordKey), keyed per account and app, so two apps can both
- write a record for one host;
-- a global hostname claim at `domains/<host>.json` (domains.ClaimKey),
- keyed by host alone, create-only, written by the domain attach doors;
-- a registry route a box minted from an app's name (its platform apex and
- a static app's canary hosts), with no record and no claim.
-
-The instance create door checks the first two before it writes
-(`hostHolder`, `hostClaimHolder` in internal/console/apex.go, 04cd2d78),
-then writes a record whose conditional create protects only its own key.
-The check and the write are two steps, so two concurrent creates of one
-host from two apps can both get 201. The same shape exists between an
-alias attach and an instance create.
-
-The design:
-
-1. **Record-backed hosts** (instances, replicas, static records, sidecars)
- are owned by the **lifecycle claim's holder** at
- `control/hosts/<host>.json`. The lifecycle create protocol's claim CAS
- is the atomic gate. There is no second ownership object for these
- hosts, and this spec adds no field to that claim.
-2. **Aliases** (hosts with a claim and no record) are owned by the
- **global hostname claim** at `domains/<host>.json`. The claim gains
- one additive block, `acq`, so an acquisition has an immutable identity
- and kind and is written pending, verified, then committed. Only
- committed claims materialise or count as held by their owner.
-3. The two families meet at one rule: **write your own authority first,
- re-read every other authority, back off if any is held, and only then
- commit.** With S3's strong read-after-write consistency at least one
- of two racing writers sees the other. Every back-off deletes only the
- exact object version it wrote.
-4. **Minted hosts** (apex `<app>.<sqid>.<domain>`, static canary
- `<name>.<app>.<sqid>.<domain>`) are owned by the app whose name they
- carry. One namespace rule, applied by every door, says so from the
- name alone. The console's unique (account, app) row is the atomic gate
- for the name, and it too is written then verified.
-5. Legacy hosts get a claim from a backfill verb that runs after the
- version floor, during a short freeze of every create door, so no
- writer can add an unclaimed record behind it. The door treats an
- absent claim as free only once the backfill has written its completion
- marker. There is no lazy adoption.
+1. **One object.** `domains/<host>.json` owns every hostname: instance
+ and replica records, static records (including first-route attaches),
+ sidecars, aliases, platform domains, and the minted app apex. It is
+ acquired with an `If-None-Match` create. A canary host
+ `<name>.<app>.<sqid>.<apps>` is covered by its app's apex ownership
+ and has no object of its own.
+2. **Acquisitions** carry an immutable id and family and move
+ `pending → committed → releasing → gone`. Every update and delete is
+ conditional on the ETag read. A same-owner retry resumes its own
+ acquisition or takes over a stale one by CAS, which fences the writer
+ it displaces.
+3. **The lifecycle claim** (`control/hosts/<host>.json`) keeps
+ execution, parking and retirement. It is bound to the ownership
+ acquisition: the create's `op.intent` names the acquisition id, and a
+ serving holder's incarnation is the one the committed ownership
+ records. It is no longer an ownership authority, and its fencing
+ history is never deleted when a hostname is released (C5a).
+4. **Release** follows lifecycle retirement: ownership goes `releasing`
+ before the lifecycle delete starts and is deleted only once the
+ lifecycle claim is free. A paused creator whose pending reservation was
+ reclaimed cannot reach serving: its commit CAS fails and the lifecycle
+ recovery pass refuses to complete a create whose acquisition the
+ ownership object does not name.
+5. **Detach dispatches by family.** `Route == ""` grants nothing.
+6. **Per-app index.** `apps/<acct>/<app>/owned/<host>.json` lists the
+ hosts an app owns, written before commit and removed after release, so
+ purge and materialization enumerate completely and fail closed.
+7. **App incarnation.** Every app row carries a random incarnation;
+ ownership records it; a late writer for a purged and recreated name is
+ refused at commit.
+8. **Fleet folds** filter by family, and a console-derived alias index
+ replaces the per-box listing once `domains/` exceeds one page.
+9. **Migration** is a freeze with a demonstrated write fence, a
+ reconciliation of records, aliases, app rows and routes, quarantine
+ for every collision, and an operator-chosen winner. Destructive
+ orphan cleanup is disabled until the backfill marker exists.
## 1. The problem, in the code
-`handleAPIInstanceCreate` (apiinstances.go:772-792) asks `hostHolder`,
-which reads this box's registry and then one `GetRecord` per other app
-of the account (live and trashed, SQL-listed), and `hostClaimHolder`,
-which reads `domains/<host>.json`. Then `provision.CreateRecord`
-(provision.go:820) does `Put(key, body, "")`, create-only on
-`apps/<acct>/<app>/instances/<host>.json`.
-
-Consequences:
-
-- Two creates for one host from apps A and B both pass the checks and
- both write, because their keys differ. The provisioning pass keeps the
- record it lists first and logs `duplicate host` for the other
- (provision/reconcile.go:819). The loser looks created and serves
- nothing, or a different box serves it.
-- An alias attach (`domains.AttachClaim`, apidomains.go:444 and
- domainattach.go:153) and an instance create for the same host each
- check the other's object before writing their own. The same window.
-- The record scan costs one sequential GET per other app, inside the
- handler's 15 s budget; it sees only SQL-listed apps and only this
- console box's registry; direct-bucket records of apps not in SQL are
- outside it.
-- The checks run before later validation (replica rails at :887, shape
- at :1092), so an invalid request answers 409 for a held host and 400
- for a free one: an occupancy oracle.
+`handleAPIInstanceCreate` (apiinstances.go:772-792) asks `hostHolder`
+(one `GetRecord` per other app of the account, this box's registry) and
+`hostClaimHolder` (`domains/<host>.json`), then `provision.CreateRecord`
+(provision.go:820) writes create-only on a key that includes the app. Two
+apps racing for one host both pass and both write; the provisioning pass
+keeps the record it lists first (`duplicate host`, provision/reconcile.go:819).
+Alias attach (`domains.AttachClaim`, apidomains.go:444, domainattach.go:153)
+and instance create check each other's object before writing their own:
+the same window. The checks run before later validation (:887, :1092), an
+occupancy oracle. The scan sees only SQL-listed apps of one account and
+costs N sequential GETs.
## 2. Writer inventory
-Every path that puts a hostname on the platform or takes one off. The
-**Role** column says how it relates to ownership: *decides* (it is an
-authority write), *derives* (it reads an authority and materialises),
-*operator* (outside the guarantee, by hand), or *no change* (host and
-holder unchanged).
-
-| Writer | Hosts | Writes today | Role | Gap today |
-|---|---|---|---|---|
-| Instance create (`handleAPIInstanceCreate`) | `<label>.<m.Domain>` | record, create-only | decides | check then write; scan cost; local registry; ordering oracle |
-| Instance delete (:1675) | same | `DeleteRecord` | decides | nothing host-scoped to release yet |
-| Replica create (same door, `CreateRecordReplica`) | one label + the app's domain | record | decides | as create |
-| Sidecar create and delete (apisidecars.go:285, :347; `CreateSidecarRecord`, `DeleteSidecarRecord`, sidecar.go:322, :398) | `<name>.<app>.<sqid>.sidecar.internal` | sidecar record at `sidecars/<name>.json`; its own primitives | decides (lifecycle family `sidecar`) | none for ownership: the host embeds account and app; never routable |
-| Canary (`carlos canary`, canary.go:231) | `<canary>-<app>.<sqid>.<apps>` | the create door | decides, via the door | flat label makes collisions easy (#207) |
-| Platform apex mint (domains/reconcile.go:266) | `<app>.<sqid>.<apps>` | registry route via `Upsert`; candidates come from routes (`candidateApps`, :757), not SQL | derives from the app name | defers to any route already at the host; mints for apps with no SQL row |
-| Static canary mint (domains/reconcile.go:328-337) | `<name>.<app>.<sqid>.<apps>` | registry `Upsert`, no `byHost` check; `Upsert` rewrites `app` and `account` on conflict (registry.go:990) | derives from the app name | can overwrite an unrelated route |
-| Trash and retired-canary route removal (domains/reconcile.go:207, :356) | the app's routes | `reg.Delete` | derives | none |
-| Instance and sidecar route mint and removal (provision/reconcile.go:965, :1178; sidecar_reconcile.go:403, :450) | record hosts | registry rows | derives from records | first-listed record wins a duplicate host |
-| App create, rename, move (server.go:3667, :3707; appsettings.go:802, :827, :1305, :1350; store.go:1801-1905) | the apex | SQL app row and related rows (credentials, grants) | decides the name | check then write against records; no crash recovery after the row; rename and move release the old name at once |
-| Startup owners backfill (store.go:676) | the apex | `INSERT OR IGNORE` ops-account rows for legacy `owners` entries with no row | decides the name, by hand | legacy-only; does not register bucket-derived apps |
-| Console materialization (materialize.go:391) | the apex | removes a trashed app's SQL row when its bucket prefix is gone (after box auto-purge) | releases the name | releases it while the app's `domains/` claims still exist, so a new app of the name inherits them |
-| Domain attach, alias branch (apidomains.go:444; domainattach.go:153) | any hostname | `domains/<host>.json`, create-only; same-owner `Route` converge in place (claims.go:150) | decides | does not read records, the lifecycle claim or app rows; converge can turn a first-route claim into an alias |
-| Domain attach, first-route branch (domainattach.go:183-192) | any hostname | claim (`Route ""`) then static record | decides, both families | the record half has the create door's race |
-| Attach rollback (`rollbackAttach`, domainattach.go:483-499) | | `DeleteRecord`, `DetachClaim` | cleanup | unconditional deletes |
-| Platform attach (`--platform`, `attachDomainAs`) | names in the deployment's zones | the same two branches | decides | the same |
-| Domain detach (apidomains.go:853; domainattach.go:288-314) | | `DeleteRecord` when `Route ""`, then `DetachClaim` (read, then unconditional delete, claims.go:182) | decides | unconditional delete; the record to delete is chosen from a claim read that nothing fences |
-| Claim capabilities and paths (`SetClaimCapabilities`, `SetClaimPaths`, claims.go:302, :398) | | fields of an existing claim, owner-gated CAS | no change to ownership | must refuse a pending claim |
-| Domains pass (`ReconcileAliases`, aliases.go:89-133): alias, wildcard, catch-all and path derivation | | `route_aliases`, `wildcard_domains`, `route_paths` | derives from claims | never reads the lifecycle claim; must skip pending claims |
-| Operator routes (`carlos route add`, `carlos remove`; cli.go:255, :278) | any | registry `Upsert`, `Delete` on one box | operator | covered by the fleet fold, not by any object |
-| Moves (apiinstancemove.go, `SetRecordMove`) | unchanged | record fields | no change | |
-| Placement (`SetRecordRegion`) and pools | unchanged | record Region | no change | |
-| Account migrate (accounts.go:312-470) | unchanged | copies `apps/<src>/<app>/` to `apps/<dst>/<app>/` create-only; only channel pointers are rebound (:369), records are copied unchanged (:416); stamps routes' account (:456); deletes nothing | operator | the copied record names the source account, which `GetRecord` refuses (provision.go:1145); two records per host; `domains/` claims still name the source; the plan is built from a listing nothing fences |
-| App purge (`release.PurgeApp`, trash.go:182-278; console door ownership.go:442; box auto-purge domains/reconcile.go:232) | all of an app's hosts | deletes every key under the app prefix and the marker | decides (release) | leaves the app's `domains/` claims |
-| Mail domains (`email/domain-claims/<domain>.json`) | email sending domains | its own create-only claim | separate namespace | out of scope |
-
-Every record-backed writer funnels through `CreateRecord`,
-`CreateSidecarRecord`, `DeleteRecord`, `DeleteSidecarRecord` or
-`PurgeApp`, and the lifecycle claim spec wraps exactly those with a
-per-host CAS. Every alias writer funnels through `AttachClaim` and
-`DetachClaim`.
+Every path that puts a hostname on the platform or takes one off, and its
+place in the new model. **Acquires** means it writes the ownership object;
+**derives** means it reads ownership and materialises; **operator** is
+outside the guarantee.
-## 3. The objects
+| Writer | Hosts | Writes today | Rev 4 |
+|---|---|---|---|
+| Instance create (`handleAPIInstanceCreate`) | `<label>.<m.Domain>` | record | acquires, family `instance` (§4.2) |
+| Instance delete (:1675) | | `DeleteRecord` | releases (§4.4) |
+| Replica create (same door) | | record | acquires, family `instance` |
+| Sidecar create and delete (apisidecars.go:285, :347; `CreateSidecarRecord`, `DeleteSidecarRecord`) | `<name>.<app>.<sqid>.sidecar.internal` | sidecar record | acquires and releases, family `sidecar` |
+| Canary (`carlos canary`) | `<canary>-<app>.<sqid>.<apps>` | the create door | via the door |
+| Platform apex mint (domains/reconcile.go:266; candidates from routes, :757) | `<app>.<sqid>.<apps>` | registry `Upsert` | derives from the `apex` ownership of (account, app) |
+| Static canary mint (domains/reconcile.go:328-337; `Upsert` rewrites app and account, registry.go:990) | `<name>.<app>.<sqid>.<apps>` | registry `Upsert`, no `byHost` check | derives from the app's `apex` ownership; `byHost` check added |
+| Trash and retired-canary route removal (:207, :356) | | `reg.Delete` | derives, unchanged |
+| Instance and sidecar route mint and removal (provision/reconcile.go:965, :1178; sidecar_reconcile.go:403, :450) | | registry rows | derives from records and lifecycle admission, unchanged |
+| App create, rename, move (server.go:3667, :3707; appsettings.go:802-840, :1305-1350; store.go:1801-1905) | the apex | SQL app row and related rows | acquires, family `apex` (§4.6) |
+| Startup owners backfill (store.go:676, run at server.go:1357 on every start) | the apex | `INSERT OR IGNORE` ops rows | retired above the floor; replaced by `carlos ops apps register` (§4.6, C-1) |
+| `console migrate-app` (cmd/console/main.go:228, `MoveApp`) | the apex | SQL row move | goes through the apex gate's move path (C-1) |
+| Console materialization (materialize.go:391, `PurgeAppRow`) | the apex | releases a trashed row | releases only after the app's `owned/` index is empty (§5) |
+| Domain attach, alias branch (apidomains.go:444; domainattach.go:153; converge, claims.go:150) | any | `domains/<host>.json` | acquires, family `alias` (§4.3) |
+| Domain attach, first-route branch (domainattach.go:183-192) | any | claim then static record | acquires, family `instance` with `first_route` (§4.3) |
+| Platform attach (`--platform`) | deployment zones | the same two branches | the same, `platform` recorded |
+| Attach rollback (domainattach.go:483-499) | | unconditional deletes | conditional (§4.1) |
+| Domain detach (apidomains.go:853; domainattach.go:288-314) | | `DeleteRecord` when `Route ""`, `DetachClaim` | dispatch by family (§4.5) |
+| Claim capabilities and paths (claims.go:302, :398) | | fields of an existing claim | unchanged; refuse unless committed `alias` |
+| Domains pass (`ReconcileAliases`, aliases.go:89-133) | | `route_aliases`, wildcard, catch-all, paths | derives from committed `alias` ownership (§4.8) |
+| Operator routes (cli.go:255, :278) | any | registry on one box | operator; covered by the fleet fold |
+| Moves, placement, pools | unchanged host | record fields | no change to ownership |
+| Account migrate (accounts.go:312-470; records copied unchanged :416, refused by `GetRecord` provision.go:1145) | unchanged | copies objects, stamps routes | refused for apps with records; quiesced through admission (§4.7) |
+| App purge (trash.go:182-278; console door ownership.go:442; box auto-purge domains/reconcile.go:232) | all the app's hosts | deletes the app prefix | releases every owned host first; `PurgeApp` keeps `owned/` (§5) |
+| Mail domains (`email/domain-claims/`) | sending domains | its own claim | separate namespace, out of scope |
-### 3.1 The global hostname claim, `domains/<host>.json`
+## 3. The objects
-domains.Claim: `{schema, host, account, app, route, created_at,
-created_by, wildcard, catchall, paths}`. `AttachClaim` reads, then
-`Put(key, body, "")` when absent; another owner gets
-`ErrHostUnavailable`, which never names the holder; the same owner
-converges `Route` in place. `DetachClaim` is owner-gated and quiet on
-mismatch. Every box folds `domains/` on its fast-converge leg through
-`ClaimCache`, and `carlos domains list` and app Settings list it too.
+### 3.1 Ownership: `domains/<host>.json`
-This spec adds one block, additive and `omitempty`:
+Today's domains.Claim, `{schema 1, host, account, app, route,
+created_at, created_by, wildcard, catchall, paths}`, extended with
+additive `omitempty` fields. The key is the same for every family, so one
+`If-None-Match` create decides the host.
```
-"acq": {
- "id": "<random, 128-bit>",
- "kind": "alias" | "first-route",
- "state": "pending" | "committed",
- "incarnation": "<hex>", // first-route only, set at commit
- "started_at": "<RFC3339>"
+{
+ "schema": 1,
+ "host": "shop.bab.oncarlos.com",
+ "account": "bab", "app": "jam",
+ "app_incarnation": "<hex, from the apps row>",
+ "family": "instance" | "sidecar" | "alias" | "apex",
+ "acq": {
+ "id": "<random 128-bit>",
+ "state": "pending" | "committed" | "releasing",
+ "started_at": "<RFC3339>",
+ "by": "<principal>",
+ "record_key": "apps/bab/jam/instances/shop....json", // instance, sidecar
+ "incarnation": "<hex>", // instance, sidecar, set at commit
+ "release_op": "<lifecycle op id>" // releasing, instance and sidecar
+ },
+ "first_route": true, // instance acquired through the domains surface
+ "platform": true, // acquired through the --platform door
+ "route": "...", "wildcard": ..., "catchall": ..., "paths": ... // alias only
}
```
Rules:
-- `id` and `kind` are **immutable for the life of the claim**. A repoint
- preserves them; a takeover (§4.4) replaces the whole claim including
- `id`, which is how it fences the writer it displaces. Two acquisitions
- of one host therefore never produce identical bytes, so their ETags
- differ on every store (B-7).
-- `state: pending` reserves the key against everyone but is **not
- held** by its owner: the alias pass derives nothing from it (alias,
- wildcard, catch-all or path), Settings and `domains list` show it as
- "attaching", and `SetClaimCapabilities`, `SetClaimPaths` and the
- repoint path refuse it.
-- `kind` fixes the family the claim is for. An `alias` acquisition can
- never become a first-route marker or the reverse (B-1).
-- A claim with no `acq` block is a legacy committed alias, or a legacy
- first-route marker when `Route == ""`. The backfill (§8.1) stamps
- `acq` onto every legacy claim it validates.
-
-The block must be declared in `Claim` in the same change that first
-writes it (the converge rule in claims.go's own comment), and the floor
-guarantees no reader below this release folds a pending claim.
-
-It remains the right owner for aliases and the wrong owner for instance
-hosts: the fleet folds this prefix every few seconds on every box and
-`ListObjectsV2` pages at 1000 keys; the detach door deletes the record
-when `Route == ""`; and the lifecycle claim already is a per-host,
-create-only, holder-naming object for record-backed hosts, with crash
-recovery designed and reviewed.
-
-### 3.2 The lifecycle claim, `control/hosts/<host>.json`
-
-hostclaim.Claim: `{v, host, gen, holder{family, key, account, app, name,
-incarnation, created_at, location, ...}, op{kind, id, ...}, touched,
-inventory, obligations, lost, retired_incarnations, quiet_after,
-cancelled_creates}`. `hostclaim.Update` writes the whole object with
-`If-Match` on the ETag read, or `If-None-Match` when it read absence. The
-object is never deleted (C5a); free is a written state. `BeginCreate`
-refuses unless `c.Free()`. `CompleteCreate` moves `op` to `serving` and
-drops the create's op id, after which `CancelCreate` is illegal
-(create.go:93, :104). `IsOrphan` treats a record whose incarnation an
-existing claim does not name as an orphan (orphan.go:10). `Adopt` writes
-a claim only for a host that has never had one (adopt.go). A purge fence
-is a holderless `purging` op (claim.go, CC21).
-
-Spec §5.1 is the create protocol: step 0 reads the claim and keeps its
-ETag or absence, then registers the create in the app's `admission.json`
-as a lease, which succeeds only while the app's admission is `open`;
-step 1 assigns `CreatedAt`; step 2 CASes the claim to `{holder: new, op:
-creating}` conditional on exactly what step 0 read, and a 412 restarts
-from step 0 with a new registration and a new `CreatedAt`; step 3 is
-`CreateRecord`; step 4 CASes to `serving`; step 5 is the recovery pass.
-§5.2 is delete; §5.3 purge; §9 legacy adoption; §10 the version floor.
-
-**The lifecycle claim's holder is the ownership object for every
-record-backed host.** This spec adds nothing to it and changes none of
-its invariants. It specifies where the console's doors read the other
-authorities inside that protocol, and asks the lifecycle spec for three
-things (§6), two of which are rollout prerequisites.
-
-### 3.3 Readers that change
-
-The record stays the provisioning authority and the registry stays
-derived. The readers this spec changes (B-15):
-
-- **Box:** `ReconcileAliases` skips any claim with `acq.state ==
- pending` before every derivation (alias, wildcard, catch-all, paths).
- The static canary mint checks `byHost` (§4.6). Nothing else.
-- **Console:** Settings and `domains list` render pending as
- "attaching"; `SetClaimCapabilities`, `SetClaimPaths` and the repoint
- path refuse pending; `apexHolder`, `hostHolder` and
- `ReportApexSquats` read the lifecycle claim instead of scanning.
+- `family`, `acq.id`, `account`, `app`, `app_incarnation`, `first_route`
+ and `platform` are **immutable** for the life of the object. A takeover
+ (§4.1) replaces the object by CAS and so replaces `acq.id`; nothing
+ else does.
+- `acq.state` moves only `pending → committed → releasing`, each by CAS
+ on the ETag read, and `releasing` ends in `DeleteIfMatch`. A `pending`
+ object may also be deleted by its own writer's rollback or by the
+ sweep (§5). Nothing moves backwards.
+- **Held** means the object exists in any state. Only a `committed`
+ object is **owned**: materialised by readers, reported as attached,
+ eligible for repoint, capability and path changes, and for release.
+- A legacy object (no `family`) is an alias when `route != ""` and a
+ first-route marker when `route == ""`. The backfill stamps every legacy
+ object (§8); above the marker none remains.
+- Readers below this release refuse nothing (schema stays 1); the floor
+ (§8) guarantees no such reader folds a pending object.
+
+### 3.2 The per-app index: `apps/<acct>/<app>/owned/<host>.json`
+
+`{host, acq_id, family}`. Written **before** the ownership commit and
+deleted **after** the ownership delete, both conditional. An index entry
+without an owned object is harmless (release finds nothing); an owned
+object without an index entry cannot arise except by a crash between
+index write and commit, which the sweep repairs from the full fold (§5).
+Purge and materialization enumerate an app's hosts from this prefix with
+a plain listing and a GET per entry, and **fail closed**: any read error
+aborts the purge step (C-3).
+
+### 3.3 The app row's incarnation
+
+`apps` gains `incarnation TEXT` (random, set on insert) and
+`apex_acq TEXT` (the apex acquisition id, §4.6). Every door reads the
+app's incarnation at its gate and writes it into the acquisition. A
+commit re-reads the row and refuses when the incarnation differs or the
+row is not live (C-3): a writer paused across a purge and a recreate of
+the same name cannot commit against the new app.
+
+### 3.4 The lifecycle claim, `control/hosts/<host>.json` (rev 6.16)
+
+Unchanged in shape. This spec uses it as rev 6.16 specifies: §5.1 create
+(admission registration as a lease, `CreatedAt` assigned by the console,
+CAS from free or absent, `CreateRecord`, CAS to serving, recovery after
+five minutes), §5.2 delete and park, §5.3 purge with its reservation
+journal, drain and purge fence, §6.2a the holderless sweep, §9 adoption,
+§10 the floor. Its holder's incarnation is the binding to ownership:
+`acq.incarnation == holder.incarnation` for every serving host. The
+object is never deleted (C5a), so a released hostname keeps its
+`retired_incarnations`, `quiet_after` and `lost` history. What this spec
+asks of that spec is in §6.
+
+### 3.5 Readers
+
+- **Boxes.** `ReconcileAliases` derives aliases, wildcards, catch-alls
+ and paths from `committed` objects of family `alias` only. The apex
+ mint and the static canary mint run only when the `apex` object for
+ (account, app) is `committed`, and the canary mint checks `byHost`
+ before `Upsert`. Instance and sidecar routes derive from records and
+ lifecycle admission as today; boxes never read `instance` or `sidecar`
+ ownership. Fold cost and the alias index are §4.8.
+- **Console.** Settings and `domains list` show `pending` as
+ "attaching" and `releasing` as "detaching". `SetClaimCapabilities` and
+ `SetClaimPaths` refuse anything but a committed `alias`. `apexHolder`,
+ `hostHolder` and `ReportApexSquats` are replaced by one GET.
## 4. The design
-### 4.1 The rule
-
-Each hostname has exactly one **ownership authority** by family:
-
-| Family | Authority | Held means | Atomic step |
-|---|---|---|---|
-| record-backed (instance, replica, static record, sidecar) | lifecycle claim | `holder != nil` or `op.kind != ""` | `Update(BeginCreate)` from absent or free |
-| alias | `domains/<host>.json` | a claim exists (pending or committed; a pending one is held against everyone but its own acquisition) | `AttachClaim`'s create-only `Put` |
-| minted (apex, static canary host) | the app named in the host | the (account, app) row exists, live, trashed or pending | the console's unique row insert |
-
-A host can be in only one family at a time. The cross-family rule, for
-every writer:
-
-> **Write your own authority first, unverified. Then re-read every other
-> authority for the host. If any is held, delete exactly the object
-> version you wrote and answer 409. Otherwise commit: the record, the
-> committed alias, the verified app row.**
-
-Why it is enough: S3 gives strong read-after-write consistency for
-`PutObject` and `GetObject` on a key. Take two writers W1 (family X) and
-W2 (family Y) racing for host H, writing at t1 and t2. If t1 < t2, W2's
-re-read of X is after t2 > t1 and sees W1's object. Symmetrically for
-t2 < t1. So at least one backs off. Both may back off; that is two 409s
-and no residue, and either retry then wins. A deterministic tie-break was
-considered and rejected: the side that must yield cannot tell whether
-the other side also saw it, so any rule that lets one side keep going is
-unsound in some interleaving.
-
-Corollaries:
-
-- **Nothing is committed before verification.** An unverified authority
- (a `creating` lifecycle op, a pending alias claim, a pending app row)
- reserves the name against third parties but is not success, is not
- materialised, and is not what a same-owner retry may report as done.
- A retry that meets its owner's unverified authority finishes the
- verification itself or refuses; it never short-circuits (A-3, A-4).
-- **Verification covers every other authority**, not only the other
- family: an alias verifies the lifecycle claim **and** the app row of
- the host's label; a create verifies `domains/` **and** the app row;
- an app row verifies the lifecycle claim **and** `domains/` (B-5).
-- **Cleanup is conditional.** Every back-off and rollback deletes with
- `release.DeleteIfMatch` on the ETag the write returned, and refuses
- when the backend cannot enforce it (A-5).
-- **Displacing a writer is a CAS.** Taking over a stale acquisition
- rewrites the object under `If-Match`, so the displaced writer's own
- commit or cleanup fails its condition (B-1).
-- **Family does not change in place.** Conversion is detach then attach,
- each through its own door (A-4).
-
-The pre-write check stays as a cheap filter for the sequential case, but
-the design never relies on it.
-
-### 4.2 Instance create
-
-Replaces `hostHolder` and `hostClaimHolder` in `handleAPIInstanceCreate`.
-Replicas, static records and canary instances go through the same door.
-The protocol is the lifecycle spec's §5.1 **in full**, with the
-cross-family reads inserted at two points (A-9).
-
-**Placement.** Every ownership read and write sits after every other
-validation the door performs, immediately before the record write, where
-`leftoverReplicaRefusal` sits today. An invalid request answers its 400
-whether or not the host is held (A-13).
-
-**Gate.** Above the floor and before the marker (§8.2) the door answers
-503 `msgOwnershipFreeze` and does nothing. Below the floor it runs
-today's path.
-
-1. Shape and the namespace rule of §4.6, early. The rule's own-app
- exception applies: an app may create at its own apex label.
-2. **Pre-check, cheap, after all validation.** `GET domains/<host>`: any
- claim is a refusal (§7). One `registry.Get` on this box for an
- operator route with no record. `edgesfile.DeploymentRoutes` for a
- route a live box publishes that no record explains; best-effort and
- permissive on read failure, as the attach door uses it.
-3. **Lifecycle steps 0 and 1.** `hostclaim.Read`, keep the ETag or
- absence; register the create in `admission.json`, holding its lease;
- assign `CreatedAt`.
-4. **Lifecycle step 2.** `Update(BeginCreate)` conditional on step 3's
- read. On `ErrConflict`: re-read once to classify (§7), answer 409 for
- a hold or a fence, and for free-again **restart from step 3** with a
- new registration and a new `CreatedAt`, at most once per request.
- Never retry the CAS alone, never wait inside the handler.
-5. **Verify.** `GET domains/<host>` again, and `AppNameState` for the
- host's label when the host is `<l>.<sqid>.<apps>`. Any `domains/`
- object, or a live, trashed or pending app at the label **other than
- the creating app itself** (B-10): `Update(CancelCreate)` (legal: the
- op is still `creating`), release the registration, 409.
-6. **Lifecycle step 3.** `CreateRecord`, create-only.
-7. **Lifecycle step 4.** `Update(CompleteCreate)`. From here the host is
- serving and the only way back is delete and park (§4.3). Nothing after
- this step may call `CancelCreate` (A-6).
-
-The recovery pass completes any `creating` claim whose record is present
-without repeating door predicates: a record exists only after step 5
-verified, and the one write that could still race (an app row or alias
-appearing after step 5) is caught on the other side, because those
-writers write and then read this claim (§4.4, §4.6).
-
-### 4.3 Instance delete
-
-Lifecycle §5.2 unchanged: CAS `deleting`, `DeleteIfMatch` the record,
-park, free. Ownership is released when the claim reaches free. A host
-that is `deleting` or `parking` is held for the create door.
-
-### 4.4 Alias attach and detach
-
-In `handleAPIDomainAttach`'s alias branch and `attachDomainAs`'s alias
-branch, around the existing `AttachClaim`. Policy and
-`deploymentRouteTaken` stay first and in their existing order; the Task 5
-ordering rules stand, and the new reads sit below them, immediately
-around the write. Every refusal below is §7's mapping; the sentences
-named here are its rows.
-
-1. `GET domains/<host>` with its ETag `E0`.
- - **Committed, own, same route:** done, 200. The titogo reload steady
- state: one GET.
- - **Committed, own, `kind alias`, different route:** repoint by CAS on
- `E0`, preserving `acq`. Ownership was decided at commit.
- - **Committed, own, `kind first-route`:** refuse (§7: own-app
- conflict). No in-place conversion.
- - **Pending, `kind alias`, own acquisition id:** a retry of this
- request. Resume at step 4 with `E0`.
- - **Pending, `kind alias`, own owner, another id, `started_at` older
- than the acquisition timeout (60 s):** **takeover.** CAS the whole
- claim on `E0` with this request's id and `started_at`, keep `E1` =
- the result, then step 4. The displaced writer's commit or cleanup
- now fails its `If-Match` (B-1).
- - **Pending, own owner, another id, younger than the timeout:** refuse
- (§7: attach in progress).
- - **Pending, `kind first-route`, own owner:** refuse (§7: own-app
- conflict, "a first-route attach of this host is in progress"). An
- alias request never touches a first-route acquisition (B-1).
- - **Any claim of another owner:** refuse (§7: foreign).
- - **Absent:** continue.
-2. **Pre-check, cheap.** `hostclaim.Read`: a holder or any op is a
- refusal. `AppNameState` of the label when the host is
- `<l>.<sqid>.<apps>`: any row other than the attaching app's own is a
- refusal.
-3. **Write pending.** `Put(domains/<host>, claim{acq: {id, alias,
- pending, started_at}}, "")`, create-only. Keep `E1`. A CAS conflict
- re-reads and re-enters step 1 once.
-4. **Verify.** `hostclaim.Read` again, and `AppNameState` of the label
- again (B-5), and the attaching app's own row is still live. Any hold,
- any other app's row (live, trashed or pending), or own app not live:
- `DeleteIfMatch(domains/<host>, E1)`, refuse. A failed conditional
- delete leaves a pending claim, which never materialises and which the
- sweep clears (§5).
-5. **Commit.** `Put(domains/<host>, claim{acq.state: committed}, E1)`.
- Keep `E2`. A conflict means a takeover or cleanup moved the object:
- re-read and re-enter step 1 once.
-6. Audit and serial, only on a commit.
-
-Detach of an alias: `GET` the claim with ETag, owner-matched, then
-`DeleteIfMatch` on that ETag, quiet on mismatch and on a lost condition
-(A-5). A pending claim of the caller's own may be detached the same way.
-
-### 4.5 First-route attach and detach
-
-The first-route branch writes a claim with `Route == ""` and `kind:
-first-route`, then a static record. The record makes the host
-record-backed; the lifecycle holder is its authority and the `domains/`
-claim is the attach surface's marker. The marker is committed only when
-the record is serving, and the commit stamps `acq.incarnation` with the
-record's `hostclaim.Incarnation(account, app, host, CreatedAt)` (B-6).
-
-**Attach**, after policy, `attachChannel` and `deploymentRouteTaken`:
-
-1. `GET domains/<host>` with `E0`. The branch table (B-11):
- - **Absent:** write pending `{id, first-route, pending}` create-only,
- keep `E1`. Continue at step 2.
- - **Pending first-route, own id:** retry; `E1 = E0`, continue at step
- 2.
- - **Pending first-route, own owner, another id, older than the
- timeout:** takeover by CAS as in §4.4, then step 2.
- - **Committed first-route, own:** read the lifecycle claim. Holder is
- this app's record with `attachWrote` shape and
- `holder.incarnation == acq.incarnation`: idempotent success, no
- write. Anything else: refuse (§7: own-app conflict, operator
- repair).
- - **Any alias claim, own or not, or a pending first-route younger than
- the timeout by another id:** refuse per §7.
-2. `hostclaim.Read`. By state:
- - **absent or free:** run §4.2 steps 3 to 7 for the static record. Its
- verify (step 5) finds this call's own pending marker, recognised by
- id, which is not a conflict; any other `domains/` state or app row
- is. If the CAS loses or the verify fails: `DeleteIfMatch(domains/,
- E1)`, refuse.
- - **`creating` with holder this app's record (a crashed earlier
- attempt):** do not write; answer 409 "attach in progress, retry in
- a minute". The recovery pass settles the record half within five
- minutes; the next retry meets `serving` or `free`.
- - **`serving` with holder this app's static record of `attachWrote`
- shape:** a lost commit. Go to step 3.
- - **held by another, or any other op:** `DeleteIfMatch(domains/, E1)`,
- refuse per §7.
-3. **Commit** the marker: `Put(..., claim{committed, incarnation:
- holder.incarnation}, E1)`.
-4. Audit and serial.
-
-`attachWrote` and its shape restriction stay (A-4): a record is this
-door's only when it is kind static, no slug, no region, no pool, **and**
-the lifecycle holder is this app's record **and** the committed marker's
-incarnation matches. An instance record of the same app can sit at the
-same key under a custom manifest domain, and detach must never gain
-authority over it.
-
-**Detach** of a first-route marker (B-6):
-
-1. `GET domains/<host>` with `E0`; owner-matched, `kind first-route`,
- committed, with incarnation `I`. Anything else: the alias detach path
- or §7's refusal.
-2. `hostclaim.Read` with its ETag. `holder.incarnation != I`, or any op
- other than serving: refuse (§7: operator repair). The marker names a
- record that is not the one serving.
-3. Lifecycle delete (§4.3), with `BeginDelete` conditional on the claim
- ETag read in step 2, so a successor holder fails the CAS.
-4. `DeleteIfMatch(domains/<host>, E0)`.
-
-A detach that paused between steps 1 and 3 while another detach and a
-new attach ran meets a different incarnation in step 2 or a failed CAS
-in step 3, and retires nothing.
-
-### 4.6 Minted hosts and the namespace rule
-
-The apex `<app>.<sqid>.<apps>` and a static app's canary host
-`<name>.<app>.<sqid>.<apps>` are minted by boxes from the app's name and
-channel pointers. They have no record and no claim, and should not get
-one: the name is the ownership.
-
-**The namespace rule**, decided from the name and the account's app
-rows alone, so it is never an oracle (A-7):
-
-| Host shape under `<sqid>.<apps>` | Belongs to | Allowed |
-|---|---|---|
-| `<l>.<sqid>.<apps>` where `<l>` is a live, trashed or pending app of the account | app `<l>` | an instance of `<l>` itself (the own-app exception, B-10), nothing else: no alias, no first-route, no instance of another app |
-| `<l>.<sqid>.<apps>` where `<l>` is no app | nobody yet | an instance of any app of the account (today's canary and slug hosts); an alias only on deployments with reservation off, as today, and only with the app-row verification of §4.4 |
-| `<x>.<app>.<sqid>.<apps>` | app `<app>` | nothing through any door. Minted only |
-| a manifest `Domain` that is `<app>.<sqid>.<apps>` or deeper | app `<app>` | refused for any other app at the manifest door and at the create door |
-
-Applied by: the instance create door (§4.2 step 1 and step 5), both
-attach branches (before any write, and again at verify), and the
-manifest door.
-
-**The apex gate** follows the write-then-verify rule with an operation
-id (A-10, B-9):
-
-1. `claimApp`, rename and move each start by inserting a row in
- `pending_app_ops` `{op id, kind, app id, from, to, journal}` and, in
- the same transaction, the name reservation: for create, the app row
- with `pending = op id`; for rename and move, a **second** row for `to`
- with `pending = op id` while the `from` row stays live. Both names are
- reserved until the operation finishes (B-9). The unique (account, app)
- constraint is the atomic step.
-2. Read the lifecycle claim of the new apex and `GET domains/<apex>`.
- Either held (a holder, an op, or any claim, pending or committed):
- compensate, in one transaction conditional on `pending_app_ops.op id`
- still present: delete the pending row, delete the op row, refuse.
-3. Finalise, in one transaction conditional on the op id: for create,
- clear `pending`; for rename and move, apply the related-row changes
- the journal lists (credentials, grants, store.go:1801-1905), drop the
- `from` row (rename) or re-home it (move), clear `pending` on `to`,
- delete the op row. A handler that paused and finds its op row gone
- does nothing: recovery already settled it.
-
-The instance and alias doors do the other half (§4.2 step 5, §4.4 step
-4). Rows and ops left pending by a crash are settled by the app-name
-sweep (§5) with the same conditional transactions. A pending row is not
-an app a member can deploy to: the doors treat it as "being created".
+### 4.1 The acquisition protocol
+
+Every door that puts a hostname on the platform runs the same four
+steps; the family-specific work sits in step 3.
+
+1. **Gate and validation.** Every check the door makes today, in its
+ order, including the app gate (which yields `app_incarnation`), shape,
+ policy (`domainHostPolicy`), the namespace rule (§4.6), and for
+ aliases `deploymentRouteTaken`. No ownership read happens before this
+ step completes (A-13, B-14).
+2. **Acquire.** `GET domains/<host>` with its ETag `E0`, classify (§7),
+ then one of:
+ - **absent:** `Put(..., {family, account, app, app_incarnation, acq:
+ {id, pending, started_at, by}}, "")`. Keep `E1`. A conflict re-reads
+ once and re-classifies.
+ - **own `pending`, own id:** resume with `E1 = E0`.
+ - **own `pending`, same family and owner, other id, `started_at`
+ older than the acquisition timeout (60 s), and not referenced by a
+ lifecycle `creating` op (§4.2):** **takeover.** CAS the whole object
+ on `E0` with this request's id and `started_at`; keep `E1`. The
+ displaced writer's later commit, rollback and index write fail their
+ conditions.
+ - **own `committed`, same family:** the idempotent branch of the
+ family (repoint, no-op, or "already exists").
+ - **anything else:** refuse per §7. No write.
+3. **Write the index entry** `owned/<host>.json {host, acq_id, family}`,
+ then do the **family's work** (§4.2, §4.3, §4.6). The family's work
+ ends with the object's `acq.incarnation` known where the family has
+ one.
+4. **Commit.** Re-read the app row: live, same `app_incarnation`,
+ otherwise roll back. `Put(..., {state: committed, incarnation}, E1)`.
+ A conflict means a takeover or a sweep moved the object: roll back the
+ family's work, refuse "attach in progress, retry".
+
+**Rollback** of a failed acquisition undoes the family's work in reverse,
+then `DeleteIfMatch(owned/<host>, its ETag)`, then
+`DeleteIfMatch(domains/<host>, E1)`. Every delete is conditional
+(`release.DeleteIfMatch`, conditional_delete.go:28) and refuses where the
+backend cannot enforce it. A rollback that cannot finish leaves a
+`pending` object for the sweep (§5).
+
+**Why it is atomic.** There is one key per hostname and one create-only
+write. Two writers for one host, any families, cannot both get `E1`.
+There is no cross-authority verification and no interleaving proof to
+maintain. The remaining races are same-owner retries and reclaimed
+reservations, which the immutable id, the CAS takeover and the lifecycle
+binding (§4.2) handle.
+
+### 4.2 Family `instance` and `sidecar` (record-backed hosts)
+
+The family's work is the lifecycle create protocol, rev 6.16 §5.1, in
+full, with the ownership acquisition bound into it:
+
+1. **Lifecycle step 0.** `hostclaim.Read`, keep the ETag or absence;
+ register the create in `admission.json` as a lease.
+2. **Lifecycle step 1.** Assign `CreatedAt`. The incarnation
+ `I = hostclaim.Incarnation(account, app, host, CreatedAt)` is now
+ known.
+3. **Lifecycle step 2.** `Update(BeginCreate)` conditional on step 1's
+ read, with `op.intent.ownership = acq.id` (§6 ask 1). On
+ `ErrConflict`: re-read once; a hold or a purge fence refuses per §7;
+ free again restarts from step 1 with a new registration and
+ `CreatedAt`, at most once per request. Never wait in the handler.
+4. **Lifecycle step 3.** `CreateRecord`, create-only.
+5. **Commit ownership** (§4.1 step 4) with `acq.incarnation = I`. If the
+ CAS fails, the reservation was reclaimed (a takeover or the sweep):
+ `Update(CancelCreate)` (legal, the op is still `creating`),
+ `DeleteIfMatch` the record, release the registration, refuse. Nothing
+ reaches serving under a reservation it does not hold.
+6. **Lifecycle step 4.** `Update(CompleteCreate)`. From here the host is
+ serving and the only way back is release (§4.4).
+
+**Binding invariant (O1).** A `serving` lifecycle holder exists only
+while `domains/<host>` is `committed` or `releasing` with
+`acq.incarnation == holder.incarnation`. Maintained by: commit before
+`CompleteCreate` (step 5 before 6); the recovery pass's completion rule
+(§6 ask 1): a `creating` claim with its record present is completed only
+if the ownership object's `acq.id` equals `op.intent.ownership` and its
+state is `pending` or `committed`; otherwise the pass cancels and deletes
+the record. And by release (§4.4), which deletes ownership only once the
+lifecycle claim is free.
+
+**A delayed creator** that acquired `pending`, paused, lost its
+reservation (sweep after 60 s, or takeover) and resumes: its
+`BeginCreate` may still succeed if the lifecycle claim is free, and its
+record may land, but its commit CAS fails (step 5) and it cancels. If it
+pauses again between steps 4 and 5 for over five minutes, the recovery
+pass finds `op.intent.ownership` naming an acquisition the ownership
+object no longer has and cancels it. A successor meanwhile meets a
+`creating` claim and is refused "in progress"; it retries. The sweep
+never deletes a `pending` object that a `creating` lifecycle op names
+(§5), so a live creator's reservation is not reclaimed under it.
+
+Sidecars: the same steps with `CreateSidecarRecord` and family
+`sidecar`; `holderKeyMatches` already binds the sidecar key and host.
+
+### 4.3 Family `alias`, and the first-route attach
+
+**Alias.** Steps 1 and 2 of §4.1; `deploymentRouteTaken` stays
+immediately before the acquire write, as the Task 5 ordering rule
+requires. The family's work is nothing: the index entry, then commit
+(§4.1 step 4), whose app-row re-read is what refuses a writer paused
+across a purge (C-3). Two PUTs on a fresh attach; the committed object
+carries `route`, and `wildcard`, `catchall`, `paths` are written later by
+their own owner-gated CAS, which requires `committed`.
+
+Idempotent branch (own committed `alias`): same route, no write, 200;
+different route, CAS the repoint on `E0` preserving every immutable
+field. This is titogo's reload steady state: one GET per host.
+
+**First-route.** The host becomes record-backed, so the family is
+`instance` with `first_route: true`, and the work is §4.2 for a static
+record (`attachWrote` shape: kind static, no slug, region or pool). The
+flag is what the domains surface dispatches on at detach (§4.5). An own
+committed `instance` without the flag at that host is the instance
+door's, and the attach refuses (§7 row 5). Conversion between alias and
+first-route is detach then attach.
+
+**Platform domains** (`--platform`) take either branch with
+`platform: true` recorded; the operator gate stays name-derived
+(`domainHostPolicy`) and is applied before step 2.
+
+### 4.4 Release (instance and sidecar delete)
+
+Release follows lifecycle retirement and is recoverable. The
+instance delete door, the sidecar delete door, first-route detach
+(§4.5), purge (§5) and `provision.RetireRecord`'s callers all run it.
+
+1. `GET domains/<host>` with `E0`: `committed`, this app, this
+ `app_incarnation`, family `instance` or `sidecar`, incarnation `I`.
+ Anything else per §7.
+2. `hostclaim.Read` with its ETag: `holder.incarnation == I`, op
+ serving. Otherwise refuse (§7 row 7): the record serving is not the
+ one this ownership names.
+3. **CAS ownership `committed → releasing`** on `E0`, writing
+ `release_op = <new lifecycle op id>`. From here the host is "being
+ released" to every door and nothing can repoint or re-attach it.
+4. **Lifecycle §5.2**: `BeginDelete(release_op, ...)` conditional on the
+ ETag read in step 2, `DeleteIfMatch` the record, `parking` with
+ obligations, each owner's chore, `free`.
+5. When the lifecycle claim is **free**, or names a different holder
+ with no op naming `release_op`: `DeleteIfMatch(domains/<host>, E)`,
+ then `DeleteIfMatch(owned/<host>)`. The lifecycle claim itself is
+ never deleted (C5a).
+
+Recovery is the **ownership sweep** (§5): a `releasing` object whose
+lifecycle claim is still `serving` with holder `I` restarts step 4 with
+`release_op` (the member already asked, durably); one whose lifecycle
+claim is `deleting` or `parking` under `release_op` waits for lifecycle
+recovery; one whose lifecycle claim is free does step 5. A new creator
+for the host meets a held object (`releasing`) and is refused "being
+released" until step 5 completes; a lifecycle claim in `parking` would
+refuse it anyway.
+
+### 4.5 Detach dispatches by family
+
+`handleAPIDomainDetach` and `detachDomain`, after the control-plane gate
+and the name-derived platform gate:
+
+| `GET domains/<host>` says | Action |
+|---|---|
+| absent, another owner, another `app_incarnation`, or `pending` of another request | 404 "not attached" (today's neutrality) |
+| own `alias`, committed | CAS `releasing` on `E0`, then `DeleteIfMatch`, then the index entry. Audit, serial |
+| own `alias`, pending, own request | roll back as §4.1 |
+| own `instance`, `first_route: true`, committed | release (§4.4) |
+| own `instance` without `first_route`, or `sidecar`, or `apex` | 409 "this hostname is an instance (or sidecar, or the app's address) of this app; use its own door" |
+| own, `releasing` | 200 "already detaching" |
+
+`Route == ""` is read by nothing (C-5's root). The `releasing` state is
+what makes a crashed first-route detach finishable: a retry finds
+`releasing` and the sweep drives §4.4 to its end whether or not the
+lifecycle holder still matches, because the intent was recorded before
+the lifecycle delete began.
+
+### 4.6 Family `apex`: app names, and the namespace rule
+
+The apex `<app>.<sqid>.<apps>` is owned by the app, and the ownership
+object is acquired by the console when the name is taken. A canary host
+`<name>.<app>.<sqid>.<apps>` is **covered** by the apex object: boxes
+mint it only when the apex object is committed for (account, app), and
+every door refuses hosts under another app's apex. No per-canary object.
+
+**App create** (`claimApp`):
+
+1. Gate and validation as today.
+2. Acquire `domains/<app>.<sqid>.<apps>` as family `apex`, `pending`
+ (§4.1 step 2). A hold by another app or an alias at the apex refuses
+ the name (today's `apexHolder` answer, from one GET).
+3. Family work: insert the app row in one transaction with
+ `incarnation` and `apex_acq = acq.id`. A unique-constraint failure
+ (the name was taken between gate and insert) rolls back the
+ acquisition.
+4. Commit (§4.1 step 4), `app_incarnation` now known.
+
+**Rename and move** reserve both names until finished (B-9, C-4):
+
+1. Acquire the **new** apex `pending`, as above.
+2. In one transaction: insert `pending_app_ops {op_id, kind, app_id,
+ from, to, journal}` and mark the source row `closed = op_id`. A closed
+ row keeps its name and its apex object but **every door refuses new
+ work for it** (create, attach, ship, add), and in-flight work is
+ drained: the door re-reads `closed` at its commit step and rolls back
+ if set. Then re-validate the source is empty (`appDeployEmpty`,
+ appsettings.go:832's check, now after the close).
+3. Finalise in one transaction conditional on `op_id`: rename or re-home
+ the row, apply the journaled related rows (credentials, grants),
+ clear `closed`, delete the op row. Then CAS the **old** apex object
+ `releasing` and delete it; commit the new one with the row's
+ incarnation.
+4. Compensation, conditional on `op_id`: clear `closed`, delete the op
+ row, roll back the new apex acquisition.
+
+The app-name sweep settles pending ops older than a minute by the same
+conditional transactions. `console migrate-app` (cmd/console/main.go:228)
+runs the move path; it does not call `MoveApp` directly (C-1).
+
+**Trash and purge.** A trashed app keeps its apex object until purge
+releases it (§5), so its name stays reserved as today.
+
+**The namespace rule**, from the name and ownership alone:
+
+| Host under `<sqid>.<apps>` | Allowed |
+|---|---|
+| `<l>.<sqid>.<apps>` | whatever the ownership object says: an `apex` of app `<l>` admits only `<l>`'s own instance record at that host (the own-app exception, B-10); absent admits any family |
+| `<x>.<app>.<sqid>.<apps>` | nothing through any door; minted only, under a committed apex of `<app>` |
+| a manifest `Domain` of `<app>.<sqid>.<apps>` or deeper | refused for any app but `<app>` |
**Apps with no SQL row.** The box mints an apex for any (account, app)
-that has routes (`candidateApps`), including an app that exists only in
-the bucket. Such a name is reserved by nothing (B-10). The backfill
-(§8.1) lists every app prefix with records or routes and no row, and
-**blocks completion** until each is registered (`carlos ops apps
-register <sqid> <app>`, which inserts the row through the apex gate) or
-excluded from minting by an operator deleting its records. The startup
-owners backfill at store.go:676 is legacy-only and is not that path.
-
-**The static canary mint** must not overwrite a route. The box-side
-change: before `reg.Upsert(nr)` at domains/reconcile.go:337, skip and
-log when `byHost` has the host and the existing route's app or account
-differ from the mint's.
-
-### 4.7 Replicas, sidecars, fleets, pools, moves
-
-- A replica is a record with its own host, created by the same door:
- §4.2 applies unchanged. Its lifecycle is the moves spec's.
-- Sidecars: the lifecycle claim covers them (family `sidecar`). For
- ownership nothing is needed: the host embeds account and app, the
- attach door refuses the suffix, and the edge never routes it.
-- Fleets, pools and placement change `Region` on an existing record and
- never the host or the holder.
-- Moves change `op` and `holder.location`, never `holder.account` or
- `holder.app`. A host under a move is held; the create door refuses.
-
-### 4.8 Account migrate
-
-`carlos accounts migrate --app A --to <dst>` copies the app's objects to
-the destination prefix create-only and deletes nothing. Records are
-copied with their body unchanged, so the destination copy names the
-source account and `GetRecord` refuses it (provision.go:1145). The verb
-has never produced a valid destination record (A-12).
-
-Under this design a host's lifecycle holder names the source record's
-exact key, the incarnation hashes the source account, `holder.location`
-is incarnation-scoped, and a sidecar's host embeds the source sqid. A
-transfer needs a protocol for record identity, storage location,
-execution authority, sidecars and `domains/` claims, and it is not
-designed here.
-
-**Rule.** `accounts migrate`:
-
-1. Closes the source app's admission (`admission.json` to `closed`, the
- lifecycle spec's own quiesce: create step 0 then refuses), and waits
- out every outstanding registration lease (B-13).
-2. Lists the app's `instances/` and `sidecars/` prefixes. Any record:
- reopen admission, refuse with a sentence naming this spec.
-3. Otherwise migrates release objects and pointers as today, **without**
- stamping routes (an app with no records has no provisioned routes to
- stamp; an operator route is the operator's), then reopens admission.
-
-The follow-up thread owns the transfer and the `domains/` rebind.
-
-## 5. Crash recovery
-
-Each crash window, what it leaves, and who clears it. No step deletes
-anything it did not create or cannot name by exact version, and nothing
-frees a host that has a record with the holder's incarnation.
-
-| Crash after | Left behind | Cleared by |
+with routes, including bucket-only apps. Above the marker a mint needs a
+committed `apex` object, which only the console writes, so bucket-only
+apps stop minting until registered. `carlos ops apps register <sqid>
+<app>` is a **migration-only registration**, admitted during the freeze
+(C-6): it acquires the apex object requiring exact same-owner evidence
+(records or routes for exactly that account and app, no other authority
+at the host) and inserts the row through the apex gate. The startup
+owners backfill (store.go:676) is retired above the floor (C-1).
+
+### 4.7 Moves, placement, migrate
+
+Moves and placement change `op`, `holder.location` and `Region`, never
+the owner. A host under a move is `committed` and its lifecycle claim is
+not serving; release refuses until the move ends.
+
+`accounts migrate`: closes the source app's admission and drains its
+registrations per rev 6.16 §5.3 step 1b (not a lease wait, B-13), lists
+`instances/` and `sidecars/`, refuses with a sentence naming this spec if
+any record exists, otherwise migrates release objects and pointers only,
+without stamping routes, and reopens admission. Ownership transfer across
+accounts (record identity, location, sidecar hosts, aliases) is a
+follow-up.
+
+### 4.8 Fleet folds
+
+Today every box lists `domains/` through `ClaimCache` on the fast-converge
+leg (one `ListMod` per pass; `ListObjectsV2` pages at 1000 keys). After
+rev 4 the prefix holds every hostname: on flagship, aliases plus
+instances, sidecars and apexes, several thousand objects and several
+pages per pass per box.
+
+- **Family filter.** Boxes derive only from `alias` and `apex` objects;
+ `instance` and `sidecar` objects are skipped on the key list's body
+ read, but still listed.
+- **Derived alias index.** The console's ownership sweep writes
+ `control/alias-index.json` `{generation, serial, aliases: [committed
+ alias objects' routing fields], apexes: [(account, app)]}` on every
+ alias or apex commit and release (the same moments that bump the
+ serial). Boxes fold the index with a conditional GET (`If-None-Match`
+ on its ETag, 304 in the steady state) instead of listing `domains/`.
+ The full fold remains the sweep's consistency check and the fallback
+ when the index is missing or older than ten minutes.
+- **Switch.** The index is written from day one; boxes prefer it once
+ `domains/` exceeds one page, which the sweep records in the index.
+
+Readers of the index are the alias derivation and the two mints. Nothing
+box-side writes `domains/`.
+
+## 5. Crash recovery and the sweeps
+
+Two console passes, each every minute, each reading exact objects with
+ETags before acting. `ClaimCache` and listings are discovery only.
+
+**The ownership sweep**, over `domains/`:
+
+| State found | Lifecycle says | Action |
|---|---|---|
-| §4.2 step 4 (claim `creating`, no record) | an orphaned claim | the lifecycle recovery pass (§5.1 step 5): record absent, `CancelCreate`, incarnation retired. Never steals: a record present with this incarnation completes instead |
-| §4.2 step 6, before `CompleteCreate` | claim `creating`, record present | the same pass completes it. Safe without re-running door predicates: the record exists only after step 5 verified |
-| §4.2 step 5 back-off, before `CancelCreate` | claim `creating`, another authority present | the pass cancels (record absent). The other writer was earlier and stands |
-| §4.4 step 3 or 4 (pending alias, not committed) | a pending claim | never materialised. The **claim sweep** (console, every few minutes): `ClaimCache` for discovery only; for each pending claim older than the timeout, `GET` the exact object with its ETag, read the lifecycle claim and the label's app row, then **delete only**: `DeleteIfMatch` when the host has any other authority, **or** when none and the claim is still pending at that ETag. The sweep never commits on an owner's behalf (B-12, answers rev 2's Q4): a pending claim is a request that did not finish, and its owner's retry is the only thing that may finish it. Deleting it removes nothing anyone was told succeeded |
-| §4.4 step 5 (verified, commit PUT lost) | a pending claim | the owner's retry (own id, step 4 then 5) within the timeout; after it, the sweep deletes it and the owner attaches again |
-| §4.5 pending marker, lifecycle absent or free, no record | a pending first-route claim | the sweep deletes it as above |
-| §4.5 pending marker, lifecycle `creating` | | the recovery pass settles the record half; the sweep leaves a pending first-route marker whose host is `creating` or `serving` **for this app's static record** alone and logs "pending first-route, re-attach to finish"; the owner's retry commits it (§4.5 step 2, lost commit) |
-| §4.5 pending marker, lifecycle held by another | | the sweep deletes the marker |
-| §4.6 step 1 (pending row or op, unverified) | names reserved by an op nobody finished | the **app-name sweep** (console, startup and every minute): for every `pending_app_ops` row older than a minute, re-run §4.6 step 2 and then step 3 or its compensation, each conditional on the op id |
-| delete (§4.3, §4.5 detach) at any step | the lifecycle spec's recovery owners | unchanged |
-| console purge door | records gone; claims | the door detaches the app's `domains/` claims with `DeleteIfMatch` **before** `PurgeApp`, discovered through `ClaimCache`, each read exactly first. An alias acquisition in flight for the app fails its §4.4 step 4 own-app-live check, because the app is trashed before purge |
-| box auto-purge | records and marker gone; the trashed SQL row; the app's claims | **console materialization** (materialize.go:391) detaches the app's claims, as the purge door does, **before** it removes the trashed row. The name is released only once no claim names the app (B-8, A-14). Until then the row stays trashed and the name reserved |
-
-Old-console deletes cannot happen above the floor (§8.3). A `serving`
-claim whose record is gone can arise from a crashed new-console delete,
-and the lifecycle spec's recovery owners cover it.
-
-**Quarantine guard (B-3).** The recovery pass never deletes an orphan
-whose host is listed in `control/hosts-quarantine.json` (§8.1). The
-guard is a rollout prerequisite, not a follow-up.
-
-## 6. Composition with the lifecycle claim
-
-Ownership and lifecycle are **one object for record-backed hosts** and
-**two for aliases**, and the two never both own a host.
-
-- This spec uses `internal/hostclaim` as specified: `Read`, `Update`,
- `BeginCreate`, `CancelCreate` (only while `creating`), `CompleteCreate`,
- `BeginDelete` conditional on the ETag read, `Free`, and the recovery
- pass. "Who holds H" is `Read(H).Claim.Holder` and the op, nothing else.
- The create door runs the lifecycle protocol in full, admission lease
- included, and migrate uses admission to quiesce.
-- It asks the lifecycle spec for **three changes**:
- 1. **(prerequisite)** `Adopt` requires deployment-wide evidence: the
- adopting actor has listed every `apps/*/*/instances/<host>.json` and
- `apps/*/*/sidecars/*.json` for the host and found exactly one
- record, and that record's host is consistent with `domains/` and
- the namespace rule (§8.1 step 3). In this design the **backfill verb
- is the only adopting actor**; reconcile, the doors, purge and the
- recovery pass, which §9 names as adopting actors, refuse a host with
- no claim and log it (A-1, B-3). Above the marker no such host
- exists.
- 2. **(prerequisite)** The recovery pass honours the quarantine guard
- (§5) before `DeleteIfMatch` on an orphan.
- 3. Account migrate is refused for hosts with records until a transfer
- transition exists (§4.8). A ruling, not a mechanism.
-- It adds console-side protocol around the lifecycle CAS: the reads of
- `domains/` and app rows, the alias doors' reads of `control/hosts/`,
- and the two sweeps. The lifecycle claim does not know aliases exist.
-- The lifecycle floor gates this design. Below the floor the doors keep
- today's checks; the `acq` block is written only above it.
-
-The alternative, an `alias` holder family in the lifecycle claim so that
-one object owns every routable name, was considered (§12).
+| `pending`, older than 60 s | absent or free, or a `creating` op whose `intent.ownership` is a different id | `DeleteIfMatch` the object and its index entry: a request that did not finish. Nothing anyone was told succeeded is removed |
+| `pending` | `creating` naming this `acq.id` | leave it: the lifecycle recovery pass owns it (complete or cancel) |
+| `pending`, family `apex`, older than 60 s | | row exists with `apex_acq == acq.id` and live: commit on its behalf (the row is the family's atomic step, and the id ties them). Else delete |
+| `committed`, family `instance` or `sidecar` | no holder, or holder incarnation differs, and no `releasing` | report "committed ownership without its holder" for the operator; never delete (a record may be mid-create or the lifecycle claim mid-recovery) |
+| `releasing` | serving with holder `I` | restart §4.4 step 4 with `release_op` |
+| `releasing` | `deleting`, `parking` under `release_op` | wait |
+| `releasing` | free, or another holder and no op naming `release_op` | §4.4 step 5 |
+| `committed` whose app row is gone or has another incarnation | | report only; SQL absence is not proof (direct-bucket apps) |
+| index entry without an object | | delete the entry |
+| object without an index entry | | write the entry |
+
+**The lifecycle recovery pass** (rev 6.16 §5.1 step 5) gains the
+ownership check of §4.2 (ask 1) and two gates: no destructive orphan
+deletion before `control/hosts-backfill.json` exists, and none for a
+quarantined host (C-2, B-3).
+
+**Purge** (console door, rev 6.16 §5.3 phases): after `retired`, a new
+phase `released`: list `owned/` (fail closed), for each entry `GET
+domains/<host>`; owned by this app and incarnation: §4.4 steps 3 to 5
+(instance, sidecar), or `releasing` then delete (alias, apex). Then
+`done`. `PurgeApp` deletes everything under the app prefix **except
+`owned/`**, which the `released` phase empties last. Box auto-purge, where
+it still runs, therefore leaves the index behind, and **console
+materialization** (materialize.go:391) runs the `released` phase for a
+trashed row whose prefix is otherwise gone **before** `PurgeAppRow`.
+The name is released only when `owned/` is empty (C-3, B-8). A late
+alias writer for the purged app fails its commit on `app_incarnation`
+(§3.3).
+
+**Writer crash windows:**
+
+| Crash after | Left | Cleared by |
+|---|---|---|
+| §4.1 step 2 (pending, nothing else) | pending object | sweep, after 60 s |
+| index write, before lifecycle `BeginCreate` | pending object and entry | sweep |
+| `BeginCreate`, before the record | `creating` claim naming the acquisition; pending object | lifecycle recovery cancels after five minutes (record absent); the sweep then deletes the pending object |
+| the record, before the ownership commit | `creating`, record, pending | lifecycle recovery: ownership names this acquisition and is pending, so it **commits the ownership object first, then completes** (ask 1); or, if the object no longer names it, cancels and deletes the record |
+| the ownership commit, before `CompleteCreate` | `creating`, record, committed | lifecycle recovery completes |
+| §4.4 step 3, before `BeginDelete` | `releasing`, serving | sweep restarts the lifecycle delete |
+| any lifecycle delete step | | lifecycle recovery owners |
+| lifecycle free, before §4.4 step 5 | `releasing`, free | sweep deletes |
+| rename or move, any step | closed source row, pending op, pending new apex | app-name sweep, conditional on `op_id` |
+| purge, any phase | the purge journal's phase | the purge recovery pass, as rev 6.16 §5.3 |
+
+## 6. What this asks of the lifecycle claim spec (rev 6.16)
+
+Exactly these, for platform-1e:
+
+1. **§5.1 create, binding.** `BeginCreate` records `op.intent.ownership
+ = <acquisition id>` (a map entry; the library's `BeginCreate` takes it
+ as a parameter or the door adds it to the intent). The step 4 CAS to
+ serving, by the door and by the recovery pass, requires
+ `domains/<host>` to exist with `acq.id == op.intent.ownership` in
+ state `pending` or `committed`; the recovery pass commits a `pending`
+ object first (CAS on its ETag), then completes. Otherwise it cancels
+ the create and deletes the record with `DeleteIfMatch`.
+2. **§5.1 step 5, gates.** Orphan deletion is disabled until
+ `control/hosts-backfill.json` exists, and never acts on a host listed
+ in `control/hosts-quarantine.json`.
+3. **§9 adoption.** Before the marker, only the backfill verb adopts
+ (§8). Reconcile, the doors, purge and the recovery pass refuse a host
+ with no claim and log it. Above the marker no such host exists, so the
+ routine is unchanged for them.
+4. **§5.3 purge.** A phase `released` between `retired` and `done`, as
+ §5 describes, and `PurgeApp` keeps `owned/`.
+5. **Nothing else.** No new field, no change to C1 to C7, to admission
+ (C4 may optionally also check O1), to parking, to the holderless
+ sweep, or to the floor. Release never deletes the lifecycle claim.
## 7. Privacy
-**Placement.** Every ownership read sits after every other validation
-and immediately before the write it protects, on every door (A-13).
+**Placement.** Every ownership read is §4.1 step 2, after every other
+validation; a request with an invalid later field answers its 400 before
+any ownership read (C's placement note, apiinstances.go:736 moves below
+:1092).
-**The response mapping**, evaluated in this order, first match wins, for
-every door (B-14). "Foreign" means the authority names an account other
-than the caller's, or names no account (a holderless purge fence, a
-malformed object).
+**Response mapping**, in order, first match wins, for every door.
+"Foreign" means another account, no account, or an object that does not
+decode.
-| # | Observation | Instance and app-name doors | Alias doors |
+| # | Observation | Instance, sidecar and app-name doors | Alias and detach doors |
|---|---|---|---|
-| 1 | a read failed (bucket, registry, SQL) | 502 "could not check the host, try again". Never an occupancy answer | 502, the same |
-| 2 | a claim or lifecycle object does not decode, or `Validate` fails | 500 "could not check the host"; logged for the operator | the same |
-| 3 | held by another **account**, any family, any op; or a holderless op; or a claim naming no account | 409 `hostUnavailableMsg`, byte for byte | 409 `hostUnavailableMsg` |
-| 4 | the caller's **own app**, same family, already there (its own record, its own committed alias) | 409 "instance already exists" (today's sentence) | 200 no-op, or the repoint |
-| 5 | the caller's own app, other family or an unfinished acquisition | 409 naming the conflict and the step that frees it ("detach the alias first", "a first-route attach is in progress", "attach in progress, retry") | the same |
-| 6 | another app of the caller's account, serving or committed | 409 naming the app, as `hostHeldSentence` does today | 409 `hostUnavailableMsg` (apidomains.go:458's stronger neutrality kept) |
-| 7 | another app of the caller's account, non-serving op or pending | 409 naming the app and "is being changed; try again later" | 409 `hostUnavailableMsg` |
-| 8 | an operator route whose account column is the caller's, with no record and no claim | 409 "operator repair", as today | 409 `hostUnavailableMsg` |
-| 9 | an operator route naming another account or none | 409 `hostUnavailableMsg` (today's "routed for account X" is retired) | 409 `hostUnavailableMsg` |
-| 10 | the ownership freeze (§8.2) | 503 `msgOwnershipFreeze` | 503, the same |
-
-Rows 1 and 2 come first so that unavailable evidence is never reported
-as a conflict or as free. Nothing a member receives distinguishes a
-foreign hold from a foreign alias claim, and §4.4 and §4.5 name only
-rows of this table.
-
-**Logs and sweeps.** The two sweeps and the backfill verb name hosts,
-accounts and keys in the operator log and the verb's stdout only.
-
-**Buckets.** `control/hosts/` and `domains/` are read by consoles and
-boxes. No door returns a claim body to a member.
+| 1 | a read failed (bucket, registry, SQL) | 502 "could not check the host, try again" | 502, the same |
+| 2 | the object does not decode | 500 "could not check the host"; operator log | the same |
+| 3 | foreign, any family, any state; or a lifecycle purge fence; or an operator route naming another account or none | 409 `hostUnavailableMsg`, byte for byte | 409 `hostUnavailableMsg` (attach); 404 "not attached" (detach) |
+| 4 | own app, same family, committed | 409 "instance already exists" (today) | 200 no-op or repoint |
+| 5 | own app, other family or `first_route` mismatch | 409 naming the conflict and the door that frees it | the same |
+| 6 | own app, `pending` of another request, or `releasing` | 409 "attach in progress, retry" / "being released, retry later" | the same |
+| 7 | own app, committed, but the lifecycle holder does not match (`acq.incarnation`) | 409 "this address needs an operator's attention" | the same |
+| 8 | another app of the caller's account, committed | 409 naming the app, as `hostHeldSentence` does today | 409 `hostUnavailableMsg` (apidomains.go:458's neutrality kept) |
+| 9 | another app of the caller's account, `pending` or `releasing` | 409 naming the app and "is being changed; try again later" | 409 `hostUnavailableMsg` |
+| 10 | an operator route whose account column is the caller's, no object | 409 "operator repair", as today | 409 `hostUnavailableMsg` |
+| 11 | the freeze (§8) | 503 `msgOwnershipFreeze` | 503 |
+
+Nothing a member receives distinguishes a foreign hold of one family
+from another, and read failures are never occupancy answers. Sweeps, the
+backfill and the quarantine file name accounts only in operator output.
## 8. Migration
-### 8.1 Backfill
-
-A console-side verb, `carlos ops hosts backfill [--dry-run]`, runnable
-only above the version floor and only while the freeze (§8.2) is in
-force:
-
-1. List the bucket, not SQL: every `apps/*/*/instances/*.json`,
- `apps/*/*/sidecars/*.json`, `domains/*.json`, and every route a live
- box publishes (`edgesfile`). This covers direct-bucket records of apps
- SQL does not list.
-2. For each host with exactly one record, no lifecycle claim, no
- `domains/` claim or only a legacy first-route marker owned by the
- record's app with `attachWrote` shape, and a host consistent with the
- namespace rule: `claim.Adopt` with that record as holder (the lifecycle
- spec's §9, discovery included), then stamp the legacy marker
- `{acq: committed, first-route, incarnation}`.
-3. **Quarantine** (B-3, B-4): any of
- - two or more records for a host, claimed or not;
- - a record and an alias claim (`Route != ""`) at one host;
- - a record whose lifecycle holder names a different incarnation;
- - a record at `<l>.<sqid>.<apps>` where `<l>` is an app other than
- the record's, or at `<x>.<app>.<sqid>.<apps>`;
- - a legacy first-route marker whose owner has no matching record.
-
- Quarantined hosts are listed in `control/hosts-quarantine.json`,
- rewritten whole by the verb, with every key involved. The create door
- refuses them (an absent or present claim with a quarantine entry is
- held, row 3 of §7), the recovery pass never deletes an orphan there
- (§5), and the operator repairs by deleting the loser or detaching the
- alias, then re-runs.
-4. A record or claim that does not decode, or whose body disagrees with
- its key, is reported and blocks completion.
-5. **Legacy alias claims** with no record at the host are stamped
- `{acq: committed, alias}` by CAS on their ETag. A legacy claim whose
- owner has no app row is reported (purge inheritance) and blocks
- completion until detached.
-6. **Apps with no SQL row** that have records or routes block completion
- (§4.6).
-7. **Completion** needs two consecutive clean passes under the freeze:
- no unclaimed record, no quarantine, no undecodable object, no
- unregistered app, and the second listing showing nothing the first
- did not. The verb then writes `control/hosts-backfill.json` with the
- counts, the floor version and the time, and the freeze lifts.
-
-### 8.2 The freeze, and the door before and after the marker
-
-There is no lazy adoption (A-1). Below the floor the create door, both
-attach branches and the apex gate run **today's** paths. Above the floor
-and before the marker they answer 503 `msgOwnershipFreeze` ("the
-platform is finishing a maintenance step; try again in a few minutes")
-and write nothing (B-2). After the marker they run §4.
-
-So no record, claim or app row can be written between the floor and the
-marker by any binary: old binaries are refused by the floor, new ones by
-the freeze. The two clean passes are defence in depth, not the mechanism.
-The freeze is minutes on flagship: one listing of a few thousand keys and
-one `Adopt` per host.
-
-The marker cannot go stale: nothing above the floor writes an unclaimed
-record. If the floor is ever lowered, the marker is deleted first and the
-doors fall back to today's paths.
-
-### 8.3 Mixed-version consoles
-
-Below the floor, old and new consoles both run today's path, so the race
-remains open until the floor is published; that is the lifecycle spec's
-rollout order and the roll is consoles first. Above the floor no old
-binary runs (lifecycle §10 refuses them, including boot recovery). A
-`serving` claim whose record is gone can still arise from a crashed
-new-console delete, and the lifecycle recovery owners cover it.
-
-### 8.4 Cross-account
-
-- Backfill adopts whatever single record exists, including records under
- accounts the console does not know. It reports those and never
- rewrites, re-homes or deletes a record.
-- Nothing in this design deletes a legacy record: the recovery pass
- deletes only orphans established under §5.1 step 5 outside quarantine,
- and under the marker every legacy record has the claim the backfill
- gave it, so none is an orphan.
-- Refusals follow §7's mapping.
+### 8.1 Order
+
+1. Land the lifecycle library with §6's asks. Ship rev 4's code dormant
+ below the floor: every door runs today's path; the verbs refuse.
+2. Roll consoles, then boxes; publish the floor. **The freeze begins**
+ with the floor: above it and before the marker every create, attach,
+ detach, app-name and delete door answers 503 `msgOwnershipFreeze`.
+3. **Drain** (B-2, C's cutover note). The backfill verb writes
+ `control/ownership-freeze.json`. Every new-binary door re-reads that
+ object **immediately before** its record, claim or row write and
+ refuses if present, so a handler that chose today's path before the
+ floor and paused cannot write after the fence. The verb then waits
+ twice the longest handler deadline (`instanceStoreTimeout` and the
+ browser form's) before listing. Old binaries cannot run above the
+ floor (rev 6.16 §10, including boot recovery).
+4. Backfill (§8.2), dry then live, until two clean passes write
+ `control/hosts-backfill.json`. The freeze lifts; the doors run §4.
+
+### 8.2 Backfill: `carlos ops hosts backfill [--dry-run]`
+
+Runs on a console above the floor with the fence in place. Inputs: every
+`apps/*/*/instances/*.json` and `apps/*/*/sidecars/*.json`; every
+`domains/*.json`; every app row (live and trashed) of every account the
+console knows; every route a live box publishes (`edgesfile`); every
+lifecycle claim.
+
+For each hostname that appears in any input, build its **candidates**:
+records (one per app), a legacy alias or first-route object, an apex
+(from an app row whose name is the host's label under its account's
+sqid), and routes. Then:
+
+- **Exactly one authority, consistent:** write the ownership object
+ `committed` (create-only, or CAS onto the legacy object preserving its
+ alias fields), write the index entry, and for a record with no
+ lifecycle claim run `claim.Adopt`. Consistent means: a record's host
+ obeys the namespace rule and its lifecycle holder, if any, names its
+ incarnation; a legacy alias has no record and no live app row at its
+ label other than its owner (B-4); an apex has no other authority at the
+ host; a first-route marker has exactly its owner's static record.
+- **Anything else is quarantined** in `control/hosts-quarantine.json`
+ with every key involved: two or more records; a record and an alias; a
+ record or alias at another app's apex; a lifecycle holder naming a
+ different incarnation; a marker without its record; an object that
+ does not decode. A quarantined host is held (§7 row 3) and the
+ recovery pass never deletes there (§6 ask 2). The operator picks the
+ winner with `carlos ops hosts resolve <host> --keep <key>`, which
+ deletes the losers' records or objects conditionally and re-runs the
+ host.
+- **Apps with no SQL row** that have records or routes block completion
+ until `carlos ops apps register` (§4.6) or the operator deletes them.
+- **Legacy aliases whose owner has no app row** are reported and block
+ completion until detached by an operator (purge inheritance).
+- Every app row gets `incarnation` and, once its apex is acquired,
+ `apex_acq`.
+
+**Completion** needs two consecutive clean passes under the fence: every
+hostname has exactly one committed object and an index entry, every
+record-backed host has a lifecycle claim naming its incarnation, no
+quarantine, no undecodable input, no unregistered app. Then the marker is
+written with counts, the floor version and the time, the freeze object is
+deleted, and the doors switch. Lowering the floor deletes the marker
+first.
+
+### 8.3 Mixed versions and cross-account
+
+Below the floor old and new consoles run today's paths and the race is
+open; above it no old binary runs. Backfill writes ownership for records
+under accounts the console does not know (from the bucket) and never
+rewrites, re-homes or deletes a record; collisions involving them are
+quarantined like any other. No door's refusal names another account.
## 9. Cost
-Per path, bucket calls only (SQL reads are local). "Today" is the
-current main.
+Bucket requests per logical operation (C-7). "Fold" means
+`edgesfile.DeploymentRoutes`: box status reads plus one GET per live
+box's edges file.
-| Path | Today | After |
+| Path | Today | Rev 4 |
|---|---|---|
-| Instance create, account with N apps | registry read, 2 SQL reads, N sequential GETs, 1 claim GET, 1 record PUT | 1 `domains/` GET, 1 fold read, lifecycle step 0 (1 GET, the lease's PUT), 1 claim PUT, 1 `domains/` GET, 1 record PUT, 1 claim PUT. Independent of N. The lifecycle pair and the lease are the lifecycle spec's cost; this spec adds two `domains/` GETs and removes N. A restarted CAS (step 4) repeats the lease and the pair once |
-| Alias re-attach, no-op | 1 GET | 1 GET, unchanged |
-| Alias repoint | 1 GET + 1 PUT | 1 GET + 1 PUT, unchanged |
-| Alias fresh attach | 1 GET + 1 PUT | 1 GET, 1 lifecycle GET, 1 PUT (pending), 1 lifecycle GET, 1 PUT (commit): two GETs and one PUT more. A takeover adds one PUT |
-| First-route attach | 1 GET + 1 PUT + record PUT | pending PUT, the create path, commit PUT |
-| Detach | 1 GET + 1 DELETE | alias: the same, conditional. First-route: plus 1 lifecycle GET and the lifecycle delete's own writes |
-| App create, rename, move | the N-GET scan | 1 lifecycle GET + 1 `domains/` GET, plus SQL |
-| Claim sweep | none | one `ListMod` of `domains/` every few minutes (the `ClaimCache` steady state); per **pending** claim older than the timeout: 1 GET, 1 lifecycle GET, 1 conditional DELETE. Normally zero pending claims |
-| App-name sweep | none | SQL, plus two GETs per pending op, normally zero |
-| Backfill | none | one-off: one listing of each prefix, one `Adopt` PUT per host, one CAS per legacy claim |
-| Migrate | listing | plus the admission close and reopen (2 CAS) and two prefix listings |
-| Boxes | | the pending skip is a field test; the canary check is a map lookup |
+| Instance create, N apps in the account | registry, N GETs, 1 GET, 1 PUT | 1 GET + 1 PUT (acquire), 1 PUT (index), lifecycle step 0 (1 GET, lease PUT), 1 CAS, 1 record PUT, 1 CAS (commit), 1 CAS (complete). The fold only when the namespace rule admits an unowned platform host. Independent of N |
+| Instance delete | 1 DELETE | 1 GET, 1 lifecycle GET, 1 CAS, lifecycle delete's own writes, 1 DELETE, 1 DELETE |
+| Alias attach, no-op | 1 GET | 1 GET |
+| Alias repoint | 1 GET, 1 PUT | 1 GET, 1 PUT |
+| Alias fresh attach | 1 GET, 1 PUT, the fold | 1 GET, 1 PUT, 1 PUT (index), 1 CAS, the fold |
+| Alias detach | 1 GET, 1 DELETE | 1 GET, 1 CAS, 2 DELETE |
+| First-route attach | 1 GET, 1 PUT, 1 PUT | the instance create path with `first_route` |
+| App create | N GETs | 1 GET, 1 PUT, 1 PUT, 1 CAS, SQL |
+| Rename, move | N GETs | the above, plus 1 CAS and 1 DELETE for the old apex |
+| Ownership sweep | | one `ListMod` per page per minute; a GET per changed or pending object; the index PUT on change |
+| Boxes | one `ListMod` per page per pass | one conditional GET of the alias index per pass (304); the full fold only as fallback |
+| Purge | app prefix listing | plus one listing of `owned/` and a GET and release per host |
+| Backfill | | one-off: the listings, one PUT and one index PUT per host, one `Adopt` per record |
+
+Removing the per-app scan does not make create constant in bucket
+requests; it makes it independent of account size and of fleet size
+except where the fold is consulted.
## 10. Tests
-Against a dirstore with real CAS and conditional delete, in
-`internal/console`, foreground. Every interruption test runs the
-recovery pass and both sweeps afterwards and asserts the residual bucket
-and SQL state, not only the responses.
-
-1. **Concurrent create.** Two apps, one host, 2 and then 16 goroutines
- per app. Exactly one 201; every other response is 409 with the
- within-account sentence; exactly one record; the claim's holder names
- it; `op.kind == serving`. Seeds 1..200.
-2. **Alias versus instance, deterministic schedules.** Every interleaving
- of §4.2 steps 2 to 7 with §4.4 steps 1 to 5 under an injected
- scheduler. Invariant after both return and after recovery: never both
- 2xx; the bucket holds either one committed alias claim and a free or
- absent lifecycle claim, or a record with a matching holder and no
- `domains/` object, or neither. No pending claim survives the sweep.
-3. **Same-owner alias retries and takeover** (A-3, B-1): A1 and A2 for
- the same owner racing B, with A1 paused past the timeout so A2 takes
- over. A1's commit and cleanup fail their conditions; whoever answered
- 2xx, the object it reported exists afterwards.
-4. **Family conversion** (A-4, B-1): a pending or committed first-route
- claim and an alias request by the same owner. Refused; never both the
- record and an alias. An alias request never CASes a first-route
- acquisition.
-5. **Stale cleanup** (A-5, B-7): a rollback paused across a detach and a
- re-attach by the same principal with the same route within one
- second. The conditional delete fails; the new claim stands.
-6. **First-route detach versus successor** (B-6): a detach paused after
- reading marker A while A is detached and B attached; the paused detach
- refuses at step 2 or fails its CAS at step 3; B's record stands.
-7. **Crash between claim and record.** Inject a failure after
- `BeginCreate`; advance the clock past five minutes; recovery frees the
- claim, retires the incarnation, lists the key; a create by another app
- succeeds. The negative: record present, recovery completes, never
- frees.
-8. **Pending alias never serves** (A-8): a pending claim and a lifecycle
- holder whose route is not yet published; `ReconcileAliases` derives
- nothing for it (alias, wildcard, catch-all, paths); the sweep deletes
- it.
-9. **Alias versus app name** (B-5): with reservation off, an alias at
- `<l>.<sqid>.<apps>` racing `claimApp` for `<l>` under every
- interleaving; at most one commits; the loser's row or claim is gone
- after the sweeps.
-10. **Backfill.** Records across three apps and two accounts; one host
- with two records; one record with an alias claim; one legacy
- first-route marker with and without its record; one undecodable
- record; one app prefix with no row. The verb adopts the clean
- singletons, stamps legacy claims, quarantines each bad combination
- with every key named, blocks on the bad record and the rowless app,
- writes no marker; after repair, two clean passes write it. The create
- door answers 503 throughout the freeze and refuses quarantined hosts
- after it.
-11. **Freeze** (B-2): above the floor and before the marker, every door
- answers 503 and writes nothing; the marker verb refuses below the
- floor.
-12. **Quarantine guard** (B-3): an orphan at a quarantined host survives
- the recovery pass.
-13. **Response mapping.** Every row of §7 on every door; cross-account
- bodies byte-identical and sqid-free; read failures answer 502 for a
- held and a free host alike; an invalid request answers 400 whether or
- not the host is held.
-14. **Apex gate** (B-9): create, rename and move each crashed after step
- 1 and after step 2; both names stay reserved; the sweep settles with
- the op id; a paused handler finding its op gone does nothing.
-15. **Namespace rule**, each row of §4.6 against each door, reservation
- on and off, including the own-app exception and a manifest `Domain`
- of `<app>.<sqid>.<apps>` for another app.
-16. **Canary mint does not overwrite** a route owned by another app.
-17. **Migrate** (B-13): refused for an app with a record, with admission
- closed during the check and reopened after; a create during the
- check is refused by admission; success for a releases-only app with
- no route stamping.
-18. **Purge and materialization** (B-8): the purge door detaches claims
- before `PurgeApp`; after box auto-purge, materialization detaches
- claims before releasing the row; a new app of the name inherits
- nothing.
-
-The joint simulation (`internal/sim/hostlife`) should gain the alias
-actor, the acquisition state and the two-family invariant, if the moves
-session agrees; this spec does not depend on it.
+Dirstore with CAS and conditional delete, foreground, in
+`internal/console`. Every interruption test runs lifecycle recovery and
+both sweeps afterwards and asserts the residual bucket, index and SQL
+state.
+
+1. **Concurrent create**, two apps, one host, 16 goroutines each, seeds
+ 1..200: one 201, one object, one record, holder incarnation equals
+ `acq.incarnation`.
+2. **Alias versus instance versus app create**, every pairing, both
+ orders and concurrent: at most one commits; no cross-authority check
+ is needed to prove it, the test asserts the single object.
+3. **Delayed creator** (§4.2): pause after acquire past 60 s; the sweep
+ reclaims; a successor acquires; the original resumes through
+ `BeginCreate` and the record and fails its commit, cancels, deletes
+ its record; the successor completes on retry. Variant: pause between
+ record and commit past five minutes; recovery cancels because the
+ object names another acquisition.
+4. **Recovery completes with commit**: crash between record and commit;
+ recovery commits the object then completes; O1 holds.
+5. **Same-owner retry and takeover**: resume by id; takeover by CAS; the
+ displaced writer's commit, rollback and index write all fail.
+6. **Release**: crash after each of §4.4's steps; the sweep finishes;
+ the object is gone, the lifecycle claim is free and retains its
+ history; a new create succeeds afterwards and is refused "being
+ released" before.
+7. **Detach dispatch**: each row of §4.5; an instance without
+ `first_route` is never deleted by detach; `route == ""` has no effect.
+8. **App incarnation fence**: attach paused across purge and recreate of
+ the name; commit refused; nothing attached to the new app.
+9. **Purge enumeration**: a read failure during `owned/` enumeration
+ aborts the `released` phase; nothing is released; the name stays
+ reserved. Materialization releases the row only when `owned/` is
+ empty.
+10. **Rename with a concurrent first-route attach** (C-4): the attach's
+ commit fails on `closed`; the source stays empty; finalize succeeds.
+ Crash after each step; the app-name sweep settles by `op_id`.
+11. **Namespace rule and canary mint**: each row of §4.6's table on
+ every door; the mint skips an unrelated route and skips when the
+ apex is not committed.
+12. **Fence and freeze** (B-2): a handler paused before its write with
+ the freeze object present refuses; every door answers 503 during the
+ freeze.
+13. **Backfill**: a fixture with every quarantine case, a legacy alias at
+ a live app's label (B-4), a rowless app, an undecodable object;
+ nothing is adopted or stamped for a quarantined host; the recovery
+ pass deletes nothing before the marker (C-2); `resolve --keep`
+ clears each; two clean passes write the marker.
+14. **Response mapping**: every row of §7 on every door; foreign bodies
+ byte-identical and sqid-free; read failures answer 502 for held and
+ free alike; invalid later fields answer 400 before any ownership
+ read.
+15. **Alias index**: boxes derive the same `route_aliases` from the index
+ as from the full fold; a stale index falls back.
+16. **Migrate**: refused with a record; admission closed and drained per
+ §5.3 step 1b during the check.
## 11. Rollout
-1. Prerequisites, in the lifecycle spec and library: the adoption
- evidence rule and the quarantine guard (§6). Land `internal/hostclaim`
- on main with them.
-2. Ship the doors, the two sweeps, the backfill verb, the `acq` block,
- the namespace rule, the apex gate's op table, the purge and
- materialization detaches, the canary-mint check and the pending skip
- in `ReconcileAliases`, all **dormant below the floor**: below it every
- door runs today's path and the verb refuses to run.
-3. Roll consoles, then boxes, in the lifecycle spec's order. Publish the
- floor. The freeze begins.
-4. Run the backfill dry; read the quarantine and the reports; repair;
- run live until two clean passes write the marker. The freeze lifts.
-5. `hostHolder`'s record loop, `apexHolder`'s scan and `ReportApexSquats`'
- route walk are removed in the release after.
-6. Follow-ups filed, not built here: migrate's transfer and `domains/`
- rebind (§4.8).
+§8.1's order. After the marker, the release that follows removes
+`hostHolder`, `apexHolder`'s scan, `ReportApexSquats`' route walk, the
+startup owners backfill and the legacy-object branches.
## 12. Alternatives considered
-- **`domains/<host>.json` as the ownership object for everything.**
- Rejected: fleet-wide listing cost, the detach door's record delete on
- `Route == ""`, and duplicating the lifecycle claim's create protocol.
-- **A new `hosts/<host>.json` ownership-only object.** Rejected: it would
- duplicate the lifecycle holder for every record-backed host and need
- its own crash recovery.
-- **An `alias` holder family in the lifecycle claim.** One object for
- every routable name, no two-family rule, and the alias pass would read
- one authority. Rejected for now: it changes a settled spec and
- `holderKeyMatches`, gives every custom domain a lifecycle object the
- recovery pass lists, and still needs an acquisition state for
- same-owner retries. It remains the cleanest end state if the two-family
- rule proves hard to keep as doors are added; §13 asks.
-- **Box-side alias pass reading the lifecycle claim.** Rejected: one GET
- per alias per box per pass, or a second cache over `control/hosts/`.
- The pending state gives the pass a committed-only rule with no new
- read.
-- **A sweep that commits stale pending aliases.** Rejected (B-12): the
- sweep lacks the observations to authorise a commit, and a commit nobody
- requested is an attach nobody asked for. Delete-only.
-- **Plain `CreateRecord` between the floor and the marker.** Rejected
- (B-2): any such write after the final listing turns a legitimate record
- into a future orphan. A short freeze is cheaper than a wrong delete.
-- **Deterministic tie-break instead of both backing off.** Rejected as
- unsound (§4.1).
-- **Lazy adoption.** Rejected (A-1): the door can only see the caller's
- account, so it would create claims over unseen foreign records and the
- recovery pass would then delete them as orphans.
-- **A one-field holder transfer for migrate.** Rejected (A-12): identity,
- location, execution authority, sidecars and aliases all move with the
- account.
+- **Two authorities with write-then-verify** (revs 1 to 3). Rejected by
+ ruling: the interleaving proof, the reciprocal checks and the
+ cross-family migration cases were most of the design's risk.
+- **An `alias` family in the lifecycle claim.** Rejected: it would make
+ every alias a lifecycle object with parking and retirement, and it
+ changes a settled spec; ownership is a smaller object.
+- **A per-canary ownership object acquired by boxes.** Rejected: boxes
+ would need `domains/` write grants and cross-region races; the apex
+ covers its subtree.
+- **Boxes keep listing `domains/`.** Kept as the fallback; the derived
+ index is the steady state once the prefix exceeds a page.
+- **Release deletes ownership as soon as the record is gone.** Rejected:
+ a successor would acquire and then be refused by a `parking` lifecycle
+ claim; releasing at lifecycle free gives one refusal with one sentence.
+- **A sweep that commits stale pending objects.** Delete-only, except the
+ `apex` family where the SQL row carrying `apex_acq` is exact evidence.
## 13. Open questions for Paul
-1. Two families with acquisition-stamped alias claims (this spec), or one
- object with an `alias` family in the lifecycle claim (§12)? The second
- reopens the lifecycle spec.
-2. The lifecycle spec's §9 adoption rule and the quarantine guard are now
- rollout prerequisites (§6). Accept them into that spec, with the
- backfill verb as the only adopting actor?
-3. The freeze (§8.2): a few minutes of 503 on every create and attach
- door on flagship, once. Acceptable, or should the backfill run against
- a snapshot with a different fence?
-4. Account migrate: refuse apps with records indefinitely (this spec), or
- file the transfer protocol now?
-5. The namespace rule refuses `<x>.<app>.<sqid>.<apps>` through every
- door, and a manifest `Domain` under another app's apex. Acceptable on
- deployments with reservation off?
-6. Backfill quarantine: operator picks the winner by hand (this spec), or
- the verb keeps the record the provisioning pass currently keeps (first
- in listing order)?
-7. The acquisition timeout of 60 s, the sweep cadence, and the recovery
- age of five minutes are guesses. Shorter?
-8. Rev 3 has not been reviewed. A third Astra round before planning, or
- straight to plans with the review at plan level?
+1. The freeze: every door 503 for the minutes the backfill takes on
+ flagship, once. Acceptable?
+2. Quarantine resolution by `carlos ops hosts resolve --keep <key>` with
+ the operator choosing: confirm, or prefer a default (the record the
+ provisioning pass keeps today, first in listing order)?
+3. The derived alias index as a console-written object the boxes read:
+ acceptable as a second derived artefact beside the edges files?
+4. The 60 s acquisition timeout and the one-minute sweeps are guesses.
+5. §6 asks 1 to 4 of the lifecycle spec need platform-1e's agreement
+ before planning.