design: host ownership rev 4, one ownership record per hostname (Paul's ruling)
domains/<host>.json owns every family (instance, sidecar, alias, apex) with immutable acquisition ids and pending/committed/releasing states; the lifecycle claim keeps execution, parking and retirement, bound by op.intent.ownership and the holder incarnation; release follows lifecycle retirement and is recoverable; detach dispatches by family; per-app owned/ index for complete purge enumeration; app-row incarnation fencing; derived alias index for fleet folds; freeze with a write fence, reconciliation and quarantine for migration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+725
−869
docs/superpowers/specs/2026-10-06-host-ownership-design.md+725 −869
CI passed — run details
diff --git a/docs/superpowers/specs/2026-10-06-host-ownership-design.md b/docs/superpowers/specs/2026-10-06-host-ownership-design.md| index f51f22cbd..26690dc09 100644 |
| --- a/docs/superpowers/specs/2026-10-06-host-ownership-design.md |
| +++ b/docs/superpowers/specs/2026-10-06-host-ownership-design.md |
| @@ -1,926 +1,782 @@ |
| # Host-scoped atomic ownership |
| -Design, 2026-10-06, **rev 3**. Closes amadan #232. Design only; nothing |
| +Design, 2026-10-06, **rev 4**. Closes amadan #232. Design only; nothing |
| here is built. |
| -Review history. Rev 1 (b70d4d30) and rev 2 (519d25e9) each went to Astra |
| -for a read-only review against the code; the findings are cited `A-n` |
| -(round 1) and `B-n` (round 2) below. Rev 2 answered A-1 to A-15. Rev 3 |
| -answers B-1 to B-15 and the parts of round 1 that round 2 judged partial: |
| -the alias acquisition carries an immutable id and kind for the life of |
| -the claim, takeover is a CAS before verification, the sweep is delete-only |
| -(B-1, B-7, B-11, B-12); creates are frozen between the floor and the |
| -marker so no unclaimed record can be written after the floor (B-2); the |
| -adoption rule and a quarantine guard are rollout prerequisites and the |
| -backfill validates every host across all three authorities (B-3, B-4); |
| -alias finalizers verify the app row (B-5); the first-route marker is |
| -bound to the record's incarnation (B-6); console materialization detaches |
| -claims before it releases a purged name (B-8); app-name operations get an |
| -op id and reserve both names (B-9); the own-app apex exception is explicit |
| -and direct-bucket apps block completion (B-10); migrate quiesces the app |
| -through admission (B-13); the response mapping is ordered and complete |
| -(B-14); readers and costs are listed (B-15). Rev 3 has not been reviewed. |
| - |
| -Related specs: `2026-10-02-host-lifecycle-claim-design.md` rev 6.14 on |
| -`design-replica-ownership` (the lifecycle claim), with its library at |
| -`internal/hostclaim` on `feat/hostclaim-library`; `2026-08-12-domain-attach-design.md` |
| +**Ruling (Paul, 2026-10-06): one ownership record per hostname.** |
| +`domains/<host>.json` is the single authoritative ownership object for |
| +every family. Astra's round 3 (`~/.claude/tmp/hostown-r3.md` on DOCS-T1, |
| +§3 and §4) recommended it. Rev 4 is a rewrite around that model; revs 1 |
| +to 3 (b70d4d30, 519d25e9, 5c4c31f1) carried a two-family design whose |
| +cross-family verification this ruling removes. Their review findings are |
| +cited `A-n`, `B-n`, `C-n` where rev 4 still answers them. |
| + |
| +Related specs: `2026-10-02-host-lifecycle-claim-design.md` **rev 6.16** |
| +(1915bac2) on `design-replica-ownership`, library at `internal/hostclaim` |
| +on `feat/hostclaim-library` (47426cc1); `2026-08-12-domain-attach-design.md` |
| (the global hostname claim, #151, #82). |
| ## 0. Summary |
| -A hostname on the platform is held today by one of three things, and no |
| -single object says which: |
| - |
| -- an instance record at `apps/<acct>/<app>/instances/<host>.json` |
| - (provision.RecordKey), keyed per account and app, so two apps can both |
| - write a record for one host; |
| -- a global hostname claim at `domains/<host>.json` (domains.ClaimKey), |
| - keyed by host alone, create-only, written by the domain attach doors; |
| -- a registry route a box minted from an app's name (its platform apex and |
| - a static app's canary hosts), with no record and no claim. |
| - |
| -The instance create door checks the first two before it writes |
| -(`hostHolder`, `hostClaimHolder` in internal/console/apex.go, 04cd2d78), |
| -then writes a record whose conditional create protects only its own key. |
| -The check and the write are two steps, so two concurrent creates of one |
| -host from two apps can both get 201. The same shape exists between an |
| -alias attach and an instance create. |
| - |
| -The design: |
| - |
| -1. **Record-backed hosts** (instances, replicas, static records, sidecars) |
| - are owned by the **lifecycle claim's holder** at |
| - `control/hosts/<host>.json`. The lifecycle create protocol's claim CAS |
| - is the atomic gate. There is no second ownership object for these |
| - hosts, and this spec adds no field to that claim. |
| -2. **Aliases** (hosts with a claim and no record) are owned by the |
| - **global hostname claim** at `domains/<host>.json`. The claim gains |
| - one additive block, `acq`, so an acquisition has an immutable identity |
| - and kind and is written pending, verified, then committed. Only |
| - committed claims materialise or count as held by their owner. |
| -3. The two families meet at one rule: **write your own authority first, |
| - re-read every other authority, back off if any is held, and only then |
| - commit.** With S3's strong read-after-write consistency at least one |
| - of two racing writers sees the other. Every back-off deletes only the |
| - exact object version it wrote. |
| -4. **Minted hosts** (apex `<app>.<sqid>.<domain>`, static canary |
| - `<name>.<app>.<sqid>.<domain>`) are owned by the app whose name they |
| - carry. One namespace rule, applied by every door, says so from the |
| - name alone. The console's unique (account, app) row is the atomic gate |
| - for the name, and it too is written then verified. |
| -5. Legacy hosts get a claim from a backfill verb that runs after the |
| - version floor, during a short freeze of every create door, so no |
| - writer can add an unclaimed record behind it. The door treats an |
| - absent claim as free only once the backfill has written its completion |
| - marker. There is no lazy adoption. |
| +1. **One object.** `domains/<host>.json` owns every hostname: instance |
| + and replica records, static records (including first-route attaches), |
| + sidecars, aliases, platform domains, and the minted app apex. It is |
| + acquired with an `If-None-Match` create. A canary host |
| + `<name>.<app>.<sqid>.<apps>` is covered by its app's apex ownership |
| + and has no object of its own. |
| +2. **Acquisitions** carry an immutable id and family and move |
| + `pending → committed → releasing → gone`. Every update and delete is |
| + conditional on the ETag read. A same-owner retry resumes its own |
| + acquisition or takes over a stale one by CAS, which fences the writer |
| + it displaces. |
| +3. **The lifecycle claim** (`control/hosts/<host>.json`) keeps |
| + execution, parking and retirement. It is bound to the ownership |
| + acquisition: the create's `op.intent` names the acquisition id, and a |
| + serving holder's incarnation is the one the committed ownership |
| + records. It is no longer an ownership authority, and its fencing |
| + history is never deleted when a hostname is released (C5a). |
| +4. **Release** follows lifecycle retirement: ownership goes `releasing` |
| + before the lifecycle delete starts and is deleted only once the |
| + lifecycle claim is free. A paused creator whose pending reservation was |
| + reclaimed cannot reach serving: its commit CAS fails and the lifecycle |
| + recovery pass refuses to complete a create whose acquisition the |
| + ownership object does not name. |
| +5. **Detach dispatches by family.** `Route == ""` grants nothing. |
| +6. **Per-app index.** `apps/<acct>/<app>/owned/<host>.json` lists the |
| + hosts an app owns, written before commit and removed after release, so |
| + purge and materialization enumerate completely and fail closed. |
| +7. **App incarnation.** Every app row carries a random incarnation; |
| + ownership records it; a late writer for a purged and recreated name is |
| + refused at commit. |
| +8. **Fleet folds** filter by family, and a console-derived alias index |
| + replaces the per-box listing once `domains/` exceeds one page. |
| +9. **Migration** is a freeze with a demonstrated write fence, a |
| + reconciliation of records, aliases, app rows and routes, quarantine |
| + for every collision, and an operator-chosen winner. Destructive |
| + orphan cleanup is disabled until the backfill marker exists. |
| ## 1. The problem, in the code |
| -`handleAPIInstanceCreate` (apiinstances.go:772-792) asks `hostHolder`, |
| -which reads this box's registry and then one `GetRecord` per other app |
| -of the account (live and trashed, SQL-listed), and `hostClaimHolder`, |
| -which reads `domains/<host>.json`. Then `provision.CreateRecord` |
| -(provision.go:820) does `Put(key, body, "")`, create-only on |
| -`apps/<acct>/<app>/instances/<host>.json`. |
| - |
| -Consequences: |
| - |
| -- Two creates for one host from apps A and B both pass the checks and |
| - both write, because their keys differ. The provisioning pass keeps the |
| - record it lists first and logs `duplicate host` for the other |
| - (provision/reconcile.go:819). The loser looks created and serves |
| - nothing, or a different box serves it. |
| -- An alias attach (`domains.AttachClaim`, apidomains.go:444 and |
| - domainattach.go:153) and an instance create for the same host each |
| - check the other's object before writing their own. The same window. |
| -- The record scan costs one sequential GET per other app, inside the |
| - handler's 15 s budget; it sees only SQL-listed apps and only this |
| - console box's registry; direct-bucket records of apps not in SQL are |
| - outside it. |
| -- The checks run before later validation (replica rails at :887, shape |
| - at :1092), so an invalid request answers 409 for a held host and 400 |
| - for a free one: an occupancy oracle. |
| +`handleAPIInstanceCreate` (apiinstances.go:772-792) asks `hostHolder` |
| +(one `GetRecord` per other app of the account, this box's registry) and |
| +`hostClaimHolder` (`domains/<host>.json`), then `provision.CreateRecord` |
| +(provision.go:820) writes create-only on a key that includes the app. Two |
| +apps racing for one host both pass and both write; the provisioning pass |
| +keeps the record it lists first (`duplicate host`, provision/reconcile.go:819). |
| +Alias attach (`domains.AttachClaim`, apidomains.go:444, domainattach.go:153) |
| +and instance create check each other's object before writing their own: |
| +the same window. The checks run before later validation (:887, :1092), an |
| +occupancy oracle. The scan sees only SQL-listed apps of one account and |
| +costs N sequential GETs. |
| ## 2. Writer inventory |
| -Every path that puts a hostname on the platform or takes one off. The |
| -**Role** column says how it relates to ownership: *decides* (it is an |
| -authority write), *derives* (it reads an authority and materialises), |
| -*operator* (outside the guarantee, by hand), or *no change* (host and |
| -holder unchanged). |
| - |
| -| Writer | Hosts | Writes today | Role | Gap today | |
| -|---|---|---|---|---| |
| -| Instance create (`handleAPIInstanceCreate`) | `<label>.<m.Domain>` | record, create-only | decides | check then write; scan cost; local registry; ordering oracle | |
| -| Instance delete (:1675) | same | `DeleteRecord` | decides | nothing host-scoped to release yet | |
| -| Replica create (same door, `CreateRecordReplica`) | one label + the app's domain | record | decides | as create | |
| -| Sidecar create and delete (apisidecars.go:285, :347; `CreateSidecarRecord`, `DeleteSidecarRecord`, sidecar.go:322, :398) | `<name>.<app>.<sqid>.sidecar.internal` | sidecar record at `sidecars/<name>.json`; its own primitives | decides (lifecycle family `sidecar`) | none for ownership: the host embeds account and app; never routable | |
| -| Canary (`carlos canary`, canary.go:231) | `<canary>-<app>.<sqid>.<apps>` | the create door | decides, via the door | flat label makes collisions easy (#207) | |
| -| Platform apex mint (domains/reconcile.go:266) | `<app>.<sqid>.<apps>` | registry route via `Upsert`; candidates come from routes (`candidateApps`, :757), not SQL | derives from the app name | defers to any route already at the host; mints for apps with no SQL row | |
| -| Static canary mint (domains/reconcile.go:328-337) | `<name>.<app>.<sqid>.<apps>` | registry `Upsert`, no `byHost` check; `Upsert` rewrites `app` and `account` on conflict (registry.go:990) | derives from the app name | can overwrite an unrelated route | |
| -| Trash and retired-canary route removal (domains/reconcile.go:207, :356) | the app's routes | `reg.Delete` | derives | none | |
| -| Instance and sidecar route mint and removal (provision/reconcile.go:965, :1178; sidecar_reconcile.go:403, :450) | record hosts | registry rows | derives from records | first-listed record wins a duplicate host | |
| -| App create, rename, move (server.go:3667, :3707; appsettings.go:802, :827, :1305, :1350; store.go:1801-1905) | the apex | SQL app row and related rows (credentials, grants) | decides the name | check then write against records; no crash recovery after the row; rename and move release the old name at once | |
| -| Startup owners backfill (store.go:676) | the apex | `INSERT OR IGNORE` ops-account rows for legacy `owners` entries with no row | decides the name, by hand | legacy-only; does not register bucket-derived apps | |
| -| Console materialization (materialize.go:391) | the apex | removes a trashed app's SQL row when its bucket prefix is gone (after box auto-purge) | releases the name | releases it while the app's `domains/` claims still exist, so a new app of the name inherits them | |
| -| Domain attach, alias branch (apidomains.go:444; domainattach.go:153) | any hostname | `domains/<host>.json`, create-only; same-owner `Route` converge in place (claims.go:150) | decides | does not read records, the lifecycle claim or app rows; converge can turn a first-route claim into an alias | |
| -| Domain attach, first-route branch (domainattach.go:183-192) | any hostname | claim (`Route ""`) then static record | decides, both families | the record half has the create door's race | |
| -| Attach rollback (`rollbackAttach`, domainattach.go:483-499) | | `DeleteRecord`, `DetachClaim` | cleanup | unconditional deletes | |
| -| Platform attach (`--platform`, `attachDomainAs`) | names in the deployment's zones | the same two branches | decides | the same | |
| -| Domain detach (apidomains.go:853; domainattach.go:288-314) | | `DeleteRecord` when `Route ""`, then `DetachClaim` (read, then unconditional delete, claims.go:182) | decides | unconditional delete; the record to delete is chosen from a claim read that nothing fences | |
| -| Claim capabilities and paths (`SetClaimCapabilities`, `SetClaimPaths`, claims.go:302, :398) | | fields of an existing claim, owner-gated CAS | no change to ownership | must refuse a pending claim | |
| -| Domains pass (`ReconcileAliases`, aliases.go:89-133): alias, wildcard, catch-all and path derivation | | `route_aliases`, `wildcard_domains`, `route_paths` | derives from claims | never reads the lifecycle claim; must skip pending claims | |
| -| Operator routes (`carlos route add`, `carlos remove`; cli.go:255, :278) | any | registry `Upsert`, `Delete` on one box | operator | covered by the fleet fold, not by any object | |
| -| Moves (apiinstancemove.go, `SetRecordMove`) | unchanged | record fields | no change | | |
| -| Placement (`SetRecordRegion`) and pools | unchanged | record Region | no change | | |
| -| Account migrate (accounts.go:312-470) | unchanged | copies `apps/<src>/<app>/` to `apps/<dst>/<app>/` create-only; only channel pointers are rebound (:369), records are copied unchanged (:416); stamps routes' account (:456); deletes nothing | operator | the copied record names the source account, which `GetRecord` refuses (provision.go:1145); two records per host; `domains/` claims still name the source; the plan is built from a listing nothing fences | |
| -| App purge (`release.PurgeApp`, trash.go:182-278; console door ownership.go:442; box auto-purge domains/reconcile.go:232) | all of an app's hosts | deletes every key under the app prefix and the marker | decides (release) | leaves the app's `domains/` claims | |
| -| Mail domains (`email/domain-claims/<domain>.json`) | email sending domains | its own create-only claim | separate namespace | out of scope | |
| - |
| -Every record-backed writer funnels through `CreateRecord`, |
| -`CreateSidecarRecord`, `DeleteRecord`, `DeleteSidecarRecord` or |
| -`PurgeApp`, and the lifecycle claim spec wraps exactly those with a |
| -per-host CAS. Every alias writer funnels through `AttachClaim` and |
| -`DetachClaim`. |
| +Every path that puts a hostname on the platform or takes one off, and its |
| +place in the new model. **Acquires** means it writes the ownership object; |
| +**derives** means it reads ownership and materialises; **operator** is |
| +outside the guarantee. |
| -## 3. The objects |
| +| Writer | Hosts | Writes today | Rev 4 | |
| +|---|---|---|---| |
| +| Instance create (`handleAPIInstanceCreate`) | `<label>.<m.Domain>` | record | acquires, family `instance` (§4.2) | |
| +| Instance delete (:1675) | | `DeleteRecord` | releases (§4.4) | |
| +| Replica create (same door) | | record | acquires, family `instance` | |
| +| Sidecar create and delete (apisidecars.go:285, :347; `CreateSidecarRecord`, `DeleteSidecarRecord`) | `<name>.<app>.<sqid>.sidecar.internal` | sidecar record | acquires and releases, family `sidecar` | |
| +| Canary (`carlos canary`) | `<canary>-<app>.<sqid>.<apps>` | the create door | via the door | |
| +| Platform apex mint (domains/reconcile.go:266; candidates from routes, :757) | `<app>.<sqid>.<apps>` | registry `Upsert` | derives from the `apex` ownership of (account, app) | |
| +| Static canary mint (domains/reconcile.go:328-337; `Upsert` rewrites app and account, registry.go:990) | `<name>.<app>.<sqid>.<apps>` | registry `Upsert`, no `byHost` check | derives from the app's `apex` ownership; `byHost` check added | |
| +| Trash and retired-canary route removal (:207, :356) | | `reg.Delete` | derives, unchanged | |
| +| Instance and sidecar route mint and removal (provision/reconcile.go:965, :1178; sidecar_reconcile.go:403, :450) | | registry rows | derives from records and lifecycle admission, unchanged | |
| +| App create, rename, move (server.go:3667, :3707; appsettings.go:802-840, :1305-1350; store.go:1801-1905) | the apex | SQL app row and related rows | acquires, family `apex` (§4.6) | |
| +| Startup owners backfill (store.go:676, run at server.go:1357 on every start) | the apex | `INSERT OR IGNORE` ops rows | retired above the floor; replaced by `carlos ops apps register` (§4.6, C-1) | |
| +| `console migrate-app` (cmd/console/main.go:228, `MoveApp`) | the apex | SQL row move | goes through the apex gate's move path (C-1) | |
| +| Console materialization (materialize.go:391, `PurgeAppRow`) | the apex | releases a trashed row | releases only after the app's `owned/` index is empty (§5) | |
| +| Domain attach, alias branch (apidomains.go:444; domainattach.go:153; converge, claims.go:150) | any | `domains/<host>.json` | acquires, family `alias` (§4.3) | |
| +| Domain attach, first-route branch (domainattach.go:183-192) | any | claim then static record | acquires, family `instance` with `first_route` (§4.3) | |
| +| Platform attach (`--platform`) | deployment zones | the same two branches | the same, `platform` recorded | |
| +| Attach rollback (domainattach.go:483-499) | | unconditional deletes | conditional (§4.1) | |
| +| Domain detach (apidomains.go:853; domainattach.go:288-314) | | `DeleteRecord` when `Route ""`, `DetachClaim` | dispatch by family (§4.5) | |
| +| Claim capabilities and paths (claims.go:302, :398) | | fields of an existing claim | unchanged; refuse unless committed `alias` | |
| +| Domains pass (`ReconcileAliases`, aliases.go:89-133) | | `route_aliases`, wildcard, catch-all, paths | derives from committed `alias` ownership (§4.8) | |
| +| Operator routes (cli.go:255, :278) | any | registry on one box | operator; covered by the fleet fold | |
| +| Moves, placement, pools | unchanged host | record fields | no change to ownership | |
| +| Account migrate (accounts.go:312-470; records copied unchanged :416, refused by `GetRecord` provision.go:1145) | unchanged | copies objects, stamps routes | refused for apps with records; quiesced through admission (§4.7) | |
| +| App purge (trash.go:182-278; console door ownership.go:442; box auto-purge domains/reconcile.go:232) | all the app's hosts | deletes the app prefix | releases every owned host first; `PurgeApp` keeps `owned/` (§5) | |
| +| Mail domains (`email/domain-claims/`) | sending domains | its own claim | separate namespace, out of scope | |
| -### 3.1 The global hostname claim, `domains/<host>.json` |
| +## 3. The objects |
| -domains.Claim: `{schema, host, account, app, route, created_at, |
| -created_by, wildcard, catchall, paths}`. `AttachClaim` reads, then |
| -`Put(key, body, "")` when absent; another owner gets |
| -`ErrHostUnavailable`, which never names the holder; the same owner |
| -converges `Route` in place. `DetachClaim` is owner-gated and quiet on |
| -mismatch. Every box folds `domains/` on its fast-converge leg through |
| -`ClaimCache`, and `carlos domains list` and app Settings list it too. |
| +### 3.1 Ownership: `domains/<host>.json` |
| -This spec adds one block, additive and `omitempty`: |
| +Today's domains.Claim, `{schema 1, host, account, app, route, |
| +created_at, created_by, wildcard, catchall, paths}`, extended with |
| +additive `omitempty` fields. The key is the same for every family, so one |
| +`If-None-Match` create decides the host. |
| ``` |
| -"acq": { |
| - "id": "<random, 128-bit>", |
| - "kind": "alias" | "first-route", |
| - "state": "pending" | "committed", |
| - "incarnation": "<hex>", // first-route only, set at commit |
| - "started_at": "<RFC3339>" |
| +{ |
| + "schema": 1, |
| + "host": "shop.bab.oncarlos.com", |
| + "account": "bab", "app": "jam", |
| + "app_incarnation": "<hex, from the apps row>", |
| + "family": "instance" | "sidecar" | "alias" | "apex", |
| + "acq": { |
| + "id": "<random 128-bit>", |
| + "state": "pending" | "committed" | "releasing", |
| + "started_at": "<RFC3339>", |
| + "by": "<principal>", |
| + "record_key": "apps/bab/jam/instances/shop....json", // instance, sidecar |
| + "incarnation": "<hex>", // instance, sidecar, set at commit |
| + "release_op": "<lifecycle op id>" // releasing, instance and sidecar |
| + }, |
| + "first_route": true, // instance acquired through the domains surface |
| + "platform": true, // acquired through the --platform door |
| + "route": "...", "wildcard": ..., "catchall": ..., "paths": ... // alias only |
| } |
| ``` |
| Rules: |
| -- `id` and `kind` are **immutable for the life of the claim**. A repoint |
| - preserves them; a takeover (§4.4) replaces the whole claim including |
| - `id`, which is how it fences the writer it displaces. Two acquisitions |
| - of one host therefore never produce identical bytes, so their ETags |
| - differ on every store (B-7). |
| -- `state: pending` reserves the key against everyone but is **not |
| - held** by its owner: the alias pass derives nothing from it (alias, |
| - wildcard, catch-all or path), Settings and `domains list` show it as |
| - "attaching", and `SetClaimCapabilities`, `SetClaimPaths` and the |
| - repoint path refuse it. |
| -- `kind` fixes the family the claim is for. An `alias` acquisition can |
| - never become a first-route marker or the reverse (B-1). |
| -- A claim with no `acq` block is a legacy committed alias, or a legacy |
| - first-route marker when `Route == ""`. The backfill (§8.1) stamps |
| - `acq` onto every legacy claim it validates. |
| - |
| -The block must be declared in `Claim` in the same change that first |
| -writes it (the converge rule in claims.go's own comment), and the floor |
| -guarantees no reader below this release folds a pending claim. |
| - |
| -It remains the right owner for aliases and the wrong owner for instance |
| -hosts: the fleet folds this prefix every few seconds on every box and |
| -`ListObjectsV2` pages at 1000 keys; the detach door deletes the record |
| -when `Route == ""`; and the lifecycle claim already is a per-host, |
| -create-only, holder-naming object for record-backed hosts, with crash |
| -recovery designed and reviewed. |
| - |
| -### 3.2 The lifecycle claim, `control/hosts/<host>.json` |
| - |
| -hostclaim.Claim: `{v, host, gen, holder{family, key, account, app, name, |
| -incarnation, created_at, location, ...}, op{kind, id, ...}, touched, |
| -inventory, obligations, lost, retired_incarnations, quiet_after, |
| -cancelled_creates}`. `hostclaim.Update` writes the whole object with |
| -`If-Match` on the ETag read, or `If-None-Match` when it read absence. The |
| -object is never deleted (C5a); free is a written state. `BeginCreate` |
| -refuses unless `c.Free()`. `CompleteCreate` moves `op` to `serving` and |
| -drops the create's op id, after which `CancelCreate` is illegal |
| -(create.go:93, :104). `IsOrphan` treats a record whose incarnation an |
| -existing claim does not name as an orphan (orphan.go:10). `Adopt` writes |
| -a claim only for a host that has never had one (adopt.go). A purge fence |
| -is a holderless `purging` op (claim.go, CC21). |
| - |
| -Spec §5.1 is the create protocol: step 0 reads the claim and keeps its |
| -ETag or absence, then registers the create in the app's `admission.json` |
| -as a lease, which succeeds only while the app's admission is `open`; |
| -step 1 assigns `CreatedAt`; step 2 CASes the claim to `{holder: new, op: |
| -creating}` conditional on exactly what step 0 read, and a 412 restarts |
| -from step 0 with a new registration and a new `CreatedAt`; step 3 is |
| -`CreateRecord`; step 4 CASes to `serving`; step 5 is the recovery pass. |
| -§5.2 is delete; §5.3 purge; §9 legacy adoption; §10 the version floor. |
| - |
| -**The lifecycle claim's holder is the ownership object for every |
| -record-backed host.** This spec adds nothing to it and changes none of |
| -its invariants. It specifies where the console's doors read the other |
| -authorities inside that protocol, and asks the lifecycle spec for three |
| -things (§6), two of which are rollout prerequisites. |
| - |
| -### 3.3 Readers that change |
| - |
| -The record stays the provisioning authority and the registry stays |
| -derived. The readers this spec changes (B-15): |
| - |
| -- **Box:** `ReconcileAliases` skips any claim with `acq.state == |
| - pending` before every derivation (alias, wildcard, catch-all, paths). |
| - The static canary mint checks `byHost` (§4.6). Nothing else. |
| -- **Console:** Settings and `domains list` render pending as |
| - "attaching"; `SetClaimCapabilities`, `SetClaimPaths` and the repoint |
| - path refuse pending; `apexHolder`, `hostHolder` and |
| - `ReportApexSquats` read the lifecycle claim instead of scanning. |
| +- `family`, `acq.id`, `account`, `app`, `app_incarnation`, `first_route` |
| + and `platform` are **immutable** for the life of the object. A takeover |
| + (§4.1) replaces the object by CAS and so replaces `acq.id`; nothing |
| + else does. |
| +- `acq.state` moves only `pending → committed → releasing`, each by CAS |
| + on the ETag read, and `releasing` ends in `DeleteIfMatch`. A `pending` |
| + object may also be deleted by its own writer's rollback or by the |
| + sweep (§5). Nothing moves backwards. |
| +- **Held** means the object exists in any state. Only a `committed` |
| + object is **owned**: materialised by readers, reported as attached, |
| + eligible for repoint, capability and path changes, and for release. |
| +- A legacy object (no `family`) is an alias when `route != ""` and a |
| + first-route marker when `route == ""`. The backfill stamps every legacy |
| + object (§8); above the marker none remains. |
| +- Readers below this release refuse nothing (schema stays 1); the floor |
| + (§8) guarantees no such reader folds a pending object. |
| + |
| +### 3.2 The per-app index: `apps/<acct>/<app>/owned/<host>.json` |
| + |
| +`{host, acq_id, family}`. Written **before** the ownership commit and |
| +deleted **after** the ownership delete, both conditional. An index entry |
| +without an owned object is harmless (release finds nothing); an owned |
| +object without an index entry cannot arise except by a crash between |
| +index write and commit, which the sweep repairs from the full fold (§5). |
| +Purge and materialization enumerate an app's hosts from this prefix with |
| +a plain listing and a GET per entry, and **fail closed**: any read error |
| +aborts the purge step (C-3). |
| + |
| +### 3.3 The app row's incarnation |
| + |
| +`apps` gains `incarnation TEXT` (random, set on insert) and |
| +`apex_acq TEXT` (the apex acquisition id, §4.6). Every door reads the |
| +app's incarnation at its gate and writes it into the acquisition. A |
| +commit re-reads the row and refuses when the incarnation differs or the |
| +row is not live (C-3): a writer paused across a purge and a recreate of |
| +the same name cannot commit against the new app. |
| + |
| +### 3.4 The lifecycle claim, `control/hosts/<host>.json` (rev 6.16) |
| + |
| +Unchanged in shape. This spec uses it as rev 6.16 specifies: §5.1 create |
| +(admission registration as a lease, `CreatedAt` assigned by the console, |
| +CAS from free or absent, `CreateRecord`, CAS to serving, recovery after |
| +five minutes), §5.2 delete and park, §5.3 purge with its reservation |
| +journal, drain and purge fence, §6.2a the holderless sweep, §9 adoption, |
| +§10 the floor. Its holder's incarnation is the binding to ownership: |
| +`acq.incarnation == holder.incarnation` for every serving host. The |
| +object is never deleted (C5a), so a released hostname keeps its |
| +`retired_incarnations`, `quiet_after` and `lost` history. What this spec |
| +asks of that spec is in §6. |
| + |
| +### 3.5 Readers |
| + |
| +- **Boxes.** `ReconcileAliases` derives aliases, wildcards, catch-alls |
| + and paths from `committed` objects of family `alias` only. The apex |
| + mint and the static canary mint run only when the `apex` object for |
| + (account, app) is `committed`, and the canary mint checks `byHost` |
| + before `Upsert`. Instance and sidecar routes derive from records and |
| + lifecycle admission as today; boxes never read `instance` or `sidecar` |
| + ownership. Fold cost and the alias index are §4.8. |
| +- **Console.** Settings and `domains list` show `pending` as |
| + "attaching" and `releasing` as "detaching". `SetClaimCapabilities` and |
| + `SetClaimPaths` refuse anything but a committed `alias`. `apexHolder`, |
| + `hostHolder` and `ReportApexSquats` are replaced by one GET. |
| ## 4. The design |
| -### 4.1 The rule |
| - |
| -Each hostname has exactly one **ownership authority** by family: |
| - |
| -| Family | Authority | Held means | Atomic step | |
| -|---|---|---|---| |
| -| record-backed (instance, replica, static record, sidecar) | lifecycle claim | `holder != nil` or `op.kind != ""` | `Update(BeginCreate)` from absent or free | |
| -| alias | `domains/<host>.json` | a claim exists (pending or committed; a pending one is held against everyone but its own acquisition) | `AttachClaim`'s create-only `Put` | |
| -| minted (apex, static canary host) | the app named in the host | the (account, app) row exists, live, trashed or pending | the console's unique row insert | |
| - |
| -A host can be in only one family at a time. The cross-family rule, for |
| -every writer: |
| - |
| -> **Write your own authority first, unverified. Then re-read every other |
| -> authority for the host. If any is held, delete exactly the object |
| -> version you wrote and answer 409. Otherwise commit: the record, the |
| -> committed alias, the verified app row.** |
| - |
| -Why it is enough: S3 gives strong read-after-write consistency for |
| -`PutObject` and `GetObject` on a key. Take two writers W1 (family X) and |
| -W2 (family Y) racing for host H, writing at t1 and t2. If t1 < t2, W2's |
| -re-read of X is after t2 > t1 and sees W1's object. Symmetrically for |
| -t2 < t1. So at least one backs off. Both may back off; that is two 409s |
| -and no residue, and either retry then wins. A deterministic tie-break was |
| -considered and rejected: the side that must yield cannot tell whether |
| -the other side also saw it, so any rule that lets one side keep going is |
| -unsound in some interleaving. |
| - |
| -Corollaries: |
| - |
| -- **Nothing is committed before verification.** An unverified authority |
| - (a `creating` lifecycle op, a pending alias claim, a pending app row) |
| - reserves the name against third parties but is not success, is not |
| - materialised, and is not what a same-owner retry may report as done. |
| - A retry that meets its owner's unverified authority finishes the |
| - verification itself or refuses; it never short-circuits (A-3, A-4). |
| -- **Verification covers every other authority**, not only the other |
| - family: an alias verifies the lifecycle claim **and** the app row of |
| - the host's label; a create verifies `domains/` **and** the app row; |
| - an app row verifies the lifecycle claim **and** `domains/` (B-5). |
| -- **Cleanup is conditional.** Every back-off and rollback deletes with |
| - `release.DeleteIfMatch` on the ETag the write returned, and refuses |
| - when the backend cannot enforce it (A-5). |
| -- **Displacing a writer is a CAS.** Taking over a stale acquisition |
| - rewrites the object under `If-Match`, so the displaced writer's own |
| - commit or cleanup fails its condition (B-1). |
| -- **Family does not change in place.** Conversion is detach then attach, |
| - each through its own door (A-4). |
| - |
| -The pre-write check stays as a cheap filter for the sequential case, but |
| -the design never relies on it. |
| - |
| -### 4.2 Instance create |
| - |
| -Replaces `hostHolder` and `hostClaimHolder` in `handleAPIInstanceCreate`. |
| -Replicas, static records and canary instances go through the same door. |
| -The protocol is the lifecycle spec's §5.1 **in full**, with the |
| -cross-family reads inserted at two points (A-9). |
| - |
| -**Placement.** Every ownership read and write sits after every other |
| -validation the door performs, immediately before the record write, where |
| -`leftoverReplicaRefusal` sits today. An invalid request answers its 400 |
| -whether or not the host is held (A-13). |
| - |
| -**Gate.** Above the floor and before the marker (§8.2) the door answers |
| -503 `msgOwnershipFreeze` and does nothing. Below the floor it runs |
| -today's path. |
| - |
| -1. Shape and the namespace rule of §4.6, early. The rule's own-app |
| - exception applies: an app may create at its own apex label. |
| -2. **Pre-check, cheap, after all validation.** `GET domains/<host>`: any |
| - claim is a refusal (§7). One `registry.Get` on this box for an |
| - operator route with no record. `edgesfile.DeploymentRoutes` for a |
| - route a live box publishes that no record explains; best-effort and |
| - permissive on read failure, as the attach door uses it. |
| -3. **Lifecycle steps 0 and 1.** `hostclaim.Read`, keep the ETag or |
| - absence; register the create in `admission.json`, holding its lease; |
| - assign `CreatedAt`. |
| -4. **Lifecycle step 2.** `Update(BeginCreate)` conditional on step 3's |
| - read. On `ErrConflict`: re-read once to classify (§7), answer 409 for |
| - a hold or a fence, and for free-again **restart from step 3** with a |
| - new registration and a new `CreatedAt`, at most once per request. |
| - Never retry the CAS alone, never wait inside the handler. |
| -5. **Verify.** `GET domains/<host>` again, and `AppNameState` for the |
| - host's label when the host is `<l>.<sqid>.<apps>`. Any `domains/` |
| - object, or a live, trashed or pending app at the label **other than |
| - the creating app itself** (B-10): `Update(CancelCreate)` (legal: the |
| - op is still `creating`), release the registration, 409. |
| -6. **Lifecycle step 3.** `CreateRecord`, create-only. |
| -7. **Lifecycle step 4.** `Update(CompleteCreate)`. From here the host is |
| - serving and the only way back is delete and park (§4.3). Nothing after |
| - this step may call `CancelCreate` (A-6). |
| - |
| -The recovery pass completes any `creating` claim whose record is present |
| -without repeating door predicates: a record exists only after step 5 |
| -verified, and the one write that could still race (an app row or alias |
| -appearing after step 5) is caught on the other side, because those |
| -writers write and then read this claim (§4.4, §4.6). |
| - |
| -### 4.3 Instance delete |
| - |
| -Lifecycle §5.2 unchanged: CAS `deleting`, `DeleteIfMatch` the record, |
| -park, free. Ownership is released when the claim reaches free. A host |
| -that is `deleting` or `parking` is held for the create door. |
| - |
| -### 4.4 Alias attach and detach |
| - |
| -In `handleAPIDomainAttach`'s alias branch and `attachDomainAs`'s alias |
| -branch, around the existing `AttachClaim`. Policy and |
| -`deploymentRouteTaken` stay first and in their existing order; the Task 5 |
| -ordering rules stand, and the new reads sit below them, immediately |
| -around the write. Every refusal below is §7's mapping; the sentences |
| -named here are its rows. |
| - |
| -1. `GET domains/<host>` with its ETag `E0`. |
| - - **Committed, own, same route:** done, 200. The titogo reload steady |
| - state: one GET. |
| - - **Committed, own, `kind alias`, different route:** repoint by CAS on |
| - `E0`, preserving `acq`. Ownership was decided at commit. |
| - - **Committed, own, `kind first-route`:** refuse (§7: own-app |
| - conflict). No in-place conversion. |
| - - **Pending, `kind alias`, own acquisition id:** a retry of this |
| - request. Resume at step 4 with `E0`. |
| - - **Pending, `kind alias`, own owner, another id, `started_at` older |
| - than the acquisition timeout (60 s):** **takeover.** CAS the whole |
| - claim on `E0` with this request's id and `started_at`, keep `E1` = |
| - the result, then step 4. The displaced writer's commit or cleanup |
| - now fails its `If-Match` (B-1). |
| - - **Pending, own owner, another id, younger than the timeout:** refuse |
| - (§7: attach in progress). |
| - - **Pending, `kind first-route`, own owner:** refuse (§7: own-app |
| - conflict, "a first-route attach of this host is in progress"). An |
| - alias request never touches a first-route acquisition (B-1). |
| - - **Any claim of another owner:** refuse (§7: foreign). |
| - - **Absent:** continue. |
| -2. **Pre-check, cheap.** `hostclaim.Read`: a holder or any op is a |
| - refusal. `AppNameState` of the label when the host is |
| - `<l>.<sqid>.<apps>`: any row other than the attaching app's own is a |
| - refusal. |
| -3. **Write pending.** `Put(domains/<host>, claim{acq: {id, alias, |
| - pending, started_at}}, "")`, create-only. Keep `E1`. A CAS conflict |
| - re-reads and re-enters step 1 once. |
| -4. **Verify.** `hostclaim.Read` again, and `AppNameState` of the label |
| - again (B-5), and the attaching app's own row is still live. Any hold, |
| - any other app's row (live, trashed or pending), or own app not live: |
| - `DeleteIfMatch(domains/<host>, E1)`, refuse. A failed conditional |
| - delete leaves a pending claim, which never materialises and which the |
| - sweep clears (§5). |
| -5. **Commit.** `Put(domains/<host>, claim{acq.state: committed}, E1)`. |
| - Keep `E2`. A conflict means a takeover or cleanup moved the object: |
| - re-read and re-enter step 1 once. |
| -6. Audit and serial, only on a commit. |
| - |
| -Detach of an alias: `GET` the claim with ETag, owner-matched, then |
| -`DeleteIfMatch` on that ETag, quiet on mismatch and on a lost condition |
| -(A-5). A pending claim of the caller's own may be detached the same way. |
| - |
| -### 4.5 First-route attach and detach |
| - |
| -The first-route branch writes a claim with `Route == ""` and `kind: |
| -first-route`, then a static record. The record makes the host |
| -record-backed; the lifecycle holder is its authority and the `domains/` |
| -claim is the attach surface's marker. The marker is committed only when |
| -the record is serving, and the commit stamps `acq.incarnation` with the |
| -record's `hostclaim.Incarnation(account, app, host, CreatedAt)` (B-6). |
| - |
| -**Attach**, after policy, `attachChannel` and `deploymentRouteTaken`: |
| - |
| -1. `GET domains/<host>` with `E0`. The branch table (B-11): |
| - - **Absent:** write pending `{id, first-route, pending}` create-only, |
| - keep `E1`. Continue at step 2. |
| - - **Pending first-route, own id:** retry; `E1 = E0`, continue at step |
| - 2. |
| - - **Pending first-route, own owner, another id, older than the |
| - timeout:** takeover by CAS as in §4.4, then step 2. |
| - - **Committed first-route, own:** read the lifecycle claim. Holder is |
| - this app's record with `attachWrote` shape and |
| - `holder.incarnation == acq.incarnation`: idempotent success, no |
| - write. Anything else: refuse (§7: own-app conflict, operator |
| - repair). |
| - - **Any alias claim, own or not, or a pending first-route younger than |
| - the timeout by another id:** refuse per §7. |
| -2. `hostclaim.Read`. By state: |
| - - **absent or free:** run §4.2 steps 3 to 7 for the static record. Its |
| - verify (step 5) finds this call's own pending marker, recognised by |
| - id, which is not a conflict; any other `domains/` state or app row |
| - is. If the CAS loses or the verify fails: `DeleteIfMatch(domains/, |
| - E1)`, refuse. |
| - - **`creating` with holder this app's record (a crashed earlier |
| - attempt):** do not write; answer 409 "attach in progress, retry in |
| - a minute". The recovery pass settles the record half within five |
| - minutes; the next retry meets `serving` or `free`. |
| - - **`serving` with holder this app's static record of `attachWrote` |
| - shape:** a lost commit. Go to step 3. |
| - - **held by another, or any other op:** `DeleteIfMatch(domains/, E1)`, |
| - refuse per §7. |
| -3. **Commit** the marker: `Put(..., claim{committed, incarnation: |
| - holder.incarnation}, E1)`. |
| -4. Audit and serial. |
| - |
| -`attachWrote` and its shape restriction stay (A-4): a record is this |
| -door's only when it is kind static, no slug, no region, no pool, **and** |
| -the lifecycle holder is this app's record **and** the committed marker's |
| -incarnation matches. An instance record of the same app can sit at the |
| -same key under a custom manifest domain, and detach must never gain |
| -authority over it. |
| - |
| -**Detach** of a first-route marker (B-6): |
| - |
| -1. `GET domains/<host>` with `E0`; owner-matched, `kind first-route`, |
| - committed, with incarnation `I`. Anything else: the alias detach path |
| - or §7's refusal. |
| -2. `hostclaim.Read` with its ETag. `holder.incarnation != I`, or any op |
| - other than serving: refuse (§7: operator repair). The marker names a |
| - record that is not the one serving. |
| -3. Lifecycle delete (§4.3), with `BeginDelete` conditional on the claim |
| - ETag read in step 2, so a successor holder fails the CAS. |
| -4. `DeleteIfMatch(domains/<host>, E0)`. |
| - |
| -A detach that paused between steps 1 and 3 while another detach and a |
| -new attach ran meets a different incarnation in step 2 or a failed CAS |
| -in step 3, and retires nothing. |
| - |
| -### 4.6 Minted hosts and the namespace rule |
| - |
| -The apex `<app>.<sqid>.<apps>` and a static app's canary host |
| -`<name>.<app>.<sqid>.<apps>` are minted by boxes from the app's name and |
| -channel pointers. They have no record and no claim, and should not get |
| -one: the name is the ownership. |
| - |
| -**The namespace rule**, decided from the name and the account's app |
| -rows alone, so it is never an oracle (A-7): |
| - |
| -| Host shape under `<sqid>.<apps>` | Belongs to | Allowed | |
| -|---|---|---| |
| -| `<l>.<sqid>.<apps>` where `<l>` is a live, trashed or pending app of the account | app `<l>` | an instance of `<l>` itself (the own-app exception, B-10), nothing else: no alias, no first-route, no instance of another app | |
| -| `<l>.<sqid>.<apps>` where `<l>` is no app | nobody yet | an instance of any app of the account (today's canary and slug hosts); an alias only on deployments with reservation off, as today, and only with the app-row verification of §4.4 | |
| -| `<x>.<app>.<sqid>.<apps>` | app `<app>` | nothing through any door. Minted only | |
| -| a manifest `Domain` that is `<app>.<sqid>.<apps>` or deeper | app `<app>` | refused for any other app at the manifest door and at the create door | |
| - |
| -Applied by: the instance create door (§4.2 step 1 and step 5), both |
| -attach branches (before any write, and again at verify), and the |
| -manifest door. |
| - |
| -**The apex gate** follows the write-then-verify rule with an operation |
| -id (A-10, B-9): |
| - |
| -1. `claimApp`, rename and move each start by inserting a row in |
| - `pending_app_ops` `{op id, kind, app id, from, to, journal}` and, in |
| - the same transaction, the name reservation: for create, the app row |
| - with `pending = op id`; for rename and move, a **second** row for `to` |
| - with `pending = op id` while the `from` row stays live. Both names are |
| - reserved until the operation finishes (B-9). The unique (account, app) |
| - constraint is the atomic step. |
| -2. Read the lifecycle claim of the new apex and `GET domains/<apex>`. |
| - Either held (a holder, an op, or any claim, pending or committed): |
| - compensate, in one transaction conditional on `pending_app_ops.op id` |
| - still present: delete the pending row, delete the op row, refuse. |
| -3. Finalise, in one transaction conditional on the op id: for create, |
| - clear `pending`; for rename and move, apply the related-row changes |
| - the journal lists (credentials, grants, store.go:1801-1905), drop the |
| - `from` row (rename) or re-home it (move), clear `pending` on `to`, |
| - delete the op row. A handler that paused and finds its op row gone |
| - does nothing: recovery already settled it. |
| - |
| -The instance and alias doors do the other half (§4.2 step 5, §4.4 step |
| -4). Rows and ops left pending by a crash are settled by the app-name |
| -sweep (§5) with the same conditional transactions. A pending row is not |
| -an app a member can deploy to: the doors treat it as "being created". |
| +### 4.1 The acquisition protocol |
| + |
| +Every door that puts a hostname on the platform runs the same four |
| +steps; the family-specific work sits in step 3. |
| + |
| +1. **Gate and validation.** Every check the door makes today, in its |
| + order, including the app gate (which yields `app_incarnation`), shape, |
| + policy (`domainHostPolicy`), the namespace rule (§4.6), and for |
| + aliases `deploymentRouteTaken`. No ownership read happens before this |
| + step completes (A-13, B-14). |
| +2. **Acquire.** `GET domains/<host>` with its ETag `E0`, classify (§7), |
| + then one of: |
| + - **absent:** `Put(..., {family, account, app, app_incarnation, acq: |
| + {id, pending, started_at, by}}, "")`. Keep `E1`. A conflict re-reads |
| + once and re-classifies. |
| + - **own `pending`, own id:** resume with `E1 = E0`. |
| + - **own `pending`, same family and owner, other id, `started_at` |
| + older than the acquisition timeout (60 s), and not referenced by a |
| + lifecycle `creating` op (§4.2):** **takeover.** CAS the whole object |
| + on `E0` with this request's id and `started_at`; keep `E1`. The |
| + displaced writer's later commit, rollback and index write fail their |
| + conditions. |
| + - **own `committed`, same family:** the idempotent branch of the |
| + family (repoint, no-op, or "already exists"). |
| + - **anything else:** refuse per §7. No write. |
| +3. **Write the index entry** `owned/<host>.json {host, acq_id, family}`, |
| + then do the **family's work** (§4.2, §4.3, §4.6). The family's work |
| + ends with the object's `acq.incarnation` known where the family has |
| + one. |
| +4. **Commit.** Re-read the app row: live, same `app_incarnation`, |
| + otherwise roll back. `Put(..., {state: committed, incarnation}, E1)`. |
| + A conflict means a takeover or a sweep moved the object: roll back the |
| + family's work, refuse "attach in progress, retry". |
| + |
| +**Rollback** of a failed acquisition undoes the family's work in reverse, |
| +then `DeleteIfMatch(owned/<host>, its ETag)`, then |
| +`DeleteIfMatch(domains/<host>, E1)`. Every delete is conditional |
| +(`release.DeleteIfMatch`, conditional_delete.go:28) and refuses where the |
| +backend cannot enforce it. A rollback that cannot finish leaves a |
| +`pending` object for the sweep (§5). |
| + |
| +**Why it is atomic.** There is one key per hostname and one create-only |
| +write. Two writers for one host, any families, cannot both get `E1`. |
| +There is no cross-authority verification and no interleaving proof to |
| +maintain. The remaining races are same-owner retries and reclaimed |
| +reservations, which the immutable id, the CAS takeover and the lifecycle |
| +binding (§4.2) handle. |
| + |
| +### 4.2 Family `instance` and `sidecar` (record-backed hosts) |
| + |
| +The family's work is the lifecycle create protocol, rev 6.16 §5.1, in |
| +full, with the ownership acquisition bound into it: |
| + |
| +1. **Lifecycle step 0.** `hostclaim.Read`, keep the ETag or absence; |
| + register the create in `admission.json` as a lease. |
| +2. **Lifecycle step 1.** Assign `CreatedAt`. The incarnation |
| + `I = hostclaim.Incarnation(account, app, host, CreatedAt)` is now |
| + known. |
| +3. **Lifecycle step 2.** `Update(BeginCreate)` conditional on step 1's |
| + read, with `op.intent.ownership = acq.id` (§6 ask 1). On |
| + `ErrConflict`: re-read once; a hold or a purge fence refuses per §7; |
| + free again restarts from step 1 with a new registration and |
| + `CreatedAt`, at most once per request. Never wait in the handler. |
| +4. **Lifecycle step 3.** `CreateRecord`, create-only. |
| +5. **Commit ownership** (§4.1 step 4) with `acq.incarnation = I`. If the |
| + CAS fails, the reservation was reclaimed (a takeover or the sweep): |
| + `Update(CancelCreate)` (legal, the op is still `creating`), |
| + `DeleteIfMatch` the record, release the registration, refuse. Nothing |
| + reaches serving under a reservation it does not hold. |
| +6. **Lifecycle step 4.** `Update(CompleteCreate)`. From here the host is |
| + serving and the only way back is release (§4.4). |
| + |
| +**Binding invariant (O1).** A `serving` lifecycle holder exists only |
| +while `domains/<host>` is `committed` or `releasing` with |
| +`acq.incarnation == holder.incarnation`. Maintained by: commit before |
| +`CompleteCreate` (step 5 before 6); the recovery pass's completion rule |
| +(§6 ask 1): a `creating` claim with its record present is completed only |
| +if the ownership object's `acq.id` equals `op.intent.ownership` and its |
| +state is `pending` or `committed`; otherwise the pass cancels and deletes |
| +the record. And by release (§4.4), which deletes ownership only once the |
| +lifecycle claim is free. |
| + |
| +**A delayed creator** that acquired `pending`, paused, lost its |
| +reservation (sweep after 60 s, or takeover) and resumes: its |
| +`BeginCreate` may still succeed if the lifecycle claim is free, and its |
| +record may land, but its commit CAS fails (step 5) and it cancels. If it |
| +pauses again between steps 4 and 5 for over five minutes, the recovery |
| +pass finds `op.intent.ownership` naming an acquisition the ownership |
| +object no longer has and cancels it. A successor meanwhile meets a |
| +`creating` claim and is refused "in progress"; it retries. The sweep |
| +never deletes a `pending` object that a `creating` lifecycle op names |
| +(§5), so a live creator's reservation is not reclaimed under it. |
| + |
| +Sidecars: the same steps with `CreateSidecarRecord` and family |
| +`sidecar`; `holderKeyMatches` already binds the sidecar key and host. |
| + |
| +### 4.3 Family `alias`, and the first-route attach |
| + |
| +**Alias.** Steps 1 and 2 of §4.1; `deploymentRouteTaken` stays |
| +immediately before the acquire write, as the Task 5 ordering rule |
| +requires. The family's work is nothing: the index entry, then commit |
| +(§4.1 step 4), whose app-row re-read is what refuses a writer paused |
| +across a purge (C-3). Two PUTs on a fresh attach; the committed object |
| +carries `route`, and `wildcard`, `catchall`, `paths` are written later by |
| +their own owner-gated CAS, which requires `committed`. |
| + |
| +Idempotent branch (own committed `alias`): same route, no write, 200; |
| +different route, CAS the repoint on `E0` preserving every immutable |
| +field. This is titogo's reload steady state: one GET per host. |
| + |
| +**First-route.** The host becomes record-backed, so the family is |
| +`instance` with `first_route: true`, and the work is §4.2 for a static |
| +record (`attachWrote` shape: kind static, no slug, region or pool). The |
| +flag is what the domains surface dispatches on at detach (§4.5). An own |
| +committed `instance` without the flag at that host is the instance |
| +door's, and the attach refuses (§7 row 5). Conversion between alias and |
| +first-route is detach then attach. |
| + |
| +**Platform domains** (`--platform`) take either branch with |
| +`platform: true` recorded; the operator gate stays name-derived |
| +(`domainHostPolicy`) and is applied before step 2. |
| + |
| +### 4.4 Release (instance and sidecar delete) |
| + |
| +Release follows lifecycle retirement and is recoverable. The |
| +instance delete door, the sidecar delete door, first-route detach |
| +(§4.5), purge (§5) and `provision.RetireRecord`'s callers all run it. |
| + |
| +1. `GET domains/<host>` with `E0`: `committed`, this app, this |
| + `app_incarnation`, family `instance` or `sidecar`, incarnation `I`. |
| + Anything else per §7. |
| +2. `hostclaim.Read` with its ETag: `holder.incarnation == I`, op |
| + serving. Otherwise refuse (§7 row 7): the record serving is not the |
| + one this ownership names. |
| +3. **CAS ownership `committed → releasing`** on `E0`, writing |
| + `release_op = <new lifecycle op id>`. From here the host is "being |
| + released" to every door and nothing can repoint or re-attach it. |
| +4. **Lifecycle §5.2**: `BeginDelete(release_op, ...)` conditional on the |
| + ETag read in step 2, `DeleteIfMatch` the record, `parking` with |
| + obligations, each owner's chore, `free`. |
| +5. When the lifecycle claim is **free**, or names a different holder |
| + with no op naming `release_op`: `DeleteIfMatch(domains/<host>, E)`, |
| + then `DeleteIfMatch(owned/<host>)`. The lifecycle claim itself is |
| + never deleted (C5a). |
| + |
| +Recovery is the **ownership sweep** (§5): a `releasing` object whose |
| +lifecycle claim is still `serving` with holder `I` restarts step 4 with |
| +`release_op` (the member already asked, durably); one whose lifecycle |
| +claim is `deleting` or `parking` under `release_op` waits for lifecycle |
| +recovery; one whose lifecycle claim is free does step 5. A new creator |
| +for the host meets a held object (`releasing`) and is refused "being |
| +released" until step 5 completes; a lifecycle claim in `parking` would |
| +refuse it anyway. |
| + |
| +### 4.5 Detach dispatches by family |
| + |
| +`handleAPIDomainDetach` and `detachDomain`, after the control-plane gate |
| +and the name-derived platform gate: |
| + |
| +| `GET domains/<host>` says | Action | |
| +|---|---| |
| +| absent, another owner, another `app_incarnation`, or `pending` of another request | 404 "not attached" (today's neutrality) | |
| +| own `alias`, committed | CAS `releasing` on `E0`, then `DeleteIfMatch`, then the index entry. Audit, serial | |
| +| own `alias`, pending, own request | roll back as §4.1 | |
| +| own `instance`, `first_route: true`, committed | release (§4.4) | |
| +| own `instance` without `first_route`, or `sidecar`, or `apex` | 409 "this hostname is an instance (or sidecar, or the app's address) of this app; use its own door" | |
| +| own, `releasing` | 200 "already detaching" | |
| + |
| +`Route == ""` is read by nothing (C-5's root). The `releasing` state is |
| +what makes a crashed first-route detach finishable: a retry finds |
| +`releasing` and the sweep drives §4.4 to its end whether or not the |
| +lifecycle holder still matches, because the intent was recorded before |
| +the lifecycle delete began. |
| + |
| +### 4.6 Family `apex`: app names, and the namespace rule |
| + |
| +The apex `<app>.<sqid>.<apps>` is owned by the app, and the ownership |
| +object is acquired by the console when the name is taken. A canary host |
| +`<name>.<app>.<sqid>.<apps>` is **covered** by the apex object: boxes |
| +mint it only when the apex object is committed for (account, app), and |
| +every door refuses hosts under another app's apex. No per-canary object. |
| + |
| +**App create** (`claimApp`): |
| + |
| +1. Gate and validation as today. |
| +2. Acquire `domains/<app>.<sqid>.<apps>` as family `apex`, `pending` |
| + (§4.1 step 2). A hold by another app or an alias at the apex refuses |
| + the name (today's `apexHolder` answer, from one GET). |
| +3. Family work: insert the app row in one transaction with |
| + `incarnation` and `apex_acq = acq.id`. A unique-constraint failure |
| + (the name was taken between gate and insert) rolls back the |
| + acquisition. |
| +4. Commit (§4.1 step 4), `app_incarnation` now known. |
| + |
| +**Rename and move** reserve both names until finished (B-9, C-4): |
| + |
| +1. Acquire the **new** apex `pending`, as above. |
| +2. In one transaction: insert `pending_app_ops {op_id, kind, app_id, |
| + from, to, journal}` and mark the source row `closed = op_id`. A closed |
| + row keeps its name and its apex object but **every door refuses new |
| + work for it** (create, attach, ship, add), and in-flight work is |
| + drained: the door re-reads `closed` at its commit step and rolls back |
| + if set. Then re-validate the source is empty (`appDeployEmpty`, |
| + appsettings.go:832's check, now after the close). |
| +3. Finalise in one transaction conditional on `op_id`: rename or re-home |
| + the row, apply the journaled related rows (credentials, grants), |
| + clear `closed`, delete the op row. Then CAS the **old** apex object |
| + `releasing` and delete it; commit the new one with the row's |
| + incarnation. |
| +4. Compensation, conditional on `op_id`: clear `closed`, delete the op |
| + row, roll back the new apex acquisition. |
| + |
| +The app-name sweep settles pending ops older than a minute by the same |
| +conditional transactions. `console migrate-app` (cmd/console/main.go:228) |
| +runs the move path; it does not call `MoveApp` directly (C-1). |
| + |
| +**Trash and purge.** A trashed app keeps its apex object until purge |
| +releases it (§5), so its name stays reserved as today. |
| + |
| +**The namespace rule**, from the name and ownership alone: |
| + |
| +| Host under `<sqid>.<apps>` | Allowed | |
| +|---|---| |
| +| `<l>.<sqid>.<apps>` | whatever the ownership object says: an `apex` of app `<l>` admits only `<l>`'s own instance record at that host (the own-app exception, B-10); absent admits any family | |
| +| `<x>.<app>.<sqid>.<apps>` | nothing through any door; minted only, under a committed apex of `<app>` | |
| +| a manifest `Domain` of `<app>.<sqid>.<apps>` or deeper | refused for any app but `<app>` | |
| **Apps with no SQL row.** The box mints an apex for any (account, app) |
| -that has routes (`candidateApps`), including an app that exists only in |
| -the bucket. Such a name is reserved by nothing (B-10). The backfill |
| -(§8.1) lists every app prefix with records or routes and no row, and |
| -**blocks completion** until each is registered (`carlos ops apps |
| -register <sqid> <app>`, which inserts the row through the apex gate) or |
| -excluded from minting by an operator deleting its records. The startup |
| -owners backfill at store.go:676 is legacy-only and is not that path. |
| - |
| -**The static canary mint** must not overwrite a route. The box-side |
| -change: before `reg.Upsert(nr)` at domains/reconcile.go:337, skip and |
| -log when `byHost` has the host and the existing route's app or account |
| -differ from the mint's. |
| - |
| -### 4.7 Replicas, sidecars, fleets, pools, moves |
| - |
| -- A replica is a record with its own host, created by the same door: |
| - §4.2 applies unchanged. Its lifecycle is the moves spec's. |
| -- Sidecars: the lifecycle claim covers them (family `sidecar`). For |
| - ownership nothing is needed: the host embeds account and app, the |
| - attach door refuses the suffix, and the edge never routes it. |
| -- Fleets, pools and placement change `Region` on an existing record and |
| - never the host or the holder. |
| -- Moves change `op` and `holder.location`, never `holder.account` or |
| - `holder.app`. A host under a move is held; the create door refuses. |
| - |
| -### 4.8 Account migrate |
| - |
| -`carlos accounts migrate --app A --to <dst>` copies the app's objects to |
| -the destination prefix create-only and deletes nothing. Records are |
| -copied with their body unchanged, so the destination copy names the |
| -source account and `GetRecord` refuses it (provision.go:1145). The verb |
| -has never produced a valid destination record (A-12). |
| - |
| -Under this design a host's lifecycle holder names the source record's |
| -exact key, the incarnation hashes the source account, `holder.location` |
| -is incarnation-scoped, and a sidecar's host embeds the source sqid. A |
| -transfer needs a protocol for record identity, storage location, |
| -execution authority, sidecars and `domains/` claims, and it is not |
| -designed here. |
| - |
| -**Rule.** `accounts migrate`: |
| - |
| -1. Closes the source app's admission (`admission.json` to `closed`, the |
| - lifecycle spec's own quiesce: create step 0 then refuses), and waits |
| - out every outstanding registration lease (B-13). |
| -2. Lists the app's `instances/` and `sidecars/` prefixes. Any record: |
| - reopen admission, refuse with a sentence naming this spec. |
| -3. Otherwise migrates release objects and pointers as today, **without** |
| - stamping routes (an app with no records has no provisioned routes to |
| - stamp; an operator route is the operator's), then reopens admission. |
| - |
| -The follow-up thread owns the transfer and the `domains/` rebind. |
| - |
| -## 5. Crash recovery |
| - |
| -Each crash window, what it leaves, and who clears it. No step deletes |
| -anything it did not create or cannot name by exact version, and nothing |
| -frees a host that has a record with the holder's incarnation. |
| - |
| -| Crash after | Left behind | Cleared by | |
| +with routes, including bucket-only apps. Above the marker a mint needs a |
| +committed `apex` object, which only the console writes, so bucket-only |
| +apps stop minting until registered. `carlos ops apps register <sqid> |
| +<app>` is a **migration-only registration**, admitted during the freeze |
| +(C-6): it acquires the apex object requiring exact same-owner evidence |
| +(records or routes for exactly that account and app, no other authority |
| +at the host) and inserts the row through the apex gate. The startup |
| +owners backfill (store.go:676) is retired above the floor (C-1). |
| + |
| +### 4.7 Moves, placement, migrate |
| + |
| +Moves and placement change `op`, `holder.location` and `Region`, never |
| +the owner. A host under a move is `committed` and its lifecycle claim is |
| +not serving; release refuses until the move ends. |
| + |
| +`accounts migrate`: closes the source app's admission and drains its |
| +registrations per rev 6.16 §5.3 step 1b (not a lease wait, B-13), lists |
| +`instances/` and `sidecars/`, refuses with a sentence naming this spec if |
| +any record exists, otherwise migrates release objects and pointers only, |
| +without stamping routes, and reopens admission. Ownership transfer across |
| +accounts (record identity, location, sidecar hosts, aliases) is a |
| +follow-up. |
| + |
| +### 4.8 Fleet folds |
| + |
| +Today every box lists `domains/` through `ClaimCache` on the fast-converge |
| +leg (one `ListMod` per pass; `ListObjectsV2` pages at 1000 keys). After |
| +rev 4 the prefix holds every hostname: on flagship, aliases plus |
| +instances, sidecars and apexes, several thousand objects and several |
| +pages per pass per box. |
| + |
| +- **Family filter.** Boxes derive only from `alias` and `apex` objects; |
| + `instance` and `sidecar` objects are skipped on the key list's body |
| + read, but still listed. |
| +- **Derived alias index.** The console's ownership sweep writes |
| + `control/alias-index.json` `{generation, serial, aliases: [committed |
| + alias objects' routing fields], apexes: [(account, app)]}` on every |
| + alias or apex commit and release (the same moments that bump the |
| + serial). Boxes fold the index with a conditional GET (`If-None-Match` |
| + on its ETag, 304 in the steady state) instead of listing `domains/`. |
| + The full fold remains the sweep's consistency check and the fallback |
| + when the index is missing or older than ten minutes. |
| +- **Switch.** The index is written from day one; boxes prefer it once |
| + `domains/` exceeds one page, which the sweep records in the index. |
| + |
| +Readers of the index are the alias derivation and the two mints. Nothing |
| +box-side writes `domains/`. |
| + |
| +## 5. Crash recovery and the sweeps |
| + |
| +Two console passes, each every minute, each reading exact objects with |
| +ETags before acting. `ClaimCache` and listings are discovery only. |
| + |
| +**The ownership sweep**, over `domains/`: |
| + |
| +| State found | Lifecycle says | Action | |
| |---|---|---| |
| -| §4.2 step 4 (claim `creating`, no record) | an orphaned claim | the lifecycle recovery pass (§5.1 step 5): record absent, `CancelCreate`, incarnation retired. Never steals: a record present with this incarnation completes instead | |
| -| §4.2 step 6, before `CompleteCreate` | claim `creating`, record present | the same pass completes it. Safe without re-running door predicates: the record exists only after step 5 verified | |
| -| §4.2 step 5 back-off, before `CancelCreate` | claim `creating`, another authority present | the pass cancels (record absent). The other writer was earlier and stands | |
| -| §4.4 step 3 or 4 (pending alias, not committed) | a pending claim | never materialised. The **claim sweep** (console, every few minutes): `ClaimCache` for discovery only; for each pending claim older than the timeout, `GET` the exact object with its ETag, read the lifecycle claim and the label's app row, then **delete only**: `DeleteIfMatch` when the host has any other authority, **or** when none and the claim is still pending at that ETag. The sweep never commits on an owner's behalf (B-12, answers rev 2's Q4): a pending claim is a request that did not finish, and its owner's retry is the only thing that may finish it. Deleting it removes nothing anyone was told succeeded | |
| -| §4.4 step 5 (verified, commit PUT lost) | a pending claim | the owner's retry (own id, step 4 then 5) within the timeout; after it, the sweep deletes it and the owner attaches again | |
| -| §4.5 pending marker, lifecycle absent or free, no record | a pending first-route claim | the sweep deletes it as above | |
| -| §4.5 pending marker, lifecycle `creating` | | the recovery pass settles the record half; the sweep leaves a pending first-route marker whose host is `creating` or `serving` **for this app's static record** alone and logs "pending first-route, re-attach to finish"; the owner's retry commits it (§4.5 step 2, lost commit) | |
| -| §4.5 pending marker, lifecycle held by another | | the sweep deletes the marker | |
| -| §4.6 step 1 (pending row or op, unverified) | names reserved by an op nobody finished | the **app-name sweep** (console, startup and every minute): for every `pending_app_ops` row older than a minute, re-run §4.6 step 2 and then step 3 or its compensation, each conditional on the op id | |
| -| delete (§4.3, §4.5 detach) at any step | the lifecycle spec's recovery owners | unchanged | |
| -| console purge door | records gone; claims | the door detaches the app's `domains/` claims with `DeleteIfMatch` **before** `PurgeApp`, discovered through `ClaimCache`, each read exactly first. An alias acquisition in flight for the app fails its §4.4 step 4 own-app-live check, because the app is trashed before purge | |
| -| box auto-purge | records and marker gone; the trashed SQL row; the app's claims | **console materialization** (materialize.go:391) detaches the app's claims, as the purge door does, **before** it removes the trashed row. The name is released only once no claim names the app (B-8, A-14). Until then the row stays trashed and the name reserved | |
| - |
| -Old-console deletes cannot happen above the floor (§8.3). A `serving` |
| -claim whose record is gone can arise from a crashed new-console delete, |
| -and the lifecycle spec's recovery owners cover it. |
| - |
| -**Quarantine guard (B-3).** The recovery pass never deletes an orphan |
| -whose host is listed in `control/hosts-quarantine.json` (§8.1). The |
| -guard is a rollout prerequisite, not a follow-up. |
| - |
| -## 6. Composition with the lifecycle claim |
| - |
| -Ownership and lifecycle are **one object for record-backed hosts** and |
| -**two for aliases**, and the two never both own a host. |
| - |
| -- This spec uses `internal/hostclaim` as specified: `Read`, `Update`, |
| - `BeginCreate`, `CancelCreate` (only while `creating`), `CompleteCreate`, |
| - `BeginDelete` conditional on the ETag read, `Free`, and the recovery |
| - pass. "Who holds H" is `Read(H).Claim.Holder` and the op, nothing else. |
| - The create door runs the lifecycle protocol in full, admission lease |
| - included, and migrate uses admission to quiesce. |
| -- It asks the lifecycle spec for **three changes**: |
| - 1. **(prerequisite)** `Adopt` requires deployment-wide evidence: the |
| - adopting actor has listed every `apps/*/*/instances/<host>.json` and |
| - `apps/*/*/sidecars/*.json` for the host and found exactly one |
| - record, and that record's host is consistent with `domains/` and |
| - the namespace rule (§8.1 step 3). In this design the **backfill verb |
| - is the only adopting actor**; reconcile, the doors, purge and the |
| - recovery pass, which §9 names as adopting actors, refuse a host with |
| - no claim and log it (A-1, B-3). Above the marker no such host |
| - exists. |
| - 2. **(prerequisite)** The recovery pass honours the quarantine guard |
| - (§5) before `DeleteIfMatch` on an orphan. |
| - 3. Account migrate is refused for hosts with records until a transfer |
| - transition exists (§4.8). A ruling, not a mechanism. |
| -- It adds console-side protocol around the lifecycle CAS: the reads of |
| - `domains/` and app rows, the alias doors' reads of `control/hosts/`, |
| - and the two sweeps. The lifecycle claim does not know aliases exist. |
| -- The lifecycle floor gates this design. Below the floor the doors keep |
| - today's checks; the `acq` block is written only above it. |
| - |
| -The alternative, an `alias` holder family in the lifecycle claim so that |
| -one object owns every routable name, was considered (§12). |
| +| `pending`, older than 60 s | absent or free, or a `creating` op whose `intent.ownership` is a different id | `DeleteIfMatch` the object and its index entry: a request that did not finish. Nothing anyone was told succeeded is removed | |
| +| `pending` | `creating` naming this `acq.id` | leave it: the lifecycle recovery pass owns it (complete or cancel) | |
| +| `pending`, family `apex`, older than 60 s | | row exists with `apex_acq == acq.id` and live: commit on its behalf (the row is the family's atomic step, and the id ties them). Else delete | |
| +| `committed`, family `instance` or `sidecar` | no holder, or holder incarnation differs, and no `releasing` | report "committed ownership without its holder" for the operator; never delete (a record may be mid-create or the lifecycle claim mid-recovery) | |
| +| `releasing` | serving with holder `I` | restart §4.4 step 4 with `release_op` | |
| +| `releasing` | `deleting`, `parking` under `release_op` | wait | |
| +| `releasing` | free, or another holder and no op naming `release_op` | §4.4 step 5 | |
| +| `committed` whose app row is gone or has another incarnation | | report only; SQL absence is not proof (direct-bucket apps) | |
| +| index entry without an object | | delete the entry | |
| +| object without an index entry | | write the entry | |
| + |
| +**The lifecycle recovery pass** (rev 6.16 §5.1 step 5) gains the |
| +ownership check of §4.2 (ask 1) and two gates: no destructive orphan |
| +deletion before `control/hosts-backfill.json` exists, and none for a |
| +quarantined host (C-2, B-3). |
| + |
| +**Purge** (console door, rev 6.16 §5.3 phases): after `retired`, a new |
| +phase `released`: list `owned/` (fail closed), for each entry `GET |
| +domains/<host>`; owned by this app and incarnation: §4.4 steps 3 to 5 |
| +(instance, sidecar), or `releasing` then delete (alias, apex). Then |
| +`done`. `PurgeApp` deletes everything under the app prefix **except |
| +`owned/`**, which the `released` phase empties last. Box auto-purge, where |
| +it still runs, therefore leaves the index behind, and **console |
| +materialization** (materialize.go:391) runs the `released` phase for a |
| +trashed row whose prefix is otherwise gone **before** `PurgeAppRow`. |
| +The name is released only when `owned/` is empty (C-3, B-8). A late |
| +alias writer for the purged app fails its commit on `app_incarnation` |
| +(§3.3). |
| + |
| +**Writer crash windows:** |
| + |
| +| Crash after | Left | Cleared by | |
| +|---|---|---| |
| +| §4.1 step 2 (pending, nothing else) | pending object | sweep, after 60 s | |
| +| index write, before lifecycle `BeginCreate` | pending object and entry | sweep | |
| +| `BeginCreate`, before the record | `creating` claim naming the acquisition; pending object | lifecycle recovery cancels after five minutes (record absent); the sweep then deletes the pending object | |
| +| the record, before the ownership commit | `creating`, record, pending | lifecycle recovery: ownership names this acquisition and is pending, so it **commits the ownership object first, then completes** (ask 1); or, if the object no longer names it, cancels and deletes the record | |
| +| the ownership commit, before `CompleteCreate` | `creating`, record, committed | lifecycle recovery completes | |
| +| §4.4 step 3, before `BeginDelete` | `releasing`, serving | sweep restarts the lifecycle delete | |
| +| any lifecycle delete step | | lifecycle recovery owners | |
| +| lifecycle free, before §4.4 step 5 | `releasing`, free | sweep deletes | |
| +| rename or move, any step | closed source row, pending op, pending new apex | app-name sweep, conditional on `op_id` | |
| +| purge, any phase | the purge journal's phase | the purge recovery pass, as rev 6.16 §5.3 | |
| + |
| +## 6. What this asks of the lifecycle claim spec (rev 6.16) |
| + |
| +Exactly these, for platform-1e: |
| + |
| +1. **§5.1 create, binding.** `BeginCreate` records `op.intent.ownership |
| + = <acquisition id>` (a map entry; the library's `BeginCreate` takes it |
| + as a parameter or the door adds it to the intent). The step 4 CAS to |
| + serving, by the door and by the recovery pass, requires |
| + `domains/<host>` to exist with `acq.id == op.intent.ownership` in |
| + state `pending` or `committed`; the recovery pass commits a `pending` |
| + object first (CAS on its ETag), then completes. Otherwise it cancels |
| + the create and deletes the record with `DeleteIfMatch`. |
| +2. **§5.1 step 5, gates.** Orphan deletion is disabled until |
| + `control/hosts-backfill.json` exists, and never acts on a host listed |
| + in `control/hosts-quarantine.json`. |
| +3. **§9 adoption.** Before the marker, only the backfill verb adopts |
| + (§8). Reconcile, the doors, purge and the recovery pass refuse a host |
| + with no claim and log it. Above the marker no such host exists, so the |
| + routine is unchanged for them. |
| +4. **§5.3 purge.** A phase `released` between `retired` and `done`, as |
| + §5 describes, and `PurgeApp` keeps `owned/`. |
| +5. **Nothing else.** No new field, no change to C1 to C7, to admission |
| + (C4 may optionally also check O1), to parking, to the holderless |
| + sweep, or to the floor. Release never deletes the lifecycle claim. |
| ## 7. Privacy |
| -**Placement.** Every ownership read sits after every other validation |
| -and immediately before the write it protects, on every door (A-13). |
| +**Placement.** Every ownership read is §4.1 step 2, after every other |
| +validation; a request with an invalid later field answers its 400 before |
| +any ownership read (C's placement note, apiinstances.go:736 moves below |
| +:1092). |
| -**The response mapping**, evaluated in this order, first match wins, for |
| -every door (B-14). "Foreign" means the authority names an account other |
| -than the caller's, or names no account (a holderless purge fence, a |
| -malformed object). |
| +**Response mapping**, in order, first match wins, for every door. |
| +"Foreign" means another account, no account, or an object that does not |
| +decode. |
| -| # | Observation | Instance and app-name doors | Alias doors | |
| +| # | Observation | Instance, sidecar and app-name doors | Alias and detach doors | |
| |---|---|---|---| |
| -| 1 | a read failed (bucket, registry, SQL) | 502 "could not check the host, try again". Never an occupancy answer | 502, the same | |
| -| 2 | a claim or lifecycle object does not decode, or `Validate` fails | 500 "could not check the host"; logged for the operator | the same | |
| -| 3 | held by another **account**, any family, any op; or a holderless op; or a claim naming no account | 409 `hostUnavailableMsg`, byte for byte | 409 `hostUnavailableMsg` | |
| -| 4 | the caller's **own app**, same family, already there (its own record, its own committed alias) | 409 "instance already exists" (today's sentence) | 200 no-op, or the repoint | |
| -| 5 | the caller's own app, other family or an unfinished acquisition | 409 naming the conflict and the step that frees it ("detach the alias first", "a first-route attach is in progress", "attach in progress, retry") | the same | |
| -| 6 | another app of the caller's account, serving or committed | 409 naming the app, as `hostHeldSentence` does today | 409 `hostUnavailableMsg` (apidomains.go:458's stronger neutrality kept) | |
| -| 7 | another app of the caller's account, non-serving op or pending | 409 naming the app and "is being changed; try again later" | 409 `hostUnavailableMsg` | |
| -| 8 | an operator route whose account column is the caller's, with no record and no claim | 409 "operator repair", as today | 409 `hostUnavailableMsg` | |
| -| 9 | an operator route naming another account or none | 409 `hostUnavailableMsg` (today's "routed for account X" is retired) | 409 `hostUnavailableMsg` | |
| -| 10 | the ownership freeze (§8.2) | 503 `msgOwnershipFreeze` | 503, the same | |
| - |
| -Rows 1 and 2 come first so that unavailable evidence is never reported |
| -as a conflict or as free. Nothing a member receives distinguishes a |
| -foreign hold from a foreign alias claim, and §4.4 and §4.5 name only |
| -rows of this table. |
| - |
| -**Logs and sweeps.** The two sweeps and the backfill verb name hosts, |
| -accounts and keys in the operator log and the verb's stdout only. |
| - |
| -**Buckets.** `control/hosts/` and `domains/` are read by consoles and |
| -boxes. No door returns a claim body to a member. |
| +| 1 | a read failed (bucket, registry, SQL) | 502 "could not check the host, try again" | 502, the same | |
| +| 2 | the object does not decode | 500 "could not check the host"; operator log | the same | |
| +| 3 | foreign, any family, any state; or a lifecycle purge fence; or an operator route naming another account or none | 409 `hostUnavailableMsg`, byte for byte | 409 `hostUnavailableMsg` (attach); 404 "not attached" (detach) | |
| +| 4 | own app, same family, committed | 409 "instance already exists" (today) | 200 no-op or repoint | |
| +| 5 | own app, other family or `first_route` mismatch | 409 naming the conflict and the door that frees it | the same | |
| +| 6 | own app, `pending` of another request, or `releasing` | 409 "attach in progress, retry" / "being released, retry later" | the same | |
| +| 7 | own app, committed, but the lifecycle holder does not match (`acq.incarnation`) | 409 "this address needs an operator's attention" | the same | |
| +| 8 | another app of the caller's account, committed | 409 naming the app, as `hostHeldSentence` does today | 409 `hostUnavailableMsg` (apidomains.go:458's neutrality kept) | |
| +| 9 | another app of the caller's account, `pending` or `releasing` | 409 naming the app and "is being changed; try again later" | 409 `hostUnavailableMsg` | |
| +| 10 | an operator route whose account column is the caller's, no object | 409 "operator repair", as today | 409 `hostUnavailableMsg` | |
| +| 11 | the freeze (§8) | 503 `msgOwnershipFreeze` | 503 | |
| + |
| +Nothing a member receives distinguishes a foreign hold of one family |
| +from another, and read failures are never occupancy answers. Sweeps, the |
| +backfill and the quarantine file name accounts only in operator output. |
| ## 8. Migration |
| -### 8.1 Backfill |
| - |
| -A console-side verb, `carlos ops hosts backfill [--dry-run]`, runnable |
| -only above the version floor and only while the freeze (§8.2) is in |
| -force: |
| - |
| -1. List the bucket, not SQL: every `apps/*/*/instances/*.json`, |
| - `apps/*/*/sidecars/*.json`, `domains/*.json`, and every route a live |
| - box publishes (`edgesfile`). This covers direct-bucket records of apps |
| - SQL does not list. |
| -2. For each host with exactly one record, no lifecycle claim, no |
| - `domains/` claim or only a legacy first-route marker owned by the |
| - record's app with `attachWrote` shape, and a host consistent with the |
| - namespace rule: `claim.Adopt` with that record as holder (the lifecycle |
| - spec's §9, discovery included), then stamp the legacy marker |
| - `{acq: committed, first-route, incarnation}`. |
| -3. **Quarantine** (B-3, B-4): any of |
| - - two or more records for a host, claimed or not; |
| - - a record and an alias claim (`Route != ""`) at one host; |
| - - a record whose lifecycle holder names a different incarnation; |
| - - a record at `<l>.<sqid>.<apps>` where `<l>` is an app other than |
| - the record's, or at `<x>.<app>.<sqid>.<apps>`; |
| - - a legacy first-route marker whose owner has no matching record. |
| - |
| - Quarantined hosts are listed in `control/hosts-quarantine.json`, |
| - rewritten whole by the verb, with every key involved. The create door |
| - refuses them (an absent or present claim with a quarantine entry is |
| - held, row 3 of §7), the recovery pass never deletes an orphan there |
| - (§5), and the operator repairs by deleting the loser or detaching the |
| - alias, then re-runs. |
| -4. A record or claim that does not decode, or whose body disagrees with |
| - its key, is reported and blocks completion. |
| -5. **Legacy alias claims** with no record at the host are stamped |
| - `{acq: committed, alias}` by CAS on their ETag. A legacy claim whose |
| - owner has no app row is reported (purge inheritance) and blocks |
| - completion until detached. |
| -6. **Apps with no SQL row** that have records or routes block completion |
| - (§4.6). |
| -7. **Completion** needs two consecutive clean passes under the freeze: |
| - no unclaimed record, no quarantine, no undecodable object, no |
| - unregistered app, and the second listing showing nothing the first |
| - did not. The verb then writes `control/hosts-backfill.json` with the |
| - counts, the floor version and the time, and the freeze lifts. |
| - |
| -### 8.2 The freeze, and the door before and after the marker |
| - |
| -There is no lazy adoption (A-1). Below the floor the create door, both |
| -attach branches and the apex gate run **today's** paths. Above the floor |
| -and before the marker they answer 503 `msgOwnershipFreeze` ("the |
| -platform is finishing a maintenance step; try again in a few minutes") |
| -and write nothing (B-2). After the marker they run §4. |
| - |
| -So no record, claim or app row can be written between the floor and the |
| -marker by any binary: old binaries are refused by the floor, new ones by |
| -the freeze. The two clean passes are defence in depth, not the mechanism. |
| -The freeze is minutes on flagship: one listing of a few thousand keys and |
| -one `Adopt` per host. |
| - |
| -The marker cannot go stale: nothing above the floor writes an unclaimed |
| -record. If the floor is ever lowered, the marker is deleted first and the |
| -doors fall back to today's paths. |
| - |
| -### 8.3 Mixed-version consoles |
| - |
| -Below the floor, old and new consoles both run today's path, so the race |
| -remains open until the floor is published; that is the lifecycle spec's |
| -rollout order and the roll is consoles first. Above the floor no old |
| -binary runs (lifecycle §10 refuses them, including boot recovery). A |
| -`serving` claim whose record is gone can still arise from a crashed |
| -new-console delete, and the lifecycle recovery owners cover it. |
| - |
| -### 8.4 Cross-account |
| - |
| -- Backfill adopts whatever single record exists, including records under |
| - accounts the console does not know. It reports those and never |
| - rewrites, re-homes or deletes a record. |
| -- Nothing in this design deletes a legacy record: the recovery pass |
| - deletes only orphans established under §5.1 step 5 outside quarantine, |
| - and under the marker every legacy record has the claim the backfill |
| - gave it, so none is an orphan. |
| -- Refusals follow §7's mapping. |
| +### 8.1 Order |
| + |
| +1. Land the lifecycle library with §6's asks. Ship rev 4's code dormant |
| + below the floor: every door runs today's path; the verbs refuse. |
| +2. Roll consoles, then boxes; publish the floor. **The freeze begins** |
| + with the floor: above it and before the marker every create, attach, |
| + detach, app-name and delete door answers 503 `msgOwnershipFreeze`. |
| +3. **Drain** (B-2, C's cutover note). The backfill verb writes |
| + `control/ownership-freeze.json`. Every new-binary door re-reads that |
| + object **immediately before** its record, claim or row write and |
| + refuses if present, so a handler that chose today's path before the |
| + floor and paused cannot write after the fence. The verb then waits |
| + twice the longest handler deadline (`instanceStoreTimeout` and the |
| + browser form's) before listing. Old binaries cannot run above the |
| + floor (rev 6.16 §10, including boot recovery). |
| +4. Backfill (§8.2), dry then live, until two clean passes write |
| + `control/hosts-backfill.json`. The freeze lifts; the doors run §4. |
| + |
| +### 8.2 Backfill: `carlos ops hosts backfill [--dry-run]` |
| + |
| +Runs on a console above the floor with the fence in place. Inputs: every |
| +`apps/*/*/instances/*.json` and `apps/*/*/sidecars/*.json`; every |
| +`domains/*.json`; every app row (live and trashed) of every account the |
| +console knows; every route a live box publishes (`edgesfile`); every |
| +lifecycle claim. |
| + |
| +For each hostname that appears in any input, build its **candidates**: |
| +records (one per app), a legacy alias or first-route object, an apex |
| +(from an app row whose name is the host's label under its account's |
| +sqid), and routes. Then: |
| + |
| +- **Exactly one authority, consistent:** write the ownership object |
| + `committed` (create-only, or CAS onto the legacy object preserving its |
| + alias fields), write the index entry, and for a record with no |
| + lifecycle claim run `claim.Adopt`. Consistent means: a record's host |
| + obeys the namespace rule and its lifecycle holder, if any, names its |
| + incarnation; a legacy alias has no record and no live app row at its |
| + label other than its owner (B-4); an apex has no other authority at the |
| + host; a first-route marker has exactly its owner's static record. |
| +- **Anything else is quarantined** in `control/hosts-quarantine.json` |
| + with every key involved: two or more records; a record and an alias; a |
| + record or alias at another app's apex; a lifecycle holder naming a |
| + different incarnation; a marker without its record; an object that |
| + does not decode. A quarantined host is held (§7 row 3) and the |
| + recovery pass never deletes there (§6 ask 2). The operator picks the |
| + winner with `carlos ops hosts resolve <host> --keep <key>`, which |
| + deletes the losers' records or objects conditionally and re-runs the |
| + host. |
| +- **Apps with no SQL row** that have records or routes block completion |
| + until `carlos ops apps register` (§4.6) or the operator deletes them. |
| +- **Legacy aliases whose owner has no app row** are reported and block |
| + completion until detached by an operator (purge inheritance). |
| +- Every app row gets `incarnation` and, once its apex is acquired, |
| + `apex_acq`. |
| + |
| +**Completion** needs two consecutive clean passes under the fence: every |
| +hostname has exactly one committed object and an index entry, every |
| +record-backed host has a lifecycle claim naming its incarnation, no |
| +quarantine, no undecodable input, no unregistered app. Then the marker is |
| +written with counts, the floor version and the time, the freeze object is |
| +deleted, and the doors switch. Lowering the floor deletes the marker |
| +first. |
| + |
| +### 8.3 Mixed versions and cross-account |
| + |
| +Below the floor old and new consoles run today's paths and the race is |
| +open; above it no old binary runs. Backfill writes ownership for records |
| +under accounts the console does not know (from the bucket) and never |
| +rewrites, re-homes or deletes a record; collisions involving them are |
| +quarantined like any other. No door's refusal names another account. |
| ## 9. Cost |
| -Per path, bucket calls only (SQL reads are local). "Today" is the |
| -current main. |
| +Bucket requests per logical operation (C-7). "Fold" means |
| +`edgesfile.DeploymentRoutes`: box status reads plus one GET per live |
| +box's edges file. |
| -| Path | Today | After | |
| +| Path | Today | Rev 4 | |
| |---|---|---| |
| -| Instance create, account with N apps | registry read, 2 SQL reads, N sequential GETs, 1 claim GET, 1 record PUT | 1 `domains/` GET, 1 fold read, lifecycle step 0 (1 GET, the lease's PUT), 1 claim PUT, 1 `domains/` GET, 1 record PUT, 1 claim PUT. Independent of N. The lifecycle pair and the lease are the lifecycle spec's cost; this spec adds two `domains/` GETs and removes N. A restarted CAS (step 4) repeats the lease and the pair once | |
| -| Alias re-attach, no-op | 1 GET | 1 GET, unchanged | |
| -| Alias repoint | 1 GET + 1 PUT | 1 GET + 1 PUT, unchanged | |
| -| Alias fresh attach | 1 GET + 1 PUT | 1 GET, 1 lifecycle GET, 1 PUT (pending), 1 lifecycle GET, 1 PUT (commit): two GETs and one PUT more. A takeover adds one PUT | |
| -| First-route attach | 1 GET + 1 PUT + record PUT | pending PUT, the create path, commit PUT | |
| -| Detach | 1 GET + 1 DELETE | alias: the same, conditional. First-route: plus 1 lifecycle GET and the lifecycle delete's own writes | |
| -| App create, rename, move | the N-GET scan | 1 lifecycle GET + 1 `domains/` GET, plus SQL | |
| -| Claim sweep | none | one `ListMod` of `domains/` every few minutes (the `ClaimCache` steady state); per **pending** claim older than the timeout: 1 GET, 1 lifecycle GET, 1 conditional DELETE. Normally zero pending claims | |
| -| App-name sweep | none | SQL, plus two GETs per pending op, normally zero | |
| -| Backfill | none | one-off: one listing of each prefix, one `Adopt` PUT per host, one CAS per legacy claim | |
| -| Migrate | listing | plus the admission close and reopen (2 CAS) and two prefix listings | |
| -| Boxes | | the pending skip is a field test; the canary check is a map lookup | |
| +| Instance create, N apps in the account | registry, N GETs, 1 GET, 1 PUT | 1 GET + 1 PUT (acquire), 1 PUT (index), lifecycle step 0 (1 GET, lease PUT), 1 CAS, 1 record PUT, 1 CAS (commit), 1 CAS (complete). The fold only when the namespace rule admits an unowned platform host. Independent of N | |
| +| Instance delete | 1 DELETE | 1 GET, 1 lifecycle GET, 1 CAS, lifecycle delete's own writes, 1 DELETE, 1 DELETE | |
| +| Alias attach, no-op | 1 GET | 1 GET | |
| +| Alias repoint | 1 GET, 1 PUT | 1 GET, 1 PUT | |
| +| Alias fresh attach | 1 GET, 1 PUT, the fold | 1 GET, 1 PUT, 1 PUT (index), 1 CAS, the fold | |
| +| Alias detach | 1 GET, 1 DELETE | 1 GET, 1 CAS, 2 DELETE | |
| +| First-route attach | 1 GET, 1 PUT, 1 PUT | the instance create path with `first_route` | |
| +| App create | N GETs | 1 GET, 1 PUT, 1 PUT, 1 CAS, SQL | |
| +| Rename, move | N GETs | the above, plus 1 CAS and 1 DELETE for the old apex | |
| +| Ownership sweep | | one `ListMod` per page per minute; a GET per changed or pending object; the index PUT on change | |
| +| Boxes | one `ListMod` per page per pass | one conditional GET of the alias index per pass (304); the full fold only as fallback | |
| +| Purge | app prefix listing | plus one listing of `owned/` and a GET and release per host | |
| +| Backfill | | one-off: the listings, one PUT and one index PUT per host, one `Adopt` per record | |
| + |
| +Removing the per-app scan does not make create constant in bucket |
| +requests; it makes it independent of account size and of fleet size |
| +except where the fold is consulted. |
| ## 10. Tests |
| -Against a dirstore with real CAS and conditional delete, in |
| -`internal/console`, foreground. Every interruption test runs the |
| -recovery pass and both sweeps afterwards and asserts the residual bucket |
| -and SQL state, not only the responses. |
| - |
| -1. **Concurrent create.** Two apps, one host, 2 and then 16 goroutines |
| - per app. Exactly one 201; every other response is 409 with the |
| - within-account sentence; exactly one record; the claim's holder names |
| - it; `op.kind == serving`. Seeds 1..200. |
| -2. **Alias versus instance, deterministic schedules.** Every interleaving |
| - of §4.2 steps 2 to 7 with §4.4 steps 1 to 5 under an injected |
| - scheduler. Invariant after both return and after recovery: never both |
| - 2xx; the bucket holds either one committed alias claim and a free or |
| - absent lifecycle claim, or a record with a matching holder and no |
| - `domains/` object, or neither. No pending claim survives the sweep. |
| -3. **Same-owner alias retries and takeover** (A-3, B-1): A1 and A2 for |
| - the same owner racing B, with A1 paused past the timeout so A2 takes |
| - over. A1's commit and cleanup fail their conditions; whoever answered |
| - 2xx, the object it reported exists afterwards. |
| -4. **Family conversion** (A-4, B-1): a pending or committed first-route |
| - claim and an alias request by the same owner. Refused; never both the |
| - record and an alias. An alias request never CASes a first-route |
| - acquisition. |
| -5. **Stale cleanup** (A-5, B-7): a rollback paused across a detach and a |
| - re-attach by the same principal with the same route within one |
| - second. The conditional delete fails; the new claim stands. |
| -6. **First-route detach versus successor** (B-6): a detach paused after |
| - reading marker A while A is detached and B attached; the paused detach |
| - refuses at step 2 or fails its CAS at step 3; B's record stands. |
| -7. **Crash between claim and record.** Inject a failure after |
| - `BeginCreate`; advance the clock past five minutes; recovery frees the |
| - claim, retires the incarnation, lists the key; a create by another app |
| - succeeds. The negative: record present, recovery completes, never |
| - frees. |
| -8. **Pending alias never serves** (A-8): a pending claim and a lifecycle |
| - holder whose route is not yet published; `ReconcileAliases` derives |
| - nothing for it (alias, wildcard, catch-all, paths); the sweep deletes |
| - it. |
| -9. **Alias versus app name** (B-5): with reservation off, an alias at |
| - `<l>.<sqid>.<apps>` racing `claimApp` for `<l>` under every |
| - interleaving; at most one commits; the loser's row or claim is gone |
| - after the sweeps. |
| -10. **Backfill.** Records across three apps and two accounts; one host |
| - with two records; one record with an alias claim; one legacy |
| - first-route marker with and without its record; one undecodable |
| - record; one app prefix with no row. The verb adopts the clean |
| - singletons, stamps legacy claims, quarantines each bad combination |
| - with every key named, blocks on the bad record and the rowless app, |
| - writes no marker; after repair, two clean passes write it. The create |
| - door answers 503 throughout the freeze and refuses quarantined hosts |
| - after it. |
| -11. **Freeze** (B-2): above the floor and before the marker, every door |
| - answers 503 and writes nothing; the marker verb refuses below the |
| - floor. |
| -12. **Quarantine guard** (B-3): an orphan at a quarantined host survives |
| - the recovery pass. |
| -13. **Response mapping.** Every row of §7 on every door; cross-account |
| - bodies byte-identical and sqid-free; read failures answer 502 for a |
| - held and a free host alike; an invalid request answers 400 whether or |
| - not the host is held. |
| -14. **Apex gate** (B-9): create, rename and move each crashed after step |
| - 1 and after step 2; both names stay reserved; the sweep settles with |
| - the op id; a paused handler finding its op gone does nothing. |
| -15. **Namespace rule**, each row of §4.6 against each door, reservation |
| - on and off, including the own-app exception and a manifest `Domain` |
| - of `<app>.<sqid>.<apps>` for another app. |
| -16. **Canary mint does not overwrite** a route owned by another app. |
| -17. **Migrate** (B-13): refused for an app with a record, with admission |
| - closed during the check and reopened after; a create during the |
| - check is refused by admission; success for a releases-only app with |
| - no route stamping. |
| -18. **Purge and materialization** (B-8): the purge door detaches claims |
| - before `PurgeApp`; after box auto-purge, materialization detaches |
| - claims before releasing the row; a new app of the name inherits |
| - nothing. |
| - |
| -The joint simulation (`internal/sim/hostlife`) should gain the alias |
| -actor, the acquisition state and the two-family invariant, if the moves |
| -session agrees; this spec does not depend on it. |
| +Dirstore with CAS and conditional delete, foreground, in |
| +`internal/console`. Every interruption test runs lifecycle recovery and |
| +both sweeps afterwards and asserts the residual bucket, index and SQL |
| +state. |
| + |
| +1. **Concurrent create**, two apps, one host, 16 goroutines each, seeds |
| + 1..200: one 201, one object, one record, holder incarnation equals |
| + `acq.incarnation`. |
| +2. **Alias versus instance versus app create**, every pairing, both |
| + orders and concurrent: at most one commits; no cross-authority check |
| + is needed to prove it, the test asserts the single object. |
| +3. **Delayed creator** (§4.2): pause after acquire past 60 s; the sweep |
| + reclaims; a successor acquires; the original resumes through |
| + `BeginCreate` and the record and fails its commit, cancels, deletes |
| + its record; the successor completes on retry. Variant: pause between |
| + record and commit past five minutes; recovery cancels because the |
| + object names another acquisition. |
| +4. **Recovery completes with commit**: crash between record and commit; |
| + recovery commits the object then completes; O1 holds. |
| +5. **Same-owner retry and takeover**: resume by id; takeover by CAS; the |
| + displaced writer's commit, rollback and index write all fail. |
| +6. **Release**: crash after each of §4.4's steps; the sweep finishes; |
| + the object is gone, the lifecycle claim is free and retains its |
| + history; a new create succeeds afterwards and is refused "being |
| + released" before. |
| +7. **Detach dispatch**: each row of §4.5; an instance without |
| + `first_route` is never deleted by detach; `route == ""` has no effect. |
| +8. **App incarnation fence**: attach paused across purge and recreate of |
| + the name; commit refused; nothing attached to the new app. |
| +9. **Purge enumeration**: a read failure during `owned/` enumeration |
| + aborts the `released` phase; nothing is released; the name stays |
| + reserved. Materialization releases the row only when `owned/` is |
| + empty. |
| +10. **Rename with a concurrent first-route attach** (C-4): the attach's |
| + commit fails on `closed`; the source stays empty; finalize succeeds. |
| + Crash after each step; the app-name sweep settles by `op_id`. |
| +11. **Namespace rule and canary mint**: each row of §4.6's table on |
| + every door; the mint skips an unrelated route and skips when the |
| + apex is not committed. |
| +12. **Fence and freeze** (B-2): a handler paused before its write with |
| + the freeze object present refuses; every door answers 503 during the |
| + freeze. |
| +13. **Backfill**: a fixture with every quarantine case, a legacy alias at |
| + a live app's label (B-4), a rowless app, an undecodable object; |
| + nothing is adopted or stamped for a quarantined host; the recovery |
| + pass deletes nothing before the marker (C-2); `resolve --keep` |
| + clears each; two clean passes write the marker. |
| +14. **Response mapping**: every row of §7 on every door; foreign bodies |
| + byte-identical and sqid-free; read failures answer 502 for held and |
| + free alike; invalid later fields answer 400 before any ownership |
| + read. |
| +15. **Alias index**: boxes derive the same `route_aliases` from the index |
| + as from the full fold; a stale index falls back. |
| +16. **Migrate**: refused with a record; admission closed and drained per |
| + §5.3 step 1b during the check. |
| ## 11. Rollout |
| -1. Prerequisites, in the lifecycle spec and library: the adoption |
| - evidence rule and the quarantine guard (§6). Land `internal/hostclaim` |
| - on main with them. |
| -2. Ship the doors, the two sweeps, the backfill verb, the `acq` block, |
| - the namespace rule, the apex gate's op table, the purge and |
| - materialization detaches, the canary-mint check and the pending skip |
| - in `ReconcileAliases`, all **dormant below the floor**: below it every |
| - door runs today's path and the verb refuses to run. |
| -3. Roll consoles, then boxes, in the lifecycle spec's order. Publish the |
| - floor. The freeze begins. |
| -4. Run the backfill dry; read the quarantine and the reports; repair; |
| - run live until two clean passes write the marker. The freeze lifts. |
| -5. `hostHolder`'s record loop, `apexHolder`'s scan and `ReportApexSquats`' |
| - route walk are removed in the release after. |
| -6. Follow-ups filed, not built here: migrate's transfer and `domains/` |
| - rebind (§4.8). |
| +§8.1's order. After the marker, the release that follows removes |
| +`hostHolder`, `apexHolder`'s scan, `ReportApexSquats`' route walk, the |
| +startup owners backfill and the legacy-object branches. |
| ## 12. Alternatives considered |
| -- **`domains/<host>.json` as the ownership object for everything.** |
| - Rejected: fleet-wide listing cost, the detach door's record delete on |
| - `Route == ""`, and duplicating the lifecycle claim's create protocol. |
| -- **A new `hosts/<host>.json` ownership-only object.** Rejected: it would |
| - duplicate the lifecycle holder for every record-backed host and need |
| - its own crash recovery. |
| -- **An `alias` holder family in the lifecycle claim.** One object for |
| - every routable name, no two-family rule, and the alias pass would read |
| - one authority. Rejected for now: it changes a settled spec and |
| - `holderKeyMatches`, gives every custom domain a lifecycle object the |
| - recovery pass lists, and still needs an acquisition state for |
| - same-owner retries. It remains the cleanest end state if the two-family |
| - rule proves hard to keep as doors are added; §13 asks. |
| -- **Box-side alias pass reading the lifecycle claim.** Rejected: one GET |
| - per alias per box per pass, or a second cache over `control/hosts/`. |
| - The pending state gives the pass a committed-only rule with no new |
| - read. |
| -- **A sweep that commits stale pending aliases.** Rejected (B-12): the |
| - sweep lacks the observations to authorise a commit, and a commit nobody |
| - requested is an attach nobody asked for. Delete-only. |
| -- **Plain `CreateRecord` between the floor and the marker.** Rejected |
| - (B-2): any such write after the final listing turns a legitimate record |
| - into a future orphan. A short freeze is cheaper than a wrong delete. |
| -- **Deterministic tie-break instead of both backing off.** Rejected as |
| - unsound (§4.1). |
| -- **Lazy adoption.** Rejected (A-1): the door can only see the caller's |
| - account, so it would create claims over unseen foreign records and the |
| - recovery pass would then delete them as orphans. |
| -- **A one-field holder transfer for migrate.** Rejected (A-12): identity, |
| - location, execution authority, sidecars and aliases all move with the |
| - account. |
| +- **Two authorities with write-then-verify** (revs 1 to 3). Rejected by |
| + ruling: the interleaving proof, the reciprocal checks and the |
| + cross-family migration cases were most of the design's risk. |
| +- **An `alias` family in the lifecycle claim.** Rejected: it would make |
| + every alias a lifecycle object with parking and retirement, and it |
| + changes a settled spec; ownership is a smaller object. |
| +- **A per-canary ownership object acquired by boxes.** Rejected: boxes |
| + would need `domains/` write grants and cross-region races; the apex |
| + covers its subtree. |
| +- **Boxes keep listing `domains/`.** Kept as the fallback; the derived |
| + index is the steady state once the prefix exceeds a page. |
| +- **Release deletes ownership as soon as the record is gone.** Rejected: |
| + a successor would acquire and then be refused by a `parking` lifecycle |
| + claim; releasing at lifecycle free gives one refusal with one sentence. |
| +- **A sweep that commits stale pending objects.** Delete-only, except the |
| + `apex` family where the SQL row carrying `apex_acq` is exact evidence. |
| ## 13. Open questions for Paul |
| -1. Two families with acquisition-stamped alias claims (this spec), or one |
| - object with an `alias` family in the lifecycle claim (§12)? The second |
| - reopens the lifecycle spec. |
| -2. The lifecycle spec's §9 adoption rule and the quarantine guard are now |
| - rollout prerequisites (§6). Accept them into that spec, with the |
| - backfill verb as the only adopting actor? |
| -3. The freeze (§8.2): a few minutes of 503 on every create and attach |
| - door on flagship, once. Acceptable, or should the backfill run against |
| - a snapshot with a different fence? |
| -4. Account migrate: refuse apps with records indefinitely (this spec), or |
| - file the transfer protocol now? |
| -5. The namespace rule refuses `<x>.<app>.<sqid>.<apps>` through every |
| - door, and a manifest `Domain` under another app's apex. Acceptable on |
| - deployments with reservation off? |
| -6. Backfill quarantine: operator picks the winner by hand (this spec), or |
| - the verb keeps the record the provisioning pass currently keeps (first |
| - in listing order)? |
| -7. The acquisition timeout of 60 s, the sweep cadence, and the recovery |
| - age of five minutes are guesses. Shorter? |
| -8. Rev 3 has not been reviewed. A third Astra round before planning, or |
| - straight to plans with the review at plan level? |
| +1. The freeze: every door 503 for the minutes the backfill takes on |
| + flagship, once. Acceptable? |
| +2. Quarantine resolution by `carlos ops hosts resolve --keep <key>` with |
| + the operator choosing: confirm, or prefer a default (the record the |
| + provisioning pass keeps today, first in listing order)? |
| +3. The derived alias index as a console-written object the boxes read: |
| + acceptable as a second derived artefact beside the edges files? |
| +4. The 60 s acquisition timeout and the one-minute sweeps are guesses. |
| +5. §6 asks 1 to 4 of the lifecycle spec need platform-1e's agreement |
| + before planning. |