carlos / platform Public

Taken, with a correction to the framing and an answer to Ask 3. Design and spec are on branch replica-placements-162 (spec: docs/superpowers/specs/2026-09-21-replica-placements-design.md, reviewed once by Fable, findings committed beside it).

Ask 1 is an edge change here, not a router change. On CARLOS the sibling router is not what carries account hosts — carlos-edge owns host routing, and a request for a host a box does not serve is backhauled to the box that homes it (one HMAC-signed mesh hop, internal/edge/edge.go:1297). titogo's remote() hairpin is dormant everywhere, because nothing writes Route.Home at HEAD. The platform's answer: on the replica's box a route_aliases row maps the account's public host to the replica instance's own route, so the edge resolves it locally and never reaches the backhaul call. "A replica never forwards" is then structural rather than a rule anyone honours — it covers host-routed traffic because that is the only kind the edge has.

Ask 1's second bullet was the one real hole, and it is not where either of us was looking. Nothing on CARLOS steers an instance-kind host to every edge today: a host needs steering: "latency" on its record, it must lie under the apps domain, and the deployment must arm CARLOS_STEERING_ZONE_ID, which no root module in infra/ does. Without that the replica box mints its row, receives no buyer, and looks correct from every box. So a home record with replicas is now refused unless it is latency-steered, and arming the steering zone is a deployment prerequisite filed separately. Account hosts under a tenant-owned domain, and custom domains, must CNAME to edge.<apps-domain> or the placement is inert; custom domains on the replica box are deliberately out of the first landing.

Ask 2 is three additive record fields: Role ("" | "replica"), Replicas on the home record (the residence opt-in — never defaulted, never inferred, capped, and refused unless the home is steered) and Home on the replica record. The replica is provisioned as an ordinary instance record whose Region is its box, so it mints there and nowhere else. The instance API exposes all three, and served_by will list the replica's box beside the home's for the account host.

Ask 3: do not use the relay's /__internal/connect/ exception, and you do not need a new channel. That path forwards exactly one shape and fails closed; widening it would make a narrow verified door general. Instead: an instance dials a sibling at that box's EIP with the real hostname as SNI — what backhaul itself does — which bypasses the local edge, so a replica can reach home through the account host without landing on its own alias. The home edge presents the mirrored public certificate. The platform writes a siblings file beside each instance's data (path in CARLOS_SIBLINGS_FILE) carrying each sibling's host, placement, dial address and a per-pair key derived from the box mesh key. You sign and shape the lease and drain calls yourself; the protocol stays yours.

Three things the platform will not promise, which your lease/drain design has to absorb:

1. Version skew. Adoption runs per box on its own cadence, so home and replica differ by one release for minutes after any deploy, in either direction.
2. Leases wake sleeping replicas. A renewal through the alias wakes and bills the replica; size the renewal interval against CARLOS_HIBERNATE_IDLE or accept that a replica never sleeps while a lease is live.
3. Drain state is invisible to us. Deleting a home with replicas is refused, but sequencing a replica's deletion after its drain confirms is yours.

One request back: the siblings file is re-readable so a mesh rotation does not need a restart, which only works if you re-read it on a verification failure rather than caching the key at boot.

Implemented in PR https://github.com/carlosframework/platform/pull/409 (branch edge-replica-placements, amadan CI green at 6b3ded9e).

Covers Ask 1 and Ask 2:

• Record: Role (""|"replica", ValidRole) and Replicas (the home record's placement-set opt-in, ValidReplicas canonicalizes, MaxReplicas 8). Additive — a record without them reads as the serving home with no replicas. New internal/provision/replica.go: Replica() builds a replica record from a home record, CreateRecordReplica (create-only), ListRecordReplicas (the listing a home walks to find whom it leases to).
• Registry: routes.replicas + routes.replica mark, additive columns, converged from the record in both directions, not in Fingerprint (they change what the edges file publishes, not what the box dials).
• Reconcile: a replica record scopes to its placement box ONLY — the residence rule made structural; a replica minted anywhere else would be order data in a region nobody authorized. TestReplicaRecordMintsOnlyOnItsPlacement pins it.
• Edges file: replica:true on the replica row's own host + the account's platform apex; fail-closed when a box publishes no placement code. This is the vocabulary every reader (steering, homemap, the instances API's served-by join) uses to tell homes from checkout replicas.
• Steering: per-host latency records name HOME boxes only — a replica never steers, an all-replicas host earns nothing. edge.<apps-domain> keeps every armed edge.
• Console + CLI: carlos instances create --role replica --region uk (armed/hosting placement validated at the door), --replicas ie,uk on the home record. docs/site/cli.md names both.

Ask 3 needs no new platform surface: the relay's signed-delivery path already carries app-defined bodies (TITO_RELAY_SIGNING_KEY → TITO_RELAY_PUBKEY verify through the router's /__internal/* exception), so lease/drain runs over it as titogo's own protocol.

Residence: per-account opt-in on the home record, armed+hosting placement validated at the door, capped at 8, never silently on.

One deliberate open point for review: the edges file marks the replica row's own host and the platform apex, but not the home's host name or its custom aliases — the replica box's registry has no local row to vouch for those names, and custom domains reach the edges via CNAME to edge.<apps-domain> anyway. If the home's exact host needs to publish from the replica box too, that wants a home-link field on the replica record (a small additive follow-up).

Two contracts on your side, found while building the siblings file (spec §7, branch replica-placements-162).

1. Check the file's host field before you trust it. A CARLOS instance's data directory is either nested (/data/instances/<id>, its own) or flat (/data, shared with every other flat row on that box). On a flat row the siblings file has nowhere private to live: two flat hosts that both want one want the same path, and the loser's process would read a key minted for another account's pair — and sign a lease call with it that the sibling would accept. The platform refuses to write when it can see that collision, and removes the stale file, but it cannot close the case where a flat row with no replicas simply receives CARLOS_SIBLINGS_FILE=/data/siblings.json pointing at a neighbour's file. So the rule for the reading app: the file names its own host, and a file whose host is not yours is not yours — ignore it. Replica pairs want nested rows; a replica on a flat row gets no file and a log line rather than a key it might share.

2. Re-read the file, and accept any key in the list. keys is a list, current first, and both sides' files are rewritten by separate passes on separate boxes, so during a mesh rotation the two ends briefly sign with different firsts. Verify against any entry rather than the first, and re-read the file on a verification failure rather than caching the key at boot — that is what makes a rotation, and adding a replica, restart-free. An empty keys list means the platform has no mesh key yet: the file is armed but unsignable, and the pass writes nothing rather than handing you a blank.

Everything else is unchanged from my earlier reply. The branch now carries the record fields and their rails, the routes.replica_of mark, the alias derivation for both the account host and custom domains, the reconcile mint that gives a replica its home's channel, the edges-file role: "replica" entry, the edge's explicit ignore of that role, the console doors and their refusals, and the siblings file itself. The no-backhaul property has a test that fails when the alias row is removed (502 from a dead home EIP), so it is a proof rather than an assumption.

Fable review round landed: verdict was CHANGES REQUESTED (3 blocking, 8 follow-ups); all fixed in fc61e5e0, amadan CI green. Comment on the PR has the full accounting.

The one worth flagging here: the first steering fold would have deleted every non-home region's per-host latency record on rollout — it only credited boxes whose edges file names the host, and edges files list local rows, so a non-home edge never names a home-boxed host. Steering is any-edge-any-app (far edges backhaul); the fold now keeps that rule and removes only replica-marked entries. The consumer-list lesson again: what PARSES the shape decides, not what calls the API.

Merged. PR #409 landed on main (b5b0a53a) at 2026-09-23T22:59Z after a catch-up merge over the private-storage work (#412/#413 — two conflicts, both mechanical: the cli.md sections and a test-file brace; plus the new private-storage test's instanceCreate call carrying the replica param). amadan CI green on the branch (06039761) and on main after the mirror (b5b0a53a).

What shipped, final shape:

• Record: Role (""|"replica") + Replicas (home's placement-set opt-in, canonicalized, MaxReplicas 8), ValidateShape rails (instance-only, no addr — the everywhere-class second door closed), replica.go (Replica() runs ValidateShape, CreateRecordReplica, ListRecordReplicas).
• Registry: routes.replicas + routes.replica, both-ways convergence, not in Fingerprint.
• Reconcile: replica records scope to their placement box only; everywhereClass/everywhereRoute refuse a replica first.
• Edges file: replica:true rides the replica row's own host; fail-closed with no placement; nothing else claimed.
• Steering: keeps the any-edge-any-app rule (the review caught my first fold deleting every non-home region's records); only replica-marked entries are removed.
• Doors: create carries role/replicas with half-decision refusals, armed+hosting gate, fleet-placed+replica refused; CLI --role replica --region uk / --replicas ie,uk.

Fable review round: CHANGES REQUESTED (3 blocking, 8 follow-ups) → all fixed, accounting on the PR. Closing the ask; the residence follow-up noted earlier (home hostname/aliases from the replica box) stays open as a possible additive follow-up if the read side needs it.