carlos / platform Public

spec: relay revision 2, folding in codex round 1

Twenty findings, all taken. The contract now base64-wraps its JSON
because the config renderer refuses quotes, carries a platform-supplied
username suffix so the app never guesses its identity, and puts the
key id in the username so the relay picks the master rather than
trying two. Termination and metering come from the relay's own
RelayAddressGenerator: closing an allocation's relay socket is how
pion deletes it, and pion exposes no other way. Allocations are bound
to the identity that created them. The authority lease is renewed by
a sixty-second read, not by serial bumps. The aggregate record is
rebuilt from a listing under CAS with an hourly reconcile. Masters
live in a watched file so rotation never restarts the relay. TLS 5349
and the inter-region meter are deferred with reasons. The console's
master custody exposure is stated honestly.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Paul Campbell pushed by paul@keymail.dev 014339b1fd5295656595d0dddf0d370699a3976e parent 3d453c6
2 files changed, +634 −241
  • docs/superpowers/specs/2026-09-07-relay-stun-turn-design.md +397 −241
  • docs/superpowers/specs/CODEX-FINDINGS-RELAY.md +237 −0
diff --git a/docs/superpowers/specs/2026-09-07-relay-stun-turn-design.md b/docs/superpowers/specs/2026-09-07-relay-stun-turn-design.md
index 595edafad..ceee6269d 100644
--- a/docs/superpowers/specs/2026-09-07-relay-stun-turn-design.md
+++ b/docs/superpowers/specs/2026-09-07-relay-stun-turn-design.md
@@ -1,11 +1,12 @@
# Stage 4a: a platform STUN and TURN relay
-**Date:** 2026-09-07
-**Status:** draft for codex review, then Paul. Stage 4a of the media
-roadmap (`2026-09-07-media-platform-roadmap-design.md`), built under
-rulings 8, 10 and 11 there. One parameter differs from the roadmap's
-text and is flagged in §9 for Paul: credential life is four hours, not
-one.
+**Date:** 2026-09-07, revision 2
+**Status:** draft. Revision 1 was reviewed by GPT-6 via Codex; its 20
+findings are in `CODEX-FINDINGS-RELAY.md`, unedited, and every one is
+taken here, marked **(codex R1-N)**. Stage 4a of the media roadmap
+(`2026-09-07-media-platform-roadmap-design.md`), built under rulings 8,
+10 and 11 there. §9 lists what Paul is asked to decide, including three
+places this spec departs from the roadmap's text.
**Why now.** Oficina's Meet is peer-to-peer WebRTC and cannot name a
third-party STUN server, because suite ruling 6 forbids egress that is a
@@ -19,172 +20,258 @@ in §2.
## 1. What is built
-One new platform unit, `carlos-relay.service`, on every region box that
-the operator enables, running `carlos relay serve` from the platform
-binary, built on `pion/turn` v4, pure Go. It answers STUN binding
+One new platform unit, `carlos-relay.service`, on every region box the
+operator arms, running `carlos relay serve` from the platform binary,
+built on `pion/turn` v4.1.4, pure Go, which fetches and cross-compiles
+clean for the CI shards (checked 2026-09-07). It answers STUN binding
requests to anyone and TURN allocations to holders of a credential an
enabled app minted. It learns which apps are enabled from a record the
-console writes, meters what it relays per app, and refuses peers on
+console writes, binds every allocation to the identity that made it,
+ends allocations when their credential, app, region or its own
+authority lapses, meters what it relays per app, and refuses peers on
private networks.
Three console doors and a CLI verb, `carlos relay enable|status|disable`,
that turn the relay on for an app and deliver its keys as sealed config.
-An operator-side key per region, staged in `host.env` like the other
-platform secrets. A security-group change opening the relay's ports. A
-DNS name per region relay.
+An operator-side master key per region, staged in a file on the region's
+boxes and, as a console secret, in the console. A security-group change
+opening the relay's ports. A DNS name per region relay.
-Not built: an SFU, recording, an issuance API, client-side relay
-selection, TURN over TLS on 443, commercial quotas. §8 lists them.
+Not built: TURN over TLS on 5349 or 443, an SFU, recording, an issuance
+API, client-side relay selection, an inter-region byte meter, commercial
+quotas. §8 lists them with reasons.
## 2. The app-facing contract
This is what Meet codes against. It is fixed by this spec; the rest of
the design may change without touching it.
-**Environment.** Two variables reach the instance through the sealed
-`config.json` path, the way store credentials do:
+**Environment.** One variable reaches the instance through the sealed
+`config.json` path, the way store credentials do. Because the config
+renderer refuses quotes, backslashes and newlines in any value
+(`internal/appconfig/materialize.go:23`), the value is the **standard
+base64 encoding, with padding, of a JSON document**; the app decodes
+the base64, then parses the JSON. **(codex R1-1)**
-- `CARLOS_RELAY`: a JSON array, one element per region the app may use,
- nearest first (the instance's own region first, then the rest of the
- app's allowed list in the console's order). Absent or empty means no
+- `CARLOS_RELAY`: base64 of a JSON array, one element per region the
+ app may use, in the console's order (§5). Absent or empty means no
relay: the app shows its floor.
[{"region":"eu-west-1",
"urls":["stun:relay.eu-west-1.oncarlos.com:3478",
"turn:relay.eu-west-1.oncarlos.com:3478?transport=udp",
- "turn:relay.eu-west-1.oncarlos.com:3478?transport=tcp",
- "turns:relay.eu-west-1.oncarlos.com:5349"],
- "key":"<base64, 32 bytes>"}]
-
-- `CARLOS_RELAY_INCARNATION`: sixteen lowercase hex characters. It is
- not per region.
-
-**Minting, per call.** The app computes, for the region it wants:
-
- username = "<expiry>:<account>:<app>:<incarnation>"
- credential = base64std( HMAC-SHA256( key, username ) )
-
-where `expiry` is a Unix time in seconds no more than four hours ahead,
-`account` is the account's sqid (alphabet `[a-z0-9]`), `app` is the app
-name (`^[a-z][a-z0-9-]{0,31}$`), and `incarnation` is the value above.
-None of the fields can contain a colon, so the relay parses by splitting
-on `:` into exactly four fields. The app hands the browser an
-`RTCIceServer` list: `stun:` entries bare, `turn:` and `turns:` entries
+ "turn:relay.eu-west-1.oncarlos.com:3478?transport=tcp"],
+ "key":"<standard base64, with padding, of 32 raw bytes>",
+ "username_suffix":"abc:meet:0123456789abcdef:k1"}]
+
+ `region` is the AWS region name, which is also the vocabulary of the
+ relay hostnames. `username_suffix` is `<account>:<app>:<incarnation>:
+ <kid>`, supplied by the platform so the app never has to know its own
+ canonical account sqid or app name. **(codex R1-11, R1-12)** There is
+ no separate incarnation variable; the suffix carries it.
+
+**Minting, per call.** For the region it wants, the app computes:
+
+ key = base64_decode( element.key ) // 32 raw bytes
+ username = "<expiry>:" + element.username_suffix // ASCII
+ credential = base64_std( HMAC-SHA256( key, username ) )
+
+where `expiry` is a decimal Unix time in seconds, at most four hours
+ahead. The username has exactly five colon-separated fields:
+`expiry` (`[0-9]+`), `account` (`[a-z0-9]+`), `app`
+(`^[a-z][a-z0-9-]{0,31}$`), `incarnation` (`[0-9a-f]{16}`) and `kid`
+(`[a-z0-9]{1,8}`). No field can contain a colon. The app hands the
+browser an `RTCIceServer` list: `stun:` entries bare, `turn:` entries
carrying `username` and `credential`. STUN needs no credential.
-**Expiry** is checked by the relay on every authenticated request:
-allocate and every refresh, permission and channel bind. A credential
-that expires mid-call ends that call's relay at the next refresh, which
-pion sends every few minutes. Four hours covers any meeting; an app
-that wants longer re-mints and performs an ICE restart before the hour
-is up, and Meet does not need to. **(differs from the roadmap's one
-hour; §9)**
+**Expiry.** The relay checks the expiry on every authenticated request
+the browser's ICE agent sends (allocate, refresh, permission, channel
+bind) **and** ends the allocation at the expiry regardless of what the
+client sends (§3, "termination"), so a credential's deadline is a hard
+deadline on relayed traffic. An app that needs a call to outlive its
+credential re-mints and performs an ICE restart before the deadline.
+Four hours is the maximum, not a promise about meeting lengths.
+**(codex R1-4, R1-20; differs from the roadmap's one hour, §9)**
**What changes the incarnation.** Only `carlos relay disable` followed by
-`carlos relay enable`. Deploys, restarts, key rotation and region changes
-do not. An in-flight credential survives everything except disable.
-
-**What changes the key.** Region master rotation (§5) and the
-incarnation. Either re-renders the app's config, which restarts the
-instance within about a minute, as any config change does today. The
-relay accepts the previous master for twenty-four hours after rotation,
-so a credential minted before the restart keeps working for its life.
+`carlos relay enable`. A repeated enable, a region change, a deploy, a
+restart and a master rotation all keep it. **(codex R1-13)**
+
+**What changes the key.** A master rotation in a region changes that
+region's key and `kid`; a new incarnation changes every region's key.
+Either re-renders the app's config, which for unit-backed and
+exec-backed routes restarts the instance within about a minute, as any
+config change does today; a TCP-address route receives the file and is
+not restarted (`internal/appconfig/syncexec.go:11`), and must re-read it
+itself. The relay accepts a rotated-out master until its retirement
+time, at least 28 hours after rotation (§5), so a credential minted from
+the old key keeps working for its whole life. **(codex R1-10, R1-14)**
**Test vector.** Master is `SHA-256("carlos-relay-test-master")`,
-account `abc`, app `meet`, incarnation `0123456789abcdef`, expiry
-`1900000000`:
+account `abc`, app `meet`, incarnation `0123456789abcdef`, kid `k1`,
+expiry `1900000000`:
master 841d374df509340d6ab18a139a27ed0d9c5d61d0bdf306e4124ff913a7a92fb5 (hex)
key uMhwW4P/9O5dQUIs91t2CCHGhwJbeiM0xQMQ2G0CJLU=
- username 1900000000:abc:meet:0123456789abcdef
- credential khBx1BoommS1C69pSAgv0r0TM2H9IKDDE0M9OVr7VXQ=
+ username 1900000000:abc:meet:0123456789abcdef:k1
+ credential jRU479Y3GO45nrRHqxBf3BMeBHE6Fip1cwAccNv28MQ=
The key derivation is in §4; an app never needs it, only the key it is
-given. The vector is checked into `internal/relay/vector_test.go` and
-Meet's minting test uses the same numbers.
+given. The vector will be checked into `internal/relay/vector_test.go`
+by the plan, and Meet's minting test uses the same numbers.
+**(codex R1-20)**
**Logging.** The app logs which region it used and nothing else about
the credential. The relay logs the account, app and outcome of an
-authentication, never the credential or the derived key.
+authentication, never the credential or a key.
## 3. The relay unit
-`carlos-relay.service` is a §3 media service in the roadmap's terms: a
+`carlos-relay.service` is a media service in the roadmap's §3 terms: a
network-facing platform unit like the edge, not a codec helper.
- **Process.** `carlos relay serve`, from the platform binary, so it
- rides the same release, promote and adopt path as the edge. A new
- file in `internal/host/units/`, added to `enabledUnits` in
- `bootstrap.go` under a guard: the unit is written on every box but
- armed only where `host.env` carries `CARLOS_RELAY_MASTERS`, so a box
- without a master never listens. Bootstrap is idempotent and converges
- an existing box the way it does for every unit.
-- **Hardening.** Copied from `carlos-edge.service` and then cut down: no
- `CAP_NET_BIND_SERVICE` (3478 and 5349 are unprivileged), `User=carlos`,
+ rides the same release and promote path as the edge. A new file in
+ `internal/host/units/`, embedded and written by bootstrap like every
+ unit, with `ConditionPathExists=/etc/carlos/relay-masters` so a box
+ without a master file never listens. Added to `enabledUnits` and to
+ the list of units the systemctl plan **starts**, not only enables,
+ and to the adopt restart path beside `restartEdge`
+ (`internal/host/systemapply.go:568`), so a platform update restarts
+ the relay onto the new binary and a fresh box runs it without a
+ reboot. The staging bootstrap plan prints the same steps without
+ running them, as it does for the edge. **(codex R1-16)**
+- **Hardening.** From `carlos-edge.service` and then cut down: no
+ `CAP_NET_BIND_SERVICE` (3478 is unprivileged), `User=carlos`,
`ProtectSystem=strict`, `ReadWritePaths=/data/relay /run/carlos`,
- `RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX`,
- `MemoryHigh=20%`, `MemoryMax=25%`, `TasksMax=512`, `Restart=always`.
- It shares the box with the edge and must lose first: `Nice=5` and
- `CPUWeight=50` against the edge's default.
-- **Listeners.** UDP 3478 and TCP 3478 on the box's primary address; TLS
- 5349 when a certificate for the relay hostname is present. The relay
- allocation range is UDP 49152 to 65535, configured as
- `RelayAddressGeneratorPortRange` with `RelayAddress` set to the box's
- Elastic IP, which the process learns the way the edge does, from
- IMDS (`internal/host/agent.go:1761`), with `CARLOS_RELAY_PUBLIC_IP` as
- the override for a box without IMDS.
-- **Certificate for 5349.** The relay hostname is a platform host the
- edge already knows how to mint a certificate for; the edge adds
- `relay.<aws region>.<platform domain>` to its allowed set on boxes
- with a master, and the relay reads the certificate from the edge's
- autocert cache read-only, reloading when the file changes. If no
- certificate is present yet, the TLS listener is not started and the
- relay logs once; `turns:` is still advertised, because the browser
- falls through to the other URLs.
+ `RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX`. Memory is an
+ absolute budget, not a share: `MemoryMax=256M`, `MemoryHigh=192M`,
+ and `OOMScoreAdjust=800` so that under host memory pressure the relay
+ is the first process the kernel kills, ahead of the edge and tenants.
+ `Nice=5` and `CPUWeight=50` for CPU. The plan's flagship gate loads
+ edge, a tenant and the relay together on a `t4g.small` and confirms
+ the relay dies first. **(codex R1-19)**
+- **Listeners.** UDP 3478 and TCP 3478 on the box's primary address.
+ No TLS listener in version one (§8). The relay allocation range is
+ UDP 49152 to 65535, bound through the relay's own
+ `RelayAddressGenerator` (below) with the box's Elastic IP as the
+ advertised relay address, learned as the edge learns it, from IMDS
+ (`readPublicIP`, `internal/host/agent.go:1808`), with
+ `CARLOS_RELAY_PUBLIC_IP` as the override for a box without IMDS.
+ **(codex R1-20)**
- **STUN.** Binding requests are answered on every listener without
authentication, which is pion's default and the whole point of §0.
-- **Authentication.** A custom `AuthHandler` (§4). pion calls it on every
- authenticated request, which is where expiry is enforced.
-- **Peer policy.** A `PermissionHandler` that refuses `CreatePermission`
- and `ChannelBind` to loopback, link-local, RFC 1918 and unique-local
- ranges, the box's own VPC CIDR (from `CARLOS_RELAY_DENY_CIDRS`, set by
- the operator from tofu output), and the relay's own addresses.
- Refusals are counted per app.
-- **Limits.** Per app, per box, in the relay process: at most 64 live
- allocations, at most 30 new allocations a minute, allocation lifetime
- capped at ten minutes so a refresh is required (pion's default), and
- a byte budget of 20 GB per hour after which new allocations for that
- app are refused with 486 (allocation quota reached) until the hour
- rolls. Per box: at most 2048 allocations. These are operational
- limits, tuned by env with those defaults; commercial quotas are out
- of scope.
-- **Metering.** The relay wraps its UDP and TCP connections to count
- bytes in and bytes out per allocation, attributes them to the
- allocation's account and app, and once a minute appends a sample to
- the box's metrics chunks alongside the per-unit samples the metrics
- loop already flushes (`internal/host/metrics_loop.go`). The sample
- carries `relay_in`, `relay_out` and `relay_allocs` per app. Reading
- those samples into an app page is stage 5 work and not in this spec;
- the numbers exist from the first byte relayed.
-- **Authority record.** The relay reads `control/relay.json` (§5) from
- the deployment bucket on start and whenever `control/serial` changes,
- through the same fast-converge poll the host agent already runs
- (`internal/host/agent.go:1304`). If the record it holds is older than
- fifteen minutes it refuses new allocations and ends the ones it is
- carrying, per the roadmap's serving lease. A relay with no record at
- all serves STUN only.
+- **Our own address generator, for termination and metering.** pion has
+ no public call to end an allocation; its allocation manager is
+ internal and `Server` exposes only `AllocationCount` and `Close`. What
+ it does expose is the `RelayAddressGenerator` that creates each
+ allocation's relay socket, and its read loop deletes the allocation
+ when that socket returns an error
+ (`internal/allocation/allocation.go:301`, v4.1.4). So the relay
+ implements the generator itself: every relay socket it hands pion is
+ wrapped, recorded against the client's five-tuple, and can be closed
+ by the relay at any time, which ends that allocation inside pion
+ within one read. The wrapper also counts bytes in and out. That one
+ mechanism is how every deadline below is enforced and how metering is
+ attributed. **(codex R1-4, R1-17)**
+- **Owner binding.** pion locates a refresh, permission or channel bind
+ by the client's five-tuple and does not compare the authenticating
+ username to the one that created the allocation
+ (`internal/server/turn.go`, v4.1.4). The relay does: on
+ `OnAllocationCreated` it records five-tuple to identity, and its
+ `AuthHandler`, which receives the source address, refuses a username
+ whose identity differs from the one bound to a live allocation from
+ that source. The binding is cleared on `OnAllocationDeleted`.
+ **(codex R1-3)**
+- **Termination.** An allocation is ended, by closing its relay socket,
+ at the earliest of: its credential's expiry (a timer set at creation
+ from the username's expiry field); the app being disabled, its
+ incarnation changing, or this region leaving its list (on the next
+ authority read); the relay's authority lease expiring (below); the
+ app crossing its byte budget (below). Ending one allocation ends its
+ permissions and channels with it. pion's own maximum requested
+ lifetime is one hour; the relay does not rely on it. **(codex R1-4)**
+- **Authentication.** A custom `AuthHandler` (§4), called by pion on
+ every authenticated request, which is where the username's expiry and
+ the authority record are checked; pion verifies the message integrity
+ against the key the handler returns.
+- **Peer policy.** A `PermissionHandler` refuses `CreatePermission` and
+ `ChannelBind` to loopback, link-local (including the IMDS address),
+ RFC 1918 and unique-local ranges, the box's own private address and
+ VPC CIDR (from `CARLOS_RELAY_DENY_CIDRS`, set by the operator from
+ tofu output), and any other relay's private address. **The box's own
+ Elastic IP is allowed**: two clients that both fall back to this relay
+ hold relay candidates on that address at different ports and must
+ reach each other, and a packet from the box to its own EIP hairpins
+ through the internet gateway under the same security group as any
+ outside sender, so it can reach nothing the public cannot. Refusals
+ are counted per app. **(codex R1-8)**
+- **Limits**, all admitted under one mutex across every listener so
+ counts cannot race, with counters reset on process start:
+ - per app per box: 64 live allocations, 30 new allocations a minute;
+ - per account per box: 256 live allocations, whatever the app count;
+ - per box: 2048 live allocations;
+ - per app per box: a byte budget of 20 GB per rolling hour; crossing
+ it refuses new allocations with 486 and **ends the app's live
+ allocations**, because a budget that only stops new ones does not
+ bound anything. Failed allocations count against the per-minute
+ rate.
+ These are operational limits with those defaults, tunable by env;
+ commercial quotas are out of scope. **(codex R1-9)**
+- **Metering.** From the wrapped sockets, per allocation, attributed to
+ the identity bound at creation: bytes in from the client, bytes out
+ to peers, and allocation count. Once a minute the relay writes its
+ own chunk, `metrics/relay/<box>/<hour>.json`, a new additive key
+ with one writer, rather than appending to the host sampler's objects,
+ which that sampler rewrites from its own buffer
+ (`internal/host/metrics_loop.go`, `writeHourChunk`). The schema is
+ `{apps: {"<account>/<app>": {in, out, allocs}}, written_at}`, and the
+ raw readers and rollups learn the new key in the plan. An
+ inter-region meter is not built (§8). **(codex R1-17, R1-18)**
+- **Authority record and lease.** The relay reads `control/relay.json`
+ (§5) from the deployment bucket **every sixty seconds**, and
+ additionally whenever `control/serial` changes as seen by its own
+ poll of that object (the host agent's `fastConvergeLoop`,
+ `internal/host/agent.go:1437`, is a separate process; the relay runs
+ the same shape of poll itself). The lease clock is the time of the
+ last **successful** read; a failed read renews nothing. If that time
+ is more than fifteen minutes ago the relay refuses new allocations
+ and ends the ones it carries, per the roadmap's serving lease, and
+ resumes on the next successful read. A relay with no record yet
+ serves STUN only. Each read also writes a heartbeat,
+ `control/relay-ack/<box>.json`, carrying the record's `written_at`,
+ the read time, and which listeners are up, so status (§5) can tell a
+ live relay from a dead one. **(codex R1-5, R1-14, R1-20)**
## 4. Keys and verification
**Region master.** Thirty-two random bytes per region, generated by the
-operator with `carlos relay keygen` and staged as
-`CARLOS_RELAY_MASTERS="<kid>:<base64>[,<kid>:<base64>]"` in that
-region's `host.env`, the way `CARLOS_SECRETS_KEY` is staged today. The
-console holds every region's masters as sealed console secrets,
-`CARLOS_RELAY_MASTERS_<REGION>`, so it can derive app keys at enable.
-Two copies, both staged by hand, both rotated by hand; §5 gives the
-procedure. There is no bucket copy of a master, sealed or otherwise,
-because sealed config is decryptable on every host (roadmap §1).
+operator with `carlos relay keygen`, which prints a `kid` and the key.
+On each of the region's boxes they live in `/etc/carlos/relay-masters`,
+root-owned, mode 0640, group `carlos`, one per line:
+
+ <kid> <base64 key> <retire_at RFC 3339 or "-">
+
+The relay reads the file at start and re-reads it when it changes, so
+a rotation never restarts the process. A master past its `retire_at` is
+ignored. **(codex R1-10)**
+
+**Where the console's copy lives, and what that exposes.** The console
+must hold every region's masters to derive app keys at enable. It holds
+them as console secrets, `CARLOS_RELAY_MASTERS_<REGION>` (region name
+uppercased with hyphens as underscores, since variable names refuse
+hyphens, `internal/appconfig/appconfig.go:103`), and console secrets
+are sealed config like any app's. Sealed config is decryptable on every
+host (roadmap §1). So the honest statement is: **a compromised host can
+read every region's master**, not only the derived keys the roadmap
+already conceded, and with a master it can mint credentials for any
+app in that region, present and future, until the operator rotates.
+Per-region masters still mean a relay box itself holds only its own
+region's, and rotation of one region recovers that region. A design
+where only a region's boxes ever hold its master (the region's relay
+deriving app keys and writing them into config itself) is possible and
+is not version one, because it makes the relay a writer of app config
+with all the CAS and ordering that implies. **(codex R1-7; §9)**
**Derived key.** For an app, region and incarnation:
@@ -193,41 +280,64 @@ because sealed config is decryptable on every host (roadmap §1).
info = account || 0x00 || app || 0x00 || incarnation,
length = 32 )
-**Verification**, in the relay's `AuthHandler(username, realm, src)`:
+**Verification**, in the relay's `AuthHandler(username, realm, src)`,
+which pion calls once per authenticated request and which returns one
+key, so the master is chosen from the username, not by trial:
+**(codex R1-2)**
-1. Split the username on `:` into exactly four fields; anything else is
+1. Split the username on `:` into exactly five fields; anything else is
refused.
-2. Parse `expiry` as a decimal Unix time; refuse if it is in the past or
- more than four hours ahead.
-3. Look up `account/app` in the authority record; refuse if absent, if
- the app's incarnation differs, or if this relay's region is not in
- the app's allowed list.
-4. For the current master, then the previous one if within its
- twenty-four-hour grace: derive the key, compute the expected
- credential, and return `turn.GenerateAuthKey(username, realm,
- credential)`. pion verifies `MESSAGE-INTEGRITY` against that key, so
- a wrong credential fails inside pion; the handler itself never sees
- the credential the client sent.
-5. Refuse with the reason counted per app: bad shape, expired, unknown
- app, wrong incarnation, wrong region.
-
-The MD5 inside `GenerateAuthKey` is the long-term credential mechanism
-RFC 8489 mandates and is not ours to choose; the HMAC-SHA256 above is.
+2. Parse `expiry`; refuse if in the past or more than four hours ahead.
+3. If `src` has a live allocation bound to a different identity (§3,
+ owner binding), refuse.
+4. Look up `account/app` in the authority record; refuse if absent, if
+ the incarnation differs, or if this relay's region is not in the
+ app's list.
+5. Look up `kid` in the masters file; refuse if absent or retired.
+6. Derive the key, compute the expected credential, and return
+ `turn.GenerateAuthKey(username, realm, credential)`. pion verifies
+ `MESSAGE-INTEGRITY` against that key, so a wrong credential fails
+ inside pion; the handler never sees what the client sent.
+7. Count the refusal reason per app: bad shape, expired, bound
+ elsewhere, unknown app, wrong incarnation, wrong region, unknown
+ kid.
+
+The MD5 inside `GenerateAuthKey` is the long-term credential key of RFC
+8489's compatibility mechanism, which pion implements; RFC 8489 also
+defines a SHA-256 variant that pion does not. The HMAC-SHA256 above is
+ours. **(codex R1-20)**
## 5. Enable, disable, rotate
**Records.** Two, both additive:
-- `apps/<acct>/<app>/relay.json`, CAS-guarded like `store.json`, holding
- `{enabled, incarnation, regions, enabled_at, disabled_at}`. The
- incarnation is sixteen hex characters drawn at random at each enable,
- never a counter, so reuse across delete and name reuse is not a
- question.
-- `control/relay.json`, the authority record the relays read: `{apps:
- [{account, app, incarnation, regions}], written_at}`, rewritten by
- the console from the per-app records on every enable and disable,
- followed by a `control/serial` bump so every box notices within
- seconds.
+- `apps/<acct>/<app>/relay.json`, CAS-guarded like `store.json`:
+ `{enabled, incarnation, regions, config_generation, enabled_at,
+ disabled_at}`. The incarnation is sixteen hex characters drawn at
+ random at first enable and at each re-enable after a disable, never a
+ counter, so reuse across delete and name reuse is not a question.
+- `control/relay.json`, the authority record: `{apps: [{account, app,
+ incarnation, regions}], written_at}`. It is **always rebuilt from a
+ listing of every per-app record**, never patched from a snapshot, and
+ written with a CAS on its own etag; a conflict re-lists and retries.
+ A reconcile that does exactly that rebuild runs at console start and
+ hourly, so an operation that died between the per-app write and the
+ aggregate write is repaired within the hour and the aggregate can
+ never carry an app whose record says disabled. Followed by a
+ `control/serial` bump so relays notice within seconds; the bump is a
+ latency hint, not the lease. **(codex R1-5, R1-6)**
+
+**State transitions** for `relay.json`: **(codex R1-13)**
+
+| From | Operation | Incarnation | Keys re-derived | Allocations |
+|---|---|---|---|---|
+| none | enable | new | yes | none |
+| enabled | enable (repeat, same regions) | kept | no, no config write | untouched |
+| enabled | enable `--regions` (change) | kept | only for added regions; removed regions' entries dropped | ended in removed regions on next authority read |
+| enabled | disable | kept in record, app removed from aggregate | entries removed | ended everywhere on next authority read |
+| disabled | enable | **new** | yes, all regions | none |
+| any | app deleted | record removed with the app prefix; reconcile drops it from the aggregate | | ended on next authority read |
+| any | rotate `--region r` | kept | region `r` only, new kid | untouched; old kid accepted until retirement |
**`carlos relay enable [--regions r1,r2]`** (member, API-mode only, on
the store model in `store.go` and `internal/console/apistore.go`):
@@ -235,95 +345,135 @@ the store model in `store.go` and `internal/console/apistore.go`):
1. `apiAccountApp` gate; unknown and foreign are one byte-identical 404.
2. Refuse if the deployment has no relay regions (`CARLOS_RELAY_REGIONS`
in console config is empty) with a message naming the operator.
-3. Regions default to every region in `CARLOS_RELAY_REGIONS` ordered
- with the app's placement region first; `--regions` narrows.
-4. Write `relay.json` with a fresh incarnation and the region list.
-5. Derive one key per region from that region's current master and
- write `CARLOS_RELAY` (as `kind:"secret"`, since it carries keys) and
- `CARLOS_RELAY_INCARNATION` (as `kind:"env"`) through
- `appconfig.Apply`, the `writeEnv` shape in `internal/objstore/walk.go`.
- That bumps the app's config generation; the box re-renders and
- restarts the instance within about a minute.
-6. Rewrite `control/relay.json`, bump `control/serial`.
-7. Answer with the record; the CLI prints regions, incarnation and
- "config delivered, instance restarting".
-
-**`carlos relay status`** prints the record and, per region, whether
-that region's relay has acknowledged the current `control/relay.json`
-(each relay writes `control/relay-ack/<box>.json` with the
-`written_at` it holds, every minute), so a member can see "eu-west-1:
-current, ap-southeast-2: 3 minutes behind" without a box command.
-
-**`carlos relay disable`** clears the two env entries, marks the record
-disabled, rewrites `control/relay.json` without the app, bumps the
-serial. Every relay ends the app's allocations on its next record read.
-
-**Rotation** (`carlos relay rotate --region r`, operator):
-
-1. Operator runs `carlos relay keygen`, adds the new `kid:key` in front
- of the old one in that region's `host.env`, restarts
- `carlos-relay.service`; the relay now accepts both, new first.
-2. Operator updates the console secret for that region the same way.
-3. `carlos relay rotate` re-derives and re-delivers keys for every app
- enabled in that region; each instance restarts within a minute.
-4. After twenty-four hours the operator removes the old entry from
- `host.env` and the console secret. Any credential minted from the old
- key has expired by then.
-
-**App deletion** removes `relay.json` with the rest of the app prefix
-and the console's deletion walk drops the app from `control/relay.json`.
+3. Regions default to every region in `CARLOS_RELAY_REGIONS`, ordered
+ with the app's placement region first when the app has one, then
+ the console's order; `--regions` narrows. The order is fixed at
+ enable for the whole app; an app placed in two regions gets one
+ order, and its browsers try every listed relay anyway. **(codex R1-12)**
+4. Apply the transition above to `relay.json` (CAS).
+5. For each region needing keys, derive from that region's current
+ master and its kid, and write `CARLOS_RELAY` as one `kind:"secret"`
+ entry through `appconfig.Apply` (the `writeEnv` shape in
+ `internal/objstore/walk.go:554`). Record the resulting config
+ generation in `relay.json`. That bumps the app's config generation;
+ the box re-renders within about a minute.
+6. Rebuild `control/relay.json`, bump `control/serial`.
+7. Answer with the record and the config generation written. The CLI
+ prints regions, incarnation, and "config generation N written; the
+ instance restarts on delivery", never "delivered". **(codex R1-14)**
+
+**`carlos relay status`** reports, separately: desired state from
+`relay.json`; the config generation written; delivery, as "adopted" when
+the instances API shows the instance restarted after that generation,
+"written, awaiting delivery" otherwise, and "written, not restarted by
+the platform" for a TCP-address route; and per region, from
+`control/relay-ack/<box>.json`, the heartbeat age, whether the record it
+holds is the current one, whether its lease is live (heartbeat under
+fifteen minutes) and which listeners are up. A heartbeat older than
+five minutes is shown as "stale", so a dead relay cannot look current.
+**(codex R1-14)**
+
+**`carlos relay disable`** applies the transition, removes the
+`CARLOS_RELAY` entry through `appconfig.Apply`, rebuilds the aggregate,
+bumps the serial.
+
+**Rotation** (`carlos relay rotate --region r`, operator): **(codex R1-10)**
+
+1. Operator runs `carlos relay keygen`, which prints a new `kid` and key.
+2. Operator adds the line to `/etc/carlos/relay-masters` on each of the
+ region's boxes with `retire_at` `-`, and sets `retire_at` on the old
+ line to at least 28 hours ahead (24 hours for config to reach every
+ instance plus a four-hour credential life). The relay re-reads the
+ file; no restart, no allocation touched.
+3. Operator updates the console secret for the region to the new
+ `kid:key`.
+4. `carlos relay rotate --region r` re-derives and re-delivers keys for
+ every app enabled in that region, recording each app's new config
+ generation; `carlos relay status` for each shows when it is adopted.
+5. After every enabled app shows adopted, or after the retirement time
+ passes, the old line's `retire_at` does its work; the operator
+ removes the line at leisure. An app still minting from the old key
+ past retirement gets refused with "unknown kid" and re-reads its
+ config on restart.
+
+**App deletion** removes `relay.json` with the rest of the app prefix,
+and the reconcile drops the app from the aggregate within the hour, or
+at once if the deletion walk calls the rebuild, which the plan has it do.
## 6. Infrastructure
- `infra/modules/region-host/main.tf`: a `relay` variable, default off,
- that adds ingress for UDP 3478, TCP 3478, TCP 5349 and UDP 49152 to
- 65535 from `0.0.0.0/0`, and outputs the VPC CIDR for
- `CARLOS_RELAY_DENY_CIDRS`. Flagship turns it on per region.
+ that adds ingress for UDP 3478, TCP 3478 and UDP 49152 to 65535 from
+ `0.0.0.0/0`, and outputs the VPC CIDR for `CARLOS_RELAY_DENY_CIDRS`.
+ Flagship turns it on per region.
- DNS: an A record `relay.<aws region>.<platform domain>` to the region's
- Elastic IP, next to the existing EIP resource. One line per region.
-- `host.env` on each enabled region box gains `CARLOS_RELAY_MASTERS`
- and `CARLOS_RELAY_DENY_CIDRS`.
+ Elastic IP, next to the existing `aws_eip.host` resource.
+- Each armed box gains `/etc/carlos/relay-masters` (staged by the
+ operator, like `host.env`) and `CARLOS_RELAY_DENY_CIDRS` and
+ optionally `CARLOS_RELAY_PUBLIC_IP` in `host.env`.
- Console config gains `CARLOS_RELAY_REGIONS` and one
`CARLOS_RELAY_MASTERS_<REGION>` secret per region.
- Egress cost: every relayed byte leaves the box once. The
unit-economics spec gets a line for relay egress per region, and the
- meters in §3 are what fill it.
+ `out` meter in §3 is what fills it.
## 7. Testing
-- `internal/relay`: the key derivation and credential functions against
- the §2 test vector; the `AuthHandler` against a fixture authority
- record covering every refusal reason; the permission handler against
- each denied range; the limits against a fake clock; the meter against
- a wrapped in-memory conn.
-- An end-to-end test that starts the relay on loopback with a fixture
- master, mints a credential with the app-side code, and drives a pion
- TURN client through allocate, refresh past expiry, and a permission
- to a denied peer.
-- `internal/host`: the unit is embedded and written by bootstrap
- (`TestUnitsEmbedComplete`, `TestBootstrapWritesUnitsToStagingDir`),
- and armed only when the staging `host.env` carries a master.
-- `internal/console`: the three doors through `rawReq` and a
- `relayURL` fixture, asserting the byte-identical 404, the no-regions
- refusal, the config entries written, and `control/relay.json` after
- enable and disable.
+- `internal/relay`: derivation and credential against the §2 vector;
+ the `AuthHandler` against a fixture authority record and masters file
+ covering every refusal reason in §4, including a valid credential for
+ app B presented from a source bound to app A; the permission handler
+ against each denied range and the allowed EIP; limits against a fake
+ clock, including an account spreading allocations across apps and an
+ allocation ended by the byte budget; the meter against wrapped
+ in-memory sockets.
+- End to end, on loopback with a fixture master: a pion TURN client
+ allocates, refreshes, and keeps relaying until the credential's
+ expiry, at which point traffic stops with no further client request;
+ the same after a disable written to the fixture authority record;
+ two clients on one relay reaching each other's relay candidates; a
+ rotation, by rewriting the masters file, during which an allocation
+ on the old kid carries traffic uninterrupted; a lease lapse ending
+ allocations and a successful read restoring service.
+- `internal/host`: the unit is embedded, written and started by the
+ bootstrap plan when the staging root carries a masters file, and not
+ started when it does not; the adopt restart path names the relay.
+- `internal/console`: the three doors through `rawReq` and a `relayURL`
+ fixture: the byte-identical 404, the no-regions refusal, each row of
+ the transition table, the config entry written and removed, the
+ aggregate after concurrent enable and disable of two apps, and the
+ reconcile repairing an aggregate that disagrees with the records.
- `docs/site/cli.md` gains `### carlos relay` with a fenced example, or
the docsite gate fails.
-- Flagship gate, from the roadmap: a Meet call between two symmetric-NAT
- networks connecting through the Ireland relay; a credential minted for
- one app refused for another; a credential from before a disable
- refused after re-enable and its allocation ended; a relay with a
- record older than fifteen minutes refusing new allocations and ending
- carried ones; a permission to a private address refused; the meters
- moving; a 443-only network failing as documented.
+- Flagship gate, from the roadmap with the corrections above: a Meet
+ call between two symmetric-NAT networks connecting through the
+ Ireland relay; both browsers forced to relay and connecting through
+ it; a credential minted for one app refused for another, including on
+ an existing allocation's tuple; a credential from before a disable
+ refused after re-enable and its allocation ended; traffic stopping at
+ a credential's expiry with the browser doing nothing; a relay with a
+ lease older than fifteen minutes refusing new allocations and ending
+ carried ones; a permission to a private address refused; the relay
+ meter chunk showing the call's bytes; a rotation during a call with
+ the call unaffected; the relay dying first under memory pressure.
## 8. Not built
-- TURN over TLS on 443: needs a second address per box.
-- An issuance API: apps mint offline from delivered keys; the alternative
- is named in the roadmap if the trade ever changes.
-- Client-aware relay selection: the list is ordered by the instance's
- region; ICE picks.
+- **TURN over TLS (5349 or 443).** Certificate issuance for a non-HTTP
+ listener needs the relay hostname registered with the deployment's
+ ACME issuer, which is elected and may be another box
+ (`internal/edge/certsync.go`), and its warmer excludes such names
+ (`internal/edge/certwarm.go`); a listener that cannot start until a
+ certificate exists cannot trigger issuance by being visited. That is
+ its own small design. UDP and TCP on 3478 cover Meet's floor; the
+ networks that block 3478 mostly block 5349 too, and 443 needs a second
+ address per box. **(codex R1-15; §9)**
+- **An inter-region byte meter.** Classifying a peer address by AWS
+ region needs the published IP ranges on the box and a rule for
+ non-AWS peers; version one meters bytes in and out per app, which is
+ what the bill is made of. **(codex R1-18; §9)**
+- An issuance API: apps mint offline from delivered keys.
+- Client-aware relay selection: the list is ordered at enable; ICE picks.
- Commercial quotas and an app-page usage view: the meters exist, the
page is stage 5.
- An SFU: stage 4b, after this ships.
@@ -331,18 +481,24 @@ and the console's deletion walk drops the app from `control/relay.json`.
## 9. Decisions for Paul
-1. **Credential life four hours, not one.** The roadmap's recommendation
- said one hour. pion enforces expiry on every refresh, so a one-hour
- credential drops the relay from any call longer than an hour unless
- the app re-mints and restarts ICE. Four hours makes that Meet's
- problem never. Recommended.
-2. **Enable is member-level, no operator grant.** The store model gates
- `grant` on operators because it spends the deployment's cloud
- account. Enabling a relay spends bandwidth, bounded by the §3 limits,
- and the operator's consent is the region list in console config.
- Recommended over a per-app operator step that Meet would wait on.
-3. **Limits' defaults** (64 allocations, 30 a minute, 20 GB an hour per
- app per box). Numbers to tune; the mechanism is the decision.
-4. **`turns:` advertised before its certificate exists.** Harmless, since
- browsers fall through, and it means no config re-render when the
- certificate lands. Recommended.
+1. **Credential life four hours, not one.** The roadmap's text said one
+ hour. Expiry is now a hard deadline the relay enforces by ending the
+ allocation, so one hour would end any call longer than that unless
+ the app re-mints and restarts ICE. Recommended: four.
+2. **Enable is member-level, no operator grant.** The operator's consent
+ is the region list in console config and the limits in §3.
+ Recommended.
+3. **The console holds every region's master as sealed config**, with
+ the exposure stated in §4: a compromised host can read them all. The
+ alternative, relays deriving and writing app keys themselves, is
+ named there and deferred. Recommended for version one, with the
+ exposure written into the operator docs.
+4. **No TURN over TLS in version one.** Recommended, for the reasons in
+ §8; it returns as its own small spec when a network that needs it is
+ reported.
+5. **No inter-region meter in version one.** The roadmap named three
+ meters; this ships two. Recommended, with the third filed for stage
+ 5.
+6. **Limits' defaults** (64 and 30 a minute per app, 256 per account,
+ 2048 per box, 20 GB an hour per app). Numbers to tune; the
+ mechanisms are the decision.
diff --git a/docs/superpowers/specs/CODEX-FINDINGS-RELAY.md b/docs/superpowers/specs/CODEX-FINDINGS-RELAY.md
new file mode 100644
index 000000000..8ce662ea8
--- /dev/null
+++ b/docs/superpowers/specs/CODEX-FINDINGS-RELAY.md
@@ -0,0 +1,237 @@
+# Findings on the relay spec, revision 1
+
+Reviewed by **GPT-6 (Codex), 2026-09-07**, independently of the draft author, from the brief in `CODEX-REVIEW-BRIEF-RELAY.md`. Unedited apart from this note. Revision 2 of the spec records how each finding was taken.
+
+Reviewed commit `3d453c6b1f50b9c1fe8dc0a835e688e913e8c548`. Spec line references below refer to the [relay spec at that revision](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/docs/superpowers/specs/2026-09-07-relay-stun-turn-design.md). I independently reproduced all three cryptographic values in the test vector; they match. Pion findings use v4.1.4’s published API and source. No files were changed; no live TURN test was run.
+
+## R1-1 — P1 — The advertised JSON cannot pass through sealed config
+
+**Section:** §2, lines 44–57; §5, lines 241–246.
+
+`CARLOS_RELAY` contains JSON double quotes. The existing renderer rejects quotes in every value, including decrypted secrets. Writing it as `kind:"secret"` therefore does not solve delivery: the envelope can be stored, but rendering the app’s configuration fails. This blocks the contract Meet is supposed to implement.
+
+**Evidence:** [`ValidVarValue`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/appconfig/materialize.go:23) rejects quotes; [`RenderEnv`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/appconfig/materialize.go:49) decrypts secrets and applies that same check.
+
+**Correction:** Either specify a quote-free wire encoding and its app-side decoding, or explicitly extend config delivery to support JSON safely across both systemd and exec consumers. Test the complete seal → store → render → child-environment path before freezing §2.
+
+## R1-2 — P1 — `AuthHandler` cannot try the previous master
+
+**Section:** §4, verification step 4; §2’s rotation guarantee.
+
+The handler receives no password or request integrity value and returns exactly one key. Computing the current credential and returning its key ends the callback. Pion does not call it again with the previous master after integrity verification fails. Consequently, putting the new master first immediately rejects credentials derived from the old one.
+
+**Evidence:** Pion’s [`AuthHandler` signature](https://pkg.go.dev/github.com/pion/turn/v4#AuthHandler) returns one `[]byte`; [`authenticateRequest`](https://raw.githubusercontent.com/pion/turn/v4.1.4/internal/server/util.go) calls it once and performs one integrity check.
+
+**Correction:** Specify an implementable key-selection mechanism before declaring the username format fixed. Options include an authenticated key identifier in the credential contract or an explicit request-verification adapter supporting multiple candidate keys. Add interoperability tests exercising old and new credentials concurrently.
+
+## R1-3 — P1 — Authentication is not bound to the allocation’s owner
+
+**Section:** §3 authentication and metering; §4 verification.
+
+Pion v4.1.4 authenticates subsequent requests but locates their allocation by connection tuple without comparing the authenticated username to its original owner. A client can allocate using app A’s credential and then refresh or modify that allocation using app B’s valid credential on the same tuple. The proposed handler accepts B because B is enabled; nothing in the spec connects that decision to A’s allocation.
+
+This breaks the cross-app credential boundary and can separate the identity authorizing traffic from the identity being metered.
+
+**Evidence:** [`handleRefreshRequest`, `handleCreatePermissionRequest`, and `handleChannelBindRequest`](https://raw.githubusercontent.com/pion/turn/v4.1.4/internal/server/turn.go) use tuple-based lookup without an owner comparison. [RFC 8656 §21.3.3](https://www.rfc-editor.org/rfc/rfc8656.html#section-21.3.3) requires subsequent credentials to match the allocation’s original credentials.
+
+**Correction:** Bind each allocation immutably to its authenticated identity and enforce that binding on subsequent requests. Name the patched library or adapter that supplies this check. Test credential substitution on an existing tuple, rather than only changing fields in a new Allocate request.
+
+## R1-4 — P1 — Refusing refresh does not terminate expired traffic
+
+**Section:** §2, lines 75–81; §3 lifetime limits; §4 expiry verification.
+
+Rejecting an expired Refresh request does not delete the allocation. Existing ChannelData and Send indications bypass `AuthHandler`, so traffic can continue until existing allocation, permission, or channel state expires. The spec’s claim that the next refresh ends the relay is therefore false.
+
+The ten-minute allocation lifetime is also a default, not the stated cap: Pion accepts requested lifetimes below one hour, including values well above ten minutes.
+
+**Evidence:** Pion’s [request handlers](https://raw.githubusercontent.com/pion/turn/v4.1.4/internal/server/turn.go) return on authentication failure without deleting the allocation. Its [`allocationLifeTime`](https://raw.githubusercontent.com/pion/turn/v4.1.4/internal/server/util.go) accepts client-requested durations below one hour. The [roadmap’s stage 4a policy](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md:650) requires established allocations to end at credential expiry.
+
+**Correction:** Give allocations an absolute credential deadline and an active termination mechanism independent of client refreshes. Specify how the implementation closes individual allocations for expiry, disable, region removal, and lease expiry. Enforce the ten-minute maximum explicitly. Test continued traffic after the deadline with no further authenticated requests.
+
+## R1-5 — P1 — The serving lease has no renewal mechanism
+
+**Section:** §3 authority record; §5 records.
+
+The console rewrites the authority record only on enable and disable. Relays fetch it only at startup or after a serial change. After fifteen quiet minutes, every relay must terminate its allocations and refuse new ones—even when the deployment is healthy.
+
+Measuring age since receipt does not fix the missing refresh path. Nor may an unrelated serial bump renew authority merely by causing an old record to be reread.
+
+**Evidence:** Spec lines 169–175 and 226–230 define those triggers. [`BumpSerial`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/release/serial.go:46) is explicitly a latency hint whose failure is tolerated, not a lease renewal.
+
+**Correction:** Define periodic authority revalidation independently of serial changes, the exact freshness clock, and what successful observation renews the lease. Failed reads must not renew it. Test both a healthy deployment with no writes for more than fifteen minutes and an outage that continues past the lease deadline.
+
+## R1-6 — P1 — Concurrent rebuilds can restore revoked authority
+
+**Section:** §5 records, enable, disable, and deletion.
+
+CAS protection on each app’s `relay.json` does not serialize replacement of the shared `control/relay.json`. An enable for B can read a snapshot containing A, pause, and overwrite the aggregate after A’s disable has removed A. The replacement can carry a fresh `written_at`, restoring A’s authority.
+
+The multi-object operation also has no recovery protocol. Failure after changing the app record but before publishing the aggregate leaves contradictory state indefinitely.
+
+**Evidence:** Spec lines 221–230 give CAS semantics only to the per-app record. Lines 240–259 describe sequential writes without aggregate fencing or a reconciliation loop. The existing [`appconfig.Apply`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/appconfig/generation.go:113) guarantees one config-manifest write, not a transaction across these objects.
+
+**Correction:** Specify a fenced aggregate publisher that rebuilds after conflicts, plus durable reconciliation of incomplete operations. Include deletion and rotation in the same concurrency model. Test overlapping enable/disable operations, stale publishers, and failure after every durable write.
+
+## R1-7 — P1 — Console master custody contradicts the claimed exposure boundary
+
+**Section:** §4 region master; §5 rotation.
+
+The console holds every region’s masters as “sealed console secrets,” while the same paragraph says no master exists in the bucket, sealed or otherwise. If these use the existing sealed-config mechanism, both statements cannot be true: a compromised host can decrypt the console bundle and obtain every region’s master.
+
+That exposure is stronger than the roadmap’s conceded access to enabled apps’ derived keys. Masters also derive keys for future incarnations and future enables.
+
+**Evidence:** Spec lines 179–187. The [roadmap §1](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md:52) explicitly describes deployment-wide decryption; [stage 4a’s custody statement](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md:641) distinguishes regional masters from derived-key exposure.
+
+**Correction:** Name the console’s actual secret store, encryption authority, and readers. To preserve the stated boundary, its master copies must not use the deployment-wide sealed-config key. Keep the already-accepted derived-key exposure explicit.
+
+## R1-8 — P1 — The peer policy blocks two clients using the same relay
+
+**Section:** §3 peer policy; §7 flagship gate.
+
+Two clients forced to use TURN can receive relay candidates on the same region EIP at different allocation ports. Each then needs permission to reach the other’s candidate. The unconditional refusal of the relay’s own addresses rejects both.
+
+This affects the promised fallback when both clients cannot send UDP directly and connect to the relay over TCP or TLS. Falling through among URLs on the same server does not resolve it.
+
+**Evidence:** Spec lines 129–151 assign one public relay address and reject it as a peer. Pion’s [`PermissionHandler`](https://raw.githubusercontent.com/pion/turn/v4.1.3/server_config.go) receives a peer IP without its port, so that hook alone cannot distinguish a live relay allocation from another service on the box.
+
+**Correction:** Specify a safe relay-to-relay mechanism that preserves the prohibition on reaching arbitrary host services. An allocation-aware internal forwarding path needs an explicit design; an IP-only exception is insufficient. Add a test with both browsers forced to relay through the same region.
+
+## R1-9 — P2 — The operational limits do not bound bandwidth or accounts
+
+**Section:** §3 limits; §9 decisions 2 and 3.
+
+The 20 GB threshold refuses only new allocations. Existing allocations can keep transferring and refreshing after crossing it. One enabled app can therefore consume sustained bandwidth beyond the advertised budget.
+
+There is also no account limit. One account controlling 32 apps can fill all 2,048 allocation slots while every app remains within its individual cap.
+
+**Evidence:** Spec lines 153–160. The [roadmap’s policy](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md:650) requires allocation, rate, bandwidth, and lifetime bounds per app, account, and box.
+
+**Correction:** Define ongoing traffic enforcement and aggregate account/box limits. Specify atomic admission across all listeners, failed-allocation accounting, and restart behavior for counters. Test an established allocation crossing the byte limit and one account spreading allocations across multiple apps.
+
+## R1-10 — P2 — Rotation drops calls and has no durable retirement deadline
+
+**Section:** §2 rotation promises; §5 rotation procedure.
+
+Restarting `carlos-relay.service` destroys its in-memory allocations and sockets. Even after fixing R1-2, step 1 drops calls rather than preserving them.
+
+The master list also contains no activation or retirement timestamp. “Previous for twenty-four hours” cannot survive process restarts reliably without additional state. Finally, an old derived key can mint a fresh four-hour credential late in the grace period; removing its master at hour 24 does not imply every such credential has expired.
+
+**Evidence:** Spec lines 87–91, 181, and 263–271. Pion’s [server lifecycle](https://raw.githubusercontent.com/pion/turn/v4.1.4/server.go) keeps allocation state inside the running server.
+
+**Correction:** Specify master reload without process replacement, durable key epochs and retirement deadlines, and acknowledgements for relay installation and app key adoption. Define how minting near retirement is bounded. Test uninterrupted traffic during rotation, delayed app delivery, partial rollout, and relay restart during grace.
+
+## R1-11 — P2 — The minting contract does not supply its identity inputs
+
+**Section:** §2 environment and minting.
+
+The app receives the regional keys and incarnation but must independently know the canonical account sqid and CARLOS app name. §2 does not specify where either comes from. An app’s own team identifier or application name is not necessarily its deployment identity.
+
+The encoding instructions should also explicitly require decoding the supplied base64 key into 32 raw bytes before HMAC.
+
+**Evidence:** Spec lines 44–73. The existing [agent-owned config variables](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/appconfig/agentvars.go:19) do not supply these identities; the [exec backend’s injected variables](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/activator/backend_exec.go:660) do not supply them either.
+
+**Correction:** Deliver the canonical account/app identity, or an exact username suffix, as part of the contract. State the base64 alphabet, padding, key decoding, and username byte encoding explicitly. Supply app-side pseudocode using only documented inputs.
+
+## R1-12 — P2 — Region vocabulary and per-instance ordering are unresolved
+
+**Section:** §2 regional array; §5 enable; §6 console configuration.
+
+The example uses AWS region names, while CARLOS placement uses values such as `ie`. The spec requires the instance’s own region first but writes one ordered array into an app-wide config bundle. Two instances of one app in different regions cannot both receive their own region first through that operation alone.
+
+The master-variable template is also incomplete: literal substitution of `eu-west-1` produces a name containing hyphens, which the config API rejects.
+
+**Evidence:** [`edgesFile`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/host/edgesfile.go:72) explicitly separates AWS region from CARLOS placement. [`Apply`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/appconfig/generation.go:113) writes the app’s default bundle. [`ValidVarName`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/appconfig/appconfig.go:103) excludes hyphens.
+
+**Correction:** Choose one relay-region vocabulary, define its placement mapping and environment-variable normalization, and specify where per-instance ordering occurs. Test one app placed in two regions and the case where its local region is outside the allowed relay list.
+
+## R1-13 — P2 — Repeated enable contradicts the incarnation contract
+
+**Section:** §2 incarnation; §5 enable.
+
+§2 says incarnation changes only after disable followed by enable, and that region changes preserve it. Enable step 4 instead writes a fresh incarnation unconditionally. Retrying enable after a lost response can therefore invalidate live credentials. Using enable to change regions has the same problem.
+
+**Evidence:** Spec lines 83–85 versus 221–224 and 238–240.
+
+**Correction:** Define explicit state transitions: first enable, repeated enable, region update, repeated disable, and re-enable. Repeated enable must preserve the existing incarnation. Specify how removing a region terminates allocations there without rotating credentials in retained regions.
+
+## R1-14 — P2 — Status cannot establish delivery or current service health
+
+**Section:** §2 config restarts; §5 enable response and status.
+
+A successful config write does not prove that an instance received or loaded it. The proposed response nevertheless reports delivery and a restart. Some supported TCP-address routes receive files without ever being restarted by CARLOS; others are not eligible for delivery.
+
+The relay acknowledgement is also insufficient as specified. A dead relay’s last acknowledged `written_at` can equal the current record indefinitely. That establishes which record it once held, not whether it is alive, within lease, or listening on the advertised transports.
+
+**Evidence:** [`SyncExec`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/appconfig/syncexec.go:11) documents the route exceptions. [`store.go`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/store.go:24) explicitly distinguishes recorded writes from runtime delivery.
+
+**Correction:** Report desired state, publication state, config generation, observed delivery, fresh relay heartbeat, lease state, and listener readiness separately. Define acknowledgement expiry and supported route shapes. The member command must expose incomplete enable and disable operations without requiring box access.
+
+## R1-15 — P2 — Allowing the hostname does not cause certificate issuance
+
+**Section:** §3 certificate for 5349.
+
+Adding a relay hostname to the edge’s allowed set does not initiate an ACME order. TURN clients connect to 5349, where no listener starts until a certificate exists, so they cannot trigger the edge’s 443 certificate path.
+
+The existing proactive certificate warmer does not automatically solve this: it enumerates routed/advertised hosts and excludes hosts under the apps domain. Issuance also belongs to the elected deployment issuer, which may be another box.
+
+**Evidence:** [`warmSet`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/edge/certwarm.go:351) and [`issuerHostPolicy`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/edge/certsync.go:191).
+
+**Correction:** Register relay names with the deployment-wide issuance and challenge machinery, explicitly trigger issuance and renewal, and define certificate propagation. Specify how 5349 starts when its first certificate arrives. Test from an empty cache with a different box holding the issuer role.
+
+## R1-16 — P2 — Installing the unit does not activate or update its process
+
+**Section:** §3 process.
+
+Adding a service to `enabledUnits` enables it for boot but does not start it. Bootstrap deliberately starts only timers and paths. Platform self-update uses the same arming plan and explicitly restarts only the edge.
+
+The relay can therefore remain inactive after installation or continue running the old executable after a platform update. Sharing the binary does not provide process adoption.
+
+**Evidence:** [`systemctlPlan`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/host/bootstrap.go:116), [`systemApplier.arm`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/host/systemapply.go:661), and [`restartEdge`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/host/systemapply.go:568).
+
+**Correction:** Specify initial activation, conditional enable/disable convergence, required directory creation, and update/rollback handling for the relay process. Include the staging/bootstrap-plan path, which cannot assume it contains the live box’s `host.env`. Test installation and update without rebooting.
+
+## R1-17 — P2 — The metrics integration lacks a compatible schema and single writer
+
+**Section:** §3 metering.
+
+Existing metrics samples have no per-app relay fields. More seriously, the relay is a separate process, while the host metrics writer retains its own hourly sample buffer. Independently appending relay samples to the same objects is unsafe: subsequent host flushes can replace them using the host’s buffered view.
+
+**Evidence:** [`metrics.Sample`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/metrics/sample.go:29) contains host and unit samples only. [`writeHourChunk`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/host/metrics_loop.go:338) merges existing data only on the writer’s first write for that hour.
+
+**Correction:** Define an additive relay sample schema and one writer per object. Either let the host sampler collect relay counters over a local interface or give the relay separate chunk keys. Specify authenticated allocation attribution, counter reset semantics, and updates to raw readers and rollups.
+
+## R1-18 — P2 — The inter-region meter and acceptance requirement were dropped
+
+**Section:** §3 metering; §6 egress cost; §7 testing; §8 exclusions.
+
+`relay_allocs` does not replace the roadmap’s third meter: inter-region bytes. The proposed samples cannot distinguish those bytes from other traffic. The flagship gate also reduces the roadmap’s account-usage requirement to unspecified meters moving, while deferring the usage view.
+
+**Evidence:** The [roadmap’s metering contract](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md:662) requires received, sent, and inter-region bytes. Its [stage 4a gate](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md:931) requires those meters on the account’s usage line.
+
+**Correction:** Restore the inter-region meter, define how traffic is classified and counted without double charging, and retain allocation count as an additional metric. Align the acceptance gate and stage boundary with the parent rather than silently weakening them.
+
+## R1-19 — P2 — The unit does not establish that the relay loses first
+
+**Section:** §3 hardening.
+
+The edge already permits 80% of physical memory; the proposed relay permits another 25%, before console and other services. Those independent ceilings allow host-wide memory pressure before either service reaches its own ceiling. `Nice` and `CPUWeight` do not determine which process loses under memory pressure.
+
+**Evidence:** The [edge unit](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/host/units/carlos-edge.service:57) explains its existing memory allocation. Systemd documents [memory ceilings and CPU weights](https://raw.githubusercontent.com/systemd/systemd/main/man/systemd.resource-control.xml) separately from [OOM victim adjustment](https://raw.githubusercontent.com/systemd/systemd/main/man/systemd.exec.xml).
+
+**Correction:** Budget memory across the shared box and specify the intended memory-pressure priority explicitly. Validate that priority under combined edge, tenant, and relay load on a `t4g.small`; the listed settings alone do not establish it.
+
+## R1-20 — P2 — Remaining factual claims and references need correction
+
+**Sections:** §2, §3, §4, and §9.
+
+The following defects are separate from the mechanism failures above:
+
+| Claim | Evidence | Correction |
+|---|---|---|
+| The vector is already checked into `internal/relay/vector_test.go` (§2:103). | `internal/relay` does not exist at the reviewed commit. | State that the test will be added. Preserve the verified numerical vector. |
+| `agent.go:1761` is the public-IP lookup (§3:134). | The lookup is [`readPublicIP`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/host/agent.go:1808); line 1761 is blank. | Cite the function and its IMDS leaf read. |
+| `agent.go:1304` is the fast serial poll (§3:172). | That location is restart-failure bookkeeping. [`fastConvergeLoop`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/host/agent.go:1418) documents the actual loop. | Correct the citation and specify the relay’s separate polling or IPC integration. |
+| RFC 8489 mandates MD5 with no alternative (§4:214–215). | [RFC 8489 §18.5.1](https://www.rfc-editor.org/rfc/rfc8489.html#section-18.5.1) defines both MD5 and SHA-256; MD5 is the legacy compatibility algorithm. | Attribute MD5 to the selected Pion compatibility mechanism, not an unconditional RFC mandate. |
+| Pion sends refreshes, and longer calls must restart ICE “before the hour is up” (§2:77–80). | The app contract hands credentials to a browser; that client controls refreshes. The same section specifies four hours. | Describe client-driven refreshes and use the actual credential deadline. Remove the unsupported promise that four hours covers every meeting. |
+
+**Verdict: request changes.** The app contract cannot be delivered through the current config path, and the proposed mechanisms do not yet establish allocation ownership, timely expiry, reliable revocation, or the claimed master-rotation behavior.
\ No newline at end of file