carlos / platform Public

spec: relay revision 3 with Paul's rulings; roadmap amended to match

Six rulings recorded: four-hour credentials, member-level enable,
console-held masters with the exposure stated, no TURN over TLS, two
meters, the stated limits. Codex round 2's twelve findings taken:
etag-first aggregate publish, per-record op and phase with a
five-minute reconcile so interrupted operations finish, owner binding
keyed on the typed source address, metering at the two forwarding
legs with rejected ingress kept out of budgets, a config-ack the host's
config sync writes so status can say adopted, account and box rate
limits and token buckets, OOM arithmetic that makes the relay the
cheapest victim, EIP forwarding restricted to live relay ports, console
order for regions, rotate refusing until relays heartbeat the new kid.
The roadmap's stage 4a ingress, credential life, metering and gate are
amended in the same commit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Paul Campbell pushed by paul@keymail.dev 1 month ago 6b64d753c9e62d5a4b8550fa5d2cd317ec98c0f4 parent 014339b

3 files changed, +576 −245

docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md +24 −13
docs/superpowers/specs/2026-09-07-relay-stun-turn-design.md +347 −232
docs/superpowers/specs/CODEX-FINDINGS-RELAY-R2.md +205 −0

CI passed — run details

diff --git a/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md b/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md

index 50950e42c..512dca9bd 100644

--- a/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md

+++ b/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md

@@ -610,11 +610,13 @@ box, built on `pion/turn`, pure Go, in the same binary as the edge, a

- **Ingress.** UDP 3478 and a relay allocation range (49152 to 65535)

opened in the region security group and configured as the allocator's

- only range; TCP 3478 and TLS 5349 for networks that block UDP. The

- relay advertises the box's public Elastic IP. TURN over TLS on 443

- would need a second address on the box because 443 is the edge's;

- version one does not provide it and the docs say a 443-only network

- is the remaining case that fails. **(codex 15)**

+ only range; TCP 3478 for networks that block UDP. The relay advertises

+ the box's public Elastic IP. **No TURN over TLS in version one**

+ (Paul, 2026-09-07, stage 4a spec ruling 4): 5349 needs certificate

+ issuance for a non-HTTP listener, which is its own small design, and

+ 443 would need a second address on the box because 443 is the edge's.

+ The docs say a network that blocks 3478 is the remaining case that

+ fails. **(codex 15; amended by the stage 4a spec)**

- **Credentials, scoped per app and per region.** Each region has a

relay master key. At `carlos relay enable` the instance receives,

sealed through `config.json`, one derived key per allowed region:

@@ -622,7 +624,11 @@ box, built on `pion/turn`, pure Go, in the same binary as the edge, a

`CARLOS_RELAY_KEYS` alongside `CARLOS_RELAY_URLS`. The app mints TURN

REST credentials per call, `username =

expiry:account:app:incarnation`, `credential = HMAC(derived key,

- username)`, with a one-hour maximum. The relay holds a **relay

+ username)`, with a four-hour maximum (Paul, 2026-09-07, stage 4a

+ spec ruling 1; the relay ends the allocation at expiry, so one hour

+ would end any longer call). The exact wire shape, a base64-wrapped

+ JSON `CARLOS_RELAY` with a platform-supplied username suffix and a

+ key id, is the stage 4a spec's §2. The relay holds a **relay

authority record** it syncs from the route table the way the edge

syncs routes: for each enabled app, its allowed regions and current

incarnation. On a request it parses the username, refuses it unless

@@ -659,9 +665,12 @@ box, built on `pion/turn`, pure Go, in the same binary as the edge, a

lowest-latency guarantee and the roadmap does not claim one.

Client-aware relay selection is a separate later decision.

**(codex 15)**

-- **Metering.** Bytes received by the relay, bytes sent by it, and

- inter-region bytes are three meters, per app, because they are billed

- differently; the unit-economics spec gets the definitions. **(codex 26)**

+- **Metering.** Bytes forwarded from the client and bytes forwarded to

+ the client are two meters, per app, written by the relay to its own

+ metrics objects; the inter-region meter the first draft named, and

+ the account usage line, move to stage 5 with the usage view (Paul,

+ 2026-09-07, stage 4a spec ruling 5). The unit-economics spec gets the

+ definitions. **(codex 26; amended by the stage 4a spec)**

- **Custody language.** A platform relay is not egress under ruling 6:

the recipient is the operator the team already trusts with its pages.

A third-party relay is egress, and under doctrine §8 which third

@@ -929,14 +938,16 @@ failing, then playback working. **(codex 27, R3-9)**

job for a non-Ireland bucket refused at the door; the ladder measured,

not promised.

- **Stage 4a**: a Meet call between two symmetric-NAT networks connecting

- through the Ireland relay with a credential that expires in an hour;

- a credential minted for one app refused for another; a credential

+ through the Ireland relay with a credential that expires in four

+ hours, and both browsers forced to relay connecting through it; a

+ credential minted for one app refused for another; a credential

minted before a disable refused after re-enable, and the allocation

it held ended; a relay with an authority record older than fifteen

minutes refusing a new allocation and ending one already carrying

traffic; a relay permission to a private

- address refused; the three meters moving on the account's usage line;

- a 443-only network failing the way the docs say it will.

+ address refused; the two meters moving in the relay's metrics object;

+ a network that blocks 3478 failing the way the docs say it will. The

+ full gate is the stage 4a spec's §7.

- **Stage 4b**: a token minted for one app refused for another app's

room; a participant removed by the app dropped within a second and

unable to rejoin with its old token or a LiveKit-refreshed one, with

diff --git a/docs/superpowers/specs/2026-09-07-relay-stun-turn-design.md b/docs/superpowers/specs/2026-09-07-relay-stun-turn-design.md

index ceee6269d..a2fd9983f 100644

--- a/docs/superpowers/specs/2026-09-07-relay-stun-turn-design.md

+++ b/docs/superpowers/specs/2026-09-07-relay-stun-turn-design.md

@@ -1,12 +1,15 @@

# Stage 4a: a platform STUN and TURN relay

-**Date:** 2026-09-07, revision 2

-**Status:** draft. Revision 1 was reviewed by GPT-6 via Codex; its 20

-findings are in `CODEX-FINDINGS-RELAY.md`, unedited, and every one is

-taken here, marked **(codex R1-N)**. Stage 4a of the media roadmap

+**Date:** 2026-09-07, revision 3

+**Status:** rulings taken. Paul ruled on the six decisions in §9 on

+2026-09-07, each marked **(ruling)**; the roadmap's stage 4a text and

+gate are amended in the same commit so the two documents agree.

+Reviewed twice by GPT-6 via Codex: revision 1's 20 findings are in

+`CODEX-FINDINGS-RELAY.md`, revision 2's 12 in

+`CODEX-FINDINGS-RELAY-R2.md`, unedited; every one is taken, marked

+**(codex R1-N)** or **(codex R2-N)**. Stage 4a of the media roadmap

(`2026-09-07-media-platform-roadmap-design.md`), built under rulings 8,

-10 and 11 there. §9 lists what Paul is asked to decide, including three

-places this spec departs from the roadmap's text.

+10 and 11 there.

**Why now.** Oficina's Meet is peer-to-peer WebRTC and cannot name a

third-party STUN server, because suite ruling 6 forbids egress that is a

@@ -27,19 +30,22 @@ clean for the CI shards (checked 2026-09-07). It answers STUN binding

requests to anyone and TURN allocations to holders of a credential an

enabled app minted. It learns which apps are enabled from a record the

console writes, binds every allocation to the identity that made it,

-ends allocations when their credential, app, region or its own

-authority lapses, meters what it relays per app, and refuses peers on

+ends allocations when their credential, app, region, master or its own

+authority lapses, meters what it forwards per app, and refuses peers on

private networks.

Three console doors and a CLI verb, `carlos relay enable|status|disable`,

-that turn the relay on for an app and deliver its keys as sealed config.

-An operator-side master key per region, staged in a file on the region's

-boxes and, as a console secret, in the console. A security-group change

-opening the relay's ports. A DNS name per region relay.

-Not built: TURN over TLS on 5349 or 443, an SFU, recording, an issuance

-API, client-side relay selection, an inter-region byte meter, commercial

-quotas. §8 lists them with reasons.

+that turn the relay on for an app and deliver its keys as sealed config,

+plus an operator verb `carlos relay rotate`. An operator-side master key

+per region, staged in a file on the region's boxes and, as a console

+secret, in the console. A config-adoption acknowledgement the host's

+config sync writes, so status can say "adopted" truthfully. A

+security-group change opening the relay's ports. A DNS name per region

+relay.

+Not built, by ruling: TURN over TLS, an inter-region byte meter. Not

+built, by scope: an SFU, recording, an issuance API, client-side relay

+selection, commercial quotas. §8 lists them with reasons.

## 2. The app-facing contract

@@ -68,7 +74,9 @@ the base64, then parses the JSON. **(codex R1-1)**

relay hostnames. `username_suffix` is `<account>:<app>:<incarnation>:

<kid>`, supplied by the platform so the app never has to know its own

canonical account sqid or app name. **(codex R1-11, R1-12)** There is

- no separate incarnation variable; the suffix carries it.

+ no separate incarnation variable; the suffix carries it. The app

+ builds its ICE server list from whatever `urls` are present and

+ assumes nothing about their number or schemes.

**Minting, per call.** For the region it wants, the app computes:

@@ -77,8 +85,8 @@ the base64, then parses the JSON. **(codex R1-1)**

credential = base64_std( HMAC-SHA256( key, username ) )

where `expiry` is a decimal Unix time in seconds, at most four hours

-ahead. The username has exactly five colon-separated fields:

-`expiry` (`[0-9]+`), `account` (`[a-z0-9]+`), `app`

+ahead **(ruling 1)**. The username has exactly five colon-separated

+fields: `expiry` (`[0-9]+`), `account` (`[a-z0-9]+`), `app`

(`^[a-z][a-z0-9-]{0,31}$`), `incarnation` (`[0-9a-f]{16}`) and `kid`

(`[a-z0-9]{1,8}`). No field can contain a colon. The app hands the

browser an `RTCIceServer` list: `stun:` entries bare, `turn:` entries

@@ -91,7 +99,7 @@ client sends (§3, "termination"), so a credential's deadline is a hard

deadline on relayed traffic. An app that needs a call to outlive its

credential re-mints and performs an ICE restart before the deadline.

Four hours is the maximum, not a promise about meeting lengths.

-**(codex R1-4, R1-20; differs from the roadmap's one hour, §9)**

+**(codex R1-4, R1-20)**

**What changes the incarnation.** Only `carlos relay disable` followed by

`carlos relay enable`. A repeated enable, a region change, a deploy, a

@@ -99,13 +107,19 @@ restart and a master rotation all keep it. **(codex R1-13)**

**What changes the key.** A master rotation in a region changes that

region's key and `kid`; a new incarnation changes every region's key.

-Either re-renders the app's config, which for unit-backed and

-exec-backed routes restarts the instance within about a minute, as any

-config change does today; a TCP-address route receives the file and is

-not restarted (`internal/appconfig/syncexec.go:11`), and must re-read it

-itself. The relay accepts a rotated-out master until its retirement

-time, at least 28 hours after rotation (§5), so a credential minted from

-the old key keeps working for its whole life. **(codex R1-10, R1-14)**

+Either re-renders the app's config, which for unit-backed routes, and

+exec-backed routes whose bundle opts into file delivery, restarts the

+instance within about a minute, as any config change does today; a

+TCP-address route without file delivery receives nothing and the app

+must fetch its config another way (`internal/appconfig/syncexec.go:11`).

+**(codex R1-10, R1-14, R2-6)**

+**A credential keeps working for its whole life** provided the master

+that signed its key is not force-retired before its expiry. The

+rotation procedure in §5 guarantees that when followed: the old master

+is retired no earlier than four hours after the last instance stopped

+minting from it, as observed by the console, and no earlier than 28

+hours after rotation regardless. **(codex R2-4)**

**Test vector.** Master is `SHA-256("carlos-relay-test-master")`,

account `abc`, app `meet`, incarnation `0123456789abcdef`, kid `k1`,

@@ -119,7 +133,6 @@ expiry `1900000000`:

The key derivation is in §4; an app never needs it, only the key it is

given. The vector will be checked into `internal/relay/vector_test.go`

by the plan, and Meet's minting test uses the same numbers.

-**(codex R1-20)**

**Logging.** The app logs which region it used and nothing else about

the credential. The relay logs the account, app and outcome of an

@@ -141,56 +154,73 @@ network-facing platform unit like the edge, not a codec helper.

the relay onto the new binary and a fresh box runs it without a

reboot. The staging bootstrap plan prints the same steps without

running them, as it does for the edge. **(codex R1-16)**

-- **Hardening.** From `carlos-edge.service` and then cut down: no

- `CAP_NET_BIND_SERVICE` (3478 is unprivileged), `User=carlos`,

- `ProtectSystem=strict`, `ReadWritePaths=/data/relay /run/carlos`,

- `RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX`. Memory is an

- absolute budget, not a share: `MemoryMax=256M`, `MemoryHigh=192M`,

- and `OOMScoreAdjust=800` so that under host memory pressure the relay

- is the first process the kernel kills, ahead of the edge and tenants.

- `Nice=5` and `CPUWeight=50` for CPU. The plan's flagship gate loads

- edge, a tenant and the relay together on a `t4g.small` and confirms

- the relay dies first. **(codex R1-19)**

+- **Hardening and memory.** From `carlos-edge.service` and then cut

+ down: no `CAP_NET_BIND_SERVICE` (3478 is unprivileged),

+ `User=carlos`, `ProtectSystem=strict`, `ReadWritePaths=/data/relay

+ /run/carlos`, `RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX`.

+ Memory: `MemoryMax=256M`, `MemoryHigh=160M`, `OOMScoreAdjust=1000`.

+ The box's ceilings oversubscribe it (edge 80% of RAM, tenants' tier

+ 1 GiB, relay 256 MiB on a 2 GiB `t4g.small`), and no setting makes

+ the kernel's choice strict, so the relay is made the cheapest victim

+ by arithmetic and sheds itself first: the kernel's score is RSS as a

+ fraction of RAM in thousandths plus the adjustment, so the relay

+ scores at least 1000 and at most 1125, a tenant at most 1000 (1 GiB

+ cap, adjustment 500), the edge at most 800; and above `MemoryHigh`

+ the relay refuses new allocations and ends its oldest until it is

+ under, before the kernel is involved. `Nice=5` and `CPUWeight=50` for

+ CPU. **(codex R1-19, R2-8)**

- **Listeners.** UDP 3478 and TCP 3478 on the box's primary address.

- No TLS listener in version one (§8). The relay allocation range is

- UDP 49152 to 65535, bound through the relay's own

- `RelayAddressGenerator` (below) with the box's Elastic IP as the

- advertised relay address, learned as the edge learns it, from IMDS

- (`readPublicIP`, `internal/host/agent.go:1808`), with

- `CARLOS_RELAY_PUBLIC_IP` as the override for a box without IMDS.

- **(codex R1-20)**

+ No TLS listener **(ruling 4)**. The relay allocation range is UDP

+ 49152 to 65535, bound through the relay's own address generator

+ (below) with the box's Elastic IP as the advertised relay address,

+ learned as the edge learns it, from IMDS (`readPublicIP`,

+ `internal/host/agent.go:1808`), with `CARLOS_RELAY_PUBLIC_IP` as the

+ override for a box without IMDS.

- **STUN.** Binding requests are answered on every listener without

- authentication, which is pion's default and the whole point of §0.

-- **Our own address generator, for termination and metering.** pion has

- no public call to end an allocation; its allocation manager is

- internal and `Server` exposes only `AllocationCount` and `Close`. What

- it does expose is the `RelayAddressGenerator` that creates each

- allocation's relay socket, and its read loop deletes the allocation

- when that socket returns an error

- (`internal/allocation/allocation.go:301`, v4.1.4). So the relay

- implements the generator itself: every relay socket it hands pion is

- wrapped, recorded against the client's five-tuple, and can be closed

- by the relay at any time, which ends that allocation inside pion

- within one read. The wrapper also counts bytes in and out. That one

- mechanism is how every deadline below is enforced and how metering is

- attributed. **(codex R1-4, R1-17)**

+ authentication, which is pion's default and the reason this stage

+ exists.

+- **The relay owns every socket pion uses.** pion has no public call to

+ end an allocation; its allocation manager is internal and `Server`

+ exposes only `AllocationCount` and `Close`. What it does expose is the

+ `RelayAddressGenerator` that creates each allocation's relay socket

+ (`server_config.go:20`, returning a `net.PacketConn`), the listener

+ sockets the relay passes in, and an `OnAllocationCreated` callback

+ that supplies the relay address before pion starts reading from it

+ (`internal/allocation/allocation_manager.go:140`). pion's read loop

+ deletes the allocation when the relay socket returns an error

+ (`internal/allocation/allocation.go:301`). So the relay implements

+ the generator and wraps both kinds of socket: every relay socket is

+ recorded against its allocation at `OnAllocationCreated` and can be

+ closed by the relay at any time, which ends that allocation inside

+ pion within one read; and both relay and listener sockets count what

+ they forward and apply the token buckets below. That one mechanism is

+ how every deadline is enforced, how metering is attributed, and how

+ bandwidth is bounded. **(codex R1-4, R1-17)**

- **Owner binding.** pion locates a refresh, permission or channel bind

by the client's five-tuple and does not compare the authenticating

username to the one that created the allocation

- (`internal/server/turn.go`, v4.1.4). The relay does: on

- `OnAllocationCreated` it records five-tuple to identity, and its

- `AuthHandler`, which receives the source address, refuses a username

+ (`internal/server/turn.go:34`); it also tags every allocation's tuple

+ as UDP even for TCP clients, so the event's protocol string cannot

+ distinguish transports. The relay keys its binding on the **typed

+ source address**: the Go type of the `net.Addr` it receives

+ (`*net.UDPAddr` or `*net.TCPAddr`) plus its string, which pion passes

+ unchanged to `AuthHandler` and to the events. On

+ `OnAllocationCreated` it records that key to the identity fields

+ (account, app, incarnation); its `AuthHandler` refuses a username

whose identity differs from the one bound to a live allocation from

- that source. The binding is cleared on `OnAllocationDeleted`.

- **(codex R1-3)**

+ that key; `OnAllocationDeleted` removes only the binding whose relay

+ address matches the event's. **(codex R1-3, R2-3)**

- **Termination.** An allocation is ended, by closing its relay socket,

at the earliest of: its credential's expiry (a timer set at creation

from the username's expiry field); the app being disabled, its

incarnation changing, or this region leaving its list (on the next

- authority read); the relay's authority lease expiring (below); the

- app crossing its byte budget (below). Ending one allocation ends its

- permissions and channels with it. pion's own maximum requested

- lifetime is one hour; the relay does not rely on it. **(codex R1-4)**

+ authority read); the master named by its `kid` reaching `retire_at`

+ (on the next masters-file read or timer); the relay's authority lease

+ expiring (below); the app or account crossing a byte budget (below);

+ the relay shedding under memory pressure. Ending one allocation ends

+ its permissions and channels with it. pion's own maximum requested

+ lifetime is one hour; the relay does not rely on it. **(codex R1-4,

+ R2-4)**

- **Authentication.** A custom `AuthHandler` (§4), called by pion on

every authenticated request, which is where the username's expiry and

the authority record are checked; pion verifies the message integrity

@@ -199,35 +229,56 @@ network-facing platform unit like the edge, not a codec helper.

`ChannelBind` to loopback, link-local (including the IMDS address),

RFC 1918 and unique-local ranges, the box's own private address and

VPC CIDR (from `CARLOS_RELAY_DENY_CIDRS`, set by the operator from

- tofu output), and any other relay's private address. **The box's own

- Elastic IP is allowed**: two clients that both fall back to this relay

- hold relay candidates on that address at different ports and must

- reach each other, and a packet from the box to its own EIP hairpins

- through the internet gateway under the same security group as any

- outside sender, so it can reach nothing the public cannot. Refusals

- are counted per app. **(codex R1-8)**

-- **Limits**, all admitted under one mutex across every listener so

- counts cannot race, with counters reset on process start:

- - per app per box: 64 live allocations, 30 new allocations a minute;

- - per account per box: 256 live allocations, whatever the app count;

- - per box: 2048 live allocations;

- - per app per box: a byte budget of 20 GB per rolling hour; crossing

- it refuses new allocations with 486 and **ends the app's live

- allocations**, because a budget that only stops new ones does not

- bound anything. Failed allocations count against the per-minute

- rate.

+ tofu output), and any other relay's private address. The box's own

+ Elastic IP is allowed by the permission handler, because two clients

+ that both fall back to this relay hold relay candidates on that

+ address and must reach each other, **but only live relay allocation

+ ports are reachable there**: the permission handler sees only an IP,

+ so the wrapped relay socket enforces the port on `WriteTo`, dropping

+ any packet to the EIP whose destination port is not a currently

+ allocated relay port. Nothing else on the box is reachable through

+ the relay, whatever the security group says about source addresses.

+ Refusals and drops are counted per app. **(codex R1-8, R2-9)**

+- **Limits (ruling 6)**, all admitted under one mutex across every

+ listener so counts cannot race, with counters reset on process start

+ and the rolling-hour byte windows rebuilt from the relay's own

+ metrics object on start:

+ - per app per box: 64 live allocations, 30 new a minute, 20 GB a

+ rolling hour;

+ - per account per box: 256 live, 60 new a minute, 60 GB a rolling

+ hour, whatever the app count;

+ - per box: 2048 live, 500 new a minute;

+ - instantaneous: a token bucket of 8 Mbps per allocation and 200 Mbps

+ per box applied in the wrapped sockets on both forwarding legs,

+ dropping packets beyond the bucket, so a burst is bounded before it

+ reaches the edge's share of the NIC;

+ - crossing a byte budget refuses new allocations with 486 for the

+ app or account and **ends its live allocations**, because a budget

+ that only stops new ones bounds nothing. Failed allocations count

+ against the per-minute rates.

These are operational limits with those defaults, tunable by env;

- commercial quotas are out of scope. **(codex R1-9)**

-- **Metering.** From the wrapped sockets, per allocation, attributed to

- the identity bound at creation: bytes in from the client, bytes out

- to peers, and allocation count. Once a minute the relay writes its

- own chunk, `metrics/relay/<box>/<hour>.json`, a new additive key

- with one writer, rather than appending to the host sampler's objects,

- which that sampler rewrites from its own buffer

- (`internal/host/metrics_loop.go`, `writeHourChunk`). The schema is

- `{apps: {"<account>/<app>": {in, out, allocs}}, written_at}`, and the

- raw readers and rollups learn the new key in the plan. An

- inter-region meter is not built (§8). **(codex R1-17, R1-18)**

+ commercial quotas are out of scope. **(codex R1-9, R2-7)**

+- **Metering.** Two meters per app and one gauge, measured where pion

+ forwards and only for what it forwards: **(codex R1-17, R1-18, R2-5;

+ ruling 5)**

+ - `in`: payload bytes the relay accepted from the client and forwarded

+ to a peer, counted at the relay socket's `WriteTo`;

+ - `out`: payload bytes the relay forwarded from a peer to the client,

+ counted at the listener socket's `WriteTo` to the client's address,

+ attributed through the owner binding;

+ - `allocs`: live allocations at sample time.

+ Both meters are host egress and are what the bill is made of.

+ Peer bytes that arrive at a relay socket and are dropped for lack of

+ permission are counted separately as `rejected` and never against a

+ budget. Every sixty seconds the relay appends one sample

+ `{at, apps: {"<account>/<app>": {in, out, allocs, rejected}}}`, with

+ `in`, `out` and `rejected` as **deltas since the previous sample**,

+ to its own hourly object `metrics/relay/<box>/<hour>.json`, which it

+ alone writes: read, append, CAS-write, retry on conflict. On restart

+ it reads the current hour's object back, so earlier samples survive

+ and its rolling-hour budgets resume from them. The raw readers and

+ rollups learn the new key in the plan. An inter-region meter is not

+ built (§8).

- **Authority record and lease.** The relay reads `control/relay.json`

(§5) from the deployment bucket **every sixty seconds**, and

additionally whenever `control/serial` changes as seen by its own

@@ -240,8 +291,9 @@ network-facing platform unit like the edge, not a codec helper.

resumes on the next successful read. A relay with no record yet

serves STUN only. Each read also writes a heartbeat,

`control/relay-ack/<box>.json`, carrying the record's `written_at`,

- the read time, and which listeners are up, so status (§5) can tell a

- live relay from a dead one. **(codex R1-5, R1-14, R1-20)**

+ the read time, which listeners are up, the `kid`s currently loaded

+ from the masters file, and the last masters-file reload error if

+ any. **(codex R1-5, R1-14, R2-4)**

## 4. Keys and verification

@@ -254,24 +306,27 @@ root-owned, mode 0640, group `carlos`, one per line:

The relay reads the file at start and re-reads it when it changes, so

a rotation never restarts the process. A master past its `retire_at` is

-ignored. **(codex R1-10)**

-**Where the console's copy lives, and what that exposes.** The console

-must hold every region's masters to derive app keys at enable. It holds

-them as console secrets, `CARLOS_RELAY_MASTERS_<REGION>` (region name

-uppercased with hyphens as underscores, since variable names refuse

-hyphens, `internal/appconfig/appconfig.go:103`), and console secrets

-are sealed config like any app's. Sealed config is decryptable on every

-host (roadmap §1). So the honest statement is: **a compromised host can

-read every region's master**, not only the derived keys the roadmap

-already conceded, and with a master it can mint credentials for any

-app in that region, present and future, until the operator rotates.

-Per-region masters still mean a relay box itself holds only its own

-region's, and rotation of one region recovers that region. A design

-where only a region's boxes ever hold its master (the region's relay

-deriving app keys and writing them into config itself) is possible and

-is not version one, because it makes the relay a writer of app config

-with all the CAS and ordering that implies. **(codex R1-7; §9)**

+ignored for new requests and its live allocations are ended (§3).

+**(codex R1-10, R2-4)**

+**Where the console's copy lives, and what that exposes (ruling 3).**

+The console must hold every region's masters to derive app keys at

+enable. It holds them as console secrets, `CARLOS_RELAY_MASTERS_<REGION>`

+(region name uppercased with hyphens as underscores, since variable

+names refuse hyphens, `internal/appconfig/appconfig.go:103`), and

+console secrets are sealed config like any app's. Sealed config is

+decryptable on every host (roadmap §1). So the honest statement is: **a

+compromised host can read every region's master**, not only the derived

+keys the roadmap already conceded, and with a master it can mint

+credentials for any app in that region, present and future, until the

+operator rotates. Per-region masters still mean a relay box itself

+holds only its own region's, and rotation of one region recovers that

+region. The operator docs say this in those words. A design where only

+a region's boxes ever hold its master, the region's relay deriving app

+keys and writing them into config itself, is filed as the tightening

+and is not version one, because it makes the relay a writer of app

+config with all the CAS and ordering that implies. **(codex R1-7,

+R2-10)**

**Derived key.** For an app, region and incarnation:

@@ -288,8 +343,8 @@ key, so the master is chosen from the username, not by trial:

1. Split the username on `:` into exactly five fields; anything else is

refused.

2. Parse `expiry`; refuse if in the past or more than four hours ahead.

-3. If `src` has a live allocation bound to a different identity (§3,

- owner binding), refuse.

+3. If the typed source address has a live allocation bound to a

+ different identity (§3, owner binding), refuse.

4. Look up `account/app` in the authority record; refuse if absent, if

the incarnation differs, or if this relay's region is not in the

app's list.

@@ -309,96 +364,148 @@ ours. **(codex R1-20)**

## 5. Enable, disable, rotate

-**Records.** Two, both additive:

-- `apps/<acct>/<app>/relay.json`, CAS-guarded like `store.json`:

- `{enabled, incarnation, regions, config_generation, enabled_at,

- disabled_at}`. The incarnation is sixteen hex characters drawn at

- random at first enable and at each re-enable after a disable, never a

- counter, so reuse across delete and name reuse is not a question.

-- `control/relay.json`, the authority record: `{apps: [{account, app,

- incarnation, regions}], written_at}`. It is **always rebuilt from a

- listing of every per-app record**, never patched from a snapshot, and

- written with a CAS on its own etag; a conflict re-lists and retries.

- A reconcile that does exactly that rebuild runs at console start and

- hourly, so an operation that died between the per-app write and the

- aggregate write is repaired within the hour and the aggregate can

- never carry an app whose record says disabled. Followed by a

- `control/serial` bump so relays notice within seconds; the bump is a

- latency hint, not the lease. **(codex R1-5, R1-6)**

+**Records.** Two, both additive. **(codex R1-6, R2-1, R2-2)**

+- `apps/<acct>/<app>/relay.json`, CAS-guarded like `store.json`, is the

+ **source of truth** for the app: `{enabled, incarnation, regions,

+ op, phase, config_generation, enabled_at, disabled_at}`. `op` is a

+ counter bumped by every operation; `phase` is one of `config`,

+ `aggregate`, `done` and records how far the operation got. The

+ incarnation is sixteen hex characters drawn at random at first

+ enable and at each re-enable after a disable, never a counter, so

+ reuse across delete and name reuse is not a question.

+- `control/relay.json`, the authority record the relays read:

+ `{apps: [{account, app, incarnation, regions}], written_at}`. It is

+ **derived**, never patched. Publishing it is one fixed sequence: read

+ the aggregate's current etag; list every `relay.json`; build the

+ aggregate from records whose `enabled` is true, whatever their

+ `phase`; write with a CAS on the etag read **first**; on conflict

+ start the whole sequence again. Because the etag is read before the

+ listing, a disable that lands during the listing makes the write

+ fail rather than restore the app. Followed by a `control/serial` bump

+ so relays notice within seconds; the bump is a latency hint, not the

+ lease.

+**Every operation is a walk the reconcile can finish.** An operation

+writes its intent to `relay.json` first (`op` bumped, `phase` set),

+then does the config write, then the aggregate publish, advancing

+`phase` after each step with a CAS that carries the `op`, so a writer

+whose `op` has been superseded finds its CAS refused and stops. Before

+the config write it re-reads the record and refuses if the `op` is not

+its own. The **reconcile** lists every record every five minutes and

+at console start, and for any record not at `phase: done` performs the

+remaining steps of that record's operation idempotently (the config

+value is a pure function of the record and the masters, so re-applying

+it is harmless; the aggregate publish is always a rebuild). So an

+interrupted enable delivers its config within five minutes, an

+interrupted disable removes it within five minutes, and a repeated

+`enable` while a previous one is unfinished resumes it rather than

+skipping the config write. **(codex R2-2)**

+**When a disable is complete.** The app is out of every relay's

+authority when the aggregate has been published without it **and**

+every armed relay's heartbeat carries a `written_at` at or after that

+publish. `carlos relay status` shows "disable pending on <regions>"

+until then, and "revoked" after. Bounds: the publish is immediate on the

+happy path and within five minutes after a crash; each relay reads

+within sixty seconds of the publish; a relay that cannot read stops

+serving on its own fifteen-minute lease. **(codex R2-1)**

**State transitions** for `relay.json`: **(codex R1-13)**

|---|---|---|---|---|

**`carlos relay enable [--regions r1,r2]`** (member, API-mode only, on

-the store model in `store.go` and `internal/console/apistore.go`):

+the store model in `store.go` and `internal/console/apistore.go`;

+ruling 2):

1. `apiAccountApp` gate; unknown and foreign are one byte-identical 404.

2. Refuse if the deployment has no relay regions (`CARLOS_RELAY_REGIONS`

in console config is empty) with a message naming the operator.

-3. Regions default to every region in `CARLOS_RELAY_REGIONS`, ordered

- with the app's placement region first when the app has one, then

- the console's order; `--regions` narrows. The order is fixed at

- enable for the whole app; an app placed in two regions gets one

- order, and its browsers try every listed relay anyway. **(codex R1-12)**

-4. Apply the transition above to `relay.json` (CAS).

+3. Regions are every region in `CARLOS_RELAY_REGIONS`, **in the

+ console's order**, or `--regions` narrows them keeping that order.

+ There is no placement-first promise: placement names (`ie`,

+ `fleet/x`) are not AWS regions and an app may have several, so the

+ order is one fixed list for the app and every browser tries every

+ listed relay. **(codex R1-12, R2-11)**

+4. Apply the transition above to `relay.json` (CAS, `op` bumped,

+ `phase: config`).

5. For each region needing keys, derive from that region's current

master and its kid, and write `CARLOS_RELAY` as one `kind:"secret"`

entry through `appconfig.Apply` (the `writeEnv` shape in

- `internal/objstore/walk.go:554`). Record the resulting config

- generation in `relay.json`. That bumps the app's config generation;

- the box re-renders within about a minute.

-6. Rebuild `control/relay.json`, bump `control/serial`.

-7. Answer with the record and the config generation written. The CLI

- prints regions, incarnation, and "config generation N written; the

- instance restarts on delivery", never "delivered". **(codex R1-14)**

+ `internal/objstore/walk.go:602`). `Apply` touches the app's config

+ generation, a nonce plus timestamp (`internal/appconfig/

+ generation.go:40`); record that generation string in `relay.json`

+ and advance to `phase: aggregate`. **(codex R2-6, R2-12)**

+6. Publish `control/relay.json`, bump `control/serial`, advance to

+ `phase: done`.

+7. Answer with the record. The CLI prints regions, incarnation, and

+ "config generation <g> written; delivery is reported by `carlos

+ relay status`", never "delivered".

+**Config adoption, observed.** The host's config sync, after it renders

+an app's env file and restarts the route (or writes the file for an

+exec route that opted into file delivery), writes an additive

+acknowledgement `control/config-ack/<box>/<acct>/<app>.json` carrying

+the generation string it rendered, the time, and whether a restart

+happened. Nothing reads it today but `carlos relay status`; it is the

+first per-instance statement of "this config is what the process runs"

+the platform has, and the plan adds it to `internal/appconfig/sync.go`

+as its own small step. **(codex R1-14, R2-6)**

**`carlos relay status`** reports, separately: desired state from

-`relay.json`; the config generation written; delivery, as "adopted" when

-the instances API shows the instance restarted after that generation,

-"written, awaiting delivery" otherwise, and "written, not restarted by

-the platform" for a TCP-address route; and per region, from

-`control/relay-ack/<box>.json`, the heartbeat age, whether the record it

-holds is the current one, whether its lease is live (heartbeat under

-fifteen minutes) and which listeners are up. A heartbeat older than

+`relay.json` with its `op` and `phase`; the config generation written;

+delivery per box, "adopted at <time>" when a config-ack for that

+generation exists, "written, awaiting delivery" when it does not,

+"written, not restarted by the platform" for an exec route with file

+delivery, and "not delivered: route has no config delivery" for a

+TCP-address route without it; disable progress as above; and per

+region, from `control/relay-ack/<box>.json`, the heartbeat age, whether

+the record it holds is current, whether its lease is live, which

+listeners are up, and which `kid`s it has loaded. A heartbeat older than

five minutes is shown as "stale", so a dead relay cannot look current.

-**(codex R1-14)**

-**`carlos relay disable`** applies the transition, removes the

-`CARLOS_RELAY` entry through `appconfig.Apply`, rebuilds the aggregate,

-bumps the serial.

+**`carlos relay disable`** applies the transition (`phase: config`),

+removes the `CARLOS_RELAY` entry through `appconfig.Apply`, publishes

+the aggregate, marks `done`.

-**Rotation** (`carlos relay rotate --region r`, operator): **(codex R1-10)**

+**Rotation** (`carlos relay rotate --region r`, operator): **(codex

+R1-10, R2-4)**

1. Operator runs `carlos relay keygen`, which prints a new `kid` and key.

2. Operator adds the line to `/etc/carlos/relay-masters` on each of the

- region's boxes with `retire_at` `-`, and sets `retire_at` on the old

- line to at least 28 hours ahead (24 hours for config to reach every

- instance plus a four-hour credential life). The relay re-reads the

- file; no restart, no allocation touched.

+ region's boxes with `retire_at` `-`. The relay re-reads the file;

+ no restart, no allocation touched. Its next heartbeat lists the new

+ `kid`.

3. Operator updates the console secret for the region to the new

`kid:key`.

-4. `carlos relay rotate --region r` re-derives and re-delivers keys for

- every app enabled in that region, recording each app's new config

- generation; `carlos relay status` for each shows when it is adopted.

-5. After every enabled app shows adopted, or after the retirement time

- passes, the old line's `retire_at` does its work; the operator

- removes the line at leisure. An app still minting from the old key

- past retirement gets refused with "unknown kid" and re-reads its

- config on restart.

+4. `carlos relay rotate --region r` **refuses until every armed relay

+ in the region has heartbeated the new `kid`**, then re-derives and

+ re-delivers keys for every app enabled in that region as an

+ operation on each record, and prints the earliest permitted

+ retirement time for the old `kid`: 28 hours from now.

+5. `carlos relay status --region r` shows, per app, whether the new

+ generation is adopted. The old master's `retire_at` may be set to

+ the later of: 28 hours after rotation, and four hours after the last

+ app's adoption of the new generation. Setting it earlier is refused

+ by nothing but the docs, and ends any allocation still on the old

+ `kid`, which is the force-retirement case §2 excludes from its

+ guarantee.

+6. After `retire_at` the operator removes the line at leisure.

**App deletion** removes `relay.json` with the rest of the app prefix,

-and the reconcile drops the app from the aggregate within the hour, or

-at once if the deletion walk calls the rebuild, which the plan has it do.

+and the deletion walk publishes the aggregate so the app is gone from

+authority at once; the reconcile would catch it within five minutes

+otherwise.

## 6. Infrastructure

@@ -413,92 +520,100 @@ at once if the deletion walk calls the rebuild, which the plan has it do.

optionally `CARLOS_RELAY_PUBLIC_IP` in `host.env`.

- Console config gains `CARLOS_RELAY_REGIONS` and one

`CARLOS_RELAY_MASTERS_<REGION>` secret per region.

-- Egress cost: every relayed byte leaves the box once. The

+- Egress cost: every forwarded byte leaves the box once. The

unit-economics spec gets a line for relay egress per region, and the

- `out` meter in §3 is what fills it.

+ two meters in §3 are what fill it.

## 7. Testing

- `internal/relay`: derivation and credential against the §2 vector;

the `AuthHandler` against a fixture authority record and masters file

covering every refusal reason in §4, including a valid credential for

- app B presented from a source bound to app A; the permission handler

- against each denied range and the allowed EIP; limits against a fake

- clock, including an account spreading allocations across apps and an

- allocation ended by the byte budget; the meter against wrapped

- in-memory sockets.

+ app B presented from a typed source bound to app A, and a UDP and a

+ TCP client with the same address string holding separate bindings;

+ the permission handler against each denied range and the allowed

+ EIP; the relay socket dropping a packet to the EIP on a non-allocated

+ port; limits against a fake clock, including an account spreading

+ allocations across apps, an allocation ended by the byte budget, and

+ a token bucket dropping a burst; the meter against wrapped in-memory

+ sockets, checking that rejected peer ingress is not counted as `in`

+ and that both forwarding legs are counted; the hourly object surviving

+ a restart.

- End to end, on loopback with a fixture master: a pion TURN client

allocates, refreshes, and keeps relaying until the credential's

expiry, at which point traffic stops with no further client request;

the same after a disable written to the fixture authority record;

two clients on one relay reaching each other's relay candidates; a

rotation, by rewriting the masters file, during which an allocation

- on the old kid carries traffic uninterrupted; a lease lapse ending

- allocations and a successful read restoring service.

+ on the old kid carries traffic uninterrupted, then a `retire_at`

+ ending it; a lease lapse ending allocations and a successful read

+ restoring service.

- `internal/host`: the unit is embedded, written and started by the

bootstrap plan when the staging root carries a masters file, and not

- started when it does not; the adopt restart path names the relay.

-- `internal/console`: the three doors through `rawReq` and a `relayURL`

+ started when it does not; the adopt restart path names the relay; the

+ config sync writes a config-ack.

+- `internal/console`: the doors through `rawReq` and a `relayURL`

fixture: the byte-identical 404, the no-regions refusal, each row of

the transition table, the config entry written and removed, the

- aggregate after concurrent enable and disable of two apps, and the

- reconcile repairing an aggregate that disagrees with the records.

+ aggregate after concurrent enable and disable of two apps with the

+ etag-first ordering, an enable interrupted after `phase: config`

+ finished by the reconcile, a superseded writer refused by its CAS,

+ and `rotate` refusing until the fixture heartbeats list the kid.

- `docs/site/cli.md` gains `### carlos relay` with a fenced example, or

the docsite gate fails.

-- Flagship gate, from the roadmap with the corrections above: a Meet

- call between two symmetric-NAT networks connecting through the

- Ireland relay; both browsers forced to relay and connecting through

- it; a credential minted for one app refused for another, including on

- an existing allocation's tuple; a credential from before a disable

+- Flagship gate, as amended in the roadmap: a Meet call between two

+ symmetric-NAT networks connecting through the Ireland relay; both

+ browsers forced to relay and connecting through it **on the real

+ EIP**, and a probe to a non-allocated port on the EIP dropped; a

+ credential minted for one app refused for another, including on an

+ existing allocation's tuple; a credential from before a disable

refused after re-enable and its allocation ended; traffic stopping at

a credential's expiry with the browser doing nothing; a relay with a

lease older than fifteen minutes refusing new allocations and ending

carried ones; a permission to a private address refused; the relay

- meter chunk showing the call's bytes; a rotation during a call with

- the call unaffected; the relay dying first under memory pressure.

+ meter object showing the call's bytes on both legs; a rotation

+ during a call with the call unaffected; the relay shedding and then

+ being killed first under memory pressure with the edge still serving.

## 8. Not built

-- **TURN over TLS (5349 or 443).** Certificate issuance for a non-HTTP

- listener needs the relay hostname registered with the deployment's

- ACME issuer, which is elected and may be another box

+- **TURN over TLS (5349 or 443)** **(ruling 4)**. Certificate issuance

+ for a non-HTTP listener needs the relay hostname registered with the

+ deployment's ACME issuer, which is elected and may be another box

(`internal/edge/certsync.go`), and its warmer excludes such names

(`internal/edge/certwarm.go`); a listener that cannot start until a

certificate exists cannot trigger issuance by being visited. That is

- its own small design. UDP and TCP on 3478 cover Meet's floor; the

- networks that block 3478 mostly block 5349 too, and 443 needs a second

- address per box. **(codex R1-15; §9)**

-- **An inter-region byte meter.** Classifying a peer address by AWS

- region needs the published IP ranges on the box and a rule for

- non-AWS peers; version one meters bytes in and out per app, which is

- what the bill is made of. **(codex R1-18; §9)**

+ its own small design, to be written when a network that needs it is

+ reported. 443 additionally needs a second address per box. The

+ roadmap's stage 4a text is amended to match. **(codex R1-15, R2-10)**

+- **An inter-region byte meter** **(ruling 5)**. Classifying a peer

+ address by AWS region needs the published IP ranges on the box and a

+ rule for non-AWS peers; version one meters what is forwarded in each

+ direction, which is what the bill is made of. Filed for stage 5 with

+ the usage view; the roadmap's meters and gate are amended.

+ **(codex R1-18, R2-10)**

- An issuance API: apps mint offline from delivered keys.

-- Client-aware relay selection: the list is ordered at enable; ICE picks.

+- Client-aware relay selection: the list is in console order; ICE picks.

- Commercial quotas and an app-page usage view: the meters exist, the

page is stage 5.

- An SFU: stage 4b, after this ships.

- Any change to how apps signal: SSE plus POST stays Meet's.

-## 9. Decisions for Paul

-1. **Credential life four hours, not one.** The roadmap's text said one

- hour. Expiry is now a hard deadline the relay enforces by ending the

- allocation, so one hour would end any call longer than that unless

- the app re-mints and restarts ICE. Recommended: four.

-2. **Enable is member-level, no operator grant.** The operator's consent

- is the region list in console config and the limits in §3.

- Recommended.

-3. **The console holds every region's master as sealed config**, with

- the exposure stated in §4: a compromised host can read them all. The

- alternative, relays deriving and writing app keys themselves, is

- named there and deferred. Recommended for version one, with the

- exposure written into the operator docs.

-4. **No TURN over TLS in version one.** Recommended, for the reasons in

- §8; it returns as its own small spec when a network that needs it is

- reported.

-5. **No inter-region meter in version one.** The roadmap named three

- meters; this ships two. Recommended, with the third filed for stage

- 5.

-6. **Limits' defaults** (64 and 30 a minute per app, 256 per account,

- 2048 per box, 20 GB an hour per app). Numbers to tune; the

- mechanisms are the decision.

+## 9. Rulings

+Paul ruled on each of these on 2026-09-07. **(codex R2-10)**

+1. **(ruling)** Maximum credential life is four hours. The roadmap's

+ one-hour figure is amended.

+2. **(ruling)** Enable is member-level; the operator's consent is the

+ region list and the limits.

+3. **(ruling)** The console holds every region's master as sealed

+ config; the exposure is stated in §4 and the operator docs; the

+ relay-derives-and-writes design is filed as the tightening.

+4. **(ruling)** No TURN over TLS in version one; the roadmap's stage 4a

+ ingress text is amended.

+5. **(ruling)** Two meters in version one, in and out; the inter-region

+ meter and the usage line move to stage 5; the roadmap's metering

+ text and gate are amended.

+6. **(ruling)** The limits in §3 with their stated defaults, including

+ the token buckets.

diff --git a/docs/superpowers/specs/CODEX-FINDINGS-RELAY-R2.md b/docs/superpowers/specs/CODEX-FINDINGS-RELAY-R2.md

new file mode 100644

index 000000000..d26ee4143

--- /dev/null

+++ b/docs/superpowers/specs/CODEX-FINDINGS-RELAY-R2.md

@@ -0,0 +1,205 @@

+# Findings on the relay spec, revision 2

+Reviewed by **GPT-6 (Codex), 2026-09-07**, second pass, against revision 2 at `014339b`. Unedited apart from this note. Revision 3 records how each finding was taken.

+| Revision-1 finding | Disposition | Reason |

+|---|---|---|

+| R1-1 — JSON delivery | Closed | Standard padded base64 passes the existing config renderer. |

+| R1-2 — Master selection | Closed | `kid` selects one master before Pion’s single integrity check. |

+| R1-3 — Allocation ownership | Partial | Binding is specified, but its transport key remains ambiguous across UDP and TCP. |

+| R1-4 — Active termination | Closed | Wrapped relay sockets support the proposed expiry and revocation mechanism. |

+| R1-5 — Lease renewal | Closed | Independent sixty-second reads renew the lease; failed reads do not. |

+| R1-6 — Aggregate concurrency | Partial | Aggregate CAS helps, but ordering and incomplete-operation recovery remain defective. |

+| R1-7 — Master custody | Partial | The wider exposure is now explicit, but its departure from the parent remains a proposed decision. |

+| R1-8 — Same-relay peers | Partial | Allowing the EIP removes the blanket refusal; the hairpin security argument is insufficient. |

+| R1-9 — Operational limits | Partial | Account allocation limits and active byte-budget termination are added; aggregate bandwidth and rate bounds remain missing. |

+| R1-10 — Rotation | Partial | Hot reload and durable retirement are specified; late minting still breaks the lifetime guarantee. |

+| R1-11 — Minting inputs | Closed | `username_suffix`, raw-key decoding and byte encoding supply the required inputs. |

+| R1-12 — Regions and ordering | Partial | Vocabulary and app-wide ordering are chosen; placement-to-AWS mapping is unspecified. |

+| R1-13 — Repeated enable | Partial | Normal transitions preserve incarnation, but retries cannot repair interrupted delivery. |

+| R1-14 — Status and delivery | Partial | Heartbeat freshness improves status; config adoption has no defined observation mechanism. |

+| R1-15 — TLS certificates | Partial | Removing TLS avoids the issuance deadlock but leaves an unresolved parent-scope change. |

+| R1-16 — Process activation | Closed | Initial start and platform-update restart are now explicit implementation requirements. |

+| R1-17 — Metrics integration | Partial | Separate objects establish one writer, but measurement points and counter semantics remain incomplete. |

+| R1-18 — Inter-region usage | Open | The required meter and account-usage gate are explicitly deferred without an approved parent amendment. |

+| R1-19 — Memory priority | Partial | The ceiling is smaller and OOM bias explicit; neither establishes the claimed kill order. |

+| R1-20 — Factual claims | Partial | The substantive corrections are made; a stale code citation and nonexistent section reference remain. |

+Reviewed commit `014339b1fd5295656595d0dddf0d370699a3976e`. Section and line references below refer to the [revision-2 spec][spec]. I independently reproduced all three cryptographic vector values. No files were changed and no live TURN or EC2 test was run.

+The requested Pion checks pass: [`RelayAddressGenerator` returns `net.PacketConn`][pion-generator], permitting a wrapper; a relay-socket [read error calls `DeleteAllocation`][pion-reader]. [`OnAllocationCreated` supplies the relay address before the reader starts][pion-created], so the implementation can associate a registered wrapper with its allocation at that callback.

+## R2-1 — P1 — Aggregate CAS does not yet fence stale publication

+**Section:** §5, lines 319–328.

+The spec requires a listing and a conditional aggregate write, but does not require reading the aggregate’s ETag **before** starting that listing. This ordering is necessary:

+1. Builder B reads A’s enabled record.

+2. Disable A publishes an aggregate without A.

+3. B reads that aggregate’s new ETag.

+4. B conditionally publishes its earlier listing, restoring A.

+Every operation follows the stated text, and the final CAS succeeds.

+Even with the correct ordering, “the aggregate can never carry an app whose record says disabled” is false. A crash after the per-app disable leaves the old aggregate until reconciliation, potentially an hour later. Successful sixty-second reads keep renewing that stale authority; the fifteen-minute lease does not bound this failure.

+**Correction:** Specify ETag read → complete listing → conditional publication, restarting the entire sequence on conflict. Define when revocation is complete, how pending revocation is exposed, and a recovery bound consistent with the serving guarantee.

+## R2-2 — P2 — An interrupted enable can permanently omit credentials

+**Section:** §5, transition table and enable steps 4–6.

+Enable first records `enabled`, then writes config. If it stops between those writes, retrying the same enable takes the transition that expressly performs “no config write.” The reconciler repairs only the aggregate. The app can therefore become authorized indefinitely without ever receiving `CARLOS_RELAY`.

+Separate object CAS also does not fence stale config writers. An enable can pause before `appconfig.Apply`, a disable can finish removing credentials, and the old enable can subsequently restore them. Re-enable and rotation introduce equivalent ordering problems.

+**Evidence:** The specified sequence is at lines 335 and 353–360; [`appconfig.Apply`][apply] conditionally changes one bundle, not the relay record and bundle together.

+**Correction:** Record a durable operation version and incomplete phase. Reconcile both config and aggregate toward that version, prevent superseded operations from publishing config, and make repeated enable resume unfinished delivery.

+## R2-3 — P2 — The owner-binding key can collide across transports

+**Section:** §3 owner binding; §4 verification step 3.

+The spec moves between “five-tuple” and “source address” without defining their mapping. A global map built from Pion’s event protocol and address strings is unsafe: Pion uses separate allocation managers for UDP and TCP listeners, but constructs the allocation tuple with `Protocol: allocation.UDP` even for TCP clients. UDP and TCP connections can have identical textual source and destination endpoints.

+The callbacks therefore permit two independent allocations to collide in the proposed owner map, causing false refusals, overwritten ownership or deletion of another allocation’s binding.

+**Evidence:** Pion’s [listener-specific managers][pion-server] and [`handleAllocateRequest` tuple construction][pion-turn].

+**Correction:** Define the binding key using the actual client transport and typed source address, with listener scope where necessary. Do not use the event’s protocol string as the client transport. Define the bound identity fields explicitly and require deletion to remove only the binding belonging to that allocation. This is buildable with the specified listeners; it does not require abandoning the adapter.

+## R2-4 — P2 — Retirement still contradicts the credential-lifetime guarantee

+**Section:** §2, lines 106–108; §4 master retirement; §5 rotation.

+Twenty-eight hours of overlap does not guarantee four hours of remaining validity for every old-key credential. An instance retaining the old config can mint at hour 27 with expiry hour 31. The master retires at hour 28, so subsequent authenticated requests fail before the promised expiry.

+Retirement also is not among §3’s active termination deadlines. Thus retirement can reject refreshes while existing traffic continues on that allocation. The spec must choose the intended behavior.

+The rollout has a second gap: heartbeat records identify authority and listeners, but not loaded masters. Updating a file is not an acknowledgement that every relay successfully loaded the new `kid` before the console distributes it.

+**Correction:** Define an issuer cutoff and maximum credential deadline, or begin the final lifetime allowance only after acknowledged issuer migration. Specify forced-retirement behavior and narrow the uninterrupted-lifetime guarantee accordingly. Include loaded `kid` and reload failure state in relay status before switching issuance.

+## R2-5 — P2 — Socket counters do not measure the promised traffic

+**Section:** §3 metering, lines 221–233; §6 egress, lines 416–418.

+Relay-socket `WriteTo` measures client-to-peer payload. Relay-socket `ReadFrom` measures peer ingress **before** permission checks. Peer-to-client traffic leaves through a different socket, `TurnSocket`.

+Consequently:

+- The proposed `out` measurement misses return-leg egress.

+- Counting raw relay reads against an app’s budget lets unsolicited packets consume that budget and terminate its allocations, although Pion never forwards those packets.

+- The hourly object schema does not say whether `in`, `out` and `allocs` are minute deltas, hourly totals or process totals. It also does not specify how a restart preserves earlier measurements in the same hourly object.

+**Evidence:** Pion’s [relay read, permission checks and client writes][pion-reader]; the existing host writer’s [first-write merge behavior][metrics-writer] illustrates the restart issue that a separate object key alone does not solve.

+**Correction:** Define directions relative to the host, payload versus wire accounting, and the treatment of rejected ingress. Measure successful forwarding on both outbound legs. Specify counter intervals, allocation-count semantics, restart recovery and retry deduplication before assigning the schema to rollups.

+## R2-6 — P2 — Config adoption cannot be inferred from the named API

+**Section:** §2 config delivery; §5 enable response, status and rotation.

+The spec asks `appconfig.Apply` for a resulting config generation and asks the instances API to establish that an instance restarted after adopting it. These are not the existing contracts:

+- [`Apply`][apply] returns an unset count.

+- [`Generation`][generation] is a timestamp and nonce, not a numeric generation `N`.

+- The [instances response][instances] contains release version, health, upstreams and serving boxes, without an applied-config identity or restart acknowledgement.

+A later restart alone would not prove which config the process loaded.

+TCP-address delivery is also overstated. [`SyncExec`][syncexec] delivers only when the bound bundle opts into `exec_delivery:"file"`; an unopted route receives nothing.

+**Correction:** Specify an additive config identity and per-instance acknowledgement of successful application/restart, carried through host status and the instances API. Define hibernating and failed-delivery states. Distinguish TCP routes awaiting external restart from routes with file delivery disabled.

+## R2-7 — P2 — Account and box bandwidth and rate bounds remain absent

+**Section:** §3 limits, lines 208–220.

+The account cap bounds concurrent allocations, but not allocation churn or traffic. An account can spread requests across apps to multiply the 30-per-minute allowance and the 20 GB budget. The box likewise has no aggregate admission-rate or traffic bound.

+A rolling-hour byte allowance also permits a concentrated burst before termination; it does not establish bandwidth sharing with other apps or the edge.

+**Evidence:** The [parent’s policy][roadmap-policy] requires allocation, rate, bandwidth and lifetime bounds per app, account and box. Revision 2 supplies only some of those dimensions.

+**Correction:** Specify aggregate account and box rate/traffic limits, including burst behavior and enforcement on ongoing forwarding. Distinguish the rolling-hour allowance from the mechanism that bounds instantaneous traffic.

+## R2-8 — P2 — The memory settings do not guarantee that the relay dies first

+**Section:** §3 hardening, lines 145–153; §7 flagship gate.

+`OOMScoreAdjust=800` is a bias added to a memory-dependent score, not a strict priority. A sufficiently large tenant with the existing `+500` adjustment can outrank a smaller relay. The repository permits a [1 GiB children tier][cgroup]. Linux’s [OOM scoring implementation](https://raw.githubusercontent.com/torvalds/linux/v6.12/mm/oom_kill.c) combines memory usage and the adjustment.

+The host budget also remains incomplete. The [edge ceiling][edge-unit] is 80% of physical memory; adding 256 MiB on a 2 GiB box leaves roughly 154 MiB beneath those ceilings for the console, other services and kernel. Their requirements are not budgeted.

+**Correction:** State a complete supported-box budget and the memory-pressure conditions under which relay-first shedding is required. Establish that behavior through an explicit shedding mechanism or a justified scoring envelope. A single successful load test cannot establish the unconditional ordering currently claimed.

+## R2-9 — P2 — The EIP hairpin argument does not establish peer isolation

+**Section:** §3 peer policy, lines 197–207; §7 tests.

+The exception permits every UDP port on the EIP, not just live relay allocations. “The same security group” does not imply “nothing the public cannot reach”: rules can distinguish source addresses, and relayed traffic originates from the box’s address. The infrastructure already accepts source-restricted additional rules through [`extra_ingress`][infra]. AWS also documents [stateful connection tracking](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/security-group-connection-tracking.html), rather than a universal public-access predicate.

+This finding does **not** assert that EC2 hairpinning fails. The problem is that successful hairpinning would not prove the claimed isolation. The specified loopback test cannot establish the EC2 address translation and security-group behavior either.

+**Correction:** Restrict same-EIP forwarding to live relay allocation destinations using the wrapped socket’s destination-port visibility, or explicitly define and approve the broader public-service policy and its infrastructure constraints. Require the same-relay gate on the actual EIP deployment as well as loopback.

+## R2-10 — P2 — Proposed deferrals are not yet an authoritative stage boundary

+**Section:** §§8–9 and the parent’s stage 4a contract.

+Revision 2 clearly labels proposed departures, which improves honesty, but it still declares the app contract fixed while the parent requires:

+| Parent requirement | Revision-2 replacement |

+|---|---|

+| TCP 3478 and TLS 5349; 443-only networks are the documented remaining transport gap | No TLS listener |

+| Received, sent and inter-region bytes on the account’s usage line | Two byte counters in raw chunks; inter-region meter and usage view deferred |

+| One-hour credential maximum | Four hours |

+| Regional master custody with deployment-wide derived-key exposure | Deployment-wide master exposure through console secrets |

+**Evidence:** [Parent ingress and credentials][roadmap-ingress], [custody and policy][roadmap-policy], [metering][roadmap-meters], and [stage 4a acceptance gate][roadmap-gate]. Section 9 contains recommendations for Paul, not recorded decisions.

+**Correction:** Obtain and record the decisions, amend the parent and stage gate, and make the documents agree before freezing the implementation-plan inputs. If those decisions are not taken, retain the parent requirements. This is a contract-resolution requirement, not an objection to choosing a smaller stage.

+## R2-11 — P2 — Placement ordering still lacks its conversion rule

+**Section:** §5 enable step 3, lines 348–352.

+The spec chooses AWS names for relay regions but still orders them using “the app’s placement region.” Placement values such as `ie` and `fleet/<name>` are not AWS region names. For multiple placements, the text also does not select which one determines the first relay.

+**Evidence:** [`edgesFile`][edgesfile] explicitly distinguishes and publishes `Placement` and AWS `Region`.

+**Correction:** Specify resolution through the published placement/region mapping, with deterministic behavior for multiple placements, missing mappings and local regions outside the allowed list. Alternatively, use console order unconditionally and remove the placement-first promise.

+## R2-12 — P3 — Two references remain incorrect

+**Section:** §3 STUN; §5 enable step 5.

+The STUN paragraph refers to “§0,” which this spec does not contain. The `writeEnv` citation points to `internal/objstore/walk.go:554`, which discusses deletion concurrency; [`writeEnv` begins at line 602][writeenv].

+**Correction:** Name the intended section and cite the function at its current location.

+**Verdict: request changes.** P1 and P2 problems remain. Aggregate publication can restore revoked authority under an ordering permitted by the text; delivery recovery, rotation, metering and status still need concrete contracts before the implementation plan.

+[spec]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/docs/superpowers/specs/2026-09-07-relay-stun-turn-design.md

+[pion-generator]: /home/paulca/.claude/jobs/f1f6099c/tmp/gomodcache/github.com/pion/turn/v4@v4.1.4/server_config.go:20

+[pion-reader]: /home/paulca/.claude/jobs/f1f6099c/tmp/gomodcache/github.com/pion/turn/v4@v4.1.4/internal/allocation/allocation.go:301

+[pion-created]: /home/paulca/.claude/jobs/f1f6099c/tmp/gomodcache/github.com/pion/turn/v4@v4.1.4/internal/allocation/allocation_manager.go:140

+[pion-server]: /home/paulca/.claude/jobs/f1f6099c/tmp/gomodcache/github.com/pion/turn/v4@v4.1.4/server.go:77

+[pion-turn]: /home/paulca/.claude/jobs/f1f6099c/tmp/gomodcache/github.com/pion/turn/v4@v4.1.4/internal/server/turn.go:34

+[apply]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/appconfig/generation.go:113

+[generation]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/appconfig/generation.go:40

+[instances]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/console/apiinstances.go:132

+[syncexec]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/appconfig/syncexec.go:11

+[metrics-writer]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/host/metrics_loop.go:338

+[edge-unit]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/host/units/carlos-edge.service:57

+[cgroup]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/cgroup/cgroup.go:60

+[infra]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/infra/modules/region-host/main.tf:77

+[edgesfile]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/host/edgesfile.go:72

+[writeenv]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/internal/objstore/walk.go:602

+[roadmap-ingress]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md:611

+[roadmap-policy]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md:641

+[roadmap-meters]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md:662

+[roadmap-gate]: /home/paulca/amadan.net/carlos/platform/.claude/worktrees/media-roadmap-spec/docs/superpowers/specs/2026-09-07-media-platform-roadmap-design.md:931

\ No newline at end of file