Media on CARLOS: a staged roadmap

Date: 2026-09-07, revision 6 Status: rulings taken. Paul ruled on all eleven decisions in §7 on 2026-09-07; each is marked (ruling). Everything not so marked is a (decision) that a stage spec may overturn cheaply.

Brainstormed with Claude from discussions #107, #112 and #113 and the requirements the Meet session sent on 2026-09-07. Reviewed four times by GPT-6 via Codex, independently of the author: revision 1's 27 findings are in CODEX-FINDINGS-MEDIA.md, revision 2's 14 in CODEX-FINDINGS-MEDIA-R2.md, revision 3's 9 in CODEX-FINDINGS-MEDIA-R3.md, revision 4's 8 in CODEX-FINDINGS-MEDIA-R4.md, and revision 5's 2 in CODEX-FINDINGS-MEDIA-R5.md, all unedited. Round 5 returned accept: no P1 or P2 remained, and its two wording corrections are in this revision. Each place a finding changed the text is marked (codex N), (codex R2-N), (codex R3-N) or (codex R4-N). §10 names the two parts declined.

This is a roadmap, not an implementation spec. Each stage below gets its own design and plan when it is picked up. The roadmap exists so the first stage is built with the later ones in view, so nobody builds a stage out of order, and so the security properties are named before any URL shape is.

0. The goal in one sentence

Any app on CARLOS can serve, transform, stream and relay media at the performance people expect from a CDN or a media API, with nothing to install and no third-party service in the path, and every object can be public, secret or private.

"No third party" is not taste. Suite ruling 6 forbids egress that is a property of the product, and a media CDN, an image API, a TURN service or a hosted SFU that every app dials by default is exactly that. The platform is the one recipient a team has already chosen, so media handled by the platform is not egress at all. That distinction is what makes this a platform feature rather than a per-app settings field, and the plan keeps that language throughout.

1. Where things stand, measured on main

Discussion #107 recorded most of this on 2026-08-31. Re-checked today, and corrected where the reviews found the citations wrong.

• Apps already have object storage, bucket-per-app, credentials delivered sealed through config.json (object-storage spec §5). Reads and writes go app-to-S3 directly. The spec says at line 27 that a byte-proxy in the data path is "permanently out of scope". Reads must cross that line for a CDN to exist. Writes never will.
• Sealed config is decryptable on every host. The deployment's decryption key sits in host.env on every box (config-secrets spec §4) and internal/appconfig/materialize.go:30 renders secrets to env at adopt time. A compromised host therefore already exposes every app's store keys and secrets, not only its own residents'. Nothing in this plan makes that better or worse, and no claim below may pretend a per-region secret changes it. (codex R2-2)
• The store already scans. internal/objstore has a ClamAV scanner, a resumable sweep that tags every object with a verdict, a scan queue of one object per item under the app's own prefix, and, when a store's scan block has enforcement on, a Deny on GetObject for objects tagged infected, written into the store user's policy (internal/objstore/aws.go:211, objstore.go:82). There is no upload hook by design; the sweep counts what it has not seen. Any new reader of the bucket must carry the same deny whenever the store user does, or it is a bypass. (codex 4, 26, R4-8)
• The edge cannot read a store bucket. The region host role is scoped to the deployment bucket and its own region's data bucket (infra/modules/region-host/main.tf:106, residency argument in the header at line 6). A new grant is most of the serving design.
• The shared response cache is unsafe for anything authorised. The key is host, method, path and query; a hit returns before any handler runs (internal/edge/edge.go:1300). kind:status already opts out and caches for itself because an unbounded keyspace in a process-global LRU lets one tenant evict every other. Media has that problem and an authorisation one.
• Purge is local to one box. The purge listener is a unix socket on the host it runs on, and the Carlos-Purge response header acts on the edge that saw the response. Nothing distributes an invalidation across regions today. (codex 9)
• An edge keeps serving from last-converged state through a control bucket outage (multi-region ingress spec, line 123). Any authority media adds to the route record inherits that: a disconnected edge will honour a revoked key until told otherwise. (codex R2-6)
• The tick path is local. activatorDeliverer reaches an instance through the activator and its unix socket on the same box, and refuses routes without one (internal/host/schedule_wiring.go:220). Once-timers are filed on an instance's control socket, bound to that host (internal/schedule/control.go:28). Neither crosses a box. (codex R2-4, R2-8)
• Range and conditional handling come from http.ServeFile for kind:static, and only there. A streaming S3 handler has to build its own HTTP mapping, and S3 GetObject serves one range per request. (codex 7)
• The edge is identity-free by design (internal/edge/edge.go:1). It never reads a cookie. Suite identity is a home-minted Ed25519 assertion, audience-bound, single-use, two minutes long, delivered by auto-POST; it cannot be replayed per image. Suite doctrine ruling 3 says authorisation is never cached at any scope, and leak item 11 names a cacheable personalised response at the platform edge as a cross-person leak.
• The doctrine's visibilities are rungs plus an orthogonal link. Ruling 15: an item is private, granted or team, and may additionally carry a read-only link. team is not public. One item can be reachable by its roster and by a link at the same time, so the storage model must let both doors reach one object. (codex 11, R2-3)
• Doctrine §8 says the recipient of any egress is the app's choice, never the team's, disclosed in the README's first paragraph and on every artefact produced. A team-pasted relay URL is not admissible. (codex 12, R2-12)
• Members already control placement. carlos apps place and carlos instances create --region exist, and the instances API reports which boxes and regions serve a host (internal/console/instancesplacement.go:21). What does not exist is a runtime signal an app can read about its host's transport capabilities. (codex R2-13)
• Wildcard certificates cover exactly two labels under the apps domain (internal/edge/wildcard.go:186), and app names are ^[a-z][a-z0-9-]{0,31}$ (internal/release/release.go:239), so a one-label media name for any app of up to 26 characters is also a valid app name, and an app name may end in a hyphen, which a DNS label may not (internal/provision/provision.go:223). Hostnames in the account namespace are also claimed by the instances door (internal/console/apiinstances.go:571), which lets an app take any unclaimed label, not only its own name. (codex 24, R2-10, R4-4, R4-5)
• The fleet is small ARM. Four production regions on Graviton t4g.small, one t4g.medium, burstable at a 20% per-vCPU baseline with the same two vCPUs on both sizes; the medium adds memory, not CPU. No GPU, no ffmpeg, no package installs at bootstrap. Remote hosts are Raspberry Pis. (codex 20)
• The Go tree is pure and CI keeps it so (.amadan/ci.d/20-cross@5 sets CGO_ENABLED=0 deliberately). That rule exists so the CLI cross-compiles for the Macs people run it on. It constrains the tree, not the box, and §3 sets the rule for native helpers.
• There is no generic media job queue. The scan queue is the one bucket-backed queue in the tree and its header explains where jobs must live so app deletion finds them. Stage 3 reuses that shape rather than inventing a second one. (codex 19)
• UDP is a stated non-goal of the capabilities spec (line 931). A relay is therefore a platform service next to the edge, not an app capability.
• A tunnel-backed host cannot carry Connection: Upgrade (remote-host-pi design, line 63).
• Real demand exists. Docs ships an "insert image" menu item disabled. Meet is being designed now and needs STUN for calls to connect across most home and mobile networks. Woodstar runs its own media service as a tenant app, the shape this plan makes unnecessary.
• Apps already hold immutable keys. rastrillo's blob store is content-addressed and presigns locally. A content-addressed key can be cached forever, and a sealed (E2EE) blob is opaque bytes the platform can serve but never transform.

2. Public, secret, private

Three words, three delivery doors, two storage namespaces. The doors are how bytes reach a browser, not a new authorisation model: the app still decides who may see what, from its own grants, and the doctrine's rungs map onto the doors as the last row of the table says.

| Door | Who can fetch | What the edge checks | Bytes cached | Reads from | |---|---|---|---|---| | public /p/ | anyone with the URL | nothing | by object key, long-lived, in every allowed region | public/ only | | secret /s/<cap>/ | anyone holding a valid capability | the capability's signature, scope, namespace and expiry | by object key, once per object, in every allowed region | protected/ | | private /_carlos/media/ on the app's origin | a person the app admits, now | asks the app on every request | by object key, on the app's home edge, after the app says yes | protected/ | | doctrine | private, granted and team rungs deliver through the private door; a read-only link on any of them is a secret capability naming the same object; nothing is public unless the app's own rule makes an item world-readable | | | |

Two namespaces, not three. The media prefix has public/ and protected/. The public door reads public/ and nothing else, so rewriting a URL from one door to another cannot reach a protected object: /p/x reads public/x, which does not exist for a protected key. The secret and private doors both read protected/, which is what lets one object be reachable by its roster through the app's check and by a link through a capability at the same time, as the doctrine's orthogonal link requires. Derivatives and renditions live under the same namespace as their source. (codex 1, R2-3)

Changing visibility is a lifecycle, not a copy. Between the two namespaces it is: copy to the new namespace, delete the old key and its derivatives, append a purge covering them, fence any transform or job in flight against the old key (§4, stage 1, "publication fence"), and report completion only when every box that could hold a copy, in any region the app has ever been allowed, has either acknowledged the purge or let its serving lease expire (§4, stage 1, "purge"). Bytes a browser already holds cannot be withdrawn, and the docs say so. (codex R3-1) Within protected/, secret and private are not a move at all: the app mints a capability or stops minting them, and an existing capability dies at its expiry or when its key id is revoked. Stage 1's gate tests public-to-protected and link-to-roster transitions on cold and warm caches. (codex R2-1)

Public is a plain URL on the media host. Content-addressed keys get immutable caching. Mutable keys get whatever the app says with Cache-Control on the object, defaulting to a short lifetime, and the app can purge.

Secret is the doctrine's read-only link and what S3 presign, Mux and Cloudinary do: a bearer capability, signed by the app, naming an object or a prefix in protected/, expiring. Anyone holding it can fetch; the bytes are the same for every holder, so a popular file is cached once and revocation lags by the expiry. The capability is a path segment, not a query string, so a playlist's relative references to its segments resolve under the same capability and every child request is authorised without rewriting. (codex 6) What the signature covers is defined in stage 1, not left to the implementer. (codex 5)

Private is identity-bound, and the only way to honour ruling 3 is to not decide at the edge. On every request the edge asks the app: "may this request read this key?" The app answers from its own grants and session, in a millisecond, and the edge serves from its byte cache or fetches the object. This is the shape of nginx auth_request, with these properties the reviews forced into the open:

• Private media is served on the app's own origin, under a reserved path the edge intercepts ahead of the shared-cache lookup and ahead of any proxy branch, because that is the only place the app's session cookie reaches. A separate media host never sees a host-only cookie, a custom domain cannot set one for the apps domain, and cross-site cookies cannot be relied on across supported browsers and privacy settings. The cost is user content on the app's origin, and the mitigations in stage 1 (nosniff, a CSP sandbox on every response, attachment disposition for anything not image, audio, video, font or PDF, and no inline SVG or HTML ever) are what make that acceptable. (codex 2, R2-14)
• The check travels over the route's own backend transport, never through a cache: the activator and unix socket for an instance on the same box, the route's TCP dial for a service route, the tunnel for a remote host. A private request for an app whose instance lives on another box reaches the home edge by the existing backhaul, as every other request for that route does, and the home edge's media cache is the one that serves it. Private bytes are therefore cached on the app's home edge only; public and secret bytes are cached in every allowed region. The check fails closed on timeout, redirect, or anything but a 200, and runs before HEAD, before a 304, before a range is answered and before a cache hit is admitted. (codex 3, R2-4)
• Private responses carry Cache-Control: private, no-store. Secret responses carry Cache-Control: private, max-age=<remaining life>, must-revalidate, generated per request from the capability presented and overriding any directive on the object, so no HTTP cache may serve a stale copy past the capability's expiry. The edge's own byte cache is internal and never a substitute for either. (codex 3, R2-5)

The costs: the app's instance is in the hot path for private media, one small request per fetch, and a hibernating instance wakes for it; the app must expose the check endpoint, which rastrillo provides once for every app; the edge stays identity-free, forwarding a cookie it never reads. The edge never caches the app's answer, for any tenant, under any knob. Ruling 1 says never; a proposal to cache a positive answer comes back to §7 for a changed ruling, not to a stage spec as an option. (ruling 1 in §7; codex R3-7)

Media publication is never evidence for anything else. A sibling app seeing an object under public/ learns nothing about whether it may ingest the source item; custody keys on the app's grants, as the doctrine says. (codex 11)

3. Native helpers and media services: the rule

Images and video are the two places where pure Go is not fast enough on a t4g.small. The answer is not a cgo dependency. Two classes, with different rules. (codex 16, 22; ruling 10)

Codec helpers transform bytes and nothing else.

• The Go tree stays CGO_ENABLED=0. A helper is a separate program in whatever language wins the benchmark: Rust, C, C++, Go with wazero, or an existing tool such as vips or ffmpeg. Every candidate, including a Go one, runs as its own process. Nothing decodes inside the edge.
• A helper is built and shipped as its own artifact, linux/arm64 only, with an authenticated manifest naming its version, hash and the libraries and codecs it carries. A box with the role that needs it fetches it at bootstrap and at each promotion, and can roll back the way it rolls back the platform binary. It never enters the CLI build.
• A platform unit spawns it under a hardened systemd scope with no network, a read-only filesystem apart from scratch, and memory, CPU, wall-time, scratch-space and process limits. It talks over stdin/stdout or a unix socket, and the Go side owns the protocol, so vips today and a Rust helper tomorrow answer the same calls.

Media services are network-facing long-running units: the relay in stage 4a, LiveKit in 4b, a live packager. They are platform units like the edge, with their own units, ports and hardening, and the no-network rule above does not apply to them.

This is the answer to "could we build some dependencies in Rust or C for throughput": yes, as codec helpers, and the spikes in stages 2 and 3 decide whether the first ones are ours or borrowed.

4. The stages and how they depend on each other

Six pieces of work. The graph replaces a single sequence; anything with no arrow into it can start now. Stage 1 carries two things later stages share, the derivative-write grant and the publication fence, so that whichever of 2 or 3 ships first finds them ready. (codex 27, R2-11; ruling 11)

stage 1 serving class ─┬─▶ stage 2 image transforms ├─▶ stage 3 audio/video on demand ─▶ recording (in 4b) └─▶ stage 5 app-facing helpers stage 4a STUN + TURN ────▶ stage 4b LiveKit SFU, live streaming

| Stage | Delivers | Needs from the platform | Size | |---|---|---|---| | 1 | kind:media: serving class with the three doors | edge read grant with the scan deny, signed capabilities, streaming handler, isolated cache, distributed purge, hostname claim, derivative-write grant, publication fence, CLI | one spec, one plan | | 2 | image transforms behind preset URLs | a codec helper, limits, derivative identity | one spec, one spike first | | 3 | audio and video on demand: single rendition, then ladders, posters, waveforms | a media worker, a job queue under the app prefix, ffmpeg helper | one spec, one infra change | | 4a | STUN and TURN | UDP ingress, a relay unit, per-app per-region credentials as config, limits | one spec, one infra change; starts now | | 4b | LiveKit as a media service, recording, live streaming | a c7g.large, LiveKit's server unit, a live serving extension | one spec | | 5 | uploads, rastrillo helpers, accounting, residency disclosure | small pieces spread across 1 to 4 | folded into each stage's plan |

Stage 1: kind:media, the serving class

A new serving class, the fourth app-mintable route kind beside instance, service and static (the edge also dispatches the platform's own status), that serves an app's store bucket through the edge, cached, with the three doors of §2. This is piece A of #107 with its open questions answered. (codex R4-8)

What an app sees. carlos media enable writes an additive media block into store.json (prefix, allowed regions, key ids, hostname) and mints media-<app>.<sqid>.<apps domain>, one label so the account wildcard already covers it. (codex 24) Because media-<app> can also be a valid app name, the hostname is a media-host claim of its own, owned by the source app and recorded in the account, not an app row. It is one entry in the account's hostname namespace, and every door that acquires a hostname there consults it: new app names, renames, and the instances door that lets an app take an unclaimed label. Enable is refused, with the error saying so, when media-<x> is already an app's name or an existing route or instance host, and those doors refuse a label a media claim holds; the acquisitions are serialised on the claim so two racing requests cannot both win. An app whose name ends in a hyphen cannot form a DNS label with media- prefixed and is refused enable before any claim or grant is written, with the error saying to rename. The generated label can be up to 38 characters, inside a DNS label; the shared release.ValidApp grammar is not narrowed, because existing manifests and channel pointers parse through it. (codex R2-10, R3-4, R4-4, R4-5) The app writes objects under <prefix>/public/ or <prefix>/protected/ with the store credentials it already has, and forms URLs:

https://media-docs.abc.oncarlos.com/p/<key> public https://media-docs.abc.oncarlos.com/s/<capability>/<key or rest> secret https://docs.abc.oncarlos.com/_carlos/media/<key> private, on the app's origin

The capability is one URL-safe base64 path segment encoding the signed claims and the signature. Everything after it is resolved against the scope the claims name, so .../s/<cap>/videos/abc/index.m3u8 and the 720p/seg001.m4s it references are both authorised by the same capability. The cache key strips the capability and keys on the object.

What a capability signs. Domain separator (carlos-media-v1), the app's store identity, the key id, the namespace (protected/), the scope as either an exact key or a prefix that is matched only at a / boundary after percent-decoding and normalising the path with no .., the expiry, and in stage 2 the transform presets allowed. Maximum life is seven days (ruling 3). Rotation is a second key id with an overlap; emergency revocation removes a key id from the route record. The capability never appears in edge logs, and every media response carries Referrer-Policy: no-referrer. (codex 5)

How the edge is granted read. The provisioner gains a bucket-policy operation. At enable it attaches statements on that one bucket letting each allowed region's host role GetObject on both namespaces and PutObject and DeleteObject under derived/ within them, with the same s3:ExistingObjectTag deny the store user carries, so an object the scanner has tagged infected is unreadable through media exactly as it is through the app's credentials. Enable and disable are a reconciled walk like the store grant, with partial completion, store revocation and route replacement handled, and disable also retires cached bytes. Adopted and non-AWS stores are refused in version one. (codex 4, 25, R2-11; ruling 2)

Signing keys. An Ed25519 pair per app, minted at enable. The public half goes in the route record, which every edge already syncs. The private half reaches the app as CARLOS_MEDIA_SIGNING_KEY through the sealed config.json path that already carries store keys. The edge holds no secret. (ruling 3)

The handler. mediaHandlerFor streams: one S3 GetObject per request with a single normalised range, the body copied through, never buffered. It defines its own HTTP mapping rather than borrowing http.ServeFile's: single-range only, multi-range answered with the whole object, If-Range against the ETag, If-None-Match before bytes, HEAD answered from metadata, 416 on unsatisfiable. Every cache entry is bound to the object's ETag and published only after a complete fill, so a range from one version is never stitched to metadata from another. (codex 7) Response headers set on every media response, whatever the object carries: X-Content-Type-Options: nosniff, Content-Security-Policy: sandbox, Referrer-Policy: no-referrer, a Content-Type from the object's metadata with a small allow-list fallback, Content-Disposition: attachment for anything that is not an image, audio, video, font or PDF, the door's Cache-Control from §2, and a real CORS policy for the app's own hostnames with Vary: Origin, because Cross-Origin-Resource-Policy is not CORS and an HLS player in JavaScript needs the latter. (codex 23)

Its own cache. kind:media is exempt from the shared LRU in both directions and keeps a disk cache under /data/media-cache/ that stores objects as fixed-size blocks (1 MiB), so overlapping ranges share storage. Isolation is per box: a total disk budget with filesystem headroom, a fair share per route within it, eviction that takes first from whichever route is furthest over its share, an account-level cap across routes, and a bound on concurrent fills. The fleet-wide figure is accounting, not a limit any one box can enforce. (codex 8)

Purge, distributed, with a serving gate. A purge is a record appended to a log in the platform bucket, sequence-numbered per app. Every edge tails the log and applies it; entries carry the generation they were filled under, so an in-flight fill cannot repopulate an entry a purge already covers. A returning edge reconciles its purge cursor and its route authority before admitting any hit from the persistent cache, and if the log history it needs is gone it drops its media cache rather than guess. Serving media is a lease. An edge may serve any media door, public included, only while its media authority (the route record's media block and its purge cursor) is younger than fifteen minutes; past that every media door on that edge fails closed, because a public object that was moved to protected/, a disabled route, a removed region or an infected verdict must not outlive an outage on a box that clients can still reach. Retirement of a key, a route or a region is complete when every box that could hold a copy has acknowledged it or its lease has expired, and the API and CLI report completion per box on exactly that condition; the local Carlos-Purge header and socket stay as fast paths and never count as completion. A scan verdict turning infected files a purge for the key and its derivatives. (codex 4, 9, R2-6, R3-1)

The publication fence. Anything that writes derived output, a stage 2 transform or a stage 3 job, writes to an attempt-specific key under derived/_attempts/ and has no authority to publish it. The console is the publisher. A commit is a console door, POST /a/<sqid>/apps/<app>/media/publish, carrying the attempt, the lease and the source ETag; the console checks the lease is current in its registry, the source's purge generation and the app's incarnation match what it holds, and only then writes the per-source manifest itself. That is one serialiser with the state to refuse, which a conditional write against the manifest is not: an expired worker with no successor would find the manifest's ETag unchanged and its If-Match would succeed. The serving path reads derived output only through a committed manifest, with no exception: a synchronous transform's first requester waits for the publish door's answer and receives the bytes only if it was 200, because an attempt that stalled past its lease or across an infected verdict must not reach even the request that caused it. One console round trip on a first miss is the cost. Purge, an infected verdict, cancellation and app deletion each bump the generation in the registry, so a late commit is refused by the door. Attempt keys are swept. This lives in stage 1 because both later stages need it and neither may reinvent it. (codex 17, R2-7, R3-2, R4-3)

Residency, enforced. The media block names the regions allowed to serve and cache the app's media. The bucket policy grants only those regions' host roles, the media host's DNS records exist only in those regions, and the handler on any other edge answers 421 with the right host as a backstop, so a client addressing a foreign edge directly cannot fill a cache there. Default at enable: the regions the app's routes are already placed in, shown back by carlos media status and on the media docs page. (codex 10, R2-9; ruling 4)

Store spec amendment. Line 27 of the object-storage spec gains one sentence: reads may cross the data plane through kind:media; writes never do. Uploads stay app-to-S3 on the app's credentials.

What stage 1 must prove (§9) is the attacks in §2 and this section failing, not only playback succeeding. (codex 27)

Not in stage 1: transforms, transcoding, uploads through the edge, per-viewer analytics, signed cookies, bandwidth quotas, dashboards beyond a bytes-served counter, multipart upload helpers.

Stage 2: image transforms

/p/<key>?t=thumb, /s/<cap>/<key>?t=thumb and the private form, where thumb is a preset the app declared. A transform is synchronous, pure CPU in a codec helper, and its result is written back to the app's bucket through the publication fence so the second region never redoes the first's work.

Abuse surface first. A public transform URL is CPU amplification on the box that terminates TLS for every tenant. From the first commit, transform parameters are a named preset the app declared, or for secret objects the presets the capability names, or for private objects the presets the app's check endpoint approves. Free-form parameters are refused everywhere. (codex 5) On top of that: single-flight per derived identity, a per-box concurrency cap with per-tenant fair admission so one tenant cannot hold every slot, a queue that turns into 503 after a short wait, a source pixel-count cap, output dimension caps, and a per-route derivative byte budget. (codex 17)

Derivative identity. A derivative is named by the source key, the source ETag, the normalised preset, the resolved output format and the helper version, and committed through the fence. f_auto picks AVIF, WebP or JPEG from Accept, and the response says Vary: Accept. (codex 17)

The engine, decided by a spike. Three candidates, all run as separate processes under the §3 rule. First, a Go helper embedding wazero-hosted libwebp, libavif and jpegli (the gen2brain/* packages), which keeps the build pure Go end to end. Second, vips as a fetched helper. Third, our own Rust helper using the image and ravif crates or linking libvips, persistent over a unix socket. The spike measures a 4000×3000 JPEG to an 800-wide WebP and AVIF on a t4g.small that is also serving: thumbnail latency, peak memory, sustained throughput under eight concurrent requests, behaviour on malformed input, and the serving latency of the box while it runs. Under a second for WebP with serving latency flat is the bar, and the candidate that clears it with the least to operate wins. The bar is not a ceiling on the fleet: if nothing clears it on a t4g.small, the choices are a bigger edge box, a dedicated transform box the fleet shares, or telling apps what to expect, and the spike's numbers decide which. (codex 16; ruling 5)

What a transform does. Resize, crop modes, format, quality, and always strips metadata. EXIF GPS in a public photo is a privacy leak the platform should not require every app to remember. Animated GIF and WebP are passed through untransformed in version one. Sealed blobs are refused with 415.

Derivatives land in the tenant's bucket under <namespace>/derived/<source key>/<identity>.<fmt>, written by the region host role under stage 1's derived/ statement, and count against the tenant's storage. Purging a source key removes its derivatives. (ruling 6)

Not in stage 2: face detection, smart crop, watermarks, uploads, SVG rasterisation, anything needing a model.

Stage 3: audio and video on demand

Stage 1 already plays a well-formed MP4 or MP3 progressively: Range passes through and the browser's <video> seeks. For a short clip or a voice note that is the whole feature and it ships in stage 1. Stage 3 is for what progressive cannot do, and it has two products the review insisted be kept apart. (codex 21)

• A single rendition, fast. Probe the input; if it is H.264 and AAC in MP4 with sane keyframe spacing, remux to CMAF and one HLS playlist in seconds. If it is WebM from a browser recording, re-encode one rendition at the source's size. This is what a phone upload in Docs or a recorded call needs first, and it follows stage 1 directly.
• An adaptive ladder, slower. Re-encode to three rungs (1080p, 720p, 360p, capped at the source's height), CMAF, one multivariant playlist. Remuxing cannot make lower rungs; this always encodes. AV1 and DASH are one flag away and off. This follows the worker measurements.

Both products also write posters at three timestamps for video, and a waveform JSON and duration for audio. Everything is served by stage 1 under one prefix-scoped capability or one private door, which is why the capability is a path segment.

The media worker. A platform unit, carlos-media.service, on one box per region that has media-enabled buckets. Jobs live where the scan queue lives and for the same reason: under the app's own prefix, apps/<acct>/<app>/media-jobs/<id>.json, one object per job, so app deletion finds them and the name-claim gate sees them. (codex 19)

• Submission is a console door, POST /a/<sqid>/apps/<app>/media/jobs, reached from the app through a rastrillo helper with the app's route-scoped credential, and from carlos media transcode <key>. The door derives the source and destination scope itself; the app names a key, never a path. Delivered store credentials never gain the right to write job records. (codex 18)
• Eligibility, decided at submission. The processing region is the bucket's region, and it must be in the app's allowed list and have a worker. Version one runs one worker, in Ireland, so a job for a bucket elsewhere is refused at the door with an error naming the reason, not queued to wait for a worker that does not exist. (codex R2-9)
• Authority. The worker reads the source through the media bucket policy and writes renditions under derived/ through stage 1's statement, on buckets in its own region only.
• Leases and the fence. A worker takes a lease with an expiry; its output goes to attempt keys and is committed through the stage 1 publication fence, which refuses a commit from an expired lease, a purged source or a deleted app. Bounded retries, cancellation, a terminal failure state. (codex 18, R2-7)
• Completion is a result record the app polls in version one. Callbacks come later as a durable completion outbox that the destination host converges into a local once-timer, because nothing files a timer across boxes today. (codex R2-8)
• Limits from the first job: admitted source size and duration, concurrent jobs per box, memory and scratch caps on the ffmpeg helper, and measurement of serving latency, I/O and CPU credit consumption under sustained encoding before any promise about co-location is made. (codex 20)

Which box. A t4g.medium has the same two burstable vCPUs as a t4g.small; it is not a transcoding box. Version one runs the worker on the Ireland medium behind the limits above, and the honest expectation is that the first tenant wanting ladders moves it to a dedicated c7g.large. Encode throughput is the first sustained bottleneck; host memory and scratch pressure fail earlier if left unbounded, which is why the limits come first. (codex 20; ruling 7)

Not in stage 3: caption generation, DRM, per-viewer watermarking, server-side ad insertion, live packaging. Caption files the app supplies are just objects.

Stage 4: real time

Meet's requirements, in the order Meet needs them. Stage 4a depends on nothing above and starts now. (codex 27; ruling 11)

4a. STUN and TURN. A STUN listener the platform runs is what lets a WebRTC app connect across most home and mobile networks without an egress exception; TURN is the same listener relaying, and it is what turns "one call in ten cannot connect" into "calls connect". Both are UDP. The capabilities spec's non-goal stands: UDP is not something an app binds. The platform runs carlos-relay.service on every region box, built on pion/turn, pure Go, in the same binary as the edge, a §3 media service.

• Ingress. UDP 3478 and a relay allocation range (49152 to 65535) opened in the region security group and configured as the allocator's only range; TCP 3478 for networks that block UDP. The relay advertises the box's public Elastic IP. No TURN over TLS in version one (Paul, 2026-09-07, stage 4a spec ruling 4): 5349 needs certificate issuance for a non-HTTP listener, which is its own small design, and 443 would need a second address on the box because 443 is the edge's. The docs say a network that blocks 3478 is the remaining case that fails. (codex 15; amended by the stage 4a spec)
• Credentials, scoped per app and per region. Each region has a relay master key. At carlos relay enable the instance receives, sealed through config.json, one derived key per allowed region: HKDF(region master, account, app, incarnation), as CARLOS_RELAY_KEYS alongside CARLOS_RELAY_URLS. The app mints TURN REST credentials per call, username = expiry:account:app:incarnation, credential = HMAC(derived key, username), with a four-hour maximum (Paul, 2026-09-07, stage 4a spec ruling 1; the relay ends the allocation at expiry, so one hour would end any longer call). The exact wire shape, a base64-wrapped JSON CARLOS_RELAY with a platform-supplied username suffix and a key id, is the stage 4a spec's §2. The relay holds a relay authority record, control/relay.json, rebuilt by the console from per-app relay records and read by every relay every minute: for each enabled app, its regions and current incarnation. On a request it parses the username, refuses it unless the app is enabled and the incarnation matches the record, re-derives the key from its own region master and the username's fields, and verifies. Verification is local and per-app metering is real. Disable removes the app from the record, which ends existing allocations and makes every outstanding credential fail; re-enable draws a new random incarnation, so nothing minted before a disable ever works again and nothing is reused across deletion or name reuse. A relay whose authority record is older than fifteen minutes refuses new allocations and terminates the ones it is carrying, the same lease rule the edge applies to media: a disable the relay never heard about must not outlive the outage by the length of a credential. Rotation is a new region master and a re-render of every enabled app's config. (codex 13, R2-2, R3-3, R4-6; ruling 8)
• What that does and does not protect. Per-region masters mean a relay box holds no other region's master. They do not mean a compromised host learns nothing about other regions: sealed config is decryptable on every host (§1), and the console must hold every region's master as sealed config to derive app keys, so a compromised host can read every region's master and mint for any app in any region, present and future, until the operator rotates (Paul, 2026-09-07, stage 4a spec ruling 3). That is wider than the derived-key exposure alone, and it is stated in those words in the operator docs. Rotation of one region recovers that region. A design where only a region's boxes ever hold its master, the relay deriving and writing app keys itself, is filed as the tightening. (codex R2-2; amended by the stage 4a spec)
• Policy and limits. Peer permissions refuse loopback, link-local, RFC 1918, the VPC's ranges and the box's private addresses; the box's own Elastic IP is reachable, but only on ports holding a live relay allocation, so two clients on one relay can reach each other and nothing else on the box is exposed. Allocations, allocation rate, bandwidth and lifetimes are bounded per app, per account and per box. Established allocations end when their credential expires or the app is disabled. These are operational limits in 4a; commercial quotas wait for measurement. (codex 14)
• Placement. The app's CARLOS_RELAY lists its regions in the console's order, one list for the whole app, since placement names are not regions and an app may have several placements (stage 4a spec). The browser gathers candidates from all of them and ICE nominates a working pair by its own procedures; that is not a lowest-latency guarantee and the roadmap does not claim one. Client-aware relay selection is a separate later decision. (codex 15)
• Metering. Bytes forwarded from the client and bytes forwarded to the client are two meters, per app, written by the relay to its own metrics objects; the inter-region meter the first draft named, and the account usage line, move to stage 5 with the usage view (Paul, 2026-09-07, stage 4a spec ruling 5). The unit-economics spec gets the definitions. (codex 26; amended by the stage 4a spec)
• Custody language. A platform relay is not egress under ruling 6: the recipient is the operator the team already trusts with its pages. A third-party relay is egress, and under doctrine §8 which third party is the app's decision, never the team's.

4b. LiveKit as a media service, recording, live streaming. An SFU lifts the four-participant cap, changes the privacy story from peer-to-peer to server-in-the-path, and is where recording becomes possible. Paul ruled to host LiveKit's open-source server rather than build on pion/webrtc, as a §3 media service, carlos-sfu.service, fronted by the platform's own API. (ruling 9) LiveKit Cloud is not an option for suite apps: a third party in every call's media path is egress that is a property of the product. It is also not cheaper past about sixty room-hours a month, since both are dominated by bandwidth.

• Box. The server is one Go binary on pion and needs no Redis as a single node. A c7g.large in Ireland (2 vCPU, 4 GB, about sixty dollars a month) carries a handful of concurrent eight-person video rooms; a c7g.xlarge gives headroom for tens. CPU is spent per forwarded track, and LiveKit's own figures put thousands of tracks on sixteen vCPUs, so the box is not the limit at this scale.
• Cost is egress. Each participant downloads everyone else's stream. An eight-person video room at about 1 Mbps per stream is about 25 GB an hour, roughly two dollars at AWS rates. That line goes in the unit-economics amendment beside the relay meters.
• Ports and hostname. LiveKit's clients speak WebSocket to the server's /rtc endpoint on 7880, so the SFU has a platform hostname of its own, sfu.<region>.<platform domain>, with TLS terminated by the edge on that box and the upgrade proxied to LiveKit. The browser reaches it directly, never through the app's route, so an app on a tunnel-backed host still joins SFU rooms. Port 7880 is reachable only from that proxy. TCP 7881 and UDP 50000 to 60000 are opened directly, the same shape as the relay. (codex R3-5)
• Tenant boundary, enforced at admission. The platform holds LiveKit's API key and secret; no app ever does. Rooms are named <account>/<app>/<incarnation>/<room> and only the platform mints tokens: the app's backend calls POST /a/<sqid>/apps/<app>/media/sfu/token with its route credential, naming a room, a participant identity and grants, and the door prefixes the room, caps the grants to that app's namespace, signs a ten-minute token, and returns it for the app to hand to its browser. The proxy in front of 7880 is the admission gate: on every join and reconnect it verifies the token, checks the room's revocation list, and only then opens the upstream connection, serialised against removals so a rejoin cannot race a removal. LiveKit's own token refresh for connected clients does not matter, because admission is keyed on identity and room against platform state, not on a token being unexpired. Removal is an app-to-platform door that records the revocation, calls LiveKit's remove-participant, which is what ends the active media session, and has the proxy close that participant's signalling connection and refuse its next admission, none of it waiting for anything the client does. Webhooks are for observation and reconciliation, never enforcement. Management operations are the door's, scoped to the app's rooms. (codex R3-6, R4-1, R4-2)
• No third-party defaults. Stock LiveKit hands participants Google's and Twilio's STUN servers when none are configured and uses them for its own external-IP discovery. The platform's LiveKit is configured with the stage 4a relay as its only STUN and TURN, for the server's discovery and the client ICE configuration alike, and the 4b gate runs a call with all third-party network access blocked to prove no fallback remains. This is why 4b follows 4a. (codex R4-7)
• Recording and ingest are separate LiveKit services and are deferred until asked for, with their prerequisites named now so the launch box is not mistaken for them: LiveKit Egress needs Redis to talk to the server, its own worker of at least four vCPUs and 4 GB, and Chrome's sandbox requirements for composite recording; WHIP and RTMP ingest is a third service with the same Redis dependency. Recording writes per-track or composite files into the app's bucket and submits a stage 3 single-rendition job, so a recorded call becomes a playable asset with no new serving code. Recording depends on stage 3, not the other way round. (codex R2-11, R3-8)
• Live streaming to many viewers is a distinct long-running workload: ingest feeds a live packager, and LL-HLS needs partial segments, blocking playlist reloads and delivery directives that an object reader does not supply, so it is a named serving extension with its own latency contract, not a reuse of stage 1 unchanged. (codex 22)

Signalling. In peer-to-peer mode, with or without the stage 4a relay, offers, answers and candidates ride the app's own HTTP, and §8 answers #113 about that. In SFU mode the LiveKit client SDK carries its own signalling over the WebSocket to the SFU hostname above; the app's HTTP is then for room coordination and token refresh only. (codex R3-5)

Stage 5: what makes it usable from an app

Each of these is small and belongs in the plan of the stage that first needs it. None of them is in stage 1's gate. (codex 27)

• Direct browser uploads. Apps already presign PUTs, but a browser cannot PUT to the bucket until the bucket has a CORS rule, and S3 CORS is bucket-wide with no prefix selector. carlos media enable therefore owns the bucket's CORS configuration for the app's hostnames, merging with nothing (a store with CORS already set is refused until the operator clears it), exposing ETag and the multipart headers, and removing it at disable. Multipart upload for anything over 100 MB is a rastrillo helper. (codex 23)
• rastrillo helpers. One package, rastrillo/media: a URL builder for the three doors, a capability signer that reads CARLOS_MEDIA_SIGNING_KEY, the private-check handler that answers the edge's subrequest from the app's grants, a visibility-move helper that drives the lifecycle in §2, a presigned-upload helper with CORS-correct headers, a job filer, and a TURN credential minter. Docs' disabled "insert image" is the first consumer.
• Accounting. Bytes served and cached in stage 1 as one counter; transform seconds, transcode minutes, the three relay meters and SFU egress when their stages land; on the console's app page and in carlos media status. Quotas come after measurement, not before.
• Scanning. Nothing to hook: the sweep already tags what it finds, and the media grant carries the deny. The one addition is the verdict purge in stage 1.
• Residency disclosure. One paragraph on the media docs page and in carlos media status naming the allowed regions and, for private media, the home edge.
• Docs page. docs/site/media.md, and docs/site/cli.md headings for every new verb, because the docsite gate fails otherwise.

5. What apps might want that is deliberately not here

Named so nobody wonders whether it was forgotten.

• Signed cookies instead of path capabilities. Not needed while capabilities are prefix-scoped; revisit if a tenant needs a whole site of secret media.
• A media-session cookie on the media host for private media. Cannot be relied on across browsers and privacy settings on custom domains; the app-origin door in §2 is the answer.
• Per-viewer watermarking or DRM. Different product, different buyer.
• AI on media: captions, transcription, moderation, smart crop. Each is a model-provider egress under ruling 6 and a per-app opt-in there, not a platform primitive.
• A platform upload door through the edge. Writes never cross the data plane; uploads stay app-to-S3.
• Peer-to-peer file transfer over the relay. TURN would carry it, but it is an app protocol, not a platform one.
• A GPU tier. Nothing in stages 1 to 4 needs one on a fleet this size.
• LiveKit Cloud, or any hosted SFU. Egress that is a property of the product.

6. What must be true before an app can build on this

The review's last question was what an app hits in its first month. The answers are all inside stages 1 and 5, and are the reason stage 1's gate is a security gate, but they are worth listing plainly:

• a way to put a private image on a page that works on a custom domain in Safari (the app-origin door);
• a way to hand a video player one URL and have every segment load (the path capability);
• a way to share a roster-visible video by link without moving it (the shared protected/ namespace);
• a way to replace an object and not serve the old bytes (ETag-bound cache, distributed purge, the publication fence);
• a way to know which regions hold copies (the media block);
• an upload that does not fail at the last byte of a phone video (CORS ownership, multipart helper);
• a helper package so none of the above is hand-rolled per app.

7. Rulings

Paul ruled on each of these on 2026-09-07. They are the roadmap's fixed points; a stage spec that wants to move one comes back here first.

1. (ruling) Private means ask-the-app, per request, never cached, served on the app's own origin under a reserved path.
2. (ruling) Edge read is granted by bucket policy on the two media namespaces, per allowed region host role, carrying the infected deny, attached and removed by a reconciled walk.
3. (ruling) Ed25519 per app for capabilities, public half in the route record, private half as env through sealed config.json, key id in the capability for rotation, seven-day maximum life.
4. (ruling) Allowed regions are explicit in the media block and enforced in the bucket policy, DNS and the handler; default is the regions the app's routes are placed in.
5. (ruling) The transform engine is chosen by a measured spike among a Go wazero helper, vips and a Rust helper, all as separate processes, benchmarked on a t4g.small, with a bigger or dedicated box on the table if the numbers say so.
6. (ruling) Derivatives and renditions land in the tenant's bucket under derived/ in the source's namespace, counted as the tenant's storage, removed with their source.
7. (ruling) The media worker starts on the Ireland t4g.medium behind hard admission limits, with a dedicated c7g.large when ladders are wanted.
8. (ruling) Relay credentials are per-region HKDF-derived per app and incarnation, verified locally; the exposure through sealed config is stated, not hidden.
9. (ruling) The SFU is LiveKit's open-source server hosted as a platform media service on a c7g.large, recording and ingest services deferred until asked for.
10. (ruling) Native helpers are allowed as separate processes only, with codec helpers and media services as distinct classes.
11. (ruling) Stage 1 and stage 4a start now, independently. Stage 2 and the stage 3 single-rendition product follow stage 1. The ladder follows the worker measurements; 4b follows 4a.

8. Answers to #112 and #113

#112. Direction A: the platform runs the relay. STUN and TURN are stage 4a and start now; LiveKit as a hosted media service is 4b. UDP ingress is something the platform will carry, as a platform unit on the region boxes, never as an app-bindable port. A platform relay is not egress; a third-party one is.

One correction to Meet's interim plan: under doctrine §8 the recipient of an egress is the app's decision, never the team's, so a settings field where a team pastes any TURN URL is not admissible. What is admissible until 4a ships is either no relay, with the disclosed floor of calls that connect without one, or an app-named provider that a team can switch on, disclosed in the README's first paragraph and on every artefact the app produces while it is on, which for Meet means any recording or export. Meet should pick one of those two, not the free-text field. (codex 12, R2-12)

#113. Confirmed as a standing constraint: no Connection: Upgrade over a tunnel-backed host. The placement picture is more nuanced than the question assumed: a member can place an app with carlos apps place and carlos instances create --region, and the instances API reports which boxes serve a host, so a team can keep Meet off tunnel-backed hosts by choice. What does not exist, and is not planned before extended CONNECT, is a runtime signal the app can read saying "this host carries upgrades", or a manifest field that refuses placement without it. (codex R2-13) So SSE plus POST remains the right default: it works everywhere, its volume is small, and a socket can be adopted later on hosts the team has placed deliberately, with SSE as the fallback Meet would need anyway.

9. Acceptance gates

Each stage has its own gate; what a gate unlocks is whatever §4's graph hangs off it, not "the next stage" in this list. The gates are attacks failing, then playback working. (codex 27, R3-9)

• Stage 1, on flagship: /p/ of a protected key is 404 cold and warm; a capability with the expiry edited, the scope widened past a / boundary, the namespace changed, or the key id revoked is refused; a private fetch is refused the request after the app revokes the grant, on a warm cache; a public-to-protected move leaves nothing readable through /p/ in any region once it reports complete; an object tagged infected is unreadable through media and its cached copy is gone within a purge round; a purge issued while a fill is in flight leaves no stale entry; an edge reconnecting after an outage serves nothing from disk until its cursor is current; a revoked key, and a public object moved to protected/, both stop being served by a disconnected edge within fifteen minutes; an edge outside the allowed regions refuses to fill; a late commit from a worker whose lease expired with no successor is refused by the publish door, and the request that caused a stalled synchronous transform receives no bytes; an app named media-<x> created before enable keeps working and enable for <x> is refused with the right error; an instance host media-<x> taken by another app before enable, and one attempted after, are each refused in the right direction; an app name ending in a hyphen is refused enable. Then: a public hit served from a second region's cache, one object reached both by the app's check and by a capability, an HLS playlist and its segments loading under one capability, and a 2 GB MP4 seeking in a browser through a t4g.small without the box paging.
• Stage 2: the spike's numbers, then 9,999 distinct transform URLs against a public object producing zero decodes, and one tenant's burst leaving another's thumbnails under a second.
• Stage 3: a 1080p H.264 phone recording of two minutes uploaded through Docs and playing in Safari as a single HLS rendition within thirty seconds, with serving latency on the same box flat; a WebM browser recording of the same length playable within three minutes; a job for a non-Ireland bucket refused at the door; the ladder measured, not promised.
• Stage 4a: a Meet call between two symmetric-NAT networks connecting through the Ireland relay with a credential that expires in four hours, and both browsers forced to relay connecting through it; a credential minted for one app refused for another; a credential minted before a disable refused after re-enable, and the allocation it held ended; a relay with an authority record older than fifteen minutes refusing a new allocation and ending one already carrying traffic; a relay permission to a private address refused; the two meters moving in the relay's metrics object; a network that blocks 3478 failing the way the docs say it will. The full gate is the stage 4a spec's §7.
• Stage 4b: a token minted for one app refused for another app's room; a participant removed by the app dropped within a second and unable to rejoin with its old token or a LiveKit-refreshed one, with the webhook delayed and then lost; a browser on an app served from a tunnel-backed host joining an SFU room; a call completing with all third-party network access blocked from the SFU box; an eight-person room's egress appearing on the account's usage line.

10. Findings not accepted

• Codex 12, the part asking for a doctrine amendment to allow team-selected external relays. Not requested; the roadmap instead corrects Meet's interim plan to what §8 already allows, now including artefact disclosure.
• Codex 15, TURN over TLS on 443. Acknowledged as a real gap and documented as such; a second address per box is deferred until a tenant reports a 443-only network, because it changes the fleet's networking for every box to serve a case nobody has hit yet.

Every finding from revisions 2 to 5 was accepted.

11. Amendments to existing specs

• 2026-08-10-object-storage-primitive-design.md:27: reads may cross the data plane through kind:media; writes never do.
• 2026-08-03-app-capabilities-design.md:931: UDP remains a non-goal for apps; the platform's own relay and SFU units are the exception and are specified in stage 4.
• 2026-08-11-unit-economics-design.md: a media section, at stage 1 for served bytes, at 4a for the two relay meters (bytes forwarded in each direction), at stage 5 for the inter-region meter and the usage line, at 4b for SFU egress.
• infra/modules/region-host/main.tf:6 header: residency by construction now has a named exception, the per-region media bucket policy, and the exception is itself region-scoped.
• The console's name-claim gate (internal/console/server.go:2904) and the instances door (internal/console/apiinstances.go:571): new app names, renames and instance hosts consult the media-host claim table. The release.ValidApp grammar is unchanged. (codex R3-4, R4-4)