Edge asset cache: versioned build assets served from every edge

Branch edge-asset-cache. Status: rev 9, 2026-10-24, builder check folded in; rulings R1 to R3 (§13). Approved by Paul 2026-10-24; Astra accepted rev 6 in round 4; rev 8 accepted in round 6; rev 9 narrows one admission rule, accepted with the plans (plan review round 8). Related: media roadmap (2026-09-07-media-platform-roadmap-design.md), which this does not amend; #368 (the shared cache stores 206 responses).

0. Intent

An app's build produces images, JavaScript and stylesheets. Most of them carry a content hash in the name (app.3f9c2a.js), so the bytes behind a name never change. Paul wants an app to be able to say "these paths are build assets" and have whichever edge a visitor reaches serve them from its own disk, for as long as the app says they are good for, without asking the app (Paul, 2026-10-24). Three reasons, all of them goals:

1. Distance. A visitor far from the app's home region waits a cross-region round trip for every stylesheet and script on every uncached load.
2. Wakes. A hibernating instance is woken to serve a stylesheet.
3. Home load. The home box serves every asset byte for every region.

This is not stage 1 of the media roadmap. Stage 1 serves objects an app writes into its store bucket at runtime and treats them as untrusted user content (three doors, capabilities, bucket grants, distributed purge with a serving lease). Build assets are public, come from the app's own route, and are immutable by construction. They need almost none of that machinery, and waiting for it would delay all three goals. The media roadmap's stage 1 is untouched and still next in that line; §11 notes what it can reuse from here. (My call, playback approved by Paul 2026-10-24.)

1. Where things stand on main (fef28dc5)

• Visitors already reach their nearest edge on the flagship. The *.<apps domain> wildcard and edge.<apps domain> (custom domain CNAME target) answer by latency across the four regions; the steering converger is armed there (flagship infra docs/latency-routing-oncarlos.md:29-38; the code comments in internal/steering/steering.go:14 saying it is armed nowhere are stale). So the geographic half of this needs no DNS work.
• A far edge forwards everything for a home-boxed app and caches nothing. An instance route row exists only on its home box (internal/provision/reconcile.go:401-414). Any other edge misses lookupUpstreamPath and calls e.backhaul (internal/edge/edge.go, the lookup-miss branch), which runs before the meter, gzip and cache. The comment there says why the far edge must not cache: purge is honoured only by the home edge, so a far copy "could never be invalidated by the app that owns it".
• The shared response cache is not built for assets. It is one process-wide in-memory LRU, 4096 entries, 1 MiB per body (edge.go:61-69), lifetime clamped to five minutes (DefaultCacheMaxTTL, edge.go:77) because a lost purge must be a blip. It has no per-tenant share. Many bundles exceed 1 MiB and are never stored. It ignores Range (#368) and conditional requests.
• Far edges already learn per-host facts cheaply. Every box publishes control/edges/<box>.json (internal/host/edgesfile.go:44-91, struct edgeHost: role, platform, alias, replica, wildcard, account, app, stamp), rewritten within about 2 s of a registry change and read by every edge on a 30 s poll gated by the change index's control/edges scope (internal/edge/homemap.go). An additive field there reaches every edge with no new polling. The change index exists because per-app reads on every box cost about $3 per app per month (2026-10-21-s3-change-index-design.md §0); this design adds none.
• Declarations that change with the code live in the release manifest. Capabilities (internal/release/release.go:95-107) is the precedent: it rides the signed pointer, immutable manifest, sha256 chain, and a rollback rolls it back. carlos ship --capabilities <file> embeds it (cli.go:2167).
• Disk is tight on small boxes. US and AU flagship boxes are t4g.small with a 20 GiB root and no separate data volume. The box disk alert fires above 90% (internal/metrics/alert.go); placement wants 10% free (internal/boxchoice/policy.go:16-22). The only edge disk cache today, /data/static, is unbounded.
• Static routes are already served everywhere (everywhereClass, reconcile.go:171-174). They are out of scope here.

2. Goals, non-goals, anti-goals

Goals

• G1. An app declares path prefixes as build assets once, at ship or deploy time, and a saved project default keeps declaring them.
• G2. Any edge that receives a request under a declared prefix serves it from its own disk after the first fill, with no backhaul and no request to the app.
• G3. Lifetimes are the app's Cache-Control, up to a year, so hashed assets marked immutable stay cached until evicted or purged.
• G4. A hit never wakes a hibernating instance, on any edge including home.
• G5. A mistake is recoverable: one command drops an app's cached assets on every edge within about a minute.
• G6. Bounded disk on every box, with a fair share per app.
• G7. Hits and fills are counted as the app's usage like any other response.

Non-goals

• Serving the store bucket (media stage 1), transforms, video streaming as a target (ranges work, §5.5, but nothing is tuned for media).
• Caching HTML or anything personalised.
• Pushing assets to edges at deploy time. Pull-through only.
• Content negotiation beyond gzip (no Vary: Accept image formats, no brotli).
• A third-party CDN or CloudFront. The 2026-08-02 edge cache spec §9 rejected that as the default and this keeps the ruling.

Anti-goals (failure even if everything else works)

• A logged-in response cached and served to someone else.
• A far edge serving one app's bytes under a hostname that now belongs to another app or account.
• A cache filling a 20 GiB box's disk.
• New per-app reads of the deployment bucket on every box.
• An SPA fallback (200 index.html for an unknown path) cached as a script forever.

3. The declaration

3.1 In the release manifest

The release manifest gains an additive assets block:

"assets": {"paths": ["/assets/", "/_app/immutable/"]}

carlos ship --assets /assets/ --assets /_app/immutable/ writes it (repeatable flag). Grammar, refused at ship with an error naming the rule:

• each entry starts and ends with /, is at most 128 bytes, is already path.Clean'd (apart from the trailing slash), and contains no %, ?, #, * or ..;
• / alone is refused: it would catch the app's pages and strip their cookies (§5.2);
• at most 8 entries, no entry a prefix of another.

Old binaries ignore the key (encoding/json drops unknown top-level keys on the manifest read path, release.go:95-100). kind:static releases refuse the flag; they are already everywhere.

carlos deploy declares the same way (Paul, 2026-10-24). Deploy is ship, promote and watch in one verb, and its ship half is runShipAPI (deploy.go:1-6, :334), which already carries --capabilities. So:

• carlos deploy --assets /assets/ takes the same repeatable flag with the same grammar and passes it to the ship half.
• The project-defaults block in .carlos/config (config.go:58-65: app, kind, artifact, target) gains "assets": ["/assets/"], so a zero-arg carlos deploy keeps declaring on every release. A flag on the command line replaces the saved list for that run; --no-assets ships with none. Without either, a release declares nothing, and since paths come from the serving release (§4.1), deploying a build without them turns catching off for it. That is why the saved default matters.
• The console's ship door (API mode) puts the block in the manifest it writes and checks the same grammar itself. It does not trust the CLI.
• Deploy's watch probes the app's own URL for X-Carlos-Version. / can never be declared, so the watch never lands on a caught path and a hit can never answer it with a stale version (§5.5).
• --kind static refuses --assets and ignores a saved list, saying so once.

Why the release manifest and not the instance record: the paths an app builds assets under change with its build tooling, and a rollback to a build that used /static/ must catch /static/, not the newer build's prefix. (My call.)

3.2 Instance opt-out

The instance record gains Assets *bool (omitempty; absent means "follow the release", false means "catch nothing for this instance"). Set through the existing PATCH door (apiinstances.go, the PATCH body beside gate and steering) and a new verb, carlos instances set-assets <host> on|off, beside set-upstreams and set-channel (carlos instances update is already "install the newest build now", instances_updates.go:201). It reaches the route row through RouteFor and convergeOwned like Steering. Use: an instance whose traffic must always reach the app, or a quick off switch while debugging. (R1.)

3.3 When a route catches nothing

The home box publishes no prefixes for a hostname when any of these hold:

• the instance opt-out is false;
• the route has a gate (Route.Gate): a password-gated preview must not leak its assets to anyone who guesses a URL;
• the route, or the row the hostname resolves to, has an import gate (ImportGate, internal/edge/importgate.go:24-35), which refuses requests even when Route.Gate is empty; a warm far cache must not answer ahead of it (astra B9);
• the hostname has any path-prefix row (route_paths) at all, even when every target belongs to the same app: different instances behind one hostname can serve different releases, gates and opt-outs, and one hostname-wide declaration cannot describe them. The hostname must also have a single owner by the rule that decides edgeHost.Account/App (edgesfile.go:77-85, fold at :93-116) (astra B2);
• the route is pooled (Route.Pool != ""): pooled routes are a tenancy boundary only their pool serves, the same reason steering refuses them (apiinstances.go:2112-2116). A pooled route still uses the local disk cache on the boxes that serve it (§5.1), but publishes nothing for far edges.

4. Getting it to every edge

4.1 The home box publishes

When a box writes its edges file, each edgeHost it emits for a hostname that catches gains:

"assets": {"paths": ["/assets/"], "gen": "9f2c41d07a3be516", "at": 1792828800}

(at is the host's authority time, §4.2.)

• Only an armed box publishes. A box without CARLOS_ASSET_CACHE=1 emits no assets block, so far edges catch a host only when its home is armed and will treat their fills as fills (§5.3). This makes the arming order home-first by construction (astra B3).
• paths comes from the release manifests of the builds the route can answer with, minus the §3.3 exclusions. For a route that is not generation-managed that is the release Route.Version names. For a generation-managed route Route.Version names only what is installed, not what answers (internal/edge/edge.go:842-845; the served build comes from the generation, internal/edge/genproxy.go:298-302), so the home publishes the union of the installed build's declared paths and the current serving generation's (ServingGen(host).Build, internal/activator/genservice.go:68) (astra B5). The activator keeps one current generation per host (actEntry.gen, internal/activator/activator.go:469); a generation being replaced only finishes requests already sent to it, so no third build can answer a new request. The installed build is in the union so that a sleeping instance, which has no live generation (ServingGen is false while asleep, internal/edge/gensource.go:24-28), keeps catching: the installed build is the one a wake would start, and its assets are what a page served before the sleep asks for (astra R1-5). During an update both builds' assets are caught, and the old build's prefix drops out when its generation retires. The home reads each build's manifest from the bucket with release.GetManifest (internal/release/release.go:710) once per version and keeps only the declared paths, since a manifest is immutable. It keeps them only for builds a local route currently references, as its installed version or its serving build; each edges-file pass drops the rest, and a rollback to a dropped build reads its manifest again. Retained state is therefore bounded by twice the box's routes, whatever the deploy history (astra R5-1). It does not use the staged copy under /opt/carlos/versions (release.ReadStagedManifest, versions.go:261), which an older binary may have re-marshalled without the assets key. A manifest that cannot be read publishes no paths for that build until it can. No other box reads manifests. The edges-file writer gets the serving build through a narrow function the agent hands it, since it has no activator handle today.
• gen is the instance's asset generation (§6), copied from the instance record through the route row.
• Aliases (custom domains, and the exact names instance catch-alls derive) publish the same block as their primary, with their at computed by the same invariant (§4.2).
• Additive and omitempty: a file without it parses byte-identically, and an older reader ignores it.

A change to paths or gen must change the registry Fingerprint (or the edges file's own dirty check) so the file is rewritten within seconds, not at the five-minute refresh. So must an advance of any published at by 20 minutes or more since the last write (§4.2).

4.2 Far edges read it

The edge's home map (homemap.go) already folds every box's file into host -> home. It now also keeps host -> (account, app, paths, gen, assets_at) for hosts whose winning entry carries assets. No new object is read; the 30 s poll and the change-index gate are unchanged.

Freshness bound. A successful read is not evidence of freshness: a stopped publisher's last file stays readable forever, and every poll would renew a read-age clock (homemap.go:440-455; internal/host/edgesfile.go has no write timestamp today) (astra B8). Nor is a successful write: a publisher whose record reads are failing still writes its file from its local registry rows (edgesfile.go:420), so a write-time stamp would keep advertising an obsolete generation as fresh (astra R1-3).

So at is the host's authority time: the last moment the home box knew the records behind that host were current and applied. The box keeps a per-app records confirmed time: the start time of the last reconcile pass (provision.Reconcile, run by reconcileSerializedWith, internal/host/agent.go:1604) that read the app's records completely and applied them to the registry without error. Reconcile today returns one error for the whole pass, so it must also report which app scopes it read completely and applied. Where those reads come from depends on how the box is configured:

• with the change-index record view armed (CARLOS_CHANGE_INDEX=read, internal/host/changeindex.go:57-78), a scope counts as read completely only when it is applied and not pending in the snapshot the pass used. A refresh that returns nil while a scope stays pending (internal/provision/recordview.go:164-175) does not count;
• without it (armedRecordView nil, the bucket read directly), a scope counts only when its listing and every record body under it were read without error.

The pass's start time is the one recorded, because a record written during the pass may not have been seen. Publishing never advances it.

The invariant. A host's at may only be a time at which every input the catch decision for that host reads was known complete and applied on the publishing box. Rounds 1 to 3 of review each found one more input left out (records, then aliases, then catch-all aliases and path rows), so the spec names the inputs instead of the cases (astra R2-1, R3-1, R3-2). The catch decision (§3.3, §4.1) reads:

1. The app's records (the instance record, its route row, gate, import gate, opt-out, generation): bounded by the app's records confirmed time above.
2. Domain claims, which decide whether a hostname has path rows, which aliases exist, and which aliases instance catch-alls derive (internal/domains/aliases.go, ReconcileAliases): bounded by the box's domain confirmed time, the completion time of the last ReconcileAliases pass that was complete (again its start time). Today the pass returns an error only when the claim listing fails. Everything else is logged and the pass returns nil: the label count for a catch-all (aliases.go:162-171), instance alias upsert (:204), alias upsert (:310), catch-all upsert (:531), alias delete (:541), wildcard upsert and delete (:561, :568), path upsert (:584) and path delete (:590). Two inputs fail silently too: claim bodies the claim cache could not read are dropped (internal/domains/claimcache.go:137-145), and a failed edgesfile.DeploymentRoutes read comes back nil and is treated as "allow". So ReconcileAliases must also report completeness, and any of those makes the pass incomplete. An incomplete pass does not advance the domain confirmed time. Existing callers keep today's error behaviour; TestInstanceCatchallKeepsNamesWhenTheListingFails (instancecatchall_test.go:507) still sees nil and now also sees "incomplete".
3. Release manifests: immutable once read, so they add no bound.

Every published host, primary or alias, gets at = the older of (1) for its app's scope and (2) for the box. This applies to primary hostnames as much as aliases, because proving a hostname has no path row is a domain read too (R3-2). A box whose domain reads fail for an hour stops catching every host it serves, on every edge. Any new input a future change adds to the catch decision must bring its own completeness report into this minimum, and the plan states this as a test (§12): a table of inputs, each failed in turn for over an hour with every other input healthy, must stop catching on home and far edges.

• A far edge serves and fills a host's caught paths only while that host's at is within the last hour by the reader's clock. Past that it stops admitting hits and fills and forwards as today.
• A box serves and fills its own routes' caught paths under the same rule, against its own confirmed time, so a home cut off from the bucket stops serving entries a purge it cannot see may have retired.
• After a restart the confirmed time starts empty, so nothing is caught until the first complete convergence.

Cost: the file is rewritten when an at advances by 20 minutes or more, about three extra PUTs per publishing box per hour and the control/edges bumps they cause, which every edge already reads.

Immutable assets would be safe to serve longer, but a purge (§6) or an owner change that never arrived must not live forever on a disconnected box. One hour and twenty minutes are my calls; they are constants, not knobs.

5. Serving

5.1 Where the asset cache sits

A new disk cache, separate from the shared LRU in both directions: caught requests never read or fill the LRU, and nothing else reads or fills the asset cache.

In ServeHTTP the caught-path check runs:

• on a box with a local route row (the home box, a replica, a pool member): after lookupUpstreamPath and the gate, before the shared-cache lookup and before the activator. The catch is decided from the local route's own published state (§3.3, §4.1), not from the home map.
• on a box without one: in the lookup-miss branch, before e.backhaul, using the home map's entry (§4.2). A miss fills by backhaul (§5.3).

A request is caught when all hold: method GET or HEAD; the request path is already canonical, meaning r.URL.RawPath is empty (no percent-encoding the decoder had to preserve) and the path equals its own path.Clean with any trailing slash restored; it starts with a published prefix; the box is armed (§10). A non-canonical path under a prefix (/assets//app.js, /assets/./app.js, /assets/%61pp.js) is forwarded as today and answered Carlos-Cache: bypass. Neither the key nor the upstream request is ever normalised: lookupUpstreamPath matches cleaned and raw first segments but forwards the original path (internal/edge/edge.go:1304-1323), so a cleaned key could be filled with the answer to a different URL (astra B4, B7). Anything else under the prefix (POST, an upgrade) is forwarded exactly as today, uncached and with its headers intact.

5.2 What a caught request sends upstream

Fills send an allow-listed request: method GET, the path and raw query, Host, User-Agent, Accept, Accept-Encoding: identity, plus the hop headers when it is a backhaul. No Cookie, no Authorization, no other client header. That is the public contract a declared prefix makes, and it is what makes the anti-goal "a logged-in response served to someone else" structural rather than a hope about the app's headers. A HEAD fills with GET. The docs page says it in one sentence: files under an asset path are fetched without the visitor's cookies.

Range, If-None-Match and If-Modified-Since are answered by the edge (§5.5) and never forwarded on a fill.

5.3 Fill, admission and coalescing

Fills are bounded per box (16 concurrent) and per app (4); over either bound the request is forwarded as today, uncached, rather than queued. Fills from a far edge go through backhaulAcquire like any hop.

Every fill has a deadline, and so does every waiter (astra R3-3). Ordinary backhaul deliberately has no body deadline (edge.go:2837), so a fill must bring its own: 60 seconds from start to the last body byte. A fill that misses it is abandoned for caching; the initiating client keeps receiving the bytes as an ordinary response would, but nothing is stored and the reservation (§7) is released at the moment of abandonment, not when the stream ends. At most 32 requests wait on one fill, and each waits at most 10 seconds; a request over the cap or past its wait is forwarded as today, which puts it back under backhaulAcquire's per-home limit (edge.go:2903) instead of beside it.

Coalescing shares only an admitted entry (astra R2-5). Concurrent requests for one key wait on a single fill, but what they wait for is the entry, not the response. If the fill produces an entry, each waiter is then answered from it as a hit, with its own conditional headers, Range and encoding applied separately (§5.5). If it does not (refused at admission, no acknowledgement, an error), only the requester that started the fill gets that response, and every other waiter makes its own request, forwarded as today and subject to the same fill bounds. A refused response, which may carry Set-Cookie or vary on Accept, therefore never reaches anyone but the client it was made for, and every client delivery is a request somebody metered.

A far edge's fill is a signed request, answered with a signed acknowledgement. Ordinary backhaul is not enough: the home meters every hop and consults its shared LRU (edge.go:1678, :1768), so a plain hop would count the bytes twice and could refill a far edge from a shared-LRU entry that predates a purge (astra B6). A bare marker header is not enough either: an old far edge forwards any client header and then signs the hop, and the hop MAC covers only the host and a timestamp (internal/edge/mesh.go:178-190), so a client could have a new home skip metering through an old edge (astra R1-1). And the fill must name the generation it is filling for, or a home that has not yet reloaded a purge could answer from its old generation and the far edge would store those bytes under the new one (astra R1-2).

• Assertion. The far edge sends X-Carlos-Asset-Fill: <ts>:<mac>, where mac is HMAC-SHA256 under the mesh key (the same key, current or next, that signHop uses) over the domain carlos-asset-fill-v1, the host, the path and raw query, the account, the app, the generation and ts, each newline-separated, with the hop's ±hopSkew window. The domain separator means no hop signature is ever a valid assertion.
• Home. The home treats a request as a fill only when the hop is verified, the assertion verifies, the home is armed, and its own local route for the host has that account, app and generation current and catches that path under §4.2's freshness rule. It then answers from its own asset cache under that key, or fills it from the app (§5.2), never reads or writes its shared LRU for the request and never meters it. It first deletes any X-Carlos-Asset-Fill-Ack and X-Carlos-Asset-Expires the app's response carries, then sets its own on the response headers: - X-Carlos-Asset-Expires is the entry's absolute expiry (§5.4) when the response is storable by the header-time rules below, and 0 when it is not (no lifetime, no-store, non-200, a disallowed Vary, an undeclaring build, and so on); - X-Carlos-Asset-Fill-Ack is HMAC-SHA256 under the mesh key over carlos-asset-fill-ack-v1, the request's assertion and the X-Carlos-Asset-Expires value.

The acknowledgement means "the home handled this as a fill and did not meter it", whatever the expiry says; storability is the expiry's job (astra R2-3).

• Anything else is an ordinary hop: a missing or invalid assertion, an unarmed or old home, a generation or owner mismatch, a host past its freshness bound. The home meters it as today and sends no acknowledgement.
• Far edge. It meters every response that carries a valid acknowledgement of its own assertion, storable or not, and its own hits. It stores an acknowledged response only when X-Carlos-Asset-Expires is non-zero and its own admission checks (body size, reservation) pass. A response without a valid acknowledgement is served to the requester unmodified, answered Carlos-Cache: bypass, not stored and not metered at the far edge, because the home metered it. So every response is counted exactly once, a disarmed or lagging home can never populate a far cache, and the mixed-fleet window after a home is disarmed only costs hits.
• The assertion and acknowledgement headers are stripped from every inbound non-hop request and from every response a client receives.

A response is stored only when all hold:

• status 200 (not 206, not any other 2xx);
• no Set-Cookie header or trailer;
• Cache-Control present, not private, no-store or no-cache, with a positive s-maxage or max-age. Lifetime is s-maxage, else max-age, capped at 365 days. There is no five-minute clamp: a declared prefix plus an explicit lifetime is the app saying the bytes hold;
• the response is still fresh on arrival. On a fill from the app the edge computes the entry's absolute expiry as the response time plus the lifetime minus the corrected initial age (RFC 9111 §4.2.3: the corrected initial age already includes the response delay, so the reference point is when the response arrived, not when the request left) (astra R2-4). On an acknowledged far fill the expiry is the home's X-Carlos-Asset-Expires, never recomputed, so a copy of a nearly expired home entry expires when the home's does (astra R1-6). An already expired response is served and not stored;
• Vary absent, or exactly Accept-Encoding; anything else (Origin, Cookie, *, Accept) is refused;
• no Content-Encoding other than identity (the fill asked for identity);
• the Content-Type is present and is not HTML (text/html, application/xhtml+xml). A response with no Content-Type is not stored either: sniffing needs the body, and the home must decide storability when the headers arrive (§5.3, X-Carlos-Asset-Expires). Build assets are never pages, and an SPA fallback can carry the asset path's long Cache-Control as easily as no-cache, so the lifetime alone cannot be what refuses it (astra R3-4);
• body at most 32 MiB, measured while streaming; a larger body is streamed to the client and discarded;
• the build that answered declared the prefix (astra R2-2). The §4.1 union decides what is caught and which existing entries may be served, but during an update the proxy picks the serving generation on its own (internal/edge/genproxy.go:255), so an older build that never declared /bundles/ could answer a request for it with a cacheable fallback, and with no version in the key that entry would outlive the update. On a generation-managed route the home knows which generation answered (genproxy.go:298-302, noteServedGen) and stores only when that generation's build declared the matching prefix; on any other route the answering build is Route.Version's. A far fill inherits this through the home's X-Carlos-Asset-Expires: 0.

A refused response is still served to the requester, unmodified, so a caught prefix that the app does not mark cacheable behaves exactly as an uncaught one. That is what defuses the SPA fallback: index.html served with no-cache for an unknown asset name is never stored. Non-200 is never stored either, so a request racing a deploy that 404s does not poison the name.

5.4 Cache key and entry

Key: (account, app, gen, request host, path, raw query), the path being canonical by §5.1. Account and app come from the published owner, so a hostname that changes hands can never serve the previous owner's entries; gen makes a purge a key change (§6). The release version is not in the key: an unchanged hashed file stays warm across deploys, which is the point.

An entry holds the identity body, its absolute expiry and lifetime, a gzip copy when the response is compressible by the existing rule (compressible, internal/edge/compress.go:100-118: not already encoded, no Content-Range, no Cache-Control: no-transform, at least minCompressSize, a compressible type), made once at fill, and every end-to-end response header except: hop-by-hop headers (Connection and those it names, Keep-Alive, Transfer-Encoding, Upgrade, TE, Trailer, Proxy-*), Date (regenerated), Age (recomputed), Content-Length (recomputed per representation), the purge headers and platform X-Carlos-* headers. So Content-Security-Policy, Cross-Origin-Resource-Policy and anything else the app sets for safety reach a hit exactly as they reached the miss, the way the shared cache already keeps and replays response headers (edge.go:1907, :2121) (astra R1-4). Access-Control-Allow-Origin with any value other than * is still refused at admission, because it would need Vary: Origin. When the app sent no ETag, the edge sets a strong one from the body's sha256, identical on every edge.

5.5 Answering from an entry

• If-None-Match / If-Modified-Since answered with 304.
• One Range answered with 206 from the stored identity body; multiple ranges answered with the whole body; If-Range against the identity ETag (a weak validator never matches, so a client holding the gzip ETag gets the whole body); 416 when unsatisfiable. http.ServeContent semantics over the file are acceptable.
• gzip copy when the entry has one, Accept-Encoding allows it and no Range is present. The two representations are kept apart the way compress.go:230-245 already does: every response from an entry that has a gzip copy carries Vary: Accept-Encoding, whichever encoding it is, and the gzip copy's ETag is the identity ETag weakened (weakenETag), so a downstream cache cannot hand gzip to a client that did not ask for it and a range is never validated against compressed bytes (astra R1-7). Caught responses bypass the edge's gzip wrapper; the asset path writes the chosen representation itself.
• Added on every caught response, hit or not: Carlos-Cache: hit, miss or bypass (the last for anything forwarded uncached). A hit carries no X-Carlos-Version: an entry can outlive many deploys, so a version on it would be a false "verified" for the deploy watch (edge.go:813-847 explains the header's freshness contract).
• Age on hits: the lifetime minus the time left to the entry's expiry.

An entry past its lifetime is not revalidated in v1; it is dropped and refilled. (Revalidation with If-None-Match upstream is a later optimisation.)

6. Purge: the asset generation

carlos assets purge <app> (and a console door, POST /a/<sqid>/apps/<app>/assets/purge) draws a fresh random 64-bit token, the asset generation, for every instance record of the app. A new instance record draws its own token at create. A random token rather than a counter means a hostname deleted and recreated can never land back on a generation an edge still holds entries for, and needs no app-level state that outlives its instances (astra B1). Each record is written separately; the door and the CLI report per instance which writes succeeded and retry the rest, and a partly applied purge says so rather than reporting done. The token flows record -> convergeOwned -> route row -> edges file -> every edge's home map, and every key under the old token stops matching at once. Old entries are garbage, reclaimed by eviction and by a sweep that deletes entries whose token is not the host's current one.

Carlos-Purge from the app continues to act on the shared LRU only, on the box that saw it. It does not touch the asset cache: a per-path purge that reaches one edge of four is the trap the shared cache's five-minute clamp exists to bound, and this cache has no clamp. Apps fix a mistake with the generation.

carlos assets status <app> prints, per instance: the declared paths of the builds being served, the opt-out, the generation in the record, and the generation and assets_at its home has published in its edges file (the console reads those files already). A purge is complete at the home when the published generation matches the record. Far edges do not report back; the status states the bound plainly: every edge that can read the bucket picks the new generation up within a minute, and one that cannot stops serving the host's cached assets within an hour (§4.2).

Expected latency: under a minute (edges-file rewrite within about 2 s, 30 s read poll, plus record convergence on the home box).

7. Disk budget and fairness

• Location: $CARLOS_DATA_DIR/asset-cache/. Entries are files named by a hash of the key; an index is rebuilt by scanning at start.
• Budget: CARLOS_ASSET_CACHE_MAX bytes; default the smaller of 4 GiB and 10% of the data filesystem's size. A 20 GiB box gets 2 GiB.
• Reservations. Checking free space when a fill starts is not enough: sixteen concurrent maximum-size fills on an 8 GiB filesystem at 15% free would take it below the 10% placement floor (internal/boxchoice/policy.go:20) (astra R1-8). So before a fill starts it reserves its worst case: Content-Length if the response will declare one, else the 32 MiB cap, doubled for the gzip copy (which is abandoned if it would exceed the identity body), plus 64 KiB for metadata. A fill is admitted only when the cache's used bytes plus all outstanding reservations plus this one fit the budget (evicting first if they do not), and the filesystem's free space minus all outstanding reservations minus this one stays at or above 15% of its size. A write past the reservation aborts the fill. The reservation is released on every exit, success or failure. Temporary files live in the cache directory and are deleted at start.
• Fair share: each app with entries on the box has a share of the budget (budget divided by the number of such apps, but never more than 25% of the budget for one app). Eviction takes the least recently used entry of whichever app is furthest over its share, so one app walking unique query strings evicts itself first.
• After a restart the edge serves no entry until the home map has completed one read (for far hosts) or the box's record view has a confirmed time for the route's scope (for local routes, §4.2), so an entry is never served against a generation or owner it has not checked.
• Memory: the index only. The kernel page cache does the rest.

8. Usage

Every caught response, hit or fill, is counted as the app's request and bytes out, attributed by the published account and app, by whichever edge served it. Today the sending edge of a backhaul records nothing and the home records the hop (usagemeter.go:43-45); for caught paths the serving edge records instead, and a fill's backhaul is recorded once, not twice. Asset bytes are not free bandwidth. (R2.)

9. Provider boundary

All core. The design reads and writes nothing beyond the edges file, the release manifest and the instance record, all through existing store interfaces, and adds no cloud API call. The geographic win depends on latency DNS, which is the existing Route 53 steering adapter; a self-hosted deployment with one plain wildcard to one edge keeps G4, G6 and the home box's disk cache, and loses only nearest-edge entry. No AWS SDK import in any touched package; a test runs the full fill/hit/purge path against DirStore with no AWS configuration.

10. Rollout

• Arming. CARLOS_ASSET_CACHE=1 in a box's host.env arms its asset cache: catching, filling, serving, and publishing assets for the routes it serves (§4.1). Off by default; an unarmed box publishes nothing and forwards exactly as today. The one exception is the window after a home is disarmed (§5.3), in which it treats marked fills as ordinary hops.
• Mixed fleet. An old or unarmed home publishes nothing, so nothing is caught for its hosts anywhere. An old far edge ignores the field and backhauls. Every combination is today's behaviour or the new one, never a half.
• Order. Release with the feature dormant; roll dev boxes, dev console, check, flagship boxes, flagship console. Then arm a test app's home dev box first (only an armed home publishes, §4.1), then one dev box in another region, and run the two-edge acceptance gate (§12) before arming the rest of dev (astra R3-5). On flagship, arm the console box (which homes most apps) first, then one small box (US or AU) to watch disk, then the rest one at a time.
• CLI. carlos ship --assets, carlos deploy --assets and --no-assets, the assets project default, carlos assets purge, carlos assets status, carlos instances set-assets get docs/site/cli.md headings (docsite gate). Every user-facing string goes through copy review before it is written.
• Census. Touching internal/provision and internal/host likely needs a plan14 overlay and a #280 ruling; the plan names which units.

11. Relation to the media roadmap

Stage 1 needs a per-box disk cache with a total budget, filesystem headroom, per-route fair share and bounded fills (roadmap §4, "Its own cache"). Build §7 as a package the media handler can reuse with its own budget and its own index, not as code inside edge.go. Stage 1's distributed purge log and serving lease remain its own; nothing here weakens them or is offered as a substitute.

12. Testing and the acceptance gate

Tests go through production wiring (two edges, real edges files through a DirStore, real home map fold, real backhaul with hop auth), not fixture routes. Attacks first:

• a fill never forwards Cookie or Authorization, asserted at the app;
• a response with Set-Cookie, Vary: Cookie, Vary: Origin, Cache-Control: private, no-cache, a 206, a 404, or a body over the cap is served to the requester and not stored;
• an SPA fallback for an unknown hashed name is not stored, both with no-cache and with Cache-Control: public, max-age=31536000;
• the authority table: each input of §4.2's invariant (record scope, claim listing, catch-all label count, alias upsert, path delete) failed alone for over an hour, with every other input and the edges-file writes healthy, stops catching on home and far edges. The cases include an exact claim replacing a derived catch-all alias, and a claim adding a path row to a gated target under a primary hostname;
• a fill whose app stalls its body after admissible headers is abandoned at 60 seconds with its reservation released, at most 32 requests wait on it, and the rest are forwarded under the backhaul limit;
• a gated route, an import-gated route, a pooled route, an opted-out instance, a hostname with any path-prefix row and a hostname split between two apps publish nothing and catch nothing;
• a non-canonical path under a prefix (//, ./, percent-encoded) is forwarded and answered bypass, and never fills a canonical key;
• an acknowledged fill is metered once, at the far edge, and does not touch the home's shared LRU; an unacknowledged one is metered once, at the home, and not stored; a client-sent assertion or acknowledgement is stripped at a new edge, and one forwarded by an old far edge to a new home fails verification and is billed as an ordinary hop;
• a fill whose assertion names a generation the home has not reloaded yet, or an owner that has changed mid-fill, gets no acknowledgement and is not stored; the same for a disarmed home;
• an app response carrying its own X-Carlos-Asset-Fill-Ack or X-Carlos-Asset-Expires has them removed before the home signs;
• a nearly expired home entry filled through a far edge expires at the home's deadline on both boxes; a delayed origin whose response arrives after half its lifetime is stored and expires at response time plus lifetime minus age;
• an acknowledged fill whose response has no Cache-Control, no-store, a 404 or an oversized body is metered once at the far edge and stored nowhere;
• during an update where only the installed build declares /bundles/, a cacheable fallback answered by the current build is not stored on home or far edges; the same when only the retiring build declares it;
• concurrent requests for one key with different Accept headers, a refused response carrying Set-Cookie, and an unacknowledged fill each reach only their initiating client, and the other waiters are metered separately;
• an alias detached while the box's claim listing fails for over an hour, with record reads and edges-file writes succeeding, stops being served from cache on home and far edges;
• an SVG with a restrictive Content-Security-Policy and Cross-Origin-Resource-Policy carries both on a hit as on the miss;
• identity and gzip responses from one entry both carry Vary: Accept-Encoding, the gzip ETag is weak, If-Range with it returns the whole body, and no-transform gets no gzip copy;
• sixteen concurrent maximum-size fills near the headroom threshold never take the filesystem below 15% free, and every exit releases its reservation;
• a generation-managed instance goes warm, sleeps and restarts, and its assets are still hits on home and far edges without a wake;
• a generation-managed route mid-update catches both builds' prefixes and drops the old one when its generation retires;
• a deleted and recreated hostname draws a new generation and serves no old entry;
• a hostname moved to another app or account serves none of the old owner's entries, on a warm cache;
• a purge makes every edge miss within one home-map poll of the rewritten file; with the home's record reads failing for over an hour while its edges-file writes keep succeeding, at stops advancing and every edge, the home included, stops catching for the host; a far edge whose winning entry's at is older than an hour forwards even though the file still reads successfully;
• an edge restarted with a warm disk serves nothing until its first read;
• one app requesting unbounded unique query strings evicts its own entries before another app's, and no fill starts under 15% free disk;
• a hibernating route's caught hit does not wake it;
• / and the other bad prefixes are refused at ship, at deploy and by the console's ship door with a CLI that skipped the check;
• a zero-arg carlos deploy with a saved assets default declares it; --assets replaces it for the run; --no-assets declares none.

Then, on dev: an asset requested through the far region's edge is a miss once and a hit after, the home box's access log shows one request for it, and a 5 MiB bundle is served gzip-compressed from the far edge. On flagship, the same from a small box with the disk figures recorded.

13. Rulings (Paul, 2026-10-24)

• R1. Instance off switch only. Paths come from the release; an instance can switch catching off and cannot declare paths (§3.2).
• R2. Asset bytes count the same. Every hit and fill is the app's request and bytes out, against plan caps like any other response (§8).
• R3. It works with carlos deploy as well as carlos ship (§3.1).

14. Review log

• rev 1, 2026-10-24: first draft.
• rev 2, 2026-10-24: Astra builder check (CODEX-FINDINGS-ASSET-CACHE-BUILDER.md), all nine items accepted and verified against the tree: B1 random per-record generation; B2 no path-prefix hostnames; B3 only an armed home publishes; B4 and B7 canonical paths only, never normalised; B5 union of live generations' paths; B6 marked fills bypass the home's LRU and meter; B8 assets_at stamp and record-view bound; B9 import gate excluded. Purge status no longer claims a far-edge acknowledgement it has no channel for. Rulings R1 and R2 recorded.
• rev 3, 2026-10-24: carlos deploy declares too (Paul): --assets, --no-assets, the .carlos/config default, and the console ship door's own check (§3.1).
• rev 4, 2026-10-24: Astra review round 1 (CODEX-FINDINGS-ASSET-CACHE-R1.md, reject: 4 P1, 4 P2). All eight accepted after checking the citations: R1-1 and R1-2 the fill becomes a domain-separated signed assertion naming owner and generation, answered by a signed acknowledgement, with single metering either way; R1-3 at is the record view's per-scope confirmed time, never the write time; R1-4 every end-to-end header is kept; R1-5 the installed build is in the paths union so a sleeping instance keeps catching; R1-6 absolute expiry, age-corrected, carried to far copies; R1-7 representation rules from compress.go; R1-8 worst-case disk reservations.
• rev 5, 2026-10-24: Astra review round 2 (CODEX-FINDINGS-ASSET-CACHE-R2.md, reject: 1 P1, 4 P2; R1-3 and R1-6 partly fixed). All five accepted after checking the citations: R2-1 alias at also bounded by the last successful alias reconcile; R2-2 only a build that declared the prefix may fill it; R2-3 the acknowledgement transfers billing and X-Carlos-Asset-Expires: 0 says "not storable"; R2-4 expiry is measured from response time; R2-5 coalescing shares only an admitted entry.
• rev 6, 2026-10-24: Astra review round 3 (CODEX-FINDINGS-ASSET-CACHE-R3.md, reject: 2 P1, 2 P2, 1 P3; R2-1 and R1-3 partly fixed). All five accepted after checking the citations. The two P1s were the third round in a row to find one more input left out of the freshness bound, so §4.2 now states it as an invariant over every input the catch decision reads, requires ReconcileAliases to report completeness, and §12 tests it as a table (R3-1, R3-2). R3-3 fill and waiter deadlines; R3-4 HTML is never stored; R3-5 arm the home first.
• round 4, 2026-10-24: Astra accept at 287a59c4, no new findings (CODEX-FINDINGS-ASSET-CACHE-R4.md).
• rev 7, 2026-10-24: corrections found while writing the plan, against the code: the records confirmed time comes from reconcile's own per-app completeness, with or without the record view (which is armed only under CARLOS_CHANGE_INDEX=read); the paths union is installed build plus ServingGen's build (the activator keeps one current generation); manifests are read from the bucket, not the staged copy; every ReconcileAliases log-and-continue point and silent input is listed; the opt-out verb is carlos instances set-assets; two citation fixes.
• rev 8, 2026-10-24: Astra round 5 (CODEX-FINDINGS-ASSET-CACHE-R5.md, reject on one P2): the manifest cache keeps only declared paths, only for builds a local route references, pruned every pass (R5-1).
• round 6, 2026-10-24: Astra accept at 91debc18 (CODEX-FINDINGS-ASSET-CACHE-R6.md).
• rev 9, 2026-10-24: a response with no Content-Type is not stored (narrows rev 6's "absent and sniffed as HTML", which the header-time storability decision could not implement). Reviewed with the plans.
• plan review round 8, 2026-10-24: Astra accept of plans 1 to 3 and spec rev 9 at 4cca5784.