carlos / platform Public

Clone
git clone https://amadan.net/carlos/platform

Plain git — no account needed to clone.

Download

Download this file

Edge asset cache, plan 3: serving, the fill protocol and arming Implementation Plan

> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Serve declared asset paths from the disk store on the home box and on every far edge, fill far caches through a signed fill protocol with exactly-once metering, and arm it per box with CARLOS_ASSET_CACHE=1, ending with the two-edge acceptance gate on dev.

Architecture: All new code is in internal/edge, in five files: assetfill.go (the assertion and acknowledgement signatures), assetadmit.go (admission, expiry and stored headers), assetserve.go (answering from an entry), assetcatch.go (the catch decision, coalescing, local and far fills) and assetarm.go (EnableAssetCache, the sweep). ServeHTTP gains three hooks: a home-side fill handler for verified hops, a local catch before gzip and the shared cache, and a far catch in the lookup-miss branch before backhaul. internal/host only arms: it opens the store and hands the edge plan 1's assetPolicy as edge.LocalAssets.

Tech Stack: Go 1.26, net/http (http.ServeContent), internal/assetcache (plan 2), crypto/hmac + SHA-256.

Spec: docs/superpowers/specs/2026-10-24-edge-asset-cache-design.md (rev 9). Read §5, §7, §8, §10 and §12 before starting.

Branch: edge-asset-cache-3, cut from amadan main after plans 1 and 2 have merged.

Global Constraints

Every task's requirements include this section.

  • Production-wired acceptance. Task 9's acceptance tests run two real Edges over one DirStore with real TLS, real mesh keys, real hop auth and real backhaul (newBackhaulScene, backhaul_e2e_test.go:380), real assetcache.Stores on t.TempDir(), and edges files whose host entries are read from plan 1's golden internal/edgesfile/testdata/assets-host.golden.json, which plan 1's writer test pins. Unit tests in Tasks 1 to 8 may use recorders and fakes and say "not acceptance".
  • Targeted local runs; CI runs the full suite. go test ./internal/edge/ -run 'TestName' -count=1. Never a whole internal/edge or internal/host run locally. The e2e scene binds 127.0.0.2 and real ports: run it with the sandbox off.
  • Cross-compile gate, copy review with no em dashes, no tenant names, Go cache, merge main first, amadan tasks, test weights: as in plan 1.
  • Census overlay per branch, loaded LAST. plan14-root-reads-edge-asset-cache-3.tsv, rev3=PROPOSED-edge-asset-cache-3.
  • Header names, verbatim. X-Carlos-Asset-Fill, X-Carlos-Asset-Fill-Ack, X-Carlos-Asset-Expires, Carlos-Cache (values hit, miss, bypass). Signature domains carlos-asset-fill-v1 and carlos-asset-fill-ack-v1.
  • Spec numbers, verbatim. Lifetime cap 365 days. Fill deadline 60 s from start to last body byte. At most 32 waiters per fill, each waiting at most 10 s. Fill bounds 16 per box, 4 per app (enforced by assetcache.Reserve). Body cap 32 MiB. Freshness one hour (plan 1's AssetsFor and assetPolicy.For). Sweep every 5 minutes. Upstream request allow-list: Host, User-Agent, Accept, Accept-Encoding: identity, plus hop headers on a backhaul.

Review Focus

  1. A HEAD that arrives first. A person expects it to be answered and to leave a usable entry. It fills with GET, stores, and answers HEAD from the entry with no body. Test: Task 5, TestHeadFillsWithGetAndAnswersHead.
  2. A query string on a hashed asset (/assets/app.js?v=2). A person expects ?v=2 and ?v=3 to be different entries and ?v=2 twice to hit. Test: Task 5, TestQueryIsPartOfTheKey.
  3. A client that disconnects mid-fill. A person expects the fill to keep going for the waiters, within its 60 s deadline, and the entry to be stored when it completes. Test: Task 5, TestFillSurvivesTheInitiatorDisconnecting.
  4. An app that sends Content-Length larger than its body, or none at all. A person expects a short body never to be stored and an unknown length to be reserved at the 32 MiB worst case. Test: Task 5, TestShortBodyIsNotStored; plan 2 covers the reservation.
  5. A far edge whose clock is a few seconds off the home's. The home's absolute X-Carlos-Asset-Expires is used as given. A person expects at most that skew of extra or lost freshness, never a doubled lifetime. Test: Task 6, TestFarExpiryIsTheHomesDeadline with a 5 s clock offset.

Plan review

Astra reviewed the three plans together over eight rounds (docs/superpowers/specs/CODEX-FINDINGS-ASSET-CACHE-PLANS.md, then -PLANS-R2.md to -PLANS-R8.md; brief CODEX-REVIEW-BRIEF-ASSET-CACHE-PLANS.md) and accepted them at 4cca5784. Findings were 16, 6, 6, 3, 1, 1, 2, then none. Every finding was accepted after its citation was checked; none was declined. Findings that changed a task are marked (astra Pn-m) where they landed.

Census impact

  • Changed frozen units, re-fingerprinted @legacy with the same class: internal/edge/edge.go Edge.ServeHTTP, Edge.backhaulTo (if Task 6 shares its preamble), RunWithActivatorReady (arming), internal/edge/usagemeter.go UsageMeter.Record only if its body changes (it should not).
  • New units, if the census flags any (reads go through assetcache, so likely none): plain rows of class legacy, task accepted-legacy, basis accepted-legacy;rev3=PROPOSED-edge-asset-cache-3;... (astra P1-3).
  • Envelopes re-pinned: internal/edge, internal/host.
  • Notes: docs/superpowers/notes/census-proposal-edge-asset-cache-3.md, ruling asked on #280.

Copy strings

Added to docs/superpowers/plans/2026-10-24-edge-asset-cache-copy.md by Task 0 of this plan, reviewed before Task 10 writes them:

| id | where | what it says | |---|---|---| | EAC-23 | docs/site/cli.md, under carlos ship --assets | files under an asset path are fetched without the visitor's cookies (spec §5.2) | | EAC-24 | same | what Carlos-Cache: hit, miss and bypass mean, and that only responses marked cacheable, status 200, not HTML and without Set-Cookie are kept |

File structure

| File | Responsibility | |---|---| | internal/edge/assetfill.go | Header names, signAssetFill, verifyAssetFill, signAssetAck, verifyAssetAck. | | internal/edge/assetadmit.go | assetAdmit, assetStoredHeader, correctedInitialAge. | | internal/edge/assetserve.go | serveAssetEntry. | | internal/edge/assetcatch.go | canonicalAssetPath, assetKey, assetFlight, assetFill, the three ServeHTTP hooks. | | internal/edge/assetarm.go | EnableAssetCache, the sweep loop. | | internal/edge/edge.go (modify) | Call the hooks; strip asset headers; skip meter for acknowledged home fills. | | internal/edge/cookiescope.go (modify) | responseModifier strips app-set X-Carlos-Asset-* and Carlos-Cache. | | internal/host/agent.go (modify) | Arm: open the store, EnableAssetCache. | | internal/edge/asset_*_test.go, internal/edge/asset_e2e_test.go | Tests. |

---

Task 0: Copy rows

  • [ ] Step 1: Add EAC-23 and EAC-24 to the copy record as pending and extend the gate's inventory to EAC-1 to EAC-24.
  • [ ] Step 2: Run the copy-review skill on the two rows; apply Paul's edits verbatim; mark them approved.
  • [ ] Step 3: go test . -run 'TestAssetCacheCopy' -count=1. Expected: PASS. Commit.

Task 1: Fill assertion and acknowledgement

Files:

  • Create: internal/edge/assetfill.go, internal/edge/assetfill_test.go

Interfaces:

  • Consumes: signingKey, canonicalizeHopHost, hopSkew (mesh.go).
  • Produces: - const assetFillHeader = "X-Carlos-Asset-Fill", assetAckHeader = "X-Carlos-Asset-Fill-Ack", assetExpiresHeader = "X-Carlos-Asset-Expires", assetCacheHeader = "Carlos-Cache" - type assetFillClaims struct { Host, PathQuery, Account, App, Gen string } - func signAssetFill(key []byte, c assetFillClaims, now time.Time) string ("<unix>:<hex>", HMAC-SHA256 over carlos-asset-fill-v1, then the five fields and the timestamp, newline-separated, host canonicalized) - func verifyAssetFill(current, next []byte, c assetFillClaims, header string, now time.Time) (matched []byte, ok bool) (either key, ±hopSkew, constant-time; returns the key that verified, and the home signs the acknowledgement with that key, so a home that already holds .next answers a far edge still signing with the current key in the far edge's key (astra P1-11)) - func signAssetAck(key []byte, assertion string, expires int64) string (hex HMAC over carlos-asset-fill-ack-v1, the assertion and the decimal expiry) - func verifyAssetAck(key []byte, assertion string, expires int64, ack string) bool (the far edge checks with the key it signed the assertion with)
  • [ ] Step 1: Write the failing tests. TestAssetFillRoundTrips; TestAssetFillRefusesEachChangedClaim (table: host, path-query, account, app, gen, timestamp outside skew); TestHopSignatureIsNeverAnAssetFill (a signHop value for the same host and time fails verifyAssetFill); TestAssetFillAcceptsTheNextKey; TestAssetAckBindsAssertionAndExpiry (changing either fails); TestAckUsesTheKeyTheAssertionMatched (home holds current and next, far signs with current only: the ack verifies at the far edge).
  • [ ] Step 2: Run go test ./internal/edge/ -run 'TestAssetFill|TestHopSignatureIsNever|TestAssetAck|TestAckUsesTheKey' -count=1. Expected: FAIL.
  • [ ] Step 3: Implement.
  • [ ] Step 4: Run the same command. Expected: PASS.
  • [ ] Step 5: Commit.

Task 2: Admission, expiry and stored headers

Files:

  • Create: internal/edge/assetadmit.go, internal/edge/assetadmit_test.go

Interfaces:

  • Consumes: parseMaxAge, hasCCDirective, joinHeader, compressible (edge.go, compress.go).
  • Produces: - type assetVerdict struct { Store bool; Reason string; Expires time.Time; Lifetime time.Duration; Gzip bool } - func assetAdmit(status int, h http.Header, requestTime, responseTime time.Time, declared bool) assetVerdict: the rules of spec §5.3 that can be decided from headers, in order: status 200; no Set-Cookie; Cache-Control present, not private, no-store or no-cache, positive s-maxage else max-age, capped at 365 days; Vary absent or exactly Accept-Encoding (case-insensitive, whitespace-tolerant); Content-Encoding absent or identity; Content-Type present and not text/html or application/xhtml+xml; Access-Control-Allow-Origin absent or *; declared; Content-Length, when present, at most 32 MiB; still fresh, with Expires = responseTime + lifetime - correctedInitialAge. Gzip is compressible(h, 200). - func correctedInitialAge(h http.Header, requestTime, responseTime time.Time) time.Duration (RFC 9111 §4.2.3: max(apparent_age, Age + response_delay), apparent_age = max(0, responseTime - Date)) - func assetTrailerVeto(trailer http.Header) bool (true when the trailers carry Set-Cookie, announced in Trailer or sent unannounced with http.TrailerPrefix); every fill, home and far, checks it after the last body byte and before Commit, and the far edge reads the backhauled response's own trailers, which the transport passes through (astra P1-6) - func assetStoredHeader(h http.Header) http.Header: a clone without hop-by-hop headers (Connection and the names it lists, Keep-Alive, Transfer-Encoding, Upgrade, TE, Trailer, Proxy-*), Date, Age, Content-Length, Carlos-Purge, Carlos-Purge-Host, Carlos-Cache, and every X-Carlos-*
  • [ ] Step 1: Write the failing tests. TestAssetAdmitTable, one row per rule above, each failing alone with its Reason; rows that store: public, max-age=31536000, immutable CSS; s-maxage beating max-age; max-age over a year capped at 365 days. TestAssetAdmitRefusesALongLivedHTMLFallback (200 text/html, public, max-age=31536000). TestAssetAdmitRefusesNoContentType. TestAssetAdmitRefusesAnUndeclaringBuild. TestExpiryIsFromResponseTime (request 0, response 20, Age: 0, lifetime 30: expires at 30). TestAnAlreadyExpiredResponseIsNotStored (Age larger than lifetime). TestStoredHeaderKeepsCSPAndCORP and TestStoredHeaderDropsHopAndPlatformHeaders; TestTrailerVetoAnnouncedAndUnannounced.
  • [ ] Step 2: Run go test ./internal/edge/ -run 'TestAssetAdmit|TestExpiryIsFromResponseTime|TestAnAlreadyExpired|TestStoredHeader|TestTrailerVeto' -count=1. Expected: FAIL.
  • [ ] Step 3: Implement.
  • [ ] Step 4: Run the same command. Expected: PASS.
  • [ ] Step 5: Commit.

Task 3: Answering from an entry

Files:

  • Create: internal/edge/assetserve.go, internal/edge/assetserve_test.go

Interfaces:

  • Consumes: assetcache.Entry, Meta.Age (plan 2), acceptsGzip, weakenETag (compress.go).
  • Produces: func serveAssetEntry(w http.ResponseWriter, r *http.Request, e *assetcache.Entry, now time.Time, result string) error: copies Meta.Header; sets Age, Carlos-Cache: <result> (hit for a hit, miss for a buffered initiator answered from the entry it just filled, astra P4-3), Vary: Accept-Encoding when the entry has a gzip copy; chooses gzip when the entry has a copy, acceptsGzip(r.Header) and no Range header, then sets Content-Encoding: gzip and weakens the ETag; answers through http.ServeContent over the chosen file (conditional requests, a single Range, If-Range, 416). A request naming more than one range has its Range header removed from the request handed to ServeContent, which would otherwise answer 206 multipart/byteranges (net/http/fs.go:366), so it gets the whole identity body (spec §5.5, astra P7-2); never sets X-Carlos-Version.
  • [ ] Step 1: Write the failing tests (entries committed into a real plan 2 store on t.TempDir()). TestServeAssetEntryHit (body, Carlos-Cache: hit, Age); TestServeAssetEntryConditional (304 for the identity ETag; 304 for the weak gzip ETag with gzip accepted); TestServeAssetEntryRange (206 with the right bytes, identity even when gzip is accepted); TestServeAssetEntryIfRangeWithWeakETagGetsWholeBody; TestServeAssetEntryVaryOnBothRepresentations; TestServeAssetEntryNoTransformHasNoGzip (an entry committed without a copy); TestServeAssetEntryKeepsCSPAndCORP; TestServeAssetEntryHead; TestServeAssetEntryCarriesNoVersion; TestServeAssetEntryReportsTheResultItIsGiven; TestServeAssetEntryMultipleRangesGetTheWholeBody.
  • [ ] Step 2: Run go test ./internal/edge/ -run 'TestServeAssetEntry' -count=1. Expected: FAIL.
  • [ ] Step 3: Implement.
  • [ ] Step 4: Run the same command. Expected: PASS.
  • [ ] Step 5: Commit.

Task 4: Catch decision, keys and coalescing

Files:

  • Create: internal/edge/assetcatch.go, internal/edge/assetcatch_test.go

Interfaces:

  • Consumes: HostAssets, LocalAssets, homeMap.AssetsFor (plan 1), assetcache.Key (plan 2).
  • Produces: - func canonicalAssetPath(u *url.URL) bool (u.RawPath == "" and u.Path equals path.Clean(u.Path) with a trailing slash restored when u.Path has one) - func assetPrefixMatch(paths []string, p string) bool - func assetKey(a HostAssets, host string, u *url.URL) assetcache.Key - type assetFlight struct with func (f *assetFlight) join(id string) (leader bool, wait func(ctx context.Context) (*assetcache.Entry, bool), done func(*assetcache.Entry)): at most 32 waiters per id; a waiter's wait returns false after 10 s, on context end, or when the leader finishes with no entry; the 33rd join returns leader=false and a wait that returns false at once.
  • [ ] Step 1: Write the failing tests. TestCanonicalAssetPathTable (/assets/app.js yes; /assets//app.js, /assets/./app.js, /assets/../x, /assets/%61pp.js no; /assets/dir/ yes); TestAssetPrefixMatch; TestAssetKeyCarriesOwnerAndGen (same path, different gen or app: different ID); TestFlightSharesOnlyAnEntry (leader finishes with nil: every waiter gets false); TestFlightCapsWaiters; TestFlightWaitTimesOut (fake clock or a short test hook for the 10 s constant, named assetWaitTimeout).
  • [ ] Step 2: Run go test ./internal/edge/ -run 'TestCanonicalAssetPath|TestAssetPrefixMatch|TestAssetKey|TestFlight' -count=1. Expected: FAIL.
  • [ ] Step 3: Implement.
  • [ ] Step 4: Run the same command. Expected: PASS.
  • [ ] Step 5: Commit.

Task 5: Local catch and fill, and the home side of a far fill

Files:

  • Modify: internal/edge/assetcatch.go, internal/edge/edge.go (ServeHTTP: the home fill hook between the import gate :1635-1637 and the meter :1665; the local catch between the upgrade branch :1729 and gzip :1742; the meter defer :1668-1686 skips a request marked as an acknowledged home fill), internal/edge/cookiescope.go:76 (strip app-set asset headers)
  • Test: internal/edge/asset_local_test.go

Interfaces:

  • Consumes: Tasks 1 to 4; Edge.assets (Task 7 adds the field; this task adds it as assets *assetState with store *assetcache.Store, local LocalAssets, flight assetFlight, now func() time.Time).
  • Produces: - func (e *Edge) serveAssetLocal(w http.ResponseWriter, r *http.Request, host string, up upstream) bool (true when it answered) - func (e *Edge) serveAssetFillForHop(w http.ResponseWriter, r *http.Request, host string, up upstream) bool (true when it handled a verified, valid fill; it marks the request so the meter defer skips it) - func (e *Edge) fillFromApp(ctx context.Context, w http.ResponseWriter, r *http.Request, up upstream, k assetcache.Key, a HostAssets, onHeader func(v assetVerdict, h http.Header)) (*assetcache.Entry, error): builds the allow-listed request (Host, User-Agent, Accept, Accept-Encoding: identity; method GET even for HEAD), sends it through the activator for a hibernating route or up.proxy otherwise, reads the answering build from the response's X-Carlos-Version and asks e.assets.local.Declared(host, build, r.URL.Path) with the request hostname (astra P1-2), calls assetAdmit, calls onHeader before the first body byte, writes the body to a Fill while the verdict says store while hashing it with SHA-256, abandons the fill at the 60 s deadline, on a short body or on assetTrailerVeto, and on success sets a strong ETag of "<hex sha256>" in the stored header when the app sent none (spec §5.4; the far edge computes the same over the same bytes, so home and far validators match) and commits (astra P1-14).

The initiating request. When the initiator is a GET with no Range, If-None-Match, If-Modified-Since or If-Range, the body is teed to it as it fills. When it is a HEAD, or has any of those headers, it is buffered (astra P3-5): nothing is teed, the initiator waits for the commit and is answered by serveAssetEntry, so its conditional or range is honoured on a cold miss as on a hit (astra P1-10). If the verdict refuses at header time, the full response is streamed to it unmodified (a 200 to a range request is valid HTTP) with Carlos-Cache: bypass; a HEAD gets the headers and no body. If the fill is abandoned after the headers (deadline, short body, trailer veto), what happens depends on whether the initiator's response has started: a streamed initiator's delivery simply continues to the end of the upstream body exactly as an uncached response would, and only the caching is abandoned; nothing is ever replayed onto a started response. A buffered initiator (a HEAD, or one with Range or a conditional, which nothing has been written to yet) has its request forwarded again as today and answered bypass (astra P2-1). Billing follows the delivery (astra P3-4): the discarded attempt is metered by no one (the home did not meter an acknowledged fill, and the far edge meters an acknowledged response only when it is the one delivered), and the re-forwarded request is metered by whoever meters it today, so the client's one delivery is one request with only the delivered bytes.

The upstream must be the policy's. LocalAssets.For reads the registry, while up comes from the edge's route snapshot, which reloads on its own two-second poll (edge.go:2324), so the two can disagree for a moment after an alias changes owner. Before filling, and before acknowledging a far fill, the edge requires up.route.Account, up.route.App and up.route.AssetGen to equal the policy's account, app and gen for the key, and the request's host to resolve to the same primary route in both; any mismatch is an ordinary uncached request (astra P4-2).

On the home, serveAssetFillForHop runs only for a verified hop carrying X-Carlos-Asset-Fill; it checks verifyAssetFill, that e.assets.local.For(host) gives the asserted account, app and gen and catches the path, and the upstream rule above; on any failure it returns false and the request continues as an ordinary hop. It answers a hit from the store with X-Carlos-Asset-Expires set to the entry's expiry, or fills; in both cases it deletes any app-set X-Carlos-Asset-Fill-Ack and X-Carlos-Asset-Expires, sets X-Carlos-Asset-Expires (0 when the verdict does not store) and X-Carlos-Asset-Fill-Ack, and never touches the shared LRU.

  • [ ] Step 1: Write the failing tests (one Edge with a registry route, a fake LocalAssets, a real plan 2 store; "not acceptance"). - TestLocalMissThenHit (miss, then hit, the backend saw one request). - TestLocalFillSendsNoCookieOrAuthorization (asserted at the backend). - TestLocalRefusedResponseServedNotStored (table: Set-Cookie, Vary: Cookie, Vary: Origin, private, no-cache, 206, 404, a long-lived HTML fallback, an oversized body; each answered bypass with the backend's body, nothing stored). - TestLocalUndeclaringBuildIsNotStored (Declared false for the answering X-Carlos-Version). - TestHibernatingRouteHitDoesNotWake (the genActivator fixture from genedge_test.go:16; a hit leaves its wake count at zero). - TestHeadFillsWithGetAndAnswersHead, TestQueryIsPartOfTheKey, TestFillSurvivesTheInitiatorDisconnecting, TestShortBodyIsNotStored. - TestColdMissAnswersARangeFromTheEntry and TestColdMissAnswersIfNoneMatchFromTheEntry (the initiator gets 206 or 304, not the fetched 200); TestColdMissWithMultipleRangesGetsTheWholeBody. - TestFallbackETagIsTheBodyHash (no app ETag: the entry's is "<sha256>", a conditional hit with it is 304). - TestSetCookieTrailerVetoesTheFill (announced and unannounced). - TestStreamedInitiatorIsNotReplayedAfterATrailerVeto and TestStreamedInitiatorIsNotReplayedAfterTheDeadline (the client receives exactly one status line and one body, byte-identical to the upstream's). - TestColdHeadIsBuffered (a cold HEAD: answered from the entry after commit with the fallback ETag and Carlos-Cache: miss; the next request reports hit; on refusal, headers and no body). - TestPolicyAndSnapshotDisagreeingFillsNothing (the registry's alias target changes to another app while Edge.Reload is withheld: the request is forwarded uncached and a far fill gets no acknowledgement). - TestStalledFillIsAbandonedAtTheDeadline (deadline hook assetFillDeadline; reservation released, Stats().Reserved == 0, waiters forwarded). - TestNonCanonicalPathBypasses, TestPostUnderAPrefixPassesThrough. - TestHomeFillAcknowledgesAndSkipsTheMeter (a verified hop with a valid assertion: response has a verifiable ack, the usage meter recorded nothing, the shared LRU has no entry). - TestHomeFillWithStaleGenGetsNoAck and TestHomeFillWithChangedOwnerGetsNoAck (metered as an ordinary hop). - TestHomeFillUncacheableIsAcknowledgedWithZeroExpiry. - TestAppSetAckHeadersAreStripped (the backend sets both headers; the home's own replace them; on a plain request neither reaches the client).
  • [ ] Step 2: Run go test ./internal/edge/ -run 'TestLocal|TestHibernatingRouteHit|TestHeadFills|TestQueryIsPart|TestFillSurvives|TestShortBody|TestStalledFill|TestNonCanonicalPath|TestPostUnder|TestHomeFill|TestAppSetAck|TestColdMiss|TestColdHead|TestFallbackETag|TestSetCookieTrailer|TestStreamedInitiator|TestPolicyAndSnapshot' -count=1. Expected: FAIL.
  • [ ] Step 3: Implement.
  • [ ] Step 4: Run the same command. Expected: PASS.
  • [ ] Step 5: Commit.

Task 6: Far catch and fill

Files:

  • Modify: internal/edge/assetcatch.go, internal/edge/edge.go (call serveAssetFar on both forwarding paths: in the exact-remote precedence branch before its backhaulTo at :1526-1530, where a local wildcard matches but another box owns the exact hostname, and in the lookup-miss branch before e.backhaul at :1561; never for a verified hop or a door caller (doorCallerFrom, :1454-1458), which keep their existing guards and paths (astra P5-1); strip a client-sent X-Carlos-Asset-Fill from every non-hop request near acceptHop at :1503)
  • Test: internal/edge/asset_far_test.go

Interfaces:

  • Consumes: Tasks 1 to 5, homeMap.AssetsFor, homeMap.Home, signingKey, backhaulAcquire, backhaulTransport, signHop, UsageMeter.Record.
  • Produces: func (e *Edge) serveAssetFar(w http.ResponseWriter, r *http.Request, host string) bool: when AssetsFor(host, now) is true, the path is canonical and matches, the method is GET or HEAD, and isUpgradeRequest(r) is false (this hook runs before ServeHTTP's upgrade branch, so it must exclude upgrades itself, astra P1-12): answer a hit from the store (and meter it with Record(registry.Route{Host: host, Account: a.Account, App: a.App}, 1, n, false)); on a miss lead or join a flight; the leader sends the allow-listed request to the home through backhaulAcquire and backhaulTransport, with X-Carlos-Hop, X-Carlos-Hop-Auth and X-Carlos-Asset-Fill; with a valid ack it meters the response once and stores it when X-Carlos-Asset-Expires is non-zero and the plan 2 store admits it, using that expiry unchanged, after assetTrailerVeto on the backhauled trailers and with the same fallback ETag rule and initiating-request rule as Task 5; without a valid ack it streams the response unmodified with Carlos-Cache: bypass, does not meter, does not store. X-Carlos-Asset-Fill-Ack and X-Carlos-Asset-Expires never reach the client. Returns false (so backhaul runs as today) when the host does not catch.
  • [ ] Step 1: Write the failing tests (a far Edge and the fakeHomeEdge from homemap_test.go:709 extended to answer with or without an ack; "not acceptance"). - TestFarMissFillsThenHits and TestFarHitDoesNotDialTheHome (e2eDialCounter). - TestFarAcknowledgedFillIsMeteredOnceAtTheFarEdge, TestFarUnacknowledgedFillIsNotMeteredOrStored. - TestFarExpiryIsTheHomesDeadline (5 s clock offset between the two). - TestFarZeroExpiryIsMeteredNotStored. - TestFarStaleAtForwards (at 61 minutes old: plain backhaul, nothing stored). - TestFarFillCarriesNoCookie (asserted at the fake home). - TestClientSentFillHeaderIsStripped. - TestFarCatchesBehindALocalWildcard (this box serves *.example.com, another box homes shop.example.com exactly: the second asset request is a local hit and dials neither backend). - TestDoorCallerAndVerifiedHopAreNeverCaught. - TestFarUpgradeUnderAPrefixIsBackhauledIntact (a WebSocket GET under /assets/: not caught, upgrade headers reach the fake home). - TestFarColdMissAnswersARangeFromTheEntry and TestFarColdHeadIsBuffered. - TestFarFallbackETagMatchesTheHomes (no app ETag: home and far entries carry the same value). - TestFarCoalescedWaitersGetTheEntry and TestFarRefusedFillReachesOnlyTheInitiator (differing Accept, a Set-Cookie response, an unacknowledged fill; each waiter's request reaches the fake home on its own).
  • [ ] Step 2: Run go test ./internal/edge/ -run 'TestFar|TestClientSentFillHeader|TestDoorCallerAndVerifiedHop' -count=1. Expected: FAIL.
  • [ ] Step 3: Implement.
  • [ ] Step 4: Run the same command. Expected: PASS.
  • [ ] Step 5: Commit.

Task 7: Arming and the sweep

Files:

  • Create: internal/edge/assetarm.go, internal/edge/assetarm_test.go
  • Modify: internal/host/agent.go:926-951 (ready hook), internal/edge/edge.go (the assets field)

Interfaces:

  • Consumes: assetcache.Open, assetcache.DefaultBudget, assetcache.Statfs, Store.Retain (plan 2), plan 1's assetCacheArmed and assetPolicy.
  • Produces: - func (e *Edge) EnableAssetCache(s *assetcache.Store, local LocalAssets) (nil s leaves it off) - func (e *Edge) sweepAssets(now time.Time) int (keeps a key only when local.For or homes.AssetsFor for its host gives the same account, app and gen); run every 5 minutes by a loop started in EnableAssetCache - In internal/host: when assetCacheArmed(os.Getenv), open the store at $CARLOS_DATA_DIR/asset-cache with budget CARLOS_ASSET_CACHE_MAX (bytes, if set and positive) else DefaultBudget(total); an Open or Statfs error logs once and leaves the cache off.
  • [ ] Step 1: Write the failing tests. TestUnarmedEdgeForwardsAsToday (no EnableAssetCache: a declared path is proxied, no Carlos-Cache header); TestSweepDropsAnOldGeneration and TestSweepDropsAChangedOwner; TestServesNothingBeforeTheFirstHomeMapRead (a warm store, a fresh home map: far requests forward). In internal/host: TestAssetCacheArmingReadsTheEnv (t.Setenv, so no t.Parallel), TestAssetCacheBudgetOverride.
  • [ ] Step 2: Run go test ./internal/edge/ -run 'TestUnarmedEdge|TestSweep|TestServesNothingBefore' -count=1 and go test ./internal/host/ -run 'TestAssetCacheArming|TestAssetCacheBudget' -count=1. Expected: FAIL.
  • [ ] Step 3: Implement.
  • [ ] Step 4: Run both commands. Expected: PASS.
  • [ ] Step 5: Commit.

Task 8: Owner and generation changes on a warm cache

Files:

  • Test: internal/edge/asset_owner_test.go

The authority table itself runs in plan 1 Task 8 (internal/host, the real assetPolicy), because a test here importing internal/host would be an import cycle (internal/host/agent.go:184 imports internal/edge) and an external test package would lose this package's private fixtures (astra P2-3). This task covers the edge's half with the edge's own fixtures: what a far and a home edge do when the published owner or generation changes under a warm cache.

  • [ ] Step 1: Write the tests. TestHostMovedToAnotherAccountServesNoOldEntry (a warm far entry; the published owner changes; the next request misses and the sweep deletes the old entry); TestRecreatedHostWithANewGenerationServesNoOldEntry; TestHomeLocalCatchStopsWhenForReturnsFalse (a warm home entry; LocalAssets.For turns false; the request is forwarded, not served from disk).
  • [ ] Step 2: Run go test ./internal/edge/ -run 'TestHostMovedToAnotherAccount|TestRecreatedHost|TestHomeLocalCatchStops' -count=1. Expected: PASS, or a failure that becomes a fix in Tasks 4 to 7, committed with its own test.
  • [ ] Step 3: Commit.

Task 9: Two-edge acceptance

Files:

  • Create: internal/edge/asset_e2e_test.go
  • Modify: internal/edge/backhaul_e2e_test.go (scene routes gain Account, App, AssetGen; publishEdgesFile gains an option to write the golden asset entry)
  • [ ] Step 1: Write the tests on newBackhaulScene with real stores on both edges. Extend newBackhaulEdge to take an activator and, for the generation cases, a route of kind instance with Hibernate set and a unix socket backend, which is what selects generation routing (edge.go:740-748); a TCP service route never does (backhaul_e2e_test.go:267). Use the genActivator fixture (genedge_test.go:16) and startGenApp (genproxy_test.go:131), and have each generation test first assert that the home's lookupUpstreamPath result has genManaged true, so a fixture that silently fell back to plain proxying fails loudly (astra P2-4). Spec §12's attack matrix runs here through production wiring, not only in Tasks 5 and 6's unit tests (astra P1-15): - TestAssetE2EFarMissThenHit: an asset requested through the far edge is miss once and hit after; the home's backend saw exactly one request. - TestAssetE2EGzipFromTheFarEdge: a 5 MiB CSS bundle is served gzip-compressed from the far edge with Vary: Accept-Encoding and a weak ETag. - TestAssetE2EPurgeMakesEveryEdgeMiss: changing the generation in the published entry and converging makes both edges miss once. - TestAssetE2EOldFarEdgeForgedMarkerIsBilledAsAHop: an old far edge is simulated explicitly, because a new edge strips the client's marker whether or not it is armed: the test sends the home a hop it builds itself, with a valid X-Carlos-Hop-Auth from signHop under the scene's mesh key and a forged X-Carlos-Asset-Fill (a well-formed timestamp with a random MAC), which is exactly what an old edge forwarding a client header would send. The home's backend observes that the hop verified and the marker arrived; the home bills it as an ordinary hop (TestHopServedRequestMetersOnceWithHopCounters's counters) and the response carries no acknowledgement (astra P2-5). - TestAssetE2EDisarmedHomeFillsNothing: the home unarmed with a stale published entry still on the far edge: responses arrive bypass, metered once at the home, nothing stored. - TestAssetE2EFillCarriesNoCredentials: a client Cookie and Authorization sent to the far edge never reach the home's backend. - TestAssetE2ERefusedFillsAreStoredNowhere: Set-Cookie, a Set-Cookie trailer, Vary: Cookie, Vary: Origin, Cache-Control: private, no-cache, no-store, no Cache-Control at all, a 206, a long-lived HTML fallback and a 404, each through the far edge: answered bypass, both stores empty, metered once (astra P3-R1-15). - TestAssetE2ELateVetoOnABufferedRangeBillsOnce: a cold range request through the far edge whose acknowledged fill gets a Set-Cookie trailer: the client gets one response from the re-forward, and the two edges' meters together record one request and only the delivered bytes. - TestAssetE2ECoalescedWaitersAndRefusals: concurrent far requests for one key share an admitted entry; with a refused response, each waiter reaches the home on its own. - TestAssetE2EStalledBodyIsAbandoned: the home's backend stalls after headers; the far fill is abandoned at the deadline with both reservations released. - TestAssetE2EUndeclaringBuildIsNotStored: during a generation switch where only the installed build declares the prefix, the serving build's cacheable fallback is stored on neither edge; and the reverse, where only the retiring (still serving) build declares it, a response from the installed build after the switch is stored on neither edge (astra P4-R1-15). - TestAssetE2EWarmSleepRestart: a generation-managed instance warms, sleeps and restarts its edge process; hits continue on both edges with no wake. - TestAssetE2EFarCatchBehindALocalWildcard (the same case through the scene, with e2eDialCounter). - TestAssetE2EOversizedResponseIsStoredNowhere (a 33 MiB body, with and without Content-Length). - TestAssetE2EGenerationOrOwnerMismatchGetsNoAck (the far edge's published view names a generation, then an owner, that the home no longer holds: no acknowledgement, metered once at the home, stored nowhere) (astra P2-R1-15).
  • [ ] Step 2: Run go test ./internal/edge/ -run 'TestAssetE2E' -count=1 with the sandbox off. Expected: PASS.
  • [ ] Step 3: Commit.

Task 10: Docs, census, merge and CI

  • [ ] Step 1: Write EAC-23 and EAC-24 into docs/site/cli.md; add them to assetCacheCopyBuilt; go test . -run 'TestAssetCacheCopy' -count=1 and go test ./internal/docsite/ -count=1 PASS.
  • [ ] Step 2: git fetch origin && git merge origin/main; census test FAILs naming the units and envelopes above; write the overlay and notes file; post on #280; census PASS.
  • [ ] Step 3: Provider boundary (spec §9): TestAssetCodeImportsNoAWSSDK in internal/edge runs go list -deps github.com/carlosframework/platform/internal/assetcache (the full import path, since the test runs in the package directory) and checks no github.com/aws/ package appears, and parses the imports of asset*.go in its own directory (package-local names, excluding _test.go) with go/parser and checks the same (astra P3-2). It does not check internal/edge as a whole, which imports the AWS SDK for its Route 53 adapter (route53dns.go:25) (astra P1-4). Task 9's e2e is the no-AWS-configuration run.
  • [ ] Step 4: Cross-compiles and gofmt -l . clean; weights for any new test over a second; commit, push, CI green on the tip, merge per plan 1's Global Constraints.

Task 11: Acceptance on dev (needs Paul's go-ahead to roll)

Infrastructure changes need Paul's approval. Ask before Step 1.

  • [ ] Step 1: Release with the feature dormant and roll in the standing order: dev boxes, dev console, check, flagship boxes, flagship console.
  • [ ] Step 2: On dev, set CARLOS_ASSET_CACHE=1 on a test app's home box, then on one box in another region. Deploy the test app with carlos deploy --assets /assets/ and an immutable bundle.
  • [ ] Step 3: From the other region, request the bundle twice: Carlos-Cache: miss then hit; the home box's access log shows one request; a 5 MiB bundle arrives gzip-compressed. Record the box's asset-cache disk use.
  • [ ] Step 4: Report to Paul. Arm the rest of dev on his say-so; flagship follows spec §10's order (console box, then one small box, then the rest one at a time).