Edge asset cache, plan 2: the disk store Implementation Plan

> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: A self-contained package, internal/assetcache, that stores cached asset responses on a box's disk under a budget, with worst-case reservations, a free-space floor, per-box and per-app fill bounds, fair-share eviction, expiry and a sweep, and that nothing calls yet (plan 3 does).

Architecture: One Store per process over one directory. Each entry is a <id>.meta file (JSON) naming its versioned body files, <id>.<v>.body (identity) and optional <id>.<v>.gz, where <id> is the hex sha256 of the key and <v> a random version drawn per commit. A commit writes the new versioned bodies, then renames a temporary over <id>.meta, then deletes the previous version's bodies, so a crash at any point leaves either the old entry or the new one, never new bytes under old metadata (astra P1-9). An in-memory index is rebuilt from the .meta files at Open. A fill is a Fill holding a reservation; it writes temporaries in the same directory and renames them on Commit. Free space comes from an FS interface whose Linux implementation is statfs. The package knows nothing about HTTP semantics beyond storing headers it is given: admission, freshness rules and serving belong to plan 3. The design is spec §7, written so the media roadmap's stage 1 can reuse it with its own directory and budget (spec §11).

Tech Stack: Go 1.26 standard library (os, compress/gzip, crypto/sha256, encoding/json), golang.org/x/sys/unix for Statfs on Linux.

Spec: docs/superpowers/specs/2026-10-24-edge-asset-cache-design.md (rev 8). Read §5.3 ("Coalescing", "Every fill has a deadline"), §5.4 and §7 before starting.

Branch: edge-asset-cache-2, cut from amadan main after the spec branch has merged. Independent of plan 1.

Global Constraints

Every task's requirements include this section.

• Real files. Every test runs on a real directory from t.TempDir(). A fake FS (free-space source) is allowed and is the only stand-in; tests using it say "not acceptance" in the doc comment. Task 2's TestStatfsReportsTheTempDir is the acceptance test for the real FS and runs on Linux only.
• Targeted local runs; CI runs the full suite. go test ./internal/assetcache/ -count=1 is allowed in full (new, small package). go build ./..., go vet ./..., gofmt -l . run locally.
• Cross-compile gate. internal/edge is linked into the CLI, and plan 3 imports this package from it. statfs_linux.go carries //go:build linux; statfs_other.go carries //go:build !linux and returns ErrUnsupported. Before every push: GOOS=darwin GOARCH=arm64 go build ./... and GOOS=windows GOARCH=amd64 go build ./....
• No user-facing strings. Every error in this package is an internal fragment, so no copy rows.
• No tenant names. Keys in tests use account abc, apps shop and blog, host shop.example.com.
• Go cache. Leave GOCACHE, GOMODCACHE and GOTMPDIR at their defaults.
• Census overlay per branch, loaded LAST. plan14-root-reads-edge-asset-cache-2.tsv, rev3=PROPOSED-edge-asset-cache-2, after whatever loads last at the merge (plan 1's overlay if it has merged), previous seal checked first.
• Merge main first; amadan tasks; scratch under t.TempDir(), removed by name. As in plan 1.
• Spec numbers, verbatim. Default budget: the smaller of 4 GiB and 10% of the filesystem's size (CARLOS_ASSET_CACHE_MAX overrides, read by plan 3). Free-space floor: no fill admitted unless free space minus outstanding reservations minus this one stays at or above 15% of the filesystem's size. Reservation: Content-Length if known, else 32 MiB, doubled for the gzip copy, plus 64 KiB of metadata. Body cap: 32 MiB. Fill bounds: 16 per box, 4 per app. Fair share: budget divided by the number of apps with entries, never more than 25% of the budget for one app.

Review Focus

1. A crash between writing .body and .meta. A person expects a restart to leave no half entry and no leaked disk. Test: Task 1, TestOpenRemovesOrphanBodiesAndTemporaries.
2. A .meta file from a newer binary with fields this one does not know. A person expects it to load (unknown JSON keys ignored), not to be deleted. Test: Task 1, TestOpenKeepsMetaWithUnknownFields; a .meta that does not parse is deleted with its files.
3. A filesystem that fills from outside the cache (logs, a database) while fills are in flight. A person expects in-flight writes to stop at their reservation and new fills to be refused, never the cache pushing past the floor. Test: Task 3, TestReserveRefusesWhenSomethingElseFillsTheDisk.
4. One app over its share when a second app arrives. A person expects the second app's fill to evict the first app's oldest entries, not its own and not to be refused. Test: Task 5, TestNewAppEvictsTheAppOverItsShare.
5. Clock moving backwards. An entry stored "in the future" after a clock step. A person expects it to stay valid until its stored expiry and never to report a negative age. Test: Task 4, TestAgeNeverNegative.

Plan review

Astra reviewed the three plans together over eight rounds (docs/superpowers/specs/CODEX-FINDINGS-ASSET-CACHE-PLANS.md, then -PLANS-R2.md to -PLANS-R8.md; brief CODEX-REVIEW-BRIEF-ASSET-CACHE-PLANS.md) and accepted them at 4cca5784. Findings were 16, 6, 6, 3, 1, 1, 2, then none. Every finding was accepted after its citation was checked; none was declined. Findings that changed a task are marked (astra Pn-m) where they landed.

Census impact

• New package internal/assetcache: its file-reading declarations (Open's scan, Entry.Open) are new plain rows of class legacy, task accepted-legacy, basis accepted-legacy;rev3=PROPOSED-edge-asset-cache-2;asset cache files under $CARLOS_DATA_DIR/asset-cache, written and read only by this package, the form the loader requires (plan14_root_reads_test.go:478-485); there is no cache class (astra P1-3). Its envelope is pinned.
• Notes: docs/superpowers/notes/census-proposal-edge-asset-cache-2.md, ruling asked on #280.

File structure

| File | Responsibility | |---|---| | internal/assetcache/key.go | Key, its id. | | internal/assetcache/meta.go | Meta, its file encoding. | | internal/assetcache/store.go | Config, Open, the index, Get, Retain, Stats. | | internal/assetcache/fill.go | Reserve, Fill, Commit, Abort. | | internal/assetcache/evict.go | Fair-share eviction. | | internal/assetcache/fs.go, statfs_linux.go, statfs_other.go | Free space. | | internal/assetcache/*_test.go | Tests. |

---

Task 1: Keys, metadata and Open

Files:

• Create: internal/assetcache/key.go, meta.go, store.go, fs.go (the interface only), store_test.go

Interfaces:

• Produces: - type Key struct { Account, App, Gen, Host, Path, RawQuery string }; func (k Key) ID() string (hex sha256 over the six fields, each length-prefixed so no two keys collide); func (k Key) AppScope() string (Account + "/" + App) - type Meta struct { Key Key; Version string; Header http.Header; Lifetime time.Duration; Expires time.Time; StoredAt time.Time; Size int64; GzipSize int64 } (GzipSize == 0 means no gzip copy; Version names the body files) - type FS interface { Space(dir string) (total, free uint64, err error) } in fs.go (declared here so this task builds; Task 2 adds the implementations) (astra P1-1) - type Config struct { Dir string; Budget int64; FS FS; Now func() time.Time } - func Open(cfg Config) (*Store, error): creates Dir (0700), deletes every *.tmp-* file, deletes every .body/.gz file no parsed .meta names (orphans and superseded versions), deletes a .meta that does not parse together with its files, and indexes the rest. - type Stats struct { Entries int; Bytes int64; Reserved int64; Apps int }; func (s *Store) Stats() Stats

• [ ] Step 1: Write the failing tests. TestKeyIDIsStableAndDistinct (same key same id; moving one character between Path and RawQuery changes it); TestOpenCreatesTheDirectory; TestOpenRemovesOrphanBodiesAndTemporaries; TestOpenKeepsMetaWithUnknownFields; TestOpenDropsUnparseableMeta; TestOpenIndexesExistingEntries (write three entries' files by hand in the documented layout, Open, Stats counts them and their bytes); TestOpenDeletesSupersededVersions (a .meta naming version b beside bodies for a and b: a is removed).
• [ ] Step 2: Run go test ./internal/assetcache/ -count=1. Expected: FAIL (package missing).
• [ ] Step 3: Implement.
• [ ] Step 4: Run the same command. Expected: PASS.
• [ ] Step 5: Commit.

Task 2: Free space

Files:

• Create: internal/assetcache/statfs_linux.go, statfs_other.go, fs_test.go, fs_linux_test.go

Interfaces:

• Consumes: FS (Task 1).
• Produces: - var Statfs FS (Linux: unix.Statfs, free from Bavail * Bsize; other systems: returns ErrUnsupported) - var ErrUnsupported = errors.New("assetcache: free space unknown on this system") - func DefaultBudget(total uint64) int64 (the smaller of 4 GiB and total/10)

• [ ] Step 1: Write the failing tests. TestDefaultBudget (20 GiB gives 2 GiB, 60 GiB gives 4 GiB, 0 gives 0). TestStatfsReportsTheTempDir (Linux only; total > 0, free <= total).
• [ ] Step 2: Run go test ./internal/assetcache/ -run 'TestDefaultBudget|TestStatfs' -count=1. Expected: FAIL.
• [ ] Step 3: Implement.
• [ ] Step 4: Run the same command, then both cross-compiles. Expected: PASS, clean builds.
• [ ] Step 5: Commit.

Task 3: Reservations, the floor and fill bounds

Files:

• Create: internal/assetcache/fill.go, fill_test.go

Interfaces:

• Consumes: FS, Store (Tasks 1 and 2).
• Produces: - const MaxBody = 32 << 20, const metaReserve = 64 << 10, const maxFillsPerBox = 16, const maxFillsPerApp = 4 - var ErrNoRoom, ErrBelowFloor, ErrBusy, ErrTooLarge (sentinels) - func (s *Store) Reserve(k Key, contentLength int64) (*Fill, error): contentLength < 0 means unknown; reserves 2*n + metaReserve where n is contentLength or MaxBody; ErrTooLarge when contentLength > MaxBody; ErrBusy over either fill bound; evicts (Task 5's makeRoom, a no-op until then) then ErrNoRoom if used + reserved + r > Budget; ErrBelowFloor if free - reserved - r < total*15/100; an FS error refuses with that error. - type Fill struct implementing io.Writer for the identity body; a write past the reservation's identity half or past MaxBody returns ErrTooLarge and marks the fill failed. - func (f *Fill) Abort(): deletes temporaries, releases the reservation and the fill slot; safe to call twice and after Commit.

• [ ] Step 1: Write the failing tests (a fake FS with settable total and free; "not acceptance"). - TestReserveWorstCaseWithUnknownLength (Stats().Reserved == 2*MaxBody + 64<<10), TestReserveUsesContentLength. - TestReserveRefusesBelowTheFloor: total 8 GiB, free 15% plus one reservation fits once, the second concurrent reservation is ErrBelowFloor. This is spec §7's sixteen-fill case reduced to its rule. - TestReserveRefusesWhenSomethingElseFillsTheDisk (free drops between two calls). - TestReserveBoundsFillsPerBoxAndPerApp (17th fill and 5th per app are ErrBusy). - TestWritePastTheReservationFails, TestAbortReleasesEverything (reserved bytes and slots back to zero, no files left), TestAbortTwiceIsSafe.
• [ ] Step 2: Run go test ./internal/assetcache/ -run 'TestReserve|TestWritePast|TestAbort' -count=1. Expected: FAIL.
• [ ] Step 3: Implement. The index lock is held only to update counters, never across file I/O.
• [ ] Step 4: Run the same command. Expected: PASS.
• [ ] Step 5: Commit.

Task 4: Commit, lookup and expiry

Files:

• Modify: internal/assetcache/fill.go, store.go
• Test: internal/assetcache/entry_test.go

Interfaces:

• Produces: - const MaxMeta = metaReserve (64 KiB) - func (f *Fill) Commit(m Meta, gzipCopy bool) error, in this order: (1) gzip the identity body into a temporary when gzipCopy, abandoning the copy (GzipSize = 0) if it would exceed the identity size or the reservation's gzip half; (2) draw a fresh Version; (3) set every meta field, Version, Size, GzipSize and StoredAt = Now(); (4) encode the final meta and return ErrTooLarge (behaving as Abort) when it exceeds MaxMeta, so no write passes the reservation (astra P1-8, P3-1); (5) fsync and rename the bodies to their versioned names; (6) write and fsync the meta temporary and rename it over <id>.meta; (7) delete the previous version's bodies (subject to leases); then count the meta's bytes in the store's used bytes and release the reservation. Any error behaves as Abort. A Commit for a key that already has an entry replaces it. A reader holding the old files keeps reading them (unlinked files stay readable on Linux). - type Entry struct { Meta Meta }; func (e *Entry) Open(gzip bool) (*Reader, error); type Reader struct wrapping the *os.File (Read, Seek, ReadAt, Stat, so http.ServeContent takes it) whose Close releases a lease on the body version (astra P2-2). A version that is replaced, evicted, expired or swept while leased is unlinked at once but its bytes stay counted in the store's used bytes until its last lease closes, because an unlinked open file still holds its disk space. Stats gains Retired int64 (bytes held only by open readers). - func (s *Store) Get(k Key) (*Entry, bool): false and the entry deleted when Now() is at or past Meta.Expires; marks the entry recently used. - func (m Meta) Age(now time.Time) time.Duration (Lifetime - Expires.Sub(now), never below zero)

• [ ] Step 1: Write the failing tests. TestCommitThenGetRoundTrips (header and body bytes equal); TestCommitThenReopenThenRead (commit with a gzip copy, close, Open a new Store on the same directory, read both identity and gzip bytes back: the persisted meta names the version it was committed with); TestCommitWritesMetaLast (a crash simulated by stopping before the meta rename leaves Open with no entry); TestReplacementCrashKeepsTheOldEntry (an existing entry, a replacement stopped after its bodies are renamed but before the meta: Open serves the old bytes with the old headers and removes the new bodies); TestOversizedMetaIsRefused (a 70 KiB header value and a 70 KiB query each give ErrTooLarge, nothing left on disk, reservation released); TestEvictedBodyStaysCountedWhileAReaderHoldsIt and TestReplacedBodyStaysCountedWhileAReaderHoldsIt (Stats().Retired is the old body's size until the reader closes, and a fill that would only fit by counting those bytes as free is refused); TestGzipCopyIsAbandonedWhenLarger (random bytes); TestGzipCopyIsKept (CSS text; reading .gz through gzip.NewReader gives the identity bytes); TestGetExpires (fake Now past Expires: false, files gone); TestCommitReplacesAnEntry; TestAgeNeverNegative.
• [ ] Step 2: Run go test ./internal/assetcache/ -run 'TestCommit|TestCommitThenReopen|TestReplacementCrash|TestOversizedMeta|TestEvictedBody|TestReplacedBody|TestGzipCopy|TestGetExpires|TestAge' -count=1. Expected: FAIL.
• [ ] Step 3: Implement.
• [ ] Step 4: Run the same command. Expected: PASS.
• [ ] Step 5: Commit.

Task 5: Fair-share eviction and the sweep

Files:

• Create: internal/assetcache/evict.go, evict_test.go
• Modify: fill.go (Reserve calls makeRoom), store.go

Interfaces:

• Produces: - func (s *Store) makeRoom(need int64) (internal): while used + reserved + need > Budget, evict the least recently used entry of the app with the largest bytes - share, share = min(Budget/apps, Budget/4); stops when nothing is left to evict. - func (s *Store) Retain(keep func(Key) bool) int: deletes every entry whose key keep rejects, returns how many. Plan 3 calls it with "this key's generation and owner are the host's current ones".

• [ ] Step 1: Write the failing tests. TestEvictionTakesFromTheAppFurthestOverItsShare (app shop holds 80% of the budget, blog 10%; a blog fill evicts shop's oldest); TestOneAppWalkingUniqueQueriesEvictsItself (only shop has entries, it fills past the budget, blog's single entry survives); TestNewAppEvictsTheAppOverItsShare; TestShareNeverExceedsAQuarter (one app alone is still evicted first once it passes 25%); TestRetainDeletesRejectedKeys.
• [ ] Step 2: Run go test ./internal/assetcache/ -run 'TestEviction|TestOneApp|TestNewApp|TestShare|TestRetain' -count=1. Expected: FAIL.
• [ ] Step 3: Implement; per-app LRU lists inside the index.
• [ ] Step 4: Run go test ./internal/assetcache/ -count=1 -race. Expected: PASS.
• [ ] Step 5: Commit.

Task 6: Census, merge and CI

• [ ] Step 1: git fetch origin && git merge origin/main; re-read the last overlay.
• [ ] Step 2: Run go test ./internal/provision/ -run TestPlan14RootReadCensus -count=1. Expected: FAIL naming the new package's units and envelope.
• [ ] Step 3: Write plan14-root-reads-edge-asset-cache-2.tsv and the notes file; post on #280.
• [ ] Step 4: Census test PASS; cross-compiles and gofmt -l . clean.
• [ ] Step 5: Commit, push, CI green on the tip, merge per plan 1's Global Constraints with this branch's name.