S3 Change Index Implementation Plan

> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Stop every box and the console from re-walking the whole deployment bucket: writers bump a per-scope generation in control/changes.json, readers re-read only what moved, a 30-minute backstop walk bounds staleness.

Architecture: The bump lives inside the two concrete stores (S3Store, DirStore), enabled by deployment openers. A process-wide release.ChangeWatch polls the index with a conditional GET. provision.RecordView keeps the record families the walk reads and exposes immutable snapshots as a read-through release.Store, so Reconcile, ReconcileSidecars, the moves pass, steering and the console sweeps run unchanged over a snapshot. A release.ReadGate gates the per-app GET loops on generations plus a maximum age.

Tech Stack: Go, AWS SDK v2 S3 (existing client in internal/release), DirStore for tests, MinIO in the delivered-contract test.

Spec: docs/superpowers/specs/2026-10-21-s3-change-index-design.md (rev 3, Astra-accepted). Section numbers below (§N) refer to it.

Global Constraints

• Index key control/changes.json, body {"schema":1,"entries":{"<scope>":<uint64>}}; generations only increase; entries never removed in v1 (§3.1).
• Scopes exactly per §3.2: apps/<a>/<b> (except data prefixes logs, errors, analytics, runs, metrics), accounts/<a>, control/certs, control/edges; nothing else.
• Bump after a successful mutation only; at most 4 attempts with jitter inside a 2s budget detached from the caller's cancellation; a failed bump is logged and never fails the write (§3.3).
• Backstop interval 30 minutes, jittered ±10% (R3). Gated reads have a maximum successful-read age equal to the backstop (§5).
• Readers armed only by CARLOS_CHANGE_INDEX=read (box host.env, console environment). Unset means exactly today's behavior. Writers always bump once shipped (§7).
• A store without ConditionalGetter + ETagLister, a denied index read, or a malformed index means "unknown": walk as today, never "empty and unchanged" (§3.4).
• Core only: no AWS SDK outside internal/release's existing S3 client (§6).
• Lock order: w.pass → reconcileMu → RecordView.mu; RecordView.mu is a leaf (§4.3).
• Tests: run only touched packages locally with -run; CI runs the full suite. Leave GOCACHE at its default. The simulation runs niced, one 3000-seed sweep per revision.
• No tenant names in code, tests or comments (use shop, example.com).
• Every new top-level test stands alone: make test is about to split internal/console and the root package across separate go test processes, so no test may depend on another having run first. Root-package e2e tests call buildCarlos(t) and never write to the shared read-only binary it returns.
• internal/host tests call t.Parallel() unless they set env (t.Setenv), cwd, umask, log output or package vars, or reach edge.SecureUnixListen. Tasks 8 and 9 set the package vars armedChangeWatch/armedRecordView (and CARLOS_CHANGE_INDEX), so those tests stay serial; tests that only build their own fastDeps, genCache or gates call t.Parallel().
• Any user-facing text (release notes for R4) goes through the copy-review skill before it is written.

Review Focus

1. A deployment with no control/changes.json yet (first arm, fresh bucket): the first Refresh must rebuild by walking, and the view must never publish as empty-and-complete. Test in Task 6.
2. A malformed or future-schema index (hand edit, newer writer): readers treat it as unusable and walk; writers do not overwrite an unknown schema. Tests in Tasks 1 and 4.
3. A mixed fleet: an armed reader while an old binary writes a record without bumping. The record appears after the next Rebuild, and no route is retired in between. Tests in Tasks 6 and 7.
4. An app purge deleting hundreds of keys under one scope: bumps stay bounded (one per mutation, each a single CAS), the view drops the app once, nothing is retired before the purge's own records are gone. Test in Task 2 (bump count) and Task 6 (view).
5. Index read denied by IAM (role lacks GetObject on the key): ChangeWatch.Usable() stays false, ReadGate.Due is always true, every loop behaves as today. Tests in Tasks 4 and 9.

---

Task 1: Change scope and the index document

Files:

• Create: internal/release/changeindex.go
• Test: internal/release/changeindex_test.go

Interfaces:

• Produces: - const ChangeIndexKey = "control/changes.json" - type ChangeIndex struct { Schema int \json:"schema"\; Entries map[string]uint64 \json:"entries"\ } - func ChangeScope(key string) (scope string, ok bool) - func ReadChangeIndex(ctx context.Context, s Store) (ChangeIndex, string, error) — absent object returns ChangeIndex{Schema: 1, Entries: {}}, etag "", nil; schema ≠ 1 returns ErrChangeIndexSchema. - var ErrChangeIndexSchema = errors.New("release: change index schema unsupported") - func bumpChangeIndex(ctx context.Context, get func(context.Context, string) ([]byte, string, error), put func(context.Context, string, []byte, string) (string, error), scopes ...string) error — unexported; takes raw get/put so the store hook (Task 2) can call it without recursion.

• [ ] Step 1: Write the failing tests

func TestChangeScope(t *testing.T) {
	cases := map[string]string{
		"apps/bab/shop/instances/a.example.com.json": "apps/bab/shop",
		"apps/bab/shop/config-gen.json":              "apps/bab/shop",
		"apps/bab/shop/channels/canary/x":            "apps/bab/shop",
		"apps/bab/shop/sidecars.json":                "apps/bab/shop",
		"accounts/bab/account.json":                  "accounts/bab",
		"accounts/bab/telemetry.json":                "accounts/bab",
		"control/certs/example.com":                  "control/certs",
		"control/edges/box-a.json":                   "control/edges",
	}
	for k, want := range cases {
		got, ok := ChangeScope(k)
		if !ok || got != want { t.Errorf("ChangeScope(%q) = %q,%v want %q", k, got, ok, want) }
	}
	for _, k := range []string{
		"apps/bab/shop/logs/box/2026-10-21-10/x", "apps/bab/shop/errors/x",
		"apps/bab/shop/analytics/x", "apps/bab/shop/runs/r1/request.json",
		"apps/bab/shop/metrics/x", "apps/bab/shop", "apps/bab/", "apps/",
		ChangeIndexKey, "control/serial", "control/edge-mesh-key",
		"control/edge-mesh-key.next", "releases/x", "status/boxes/a.json",
		"fleets/f/apps/bab/shop/config.json",
	} {
		if s, ok := ChangeScope(k); ok { t.Errorf("ChangeScope(%q) = %q, want none", k, s) }
	}
}

func TestBumpChangeIndexCreatesThenIncrements(t *testing.T)   // DirStore: two bumps of "apps/bab/shop" -> 2; other entry untouched
func TestBumpChangeIndexRetriesOnConflict(t *testing.T)        // put fails with ErrCASConflict twice, then succeeds -> nil, final gen 1
func TestBumpChangeIndexGivesUpAfterFourAttempts(t *testing.T) // 4 conflicts -> non-nil error, exactly 4 puts
func TestBumpChangeIndexRefusesUnknownSchema(t *testing.T)     // existing {"schema":2,...} -> ErrChangeIndexSchema, no put
func TestReadChangeIndexAbsentIsEmpty(t *testing.T)            // Entries non-nil, len 0, etag ""

• [ ] Step 2: Run to verify they fail

Run: go test ./internal/release/ -run 'TestChangeScope|TestBumpChangeIndex|TestReadChangeIndex' -count=1 Expected: FAIL (undefined: ChangeScope)

• [ ] Step 3: Implement ChangeScope, ReadChangeIndex, bumpChangeIndex in internal/release/changeindex.go. ChangeScope requires apps/<a>/<b>/<rest> with all three non-empty segments and <rest>'s first segment not in the data set; accounts/<a>/<rest> with both non-empty. bumpChangeIndex applies all scopes in one read-modify-write; conflict is errors.Is(err, ErrCASConflict); jitter 20–120ms between attempts; it does not create its own context (Task 2 passes the detached one).

• [ ] Step 4: Run to verify they pass (same command). Expected: PASS

• [ ] Step 5: Commit git commit -m "release: change scopes and the change index document (#336)"

---

Task 2: Bump inside the concrete stores

Files:

• Modify: internal/release/store.go (S3Store struct :681, DirStore struct :342, Put :414/:787, PutFrom :456/:820, Delete :660/:970)
• Modify: internal/release/conditional_delete.go (DeleteIfMatch :63/:86)
• Test: internal/release/changeindex_store_test.go

Interfaces:

• Consumes: Task 1's ChangeScope, bumpChangeIndex.
• Produces: - func (s *S3Store) EnableChangeIndex(), func (d *DirStore) EnableChangeIndex() - func NewDeploymentS3Store(ctx context.Context, bucket string) (*S3Store, error) and func NewDeploymentS3StoreIn(ctx context.Context, bucket, region string) (*S3Store, error) — the S3 constructors plus EnableChangeIndex(). - var changeIndexBumpBudget = 2 * time.Second (var for tests).

• [ ] Step 1: Write the failing tests (all on DirStore with the index enabled)

func TestStoreBumpsAfterEveryMutation(t *testing.T) {
	// Put, PutFrom, Delete, DeleteIfMatch on "apps/bab/shop/instances/h.json"
	// each raise entries["apps/bab/shop"] by exactly 1.
}
func TestStoreDoesNotBumpFailedMutation(t *testing.T)   // Put with stale ifMatch -> ErrCASConflict, gen unchanged
func TestStoreDoesNotBumpUnscopedKeys(t *testing.T)     // logs/, control/serial, the index key itself -> no index object
func TestStoreBumpFailureDoesNotFailWrite(t *testing.T) // index object is a directory (unwritable) -> Put returns nil, record written
func TestStoreBumpDetachedFromCallerCancel(t *testing.T)// ctx cancelled right after Put returns from the backend -> bump still lands
func TestStoreWithoutEnableNeverBumps(t *testing.T)
func TestStoreCapabilitiesPreserved(t *testing.T) {
	d := NewDirStore(t.TempDir()); d.EnableChangeIndex()
	var s Store = d
	_, ok1 := s.(ConditionalDeleter); _, ok2 := s.(PutStreamer); _, ok3 := s.(Opener)
	_, ok4 := s.(SizeLister); _, ok5 := s.(PageLister); _, ok6 := s.(ModLister)
	if !(ok1 && ok2 && ok3 && ok4 && ok5 && ok6) { t.Fatal("index-enabled DirStore lost an extension") }
}
func TestPurgeBumpsBounded(t *testing.T) // 300 deletes under one app -> gen 300, each bump one CAS (count puts on a counting wrapper of the raw backend)

• [ ] Step 2: Run to verify they fail

Run: go test ./internal/release/ -run 'TestStore(Bumps|DoesNot|BumpFailure|BumpDetached|WithoutEnable|Capabilities)|TestPurgeBumps' -count=1 Expected: FAIL (undefined: EnableChangeIndex)

• [ ] Step 3: Implement. Add an unexported changeIndex bool field to both structs. Split each mutation into the existing body (renamed putRaw, putFromRaw, deleteRaw, deleteIfMatchRaw) and a public method that calls the raw one and, on success and when enabled and ChangeScope matches, calls bumpChangeIndex with getRaw/putRaw under context.WithTimeout(context.WithoutCancel(ctx), changeIndexBumpBudget), logging a failure as release: change index bump %s: %v. The struct literal &release.S3Store{Client:..., Bucket:...} elsewhere must still compile (exported fields unchanged).

• [ ] Step 4: Run to verify they pass (same command, plus go test ./internal/release/ -count=1 for regressions). Expected: PASS

• [ ] Step 5: Commit git commit -m "release: concrete stores bump the change index after each mutation (#336)"

---

Task 3: Read extensions and their forwarding

Files:

• Modify: internal/release/store.go (new interfaces beside ModLister :186-207; methods on both stores; prefixStore forwarding :113-217)
• Modify: internal/hostops/mutation_boundary.go (forwarding surface :108-130)
• Modify: the MinIO delivered-contract test (grep -rn TestDeliveredContractMinio)
• Test: internal/release/readext_test.go, internal/hostops/mutation_boundary_test.go

Interfaces:

• Produces: - type ConditionalGetter interface { GetIfChanged(ctx context.Context, key, etag string) (body []byte, newETag string, changed bool, err error) } — etag == "" behaves as Get; unchanged returns nil, etag, false, nil; absent returns an os.ErrNotExist-wrapped error. - type KeyETag struct { Key, ETag string } - type ETagLister interface { ListETags(ctx context.Context, prefix string) ([]KeyETag, error) } — sorted by key; ETags in the same form Get returns. - Both on *S3Store and *DirStore; prefixStore and mutationBoundary forward both (boundary returns ErrContract when the raw store lacks one, matching ListMod).

• [ ] Step 1: Write the failing tests

func TestGetIfChangedDirStore(t *testing.T) // put k; GetIfChanged(k,"") -> changed, etag e; GetIfChanged(k,e) -> !changed, body nil; put again; GetIfChanged(k,e) -> changed
func TestGetIfChangedAbsent(t *testing.T)   // errors.Is(err, os.ErrNotExist)
func TestListETagsMatchesGet(t *testing.T)  // for each listed key, ETag == etag from Get
func TestPrefixStoreForwardsReadExtensions(t *testing.T) // NewPrefixStore(dir,"fleets/f/") answers both, keys stripped
func TestBoundaryForwardsReadExtensions(t *testing.T)     // in hostops

In the MinIO contract test add: GetIfChanged returns changed=false for the current ETag (S3 304), and ListETags ETags equal Get ETags.

• [ ] Step 2: Run to verify they fail

Run: go test ./internal/release/ ./internal/hostops/ -run 'GetIfChanged|ListETags|ForwardsReadExtensions' -count=1 Expected: FAIL

• [ ] Step 3: Implement. S3: GetObject with IfNoneMatch: etag; a 304 surfaces from the SDK as a smithy API error with HTTP status 304 — map it to changed=false. ListETags pages ListObjectsV2 and keeps Key/ETag exactly as Get reports them (same quoting). DirStore: compute the ETag the way its Get does.

• [ ] Step 4: Run to verify they pass (same command). Expected: PASS. MinIO contract runs in CI.

• [ ] Step 5: Commit git commit -m "release: conditional GET and ETag listings, forwarded through prefix stores and the hostops boundary (#336)"

---

Task 4: ChangeWatch and ReadGate

Files:

• Create: internal/release/changewatch.go
• Test: internal/release/changewatch_test.go

Interfaces:

• Consumes: Tasks 1 and 3.
• Produces: - const ChangeIndexBackstop = 30 * time.Minute (R3; here so provision, host, edge and console share it without import cycles) - type ChangeWatch struct (safe for concurrent use) - func NewChangeWatch(s Store) *ChangeWatch - func (w *ChangeWatch) Poll(ctx context.Context) (moved []string, err error) — conditional GET of ChangeIndexKey; returns scopes whose generation rose (or appeared) since the last successful poll; on 304 returns nil, nil; any error, an unknown schema, or a store lacking ConditionalGetter marks the watch unusable and returns the error. - func (w *ChangeWatch) Gen(scope string) (gen uint64, ok bool) — ok false while unusable. - func (w *ChangeWatch) Observed() (map[string]uint64, bool) — copy of the last good entries. - func (w *ChangeWatch) Usable() bool - type ReadGate struct; func NewReadGate(w *ChangeWatch, maxAge time.Duration) *ReadGate - func (g *ReadGate) Due(scope string, now time.Time) (gen uint64, due bool) — due when the watch is nil or unusable, the scope was never marked, its last mark failed, its generation moved past the marked one, or now is at least maxAge past the last successful mark. - func (g *ReadGate) Mark(scope string, gen uint64, now time.Time, ok bool) — records the generation captured by Due before the read.

• [ ] Step 1: Write the failing tests

func TestChangeWatchFirstPollReportsAll(t *testing.T)      // index with 2 entries -> moved has both
func TestChangeWatchUnchangedIs304(t *testing.T)           // second poll, no writes -> moved nil, err nil; counting store shows GetIfChanged with etag
func TestChangeWatchReportsOnlyMoved(t *testing.T)
func TestChangeWatchAbsentIndexIsUsableEmpty(t *testing.T) // absent -> Usable true, Observed empty
func TestChangeWatchDeniedIsUnusable(t *testing.T)         // store whose Get of the key fails with access denied -> Usable false
func TestChangeWatchUnknownSchemaIsUnusable(t *testing.T)
func TestChangeWatchWithoutExtensionIsUnusable(t *testing.T)
func TestReadGateDue(t *testing.T) // table: unusable->due; never marked->due; marked ok same gen within age->!due; gen moved->due; age>=maxAge->due; last mark failed->due

• [ ] Step 2: Run to verify they fail

Run: go test ./internal/release/ -run 'TestChangeWatch|TestReadGate' -count=1 Expected: FAIL

• [ ] Step 3: Implement in changewatch.go. Absent index is usable with empty entries (Review Focus 1 is handled by RecordView, not here).

• [ ] Step 4: Run to verify they pass. Expected: PASS

• [ ] Step 5: Commit git commit -m "release: ChangeWatch polls the change index; ReadGate gates reads on generation and age (#336)"

---

Task 5: Every deployment opener enables the index

Files:

• Modify: deployment-bucket openers among the release.NewS3Store/NewS3StoreIn call sites: cli.go:1957,2067,3591, cmd/console/main.go:633 (the deployment bucket; :333 is the database bucket), internal/host/agent.go:4368,4462, internal/edge/edge.go:3407, internal/host/relocatepass.go:251,680, internal/host/activation_command_linux.go:67,92,206,283, internal/ops/ops.go:339, internal/vet/vet.go:226, internal/host/seedwiring.go:102. Classify each by reading it; switch deployment ones to NewDeploymentS3Store[In].
• Modify: internal/host/boxpublish.go only if its test shows the concrete assertion :259 failing (it should not: the type is still *release.S3Store).
• Create: internal/release/openers_test.go

Interfaces:

• Consumes: Task 2's NewDeploymentS3Store[In].

• [ ] Step 1: Write the failing test TestS3StoreOpenersClassified: parse every non-test .go file in the module with go/parser, find calls to release.NewS3Store, release.NewS3StoreIn (and the package-internal NewS3Store outside store.go), and require each file:func to appear in a table otherBucketOpeners map[string]string (value: the reason, for example "database bucket", "hibernate data bucket"). Deployment openers must call NewDeploymentS3Store[In] instead, so they never appear.

• [ ] Step 2: Run to verify it fails

Run: go test ./internal/release/ -run TestS3StoreOpenersClassified -count=1 Expected: FAIL listing the unclassified call sites

• [ ] Step 3: Switch each deployment opener to NewDeploymentS3Store[In] and list the rest in otherBucketOpeners with a reason.

• [ ] Step 4: Box publication witness TestBoxPublishThroughDeploymentOpener in internal/host: openDeployStoreEnv against a DirStore-backed fake is out of reach, so assert at the type level that NewDeploymentS3Store's result satisfies the *release.S3Store assertion used at boxpublish.go:259 (compile-time var _ *release.S3Store = mustDeploy() plus a runtime check through the same opened.(*release.S3Store) expression).

• [ ] Step 5: Run go test ./internal/release/ ./internal/host/ -run 'TestS3StoreOpenersClassified|TestBoxPublishThroughDeploymentOpener' -count=1 and go build ./.... Expected: PASS, build clean.

• [ ] Step 6: IAM check. Read infra/modules/deployment-buckets/main.tf and infra/modules/region-host policies: confirm every writing role (box, console) is allowed s3:GetObject and s3:PutObject on control/changes.json and no bucket-policy Deny covers it. If anything denies it, add an allow and a tests/*.tftest.hcl assertion in that module. Record the result (and the same check owed on the Tito infrastructure repo) in the branch's #336 thread.

• [ ] Step 7: Commit git commit -m "Deployment openers enable the change index; opener classification test (#336)"

---

Task 6: RecordView and its snapshot store

Files:

• Create: internal/provision/recordview.go, internal/provision/recordview_store.go
• Test: internal/provision/recordview_test.go

Interfaces:

• Consumes: Tasks 1, 3, 4 (including release.ChangeIndexBackstop); existing knownAccounts (reconcile.go:1760), listAppSubtree (:2022), ManifestKey/RecordsPrefix (provision.go:321), SidecarsPrefix (sidecar.go:122), release.ReadAppTombstone's key, domains.AccountManifestSqid.
• Produces: - type RecordView struct with mu sync.Mutex (leaf lock) - func NewRecordView(live release.Store, watch *release.ChangeWatch) *RecordView - func (v *RecordView) Refresh(ctx context.Context) error — §4.2 under mu: watch.Poll, then re-read every scope with applied < observed or new; an unusable watch, or a view never built, runs Rebuild instead. - func (v *RecordView) Rebuild(ctx context.Context) error — §4.4 under mu: read watch.Observed() (after a Poll) first, then walk everything, publish, set applied = the pre-walk generations. - func (v *RecordView) Snapshot() *ViewSnapshot — atomic load; nil before the first publication. - type ViewSnapshot struct { Epoch uint64; Complete bool /* plus unexported cached families */ } - func (sn *ViewSnapshot) Store(live release.Store) release.Store — read-through store, below. - func ScopeApp(scope string) (acct, app string, ok bool) — parses apps/<a>/<b>.

Cached families (what a snapshot answers; everything else goes to live):

• List("accounts/") and List("accounts/<a>/"): complete key listings per account, kept from ListETags("accounts/<a>/") on each account-scope re-read.
• Get("accounts/<a>/account.json"): cached body or not-found.
• ListDirs("apps/"), ListDirs("apps/<a>/"): accounts and apps that have at least one in-scope app scope; BucketShapedListDirs() returns true so listAppSubtree walks.
• List of apps/<a>/<b>/instances/ and apps/<a>/<b>/sidecars/: complete, from ListETags.
• Get of each record key under those prefixes, of apps/<a>/<b>/deleted.json (trash marker), instances.json and sidecars.json: cached body, cached not-found (an os.ErrNotExist-wrapped error), or the cached read error.
• Every Put/PutFrom/Delete/DeleteIfMatch goes to live, and the key, and every listing prefix it falls under, is answered from live for the rest of that snapshot store's life (read-your-writes and CAS retries).
• Optional extensions not listed above: forwarded to live.

• [ ] Step 1: Write the failing tests (DirStore with the index enabled, ChangeWatch over it)

func TestViewMatchesWalk(t *testing.T)            // 3 accounts, 5 apps, records/sidecars/manifests/trash: ListAllRecordsStrict over sn.Store(live) == over live
func TestViewRefreshRereadsOnlyMovedApp(t *testing.T) // counting store: after one record Put in shop, Refresh lists only apps/bab/shop/{instances,sidecars}/ and GETs only the changed key + that app's three manifests/marker
func TestViewRefreshNoChangeIsOneGet(t *testing.T)   // counting store: exactly one GetIfChanged
func TestViewRetriesFailedScopeOn304(t *testing.T)   // inject one List failure for shop after its bump; next Refresh (index 304) re-reads shop; Complete false until then (codex 4)
func TestViewFirstRefreshWithoutIndexRebuilds(t *testing.T) // no control/changes.json, records exist -> snapshot has them, Complete true (Review Focus 1)
func TestViewUnusableWatchRebuilds(t *testing.T)     // malformed index -> Refresh walks (Review Focus 2)
func TestViewBypassWriteAppearsAfterRebuild(t *testing.T) // record written with index disabled -> absent after Refresh, present after Rebuild (Review Focus 3)
func TestViewPurgeDropsApp(t *testing.T)            // delete every key under shop -> ListDirs("apps/bab/") no longer has shop (Review Focus 4)
func TestViewEpochsIncrease(t *testing.T)
func TestViewBackstopCannotOverwriteNewer(t *testing.T) // codex 2 interleaving: Rebuild blocked mid-walk by a hook; concurrent Refresh must wait on mu; final snapshot has the record
func TestSnapshotStoreReadYourWrites(t *testing.T)  // Put via sn.Store, then Get/List return live state
func TestSnapshotStoreErrorsMatchLive(t *testing.T) // not-found Get of a cached-family key: errors.Is(err, os.ErrNotExist)
func TestSnapshotStoreOtherKeysGoLive(t *testing.T) // Get of config.json / List("apps/bab/shop/channels/") hit live

• [ ] Step 2: Run to verify they fail

Run: go test ./internal/provision/ -run 'TestView|TestSnapshotStore' -count=1 Expected: FAIL

• [ ] Step 3: Implement recordview.go (state, Refresh, Rebuild, publication with atomic.Pointer[ViewSnapshot]) and recordview_store.go (the read-through store). Rebuild reuses knownAccounts and listAppSubtree over live (and ListETags to skip unchanged bodies already cached); an app-scope re-read uses ListETags on both prefixes and GetIfChanged for the three per-app objects. Incomplete stays incomplete until every scope with applied < observed succeeds.

• [ ] Step 4: Run to verify they pass (same command). Expected: PASS

• [ ] Step 5: Commit git commit -m "provision: RecordView keeps the record families and serves snapshots as a read-through store (#336)"

---

Task 7: Invariant simulation and mutation checks

Files:

• Create: internal/provision/recordview_sim_test.go

Interfaces:

• Consumes: Task 6.

• [ ] Step 1: Write the simulation TestRecordViewSimulation (seeded by -seed and -seeds flags, default 200 in CI, skipped under -short): random actors on a DirStore — writers that bump, writers that bypass (EnableChangeIndex off), writers whose bump fails (injected), incremental Refresh, Rebuild, and a consumer that takes snapshots under a fake w.pass and records epochs. Check after every step: (a) after a Refresh that observed generation g for a scope, every bumped write to that scope at ≤ g is in the snapshot; (b) after Rebuild, ListAllRecordsStrict(sn.Store(live)) equals ListAllRecordsStrict(live); (c) snapshot epochs only increase and no scope's contents revert; (d) a snapshot with Complete == false never returns a nil error from ListAllRecordsStrict; (e) a failed scope is re-read on the next Refresh with the index unchanged. The seed set must include the three named interleavings in §8 (codex 1, 2, 4) as fixed seeds or scripted cases.

• [ ] Step 2: Run nice -n 19 go test ./internal/provision/ -run TestRecordViewSimulation -seeds 3000 -count=1. Expected: PASS

• [ ] Step 3: Mutation checks in a scratch copy of the tree under $TMPDIR (never in the worktree): apply each mutation, run the sweep, confirm it FAILS, discard the copy. Mutations: bump before write (Task 2); record applied for failed scopes (Task 6); skip the pre-walk index read in Rebuild; publish the rebuild outside mu; consume a handed-over snapshot instead of taking one under w.pass. Record the five results in the commit message.

• [ ] Step 4: Commit git commit -m "provision: RecordView invariant simulation; five mutation checks fail as they must (#336)"

---

Task 8: Box agent reads through the view

Files:

• Modify: internal/host/agent.go (reconcileSerializedWith :1569, fast-converge deps and fastTick :2537, fullPass :2787, instancesLoop :3358, agent start-up wiring)
• Modify: internal/host/movespass.go (movesPass :1060), internal/host/movesoffline.go (refreshOfflineView :241 callers), internal/host/steering_loop.go:100
• Modify: internal/appconfig/syncexec.go (new option beside :751-810)
• Create: internal/host/changeindex.go
• Test: internal/host/changeindex_test.go, internal/appconfig/syncexec_test.go

Interfaces:

• Consumes: Tasks 4, 6.
• Produces: - func changeIndexArmed(getenv func(string) string) bool — getenv("CARLOS_CHANGE_INDEX") == "read" - var armedChangeWatch atomic.Pointer[release.ChangeWatch], var armedRecordView atomic.Pointer[provision.RecordView] (set once at start-up when armed; nil means today's behavior everywhere) - func viewStore(ctx context.Context, live release.Store) (s release.Store, epoch uint64, ok bool) — Refresh then Snapshot().Store(live); ok false when disarmed, so callers use live. - func recordViewBackstopLoop(ctx context.Context, v *provision.RecordView) — Rebuild at start and every release.ChangeIndexBackstop ±10%, then runs fullPass (unrestricted). - func appconfig.WithRouteFilter(keep func(registry.Route) bool) SyncOption - fastDeps.scopedPass func(ctx context.Context, moved []string) - movesWorker.lastEpoch uint64

• [ ] Step 1: Write the failing tests

func TestReconcileUsesViewWhenArmed(t *testing.T)    // armed, counting live store: two reconcileSerializedWith calls with no writes -> zero List calls on live
func TestReconcileWalksWhenDisarmed(t *testing.T)    // disarmed -> today's List calls (regression guard)
func TestMovesPassTakesSnapshotUnderPass(t *testing.T)   // codex 1: hook pauses a pass after Reconcile; a second pass runs moves at epoch N+1; first pass's moves then sees epoch >= N+1 (never older)
func TestMovesPassRefusesOlderEpoch(t *testing.T)    // forced older snapshot -> pass skipped with a log line, no move step taken
func TestFastTickIndexOnlyRunsScopedPass(t *testing.T)   // serial unchanged, index moved for shop -> scopedPass called with ["apps/bab/shop"], fullPass not called
func TestScopedPassTouchesOnlyMovedApp(t *testing.T)     // codex 10 witness: registry hosts shop and cafe; bump shop; counting store sees no key under apps/bab/cafe/
func TestSerialChangeStillRunsFullPass(t *testing.T)
func TestInstancesLoopFloorIsRefreshWhenArmed(t *testing.T) // nothing changed -> one GetIfChanged on live, no List
func TestSyncExecWithRouteFilter(t *testing.T)            // in appconfig: filter excluding cafe -> no read of cafe's bundle

• [ ] Step 2: Run to verify they fail

Run: go test ./internal/host/ ./internal/appconfig/ -run 'TestReconcileUsesView|TestReconcileWalksWhenDisarmed|TestMovesPass(Takes|Refuses)|TestFastTickIndexOnly|TestScopedPass|TestSerialChangeStill|TestInstancesLoopFloor|TestSyncExecWithRouteFilter' -count=1 Expected: FAIL

• [ ] Step 3: Implement. - Start-up: when armed, open the watch on the deployment store, build the view, store both pointers, start recordViewBackstopLoop. A fleet box (store door) leaves itself disarmed: its watch would be unusable and every Refresh would fall back to a full walk anyway. - reconcileSerializedWith: inside reconcileMu, s2, _, ok := viewStore(ctx, s); pass s2 (or s) to refreshOfflineView, provision.Reconcile and provision.ReconcileSidecars. - movesPass: inside w.pass, take viewStore; refuse and log agent: moves pass: snapshot epoch %d older than %d consumed if the epoch is lower than w.lastEpoch; set w.lastEpoch. - steeredRoutesFromRecords callers pass viewStore's store. - fastTick: when the watch is armed, moved, err := watch.Poll(ctx) every tick; a serial change keeps fullPass; otherwise non-empty moved (after the 5s clamp) runs scopedPass(ctx, moved). - scopedPass: reconcileSerialized, telemetryPass, runMovesPass, runRunsPass, aliases as in fullPass, then release.PassFilter with extra keeping routes whose (Account, App) is in ScopeApp(moved), then syncExecSerialized with appconfig.WithRouteFilter on the same set. - instancesLoop: unchanged structure; its reconcile now goes through the view, so a quiet floor costs one conditional GET.

• [ ] Step 4: Run to verify they pass (same command), then go test ./internal/host/ -count=1 -run 'Moves|Reconcile|FastTick|Steer' for regressions. Expected: PASS

• [ ] Step 5: Commit git commit -m "host: reconcile, sidecars, moves and steering read through the record view; index-only changes run a scoped pass (#336)"

---

Task 9: Box GET loops gated by generation and age

Files:

• Modify: internal/host/agent.go (genPollPass :1720, configGenLoop :1979, the PassFilter call :2628 and fastMembers :2214)
• Modify: internal/host/telemetry_pass.go:58
• Test: internal/host/readgate_test.go

Interfaces:

• Consumes: Task 4's ReadGate, Task 8's armedChangeWatch.
• Produces: genCache gains a gate *release.ReadGate; fastDeps gains canaryGate *release.ReadGate; telemetryPass uses a package telemetryGate plus a per-pass account cache. All gates are NewReadGate(armedChangeWatch.Load(), release.ChangeIndexBackstop); with a nil watch every read is due (today's behavior).

• [ ] Step 1: Write the failing tests

func TestConfigGenSkipsUnmovedApp(t *testing.T)     // armed, two ticks, no bump -> one GET of config-gen.json (the first)
func TestConfigGenReadsMovedApp(t *testing.T)
func TestConfigGenReadsAtMaxAge(t *testing.T)       // fake clock past 30m -> GET (codex 9)
func TestConfigGenRetriesAfterError(t *testing.T)
func TestCanaryPointerGated(t *testing.T)           // canary route stays a member; pointer GET only on move or age
func TestTelemetryAccountReadOncePerPass(t *testing.T) // 3 apps in one account -> one GET of accounts/<a>/telemetry.json
func TestTelemetryGatedWithAgeBound(t *testing.T)   // codex 9
func TestGatesDueWhenWatchUnusable(t *testing.T)    // Review Focus 5: watch denied -> every read due on every tick

• [ ] Step 2: Run to verify they fail

Run: go test ./internal/host/ -run 'TestConfigGen|TestCanaryPointerGated|TestTelemetry(Account|Gated)|TestGatesDueWhenWatchUnusable' -count=1 Expected: FAIL

• [ ] Step 3: Implement. Each read calls gen, due := gate.Due(scope, now); skip when not due; after the read gate.Mark(scope, gen, now, err == nil). For canaries, gate the pointer read inside the PassFilter membership for canary/* routes only; stable routes keep today's path.

• [ ] Step 4: Run to verify they pass (same command). Expected: PASS

• [ ] Step 5: Commit git commit -m "host: config-gen, canary pointers and telemetry read only on change or at the backstop age (#336)"

---

Task 10: Edge cert mirror and home map

Files:

• Modify: internal/edge/certsync.go (certMirror struct, Refresh :622, loop :411)
• Modify: internal/edge/homemap.go (Refresh :320, refreshHosts :264)
• Modify: the edge's wiring in internal/edge/edge.go to pass the process's watch (nil when disarmed)
• Test: internal/edge/certsync_test.go, internal/edge/homemap_test.go

Interfaces:

• Consumes: Tasks 3, 4.
• Produces: certMirror.watch *release.ChangeWatch, homeMap.watch *release.ChangeWatch; each keeps the last successful full rebuild time.

• [ ] Step 1: Write the failing tests

func TestCertMirrorSkipsWhenUnmoved(t *testing.T)     // armed, two polls, no bump -> second poll: no List, no Get
func TestCertMirrorFetchesOnlyChanged(t *testing.T)   // one cert rewritten -> ListETags + exactly one Get
func TestCertMirrorDropsRemovedKey(t *testing.T)      // deleted challenge token disappears from the snapshot
func TestCertMirrorFullRebuildAtBackstop(t *testing.T)
func TestCertMirrorBootRefreshUnchanged(t *testing.T) // boot path still a full read
func TestHomeMapKeysPolledIndependently(t *testing.T) // codex 8: rotate control/edge-mesh-key, control/edges unchanged -> new key loaded within one poll
func TestHomeMapHeartbeatOnUnchangedPoll(t *testing.T) // RefreshAttempted advances when hosts are skipped

• [ ] Step 2: Run to verify they fail

Run: go test ./internal/edge/ -run 'TestCertMirror|TestHomeMap(Keys|Heartbeat)' -count=1 Expected: FAIL

• [ ] Step 3: Implement. Cert mirror: when the watch is usable and control/certs has not moved and the last full rebuild is younger than release.ChangeIndexBackstop, the poll does nothing; when it moved, ListETags(certPrefix), GET new or changed keys, drop keys absent from the listing; otherwise today's full Refresh. Home map: same rule for refreshHosts on control/edges; refreshKeys and health.RefreshAttempted run on every poll as today.

• [ ] Step 4: Run to verify they pass (same command). Expected: PASS

• [ ] Step 5: Commit git commit -m "edge: cert mirror and home map re-read only what moved; mesh keys keep their own poll (#336)"

---

Task 11: Console sweeps

Files:

• Modify: cmd/console/main.go (arm watch and view when CARLOS_CHANGE_INDEX=read; start the backstop loop)
• Modify: internal/console/server.go (Server fields), internal/console/runsweep.go (:62-114), internal/console/seedsweep.go (:140-165), internal/console/store.go (new query beside RecordActivity :2591)
• Test: internal/console/runsweep_test.go, internal/console/seedsweep_test.go, internal/console/store_test.go

Interfaces:

• Consumes: Tasks 4, 6.
• Produces: - Server.recordView *provision.RecordView (nil when disarmed) - func (s *Store) AppsWithActionSince(action string, since time.Time) ([]AppRef, error) with type AppRef struct{ Sqid, App string } — distinct (account sqid, app) from the audit table where action = ? and the time column ≥ since. - const orphanCandidateWindow = 48 * time.Hour, const orphanFullSweepEvery = 24 * time.Hour, Server.lastOrphanFull time.Time

• [ ] Step 1: Write the failing tests

func TestRunSweepReadsViewWhenArmed(t *testing.T)        // counting releases store: no List of apps/ across two sweeps with no writes
func TestOrphanHousekeepingListsOnlyRecentRunApps(t *testing.T) // shop has run-start 1h ago, cafe none -> ListDirs only under apps/bab/shop/runs/ (codex R2-1)
func TestOrphanHousekeepingFullSweepDaily(t *testing.T)  // fake clock past 24h -> every app with records is swept
func TestAppsWithActionSince(t *testing.T)               // rows older than since excluded; distinct pairs
func TestSeedSweepSkipsWalkWithoutSnapshots(t *testing.T)// no snapshots -> no strict live walk; abandonment still sees a Seed record via the view
func TestSeedSweepWalksStrictWithSnapshots(t *testing.T) // snapshots present -> live ListAllRecordsStrict after the snapshot listing, as today

• [ ] Step 2: Run to verify they fail

Run: go test ./internal/console/ -run 'TestRunSweepReadsView|TestOrphanHousekeeping|TestAppsWithActionSince|TestSeedSweep(Skips|Walks)' -count=1 Expected: FAIL

• [ ] Step 3: Implement. runSweepPass takes its record store from recordView.Refresh + Snapshot().Store(live) when armed. Housekeeping builds its app set from AppsWithActionSince("run-start", now-48h) unless now-lastOrphanFull >= 24h, in which case it uses every app with records and sets lastOrphanFull. Seed sweep: when the snapshot listing is empty and the view is armed, read records from the view snapshot; otherwise keep the strict live walk exactly where it is today (after seedsPassBetweenListings).

• [ ] Step 4: Run to verify they pass (same command). Expected: PASS

• [ ] Step 5: Commit git commit -m "console: run sweep reads the record view; orphan housekeeping narrows to recent runs; seed sweep skips the walk with nothing to delete (#336)"

---

Task 12: Census overlay, release note, rollout notes

Files:

• Create: the plan14 root-reads census overlay for every touched covered package (recipe in the litestream-284 memory and #280), census-ruling-s3-change-index
• Modify: README.md rollout/arming section (where release recipes live), CHANGELOG or release-notes location the repo uses
• Test: the census test package (internal/provision's root-reads census test)

• [ ] Step 1: Run the census test go test ./internal/provision/ -run RootRead -count=1. Expected: FAIL naming changed packages/units.
• [ ] Step 2: Merge main first, then generate the overlay so it loads after the latest one on main (as of 2026-10-21 the overlays from ci-speed-activator and securelisten-337; check main's latest when you get there), with the scratch-test recipe (rootReadProduction, loadRootReadInventory, rootReadLegacyFingerprint, rootReadLegacySeal), add it, and post the census-ruling request on #280.
• [ ] Step 3: Run the census test again. Expected: PASS
• [ ] Step 4: Release note for R4 (old CLIs in direct-bucket mode converge at the 30-minute backstop until updated) and the arming steps from §7: draft the strings, run them through the copy-review skill, apply Paul's edits verbatim.
• [ ] Step 5: Full local gate on touched packages go test ./internal/release/ ./internal/provision/ ./internal/host/ ./internal/edge/ ./internal/console/ ./internal/appconfig/ ./internal/hostops/ -count=1 (host and console with -run filters per the targeted-tests rule if the full packages are too heavy), go vet ./..., and the cross-compile check (GOOS=darwin go build ./..., GOOS=windows go build ./...). Expected: PASS
• [ ] Step 5b: Claim invariant. If any console test added here swaps process-wide state (a package var, log output, a client transport) or uses t.Setenv inside a TestClaimConsoleMutationInvariant fixture, list it serial (claimSerialDoors or the probe's serial field) and call claimMustRunAlone(t, ...) before the swap. Task 11 adds no mutation doors, so this is a check, not expected work.
• [ ] Step 6: Shard weights. If any new console or root-package test takes several seconds, refresh .amadan/test-weights.tsv with .amadan/test-weights.sh (CI runs .amadan/test-gate.sh, which shards console 8 ways and root 3 ways). Put no CI-only script under scripts/: the root-read census counts every .sh outside dot directories.
• [ ] Step 7: Merge main, commit and push; wait for CI green on the tip (one === <job> section per job; failures listed under FAILED: at the end).