amadan
plan: plans 2, boxes read account limits
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+227
−0
docs/superpowers/plans/2026-10-21-plans-2-box-limits.md+227 −0
diff --git a/docs/superpowers/plans/2026-10-21-plans-2-box-limits.md b/docs/superpowers/plans/2026-10-21-plans-2-box-limits.md| new file mode 100644 |
| index 000000000..b0e2b9f59 |
| --- /dev/null |
| +++ b/docs/superpowers/plans/2026-10-21-plans-2-box-limits.md |
| @@ -0,0 +1,227 @@ |
| +# Plans 2: Boxes Read Account Limits Implementation Plan |
| + |
| +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
| + |
| +**Goal:** Boxes read each account's published limits and apply the plan's idle window to instances and sidecar workers. Self-hosted deployments without a plan catalog see no change. |
| + |
| +**Architecture:** The console publishes a deployment marker `control/plans.json` (enabled plus the catalog id) once at start. On every box the host agent's existing full and five-minute passes run a `plansPass` beside `telemetryPass`. It reads the marker, and only when plans are enabled does it read `accounts/<sqid>/limits.json` for the accounts this box serves, into a last-known-good in-memory cache. The activator gets an `IdleFor` hook that can only shorten an entry's idle window. A box-level switch `CARLOS_PLAN_LIMITS=off` turns all of it off with zero bucket reads. |
| + |
| +**Tech Stack:** Go; `internal/plans`, `internal/console`, `internal/host`, `internal/activator`, `internal/hostops`, `internal/edge`. |
| + |
| +**Spec:** `docs/superpowers/specs/2026-10-20-plans-caps-packs-design.md` §5.2 (idle window), §7 (how limits reach the boxes; fail open), §10 (provider boundary). Builds on Plan 1 (`docs/superpowers/plans/2026-10-20-plans-1-catalog-and-limits.md`, merged f550afa). |
| + |
| +## Self-hosters (Paul, 2026-10-21: "make sure this isn't going to interrupt them") |
| + |
| +Three guards, each pinned by a test: |
| + |
| +1. **No catalog, no change.** Without `CARLOS_PLAN_CATALOG` the console never writes `control/plans.json` as enabled. A box that finds the marker absent or disabled clears its cache and reads no limits record: one small read per pass (the marker), nothing else, and every idle window is exactly today's. |
| +2. **No record, no limits.** A box with no limits record for an account runs that account exactly as today (fail open, spec §7). |
| +3. **A box can opt out.** `CARLOS_PLAN_LIMITS=off` (also `false`, `0`, `no`) makes the box ignore plans entirely and read nothing. |
| + |
| +A deployment that turns its catalog off again: the console's next start writes the marker disabled, and boxes revert within one pass. |
| + |
| +## Scope changes from the spec's plan list (rulings, author's) |
| + |
| +- The **small memory tier** moves to Plan 2b. It touches nine allowlists across `internal/cgroup`, `internal/instancectl` and the root claim launch code, plus unit-backed apps' rendered drop-ins, and most of that is root-path code with its own census ruling. |
| +- **Throttling and the continuous-awake ceiling** move to Plan 4, where the grace engine that triggers them lives. Plan 2 carries no throttle state. |
| +- Plan 1's two deferred minors are fixed here (Task 1, Task 2). |
| + |
| +## Global Constraints |
| + |
| +- No new module dependencies. |
| +- Plans can only **shorten** an idle window: an entry's window is `min(its current window, plan idle)` (spec §5.2 "the smaller of its own and the plan's"). A plan never lengthens a box's `CARLOS_HIBERNATE_IDLE` and never touches an always-on worker. |
| +- Fail open everywhere on the box: a read error keeps the last good value, an absent record means no limits. |
| +- New box env vars go in `hostenv.Optional` (`internal/hostenv/hostenv.go:71`). |
| +- New bucket keys need a hostops classification and, if fleet boxes read them, the storedoor allowlist (`internal/edge/storedoor.go:436`). |
| +- New console workers: `claim_workers.go`, `cmd/console/main.go` RunClaimWorker, `testdata/claim-routes.txt`, a claim fixture and probe. |
| +- CI shards tests: no test depends on another top-level test having run first. New `internal/host` tests call `t.Parallel()` unless they set env, cwd, umask, log output or package vars; the cache is a package var, so tests that touch it stay serial. |
| +- Every change to covered packages re-pins the #280 root-read overlay; the record-write census layer may need entries too (Task 5). |
| +- Copy: this plan adds no human-facing text except log lines (not reviewed). |
| +- Local testing: touched packages only, with `-run`; CI runs the full suite. |
| + |
| +## Review Focus |
| + |
| +1. **Plans off.** With the marker absent, a box must make exactly one bucket read per pass (the marker) and apply no limits. Pinned in Task 3 (`TestPlansPassOffReadsOnlyTheMarker`). |
| +2. **Box switch off.** `CARLOS_PLAN_LIMITS=off` must make zero bucket reads and clear any cache. Pinned in Task 3 (`TestPlansPassBoxSwitchOffReadsNothing`). |
| +3. **Bucket blips.** A marker read error keeps the whole last-good cache; a per-account read error keeps that account's last value. Pinned in Task 3 (`TestPlansPassKeepsLastGoodOnErrors`). |
| +4. **Plans turned off again.** Marker enabled, then disabled: the next pass clears the cache and idle windows revert. Pinned in Task 3 (`TestPlansPassClearsWhenDisabled`) and Task 4 (`TestIdleForNilOrZeroChangesNothing`). |
| +5. **A plan idle longer than the box's default.** The box default stands. Pinned in Task 4 (`TestPlanIdleOnlyShortens`). |
| + |
| +--- |
| + |
| +### Task 1: The deployment marker, and checked pack arithmetic |
| + |
| +**Files:** |
| +- Create: `internal/plans/marker.go`, `internal/plans/marker_test.go` |
| +- Modify: `internal/plans/catalog.go` (`Effective`), `internal/plans/catalog_test.go` |
| + |
| +**Interfaces:** |
| +- Produces: |
| + ```go |
| + const MarkerSchema = 1 |
| + type Marker struct { |
| + Schema int `json:"schema"` |
| + Enabled bool `json:"enabled"` |
| + CatalogID string `json:"catalog_id,omitempty"` |
| + UpdatedAt time.Time `json:"updated_at"` |
| + } |
| + func MarkerKey() string // "control/plans.json" |
| + func GetMarker(ctx context.Context, s release.Store) (*Marker, string, error) // nil, "", nil when absent |
| + func PutMarker(ctx context.Context, s release.Store, m *Marker, ifMatch string) (string, error) |
| + ``` |
| + `GetMarker` refuses an unknown schema (error, so the box keeps its last good state). |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +```go |
| +func TestMarkerKey(t *testing.T) // MarkerKey() == "control/plans.json" |
| +func TestMarkerRoundTrip(t *testing.T) // PutMarker create-only then GetMarker equal; absent -> nil, "", nil |
| +func TestMarkerRefusesUnknownSchema(t *testing.T) // a stored {"schema": 2} -> GetMarker error |
| +func TestPutMarkerIsCAS(t *testing.T) // second create-only put -> release.ErrCASConflict |
| +func TestEffectiveRefusesOverflow(t *testing.T) // pack storage adds 1<<62, quantity 2, ceiling 100 -> error wrapping ErrPackQuantity, |
| + // never a negative or wrapped limit (Astra plans-1 finding 5) |
| +``` |
| + |
| +- [ ] **Step 2: Run them to verify they fail** — `go test ./internal/plans/` → FAIL (undefined `MarkerKey`). |
| + |
| +- [ ] **Step 3: Implement** `marker.go` on the pattern of `record.go`. In `Effective`, compute each pack's contribution with checked multiplication and addition (`math/bits.Mul64`/overflow checks, or compare against `math.MaxInt64 / qty`) and return `fmt.Errorf("%w: %q overflows", ErrPackQuantity, name)` on overflow. |
| + |
| +- [ ] **Step 4: Run** `go test ./internal/plans/` → PASS. |
| + |
| +- [ ] **Step 5: Commit** `plans: the deployment marker, and checked pack arithmetic` |
| + |
| +### Task 2: The console publishes the marker; plan writes need a live account |
| + |
| +**Files:** |
| +- Create: `internal/console/plansmarker.go`, `internal/console/plansmarker_test.go` |
| +- Modify: `internal/console/accountplans.go` (`SetAccountPlan`), `internal/console/accountplans_test.go`, `internal/console/apiaccountplan.go`, `internal/console/apiaccountplan_test.go` |
| +- Modify: `internal/console/claim_workers.go`, `cmd/console/main.go`, `internal/console/testdata/claim-routes.txt`, `internal/console/claim_invariant_fixtures_test.go`, `internal/console/claim_invariant_probes_worker_test.go` |
| +- Modify: `internal/hostops/mutation_classifier.go`, `internal/hostops/limits_classifier_test.go` |
| +- Modify: `internal/edge/storedoor.go` (`aclRead`), its test file |
| + |
| +**Interfaces:** |
| +- Consumes: Task 1. |
| +- Produces: |
| + ```go |
| + func (s *Server) PublishPlansMarker(ctx context.Context) // claim worker "PublishPlansMarker"; runs once |
| + var ErrAccountGone = errors.New("account deleted") // console package |
| + ``` |
| + `SetAccountPlan` returns `ErrAccountGone` and writes nothing when the account's `deleted_at` is set (Astra plans-1 finding 3); the plan door answers 404 for it. |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +```go |
| +// plansmarker_test.go |
| +func TestPublishPlansMarkerOffAndAbsentWritesNothing(t *testing.T) // no catalog, no marker: bucket unchanged |
| +func TestPublishPlansMarkerEnables(t *testing.T) // catalog set: marker Enabled, CatalogID == catalog.ID() |
| +func TestPublishPlansMarkerIsIdempotent(t *testing.T) // second run with the same catalog: etag unchanged |
| +func TestPublishPlansMarkerDisablesWhenCatalogRemoved(t *testing.T) // marker enabled, then a run with no catalog: Enabled false |
| +// accountplans_test.go |
| +func TestSetAccountPlanRefusesADeletedAccount(t *testing.T) // deleted_at set: ErrAccountGone, no row |
| +// apiaccountplan_test.go |
| +func TestAccountPlanSetOnADeletedAccountIs404(t *testing.T) // deleted between lookup and write: 404, no record published |
| +// hostops limits_classifier_test.go |
| +func TestClassifyPlansMarkerIsExact(t *testing.T) // Put on control/plans.json: non-app; Delete and lookalikes: ErrUnclassified |
| +// edge storedoor test |
| +func TestFleetBoxCanReadThePlansMarker(t *testing.T) // aclRead(sqid, fleet, "control/plans.json") == true |
| +``` |
| + |
| +- [ ] **Step 2: Run** the touched packages' tests → FAIL. |
| + |
| +- [ ] **Step 3: Implement.** `PublishPlansMarker`: with a catalog, `GetMarker`; if absent or different (`!Enabled || CatalogID != cat.ID()`), `PutMarker` with the read etag. Without a catalog, `GetMarker`; only if present and `Enabled`, write it `Enabled: false`. Log, never fail. `SetAccountPlan`: guard the insert and the update with `EXISTS (SELECT 1 FROM accounts WHERE id = ? AND deleted_at = 0)`, returning `ErrAccountGone` when no row changes. hostops: add `{exact: "control/plans.json", reason: "plans deployment marker published by the console; no app state"}` to `nonAppFamilies` (deletes false). storedoor: add `plans.MarkerKey()` to the exact case in `aclRead`. |
| + |
| +- [ ] **Step 4: Register and start the worker** (`claim_workers.go`, `cmd/console/main.go` beside `StartPlanLimitsPass`), add `worker PublishPlansMarker` to `claim-routes.txt`, a `claimNonApp` fixture and a probe asserting the marker is written when a catalog is set. |
| + |
| +- [ ] **Step 5: Run** `go test ./internal/plans/ ./internal/hostops/ ./internal/edge/ -run 'Marker|Classify|Fleet'`, `go test ./internal/console/ -run 'PlansMarker|AccountPlan|ClaimConsoleRouteCensus'`, `go test ./cmd/console/ -run TestEveryRegisteredClaimWorkerIsStarted` → PASS. |
| + |
| +- [ ] **Step 6: Commit** `console: publish the plans marker; plan writes need a live account` |
| + |
| +### Task 3: The box's plans pass and cache |
| + |
| +**Files:** |
| +- Create: `internal/host/plans_pass.go`, `internal/host/plans_pass_test.go` |
| +- Modify: `internal/host/agent.go` (call `plansPass` beside both `telemetryPass` calls, `agent.go:2809` and `:3395`), `internal/hostenv/hostenv.go` (`CARLOS_PLAN_LIMITS`) |
| + |
| +**Interfaces:** |
| +- Consumes: `plans.GetMarker`, `plans.GetLimits` (Task 1, Plan 1). |
| +- Produces: |
| + ```go |
| + func plansPass(ctx context.Context, s release.Store, reg *registry.Registry, getenv func(string) string) |
| + func planLimitsFor(sqid string) (plans.Limits, bool) // false when plans are off or the account has no record |
| + func planLimitsSwitchOn(getenv func(string) string) bool // false for off/false/0/no (case-insensitive) |
| + ``` |
| + The cache is a package-level `atomic.Pointer` to an immutable snapshot `{byAccount map[string]plans.Limits}`; `nil` means off. |
| + |
| +- [ ] **Step 1: Write the failing tests** (serial: they touch the package cache), using `release.NewDirStore(t.TempDir())` wrapped in a counting store that records every `Get` key and can fail a key once, and a registry with routes for two accounts: |
| + |
| +```go |
| +func TestPlanLimitsSwitch(t *testing.T) // "", "on": true; "off", "OFF", "false", "0", "no": false |
| +func TestPlansPassOffReadsOnlyTheMarker(t *testing.T) // no marker: Gets == ["control/plans.json"]; planLimitsFor(any) false |
| +func TestPlansPassBoxSwitchOffReadsNothing(t *testing.T) // CARLOS_PLAN_LIMITS=off with an enabled marker and records: zero Gets; cache cleared |
| +func TestPlansPassLoadsServedAccounts(t *testing.T) // enabled marker, records for A and B: planLimitsFor(A).IdleSeconds == 90; an account with no routes on this box is not read |
| +func TestPlansPassAbsentRecordMeansNoLimits(t *testing.T) // enabled, no record for B: planLimitsFor(B) false |
| +func TestPlansPassKeepsLastGoodOnErrors(t *testing.T) // after a good pass: marker Get fails -> A and B unchanged; A's record Get fails -> A unchanged, B refreshed |
| +func TestPlansPassClearsWhenDisabled(t *testing.T) // enabled then marker Enabled false: planLimitsFor(A) false |
| +``` |
| + |
| +- [ ] **Step 2: Run** `go test ./internal/host/ -run 'PlansPass|PlanLimitsSwitch'` → FAIL. |
| + |
| +- [ ] **Step 3: Implement** `plans_pass.go` after `telemetry_pass.go`'s shape: switch off → store nil, return. `GetMarker` error → log, return (keep). Absent or disabled → store nil, return. Otherwise collect distinct `release.RouteAccount(rt)` from `reg.List()` (skip status routes), `GetLimits` each: error → copy the previous value for that account if any; nil → omit; else set. Store the new snapshot. Log first failure and every tenth, as `homemap.go`'s loop does. |
| + |
| +- [ ] **Step 4: Wire** `plansPass(ctx, store, reg, os.Getenv)` after both `telemetryPass` calls; add `"CARLOS_PLAN_LIMITS"` to `hostenv.Optional` in sorted position. |
| + |
| +- [ ] **Step 5: Run** `go test ./internal/host/ -run 'PlansPass|PlanLimitsSwitch' ./internal/hostenv/` → PASS. |
| + |
| +- [ ] **Step 6: Commit** `host: read account limits each pass, marker-gated, with a box off switch` |
| + |
| +### Task 4: The activator's idle hook |
| + |
| +**Files:** |
| +- Modify: `internal/activator/activator.go` (`Service` gains `IdleFor`; the sweep at `:4646`) |
| +- Modify: `internal/activator/activator_test.go` |
| +- Modify: `internal/host/agent.go` (set `svc.IdleFor` after `activator.New`, `agent.go:490`) |
| +- Create: `internal/host/plans_idle_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: `planLimitsFor` (Task 3). |
| +- Produces: |
| + ```go |
| + // Service field, set before the service is published: |
| + IdleFor func(registry.Route) time.Duration // 0 = no opinion; nil = today's behaviour |
| + func planIdleFor(rt registry.Route) time.Duration // internal/host: Limits.IdleSeconds as a duration, 0 when none |
| + ``` |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +```go |
| +// activator_test.go, beside TestActivatorIdleSleep and TestWorkerRouteIdleWindowOverridesPlatformDefault |
| +func TestIdleForShortensAnInstance(t *testing.T) // cfg.Idle 15m, IdleFor returns 90s: the entry sleeps after 90s idle, not before |
| +func TestPlanIdleOnlyShortens(t *testing.T) // cfg.Idle 5m, IdleFor 15m: sleeps at 5m |
| +func TestIdleForShortensAWorkerToTheSmaller(t *testing.T) // worker IdleSeconds 600, IdleFor 90s: 90s; worker 60, IdleFor 90s: 60s |
| +func TestIdleForNilOrZeroChangesNothing(t *testing.T) // nil hook, and a hook returning 0: today's windows |
| +func TestIdleForLeavesAlwaysOnWorkersAlone(t *testing.T) // Hibernate=false worker with IdleFor 90s: never stopped for idleness |
| +// plans_idle_test.go (serial) |
| +func TestPlanIdleFor(t *testing.T) // cache off: 0; account with IdleSeconds 90: 90s; account absent: 0 |
| +``` |
| + |
| +- [ ] **Step 2: Run** `go test ./internal/activator/ -run IdleFor` and `go test ./internal/host/ -run TestPlanIdleFor` → FAIL. |
| + |
| +- [ ] **Step 3: Implement.** In the sweep, after the current `idleWindow` is chosen: `if f := a.IdleFor; f != nil { if d := f(e.route); d > 0 && d < idleWindow { idleWindow = d } }`. The "held for tick" check already uses `idleWindow`. `planIdleFor` reads `planLimitsFor(release.RouteAccount(rt))`. In `agent.go`, set `svc.IdleFor = planIdleFor` right after `activator.New`. |
| + |
| +- [ ] **Step 4: Run** the two commands above and `go test ./internal/activator/ -run 'Idle|Worker|AlwaysOn'` → PASS. |
| + |
| +- [ ] **Step 5: Commit** `activator: a plan can shorten an entry's idle window` |
| + |
| +### Task 5: Census, then the branch gates |
| + |
| +**Files:** `internal/provision/testdata/plan14-root-reads-plans-2-box-limits.tsv` (new), `internal/provision/plan14_root_reads_test.go`, possibly a record-write census layer beside `internal/provision/plans_caps_census_test.go`. |
| + |
| +- [ ] **Step 1:** Run `GOFLAGS=-buildvcs=false go test ./internal/provision/ -run 'TestPlan14RootReadCensus|TestPlan9B8'`. For each discrepancy, regenerate the overlay with the scratch-generator recipe used for Plan 1 (load every overlay, print `@legacy`/rows/`@coverage` for changed and new units, compute the seal), with basis text naming the executing identity: the box agent's plans pass reads two bucket objects and makes no root decision; the activator change narrows an idle window. Load it last; mark `rev3=PROPOSED-plans-2-box-limits`. |
| +- [ ] **Step 2:** Run the full claim invariant once: `nice -n 19 go test ./internal/console/ -run TestClaimConsoleMutationInvariant -timeout 30m` → ok. |
| +- [ ] **Step 3:** Commit `plans 2: census overlay`, merge current main, push, wait for CI green, ask for the census ruling on #280. |
| + |
| +## Done when |
| + |
| +- The commands in each task pass locally; CI is green on the tip after merging current main. |
| +- Review Focus 1–5 each have their named test passing. |
| +- The branch's final review (Astra, whole branch) has no open Critical or Important finding. |