amadan
Close round sixteen: an unmeasured store holds storage for its own account only
Round sixteen (Astra reject: one P1, two P2; Opus accept with fixes: one P1, one P2, two P3) found every round fifteen finding closed, and found that round fifteen's storage hold applied to every account. - Unmeasured storage days are counted per account. The account whose store could not be sized, or was being torn down, carries fewer storage source days; every other account's storage stays complete. An unswept entry with no recognisable account still holds the day for everyone. - The storage pass reads the record again after sizing and discards a size taken across the start of a teardown. - The record writer's handling of the mark's earlier place is tested. Left open and listed: a live shared bucket sized in full for each store on it (19i); the unwired scan queue's drain (19f). Each fix has a reproduction and was mutation-checked in a scratch copy. The round's reports, dispositions and a diff-only brief for round seventeen are added. No review round has passed. Billing is off; nothing attests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
9 files changed,
+423
−20
docs/superpowers/specs/CODEX-DISPOSITIONS-carloku-paid-launch-overnight.md+16 −0docs/superpowers/specs/CODEX-FINDINGS-carloku-paid-launch-overnight-round16.md+97 −0docs/superpowers/specs/CODEX-REVIEW-BRIEF-carloku-paid-launch-overnight-round17.md+26 −0docs/superpowers/specs/OPUS-REVIEW-carloku-paid-launch-overnight-round16.md+109 −0implementation/carloku-paid-launch/AGENT_HANDOFF.md+1 −1implementation/carloku-paid-launch/OUTSTANDING.md+4 −3internal/console/economicspass.go+54 −16internal/console/economicspass_test.go+95 −0internal/objstore/ownership_test.go+21 −0
CI passed — run details
diff --git a/docs/superpowers/specs/CODEX-DISPOSITIONS-carloku-paid-launch-overnight.md b/docs/superpowers/specs/CODEX-DISPOSITIONS-carloku-paid-launch-overnight.md| index 26ab3a91f..600774c3b 100644 |
| --- a/docs/superpowers/specs/CODEX-DISPOSITIONS-carloku-paid-launch-overnight.md |
| +++ b/docs/superpowers/specs/CODEX-DISPOSITIONS-carloku-paid-launch-overnight.md |
| @@ -470,3 +470,19 @@ Both found every round fourteen finding closed. Astra: one P1, one P2. Opus: one |
| | Both, claims: the mark field's doc, "an earlier build carries no mark", "resumed by running it again" stated absolutely, `Inspect` "for display", a test comment, 19h's reliance on the signing window | Corrected. | `objstore.go`, `walk.go`, `AGENT_HANDOFF.md`, `apiscan_test.go`, `OUTSTANDING.md` | none | |
| Mutation checks in a scratch copy, each failing its test: the sidecar writer's teardown check off; the storage pass sizing a store under teardown; the status showing coverage; the re-grant accepting a record with nothing provisioned. |
| + |
| +## Round sixteen (reviewed `abc9f781`; Astra rejected, Opus accepted with fixes) |
| + |
| +Reports: `CODEX-FINDINGS-carloku-paid-launch-overnight-round16.md`, `OPUS-REVIEW-carloku-paid-launch-overnight-round16.md`. |
| + |
| +Both found every round fifteen finding closed. Astra: one P1, two P2. Opus: one P1, one P2, two P3. Astra's F1 and Opus's N1 are the same fault, and it was a fault in round fifteen's own fix. Nothing that attests, reaches Stripe with a switch unset, or makes an open late-request defect reachable. |
| + |
| +| Finding | Disposition | Where | Reproduction | |
| +| --- | --- | --- | --- | |
| +| Astra F1, Opus N1: round fifteen recorded a store under teardown as unswept, and one unswept store made the storage source incomplete for every account, for that day and so for the month, with no way to correct the day afterwards | Accepted; a fault of round fifteen's fix, on top of a rule that already held every account when one store failed to size. Unmeasured storage days are now counted per account: the account whose store could not be measured carries fewer storage source days, every other account's storage stays complete. An unswept entry whose account cannot be told still holds the day for everyone. A day already written unswept stays held for that account; there is no rebuild door (item 18). | `console/economicspass.go` `sweepUsage`, `accountSourceDays` | `TestUnsweptStorageHoldsOnlyItsOwnAccount`, `TestStatementsPassUnsweptStorageIsNotComplete` | |
| +| Astra F2: a storage pass that began sizing before a retained teardown records the other account's bytes | Accepted. The record is read again after sizing; if a teardown began or the store is no longer the same, the size is discarded and the day held for that account. | `console/economicspass.go` `storagePass` | `TestStoragePassDiscardsASizeTakenAcrossATeardown` | |
| +| Opus N2: a live bucket shared by two stores is sized in full for each of them | Accepted, not fixed. It predates this branch and needs the holder inventory in the storage pass. Storage for a store on a shared bucket must not be billed until it is. Open as 19i. | `OUTSTANDING.md` 19i | none | |
| +| Astra F3, Opus N3 (latent): the queue drain checks permission once per batch and can return another account's object metadata after a retained teardown | Accepted, not fixed. Nothing calls the drain and nothing enqueues. Added to 19f, to be fixed before the queue is wired. | `OUTSTANDING.md` 19f | none | |
| +| Opus N4: the writer's handling of the mark's earlier place had no test | Accepted. | `ownership_test.go` | `TestTheMarkIsReadFromWhereAnEarlierBuildWroteIt` (now also writes a record with the earlier field set and reads the raw result) | |
| + |
| +Mutation checks in a scratch copy, each failing its test: every unswept entry holding the day for everyone; the per-account count not kept; an entry with no account not holding the day; the re-read after sizing ignoring a teardown; the writer leaving the mark in its earlier place. |
diff --git a/docs/superpowers/specs/CODEX-FINDINGS-carloku-paid-launch-overnight-round16.md b/docs/superpowers/specs/CODEX-FINDINGS-carloku-paid-launch-overnight-round16.md| new file mode 100644 |
| index 000000000..2388c5a46 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-FINDINGS-carloku-paid-launch-overnight-round16.md |
| @@ -0,0 +1,97 @@ |
| +# Round-sixteen review: `abc9f781` |
| + |
| +Reviewed against `aed42815`, concentrating on `973a9f15..abc9f781`. No files changed, tests run, network commands, or Stripe/AWS CLI calls. Test counterfactuals are based on inspection. |
| + |
| +All locations refer to `abc9f781`. **O** = `internal/objstore`; **C** = `internal/console`; **U** = `internal/usage`; **D** = `implementation/carloku-paid-launch`; **S** = `docs/superpowers/specs`. |
| + |
| +## 1. Round-fifteen dispositions |
| + |
| +| Finding or claim | Disposition | Fix and test | Would the test pass without the fix? | |
| +|---|---|---|---| |
| +| **Astra F1:** an admitted sweep publishes objects listed after retained teardown | **Closed for the reported interleaving.** The writer detects teardown and the sweep discards its result. Status suppresses coverage when its supplied record carries teardown. | `O/sweep.go:411`, `:305`; `C/apiscan.go:97`. Tests: `TestSweepInFlightDoesNotPublishAfterATeardownBegins`, `O/sweep_test.go:575`; `TestScanStatusShowsNoCoverageForAStoreUnderTeardown`, `C/apiscan_test.go:761`. | **No.** Removing either guard fails its corresponding test. Neither test covers teardown overtaking the final record check. | |
| +| **Astra F2 / Opus N1:** daily storage collection attributes a retained bucket to the departed account | **Partly closed.** A pass that initially observes teardown holds instead of sizing. An admitted pass still misattributes; the hold also affects more accounts and lasts longer than documented. | `C/economicspass.go:211`. `TestStoragePassDoesNotSizeAStoreUnderTeardown`, `C/economicspass_test.go:1535`. See new F1–F2. | **No.** Removing the guard calls the sizer and records bytes. The test does not fold the resulting day or exercise concurrent teardown. | |
| +| **Opus N2:** `Regrant` creates a first store after destruction and redeclaration | **Closed.** Missing `Provisioned` is refused under the lease. | `O/walk.go:259`; clean refusal at `:1356`. `TestRegrantNeverMakesAFirstStore`, `O/ownership_test.go:3639`. | **No.** The expected-error assertion fails; the unguarded path also creates provider resources and changes the record. | |
| +| **Opus N3:** scanner enable races retained teardown | **Accepted and left open**, as 19f. | Separate check and enable remain at `C/apiscan.go:367`, `:371`; documented at `D/OUTSTANDING.md:69`. | No fix or regression test claimed. | |
| +| **Astra observation:** an explicitly constructed legacy mark can be serialized in its old location | **Closed in source; the new writer protection is untested.** | `O/objstore.go:277` normalizes before serialization. Cited regression: `TestTheMarkIsReadFromWhereAnEarlierBuildWroteIt`, `O/ownership_test.go:3352`. | **Yes, without the new writer normalization.** That test injects raw JSON and subsequently reads through `Inspect`, which already migrates the mark at `O/objstore.go:259`. | |
| +| **Astra/Opus claim:** rotation marks before reaping and minting | **Closed as documentation.** | Corrected field comment at `O/objstore.go:158`. | No wording test; behavioral tests do not require the correction. | |
| +| **Astra claim:** earlier builds uniformly carry no mark | **Closed as documentation.** Pre-mark builds are distinguished from the build using the nested mark. | `O/objstore.go:168`; `D/AGENT_HANDOFF.md:35`. Migration regression: `O/ownership_test.go:3352`. | **Yes** without the wording changes; **no** without reader migration. | |
| +| **Astra claim:** `Inspect` also serves a display door | **Closed as documentation.** | `O/walk.go:1569` now names the holder inventory only. | No wording test. | |
| +| **Opus claim:** every rotation failing before delivery resumes without a mark | **Closed as documentation.** The lost-response exception for writing the mark is now explicit. | `D/AGENT_HANDOFF.md:35`. | No dedicated test of that exception; existing behavioral tests do not require the wording. | |
| +| **Opus claim:** the scanning regrant fixture removes the user | **Closed as documentation.** It now explicitly says the user and key remain. | `C/apiscan_test.go:733`; test begins at `:716`. | **Yes.** Only the comment changed. | |
| +| **Opus claim:** waiting out the signing window preserves overlap on every backend | **Closed as documentation.** The statement is conditional on backend enforcement, observed on AWS only. | `D/OUTSTANDING.md:68`. | No wording test; the repair remains unbuilt. | |
| +| **Astra/Opus retained-reader claims** in handoff item 5 | **Partly closed.** The reported sweep interleaving is stopped, but storage still has an in-flight reader, and the absolute publication wording exceeds the checks. | `D/AGENT_HANDOFF.md:35`; fixes and tests above. | The new tests fail without their guards but pass with the remaining cases below. | |
| + |
| +No new scanning convergence failure was found from the `Regrant` refusal. `regrantForScanning` already skips records without `Provisioned` (`C/apiscan.go:574`); provisioned stores remain eligible for convergence. |
| + |
| +## 2. New findings |
| + |
| +### F1. P1 — One ordinary teardown permanently blocks that month’s metered calculation for every account |
| + |
| +**Where:** `C/economicspass.go:211`, `:858`, `:918`, `:1649`, `:1674`; `U/statement.go:106`, `:121`, `:176`, `:210`; `C/billingcalculation.go:57`. |
| + |
| +**Scenario:** |
| + |
| +1. Account A starts an ordinary, single-owner teardown. It fails after recording its teardown block and is retried the following day. |
| +2. The daily storage pass encounters that block and persists `StoreUnswept=true`. |
| +3. `sweepUsage` examines **every app** in the daily snapshot. Any unswept store sets the single, deployment-wide `storageOK` to false. |
| +4. Every account’s statement inherits the reduced `SourceStorage` count. `accountSourceDays` specializes only durability. |
| +5. A finishes teardown. The storage day remains unchanged: daily objects are create-only, and subsequent collection returns immediately when the object exists. |
| +6. Every account’s statement for that month continues to fail storage completeness. The monthly calculation requires all five billed meters, so unrelated accounts’ overages remain blocked too. |
| + |
| +**Scope and duration:** This holds the whole month’s metered calculation for every account, including accounts with no object store. It does not stop the base subscription charge. Completing teardown stops future affected snapshots but does not repair past ones. Statements can also close with this incomplete count, after which ordinary upserts refuse to change them. |
| + |
| +A retained record repeats this across subsequent months. An ordinary teardown need only overlap one daily snapshot to produce the same permanent month-wide effect. |
| + |
| +This exceeds the accepted 19a limitation: the documentation describes the departed store’s hold while its record remains, not a persistent hold for unrelated accounts. |
| + |
| +**Smallest acceptable fix:** Track storage completeness by account, preserving the hold for the account whose measurement is uncertain. Document that already-written uncertain days remain held unless supported correction exists. Add a regression with an unrelated account, an ordinary teardown spanning a snapshot, and successful subsequent cleanup; the unrelated account must retain storage completeness. Do not manufacture a zero measurement to clear the affected account’s hold. |
| + |
| +### F2. P2 — An in-flight storage pass still records the remaining account’s bytes under the departed account |
| + |
| +**Where:** `C/economicspass.go:201`, `:222`, `:227`, `:233`, `:1875`; folding at `:1030`. |
| + |
| +**Scenario:** |
| + |
| +1. A and B share an operator-adopted bucket X. |
| +2. `storagePass` reads A’s record before teardown and enters `sizeStore`. Pause its bucket listing. |
| +3. A completes retained teardown, losing access to X. |
| +4. B adds objects to X. |
| +5. Resume the listing. The pass records X’s new size under A with `StoreUnswept=false`. |
| + |
| +The guard tests only the record read before sizing. Nothing revalidates the store after the paginated listing or before publishing the daily snapshot. Folding consequently attributes B’s new bytes to A, and the create-only snapshot preserves that attribution. |
| + |
| +Billing remains off, and other meter holds remain effective. This is incorrect persisted usage, not a demonstrated current charge. |
| + |
| +**Smallest acceptable fix:** Revalidate the store’s estate and teardown state after sizing; discard the measurement and mark it unswept if either changed or cannot be established. Add an interleaving regression that completes retained teardown inside the sizer seam before returning B’s bytes. |
| + |
| +The new test at `C/economicspass_test.go:1535` places teardown on the record before collection begins, so it misses this schedule. |
| + |
| +### F3. P2, latent — An admitted queue drain returns another account’s object metadata after teardown |
| + |
| +**Where:** `O/queue.go:276`, `:314`, `:329`, `:420`, `:437`, `:461`, `:467`. |
| + |
| +**Scenario:** |
| + |
| +1. A queues a known key in shared bucket X. |
| +2. `DrainApp` passes `queuePermitted`. |
| +3. A completes retained teardown; B replaces that key. |
| +4. The drain resumes and calls `sc.Status` using its saved bucket. |
| +5. It returns B’s version, ETag, size and verdict in A’s `Drained` result. |
| + |
| +Existing-verdict and successfully recorded scan paths return without another permission check. Even the fallback path, which does recheck, returns `out, nil` when permission is false without a read error. Its caller then appends the result as successful. |
| + |
| +**Reachability limitation:** No production caller of `DrainApp` was found in this snapshot. This is a defect in the callable drain API; current member disclosure, callback delivery or billing through it is not established. |
| + |
| +**Smallest acceptable fix:** Recheck permission before each item and before accepting its result. Permission withdrawal must suppress the result and return a distinct stand-down outcome, including the fallback path. Test teardown between admission and object status with replacement bytes. |
| + |
| +## 3. Claims the code contradicts |
| + |
| +| Location | Claim | What the code supports | |
| +|---|---|---| |
| +| `S/CODEX-DISPOSITIONS-carloku-paid-launch-overnight.md:466`; `D/OUTSTANDING.md:74` | The unswept store holds that day’s storage for its account, for as long as the retained record stands. | One unswept entry removes storage completeness for every account. Persisted affected months remain held after the record disappears: F1. | |
| +| `D/AGENT_HANDOFF.md:35`; dispositions `:466` | The daily storage pass holds a teardown store instead of sizing its bucket. | True when the initial read sees teardown. A pass already admitted still measures and attributes post-teardown contents: F2. | |
| +| `D/AGENT_HANDOFF.md:35`; dispositions `:465` | A running sweep publishes nothing after teardown begins; status shows no coverage. | `writeSidecar` checks before writing, and its CAS retry does not reread the record (`O/sweep.go:403`, `:418`, `:439`). Status checks the caller’s earlier record (`C/apiscan.go:228`, `:97`). Teardown overtaking those reads can still permit publication or historical coverage. This does **not**, by itself, demonstrate a fresh cross-account disclosure. | |
| + |
| +The corrections concerning mark placement, legacy marks, repair limitations and the AWS-only observation are supported. The acknowledged limitations in 19f, 19g and 19h are not re-reported as new findings. |
| + |
| +**Verdict: reject.** |
| \ No newline at end of file |
diff --git a/docs/superpowers/specs/CODEX-REVIEW-BRIEF-carloku-paid-launch-overnight-round17.md b/docs/superpowers/specs/CODEX-REVIEW-BRIEF-carloku-paid-launch-overnight-round17.md| new file mode 100644 |
| index 000000000..4aedf4b8b |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-REVIEW-BRIEF-carloku-paid-launch-overnight-round17.md |
| @@ -0,0 +1,26 @@ |
| +# Review brief, round 17: Carloku paid launch, overnight changes of 2026-10-01 |
| + |
| +Read the round 1 brief (`CODEX-REVIEW-BRIEF-carloku-paid-launch-overnight.md`) for the fixed rulings and the output rules, the round 3 to 16 briefs for rulings 6 to 12, then `CODEX-DISPOSITIONS-carloku-paid-launch-overnight.md` (its last section covers round 16) and `implementation/carloku-paid-launch/OUTSTANDING.md`. |
| + |
| +Review the commit named in the prompt. Read source only with `git show <commit>:<path>`. The changes since round 16 are `git diff abc9f781 <commit>`. The code changes are in `internal/console/economicspass.go` and its test, and one test in `internal/objstore`. The rest is documents. |
| + |
| +Still the fixed snapshot: round 16's findings, nothing else. Nothing may attest. The quantity worker has two switches, both unset. Billing is off. |
| + |
| +Do not report again, as findings, what `OUTSTANDING.md` lists as open: items 6 and 6a, 7 and 9, 19a to 19i. Do report anything that makes one of them worse than the document says, and any claim about them the code contradicts. |
| + |
| +## First: a disposition table |
| + |
| +For every round 16 finding (Astra F1 to F3; Opus N1 to N4): closed, partly closed, not closed, or accepted and left open as the dispositions say. File and line of the fix and of the test, and whether the test would pass without the fix. |
| + |
| +## Then: new findings only |
| + |
| +- **The per-account storage hold** (`sweepUsage`, `usageSweep.StorageExcluded`, `accountSourceDays`). The claim: a store that could not be measured holds storage for its own account and for nobody else, and an entry whose account cannot be told holds it for everyone. Find an account whose statement reads storage complete while one of its stores was not measured. Find a view or a fold that still reads the shared count where the account's own count is the right one (the platform's own statement, the economics view, the billing snapshot). |
| +- **The re-read after sizing** (`storagePass`). Find a size recorded for a store whose teardown began during the listing. |
| +- **Claims** in `AGENT_HANDOFF.md` item 5, `OUTSTANDING.md` 19, 19a, 19f, 19i, and the round 16 dispositions. This review is of the diff only; unchanged text needs no second reading. |
| + |
| +## What to return |
| + |
| +1. The disposition table. |
| +2. New findings, most severe first, in the round 1 format. |
| +3. Claims the code contradicts. |
| +4. A one-line verdict: accept, accept with fixes, or reject. |
diff --git a/docs/superpowers/specs/OPUS-REVIEW-carloku-paid-launch-overnight-round16.md b/docs/superpowers/specs/OPUS-REVIEW-carloku-paid-launch-overnight-round16.md| new file mode 100644 |
| index 000000000..081ce7f1a |
| --- /dev/null |
| +++ b/docs/superpowers/specs/OPUS-REVIEW-carloku-paid-launch-overnight-round16.md |
| @@ -0,0 +1,109 @@ |
| +# Opus production and authority review, round 16: Carloku paid launch, `aed42815..abc9f781` |
| + |
| +**Verdict: accept with fixes.** All five round 15 findings are closed for the shape reported. One new P1: the storage hold added this round is applied to every account in the deployment, not just the account whose store is being torn down. A teardown the member walks away from, or any retained one, therefore makes every customer's storage meter permanently unbillable for that month. Nobody is overcharged, but the storage meter is stuck for the whole fleet. There is also one P2 and three P3 findings. |
| + |
| +I read everything with `git show`, changed nothing, ran no tests and made no network calls. Every "fails without the fix?" answer comes from reading the test against the code. |
| + |
| +Abbreviations: **O** = `internal/objstore`, **C** = `internal/console`, **D** = `implementation/carloku-paid-launch`, **S** = `docs/superpowers/specs`. |
| + |
| +## Production authority |
| + |
| +- **Nothing attests.** `SetAttestation` and `SetAllClientsInstrumented` (`usage/durabilityledger.go:1455`, `:1475`) still have no caller outside tests. |
| +- **The quantity worker cannot reach Stripe with either switch unset.** |
| + - It needs a proration policy (`C/billingquantity.go:119-122`). |
| + - It also needs `QuantityFence == "stripe-sandbox-verified"` (`:81`). `cmd/console/billing.go:40` rejects any other non-empty value. |
| +- **Files unchanged since my round 9 to 15 readings:** `billingquantity.go`, `billingservice.go`, `internal/billingstripe`, `cmd/console`, `walklock.go`, `fence.go`, `aws.go`, `internal/sigfence` and `internal/usage`. Those readings stand: |
| + - The fence has been observed on AWS only (19d). |
| + - `TestSandboxQuantityFence` has still not been run, and no document says it has. |
| +- **Changes to walk release.** `walkEndedClean` now also treats `ErrNotProvisioned` as a clean end (`O/walk.go:1356`). I checked every walk that returns it: grant at `:260`, before any provider call; rotate at `:710` and `:827`, before the lease; `leaseForRotation` at `:879`, which releases clean itself. None returns it after a provider call. The `doubt` latch still applies (`walklock.go:225`). The fence is not weakened. |
| +- **Billing.** Off. New N1 is a fleet-wide hold, not a wrong charge. N2 is a double charge that predates this round; it applies once storage is billed. |
| +- **Data.** I found no path that destroys a customer's data. A failed teardown keeps `Provisioned` until the final `deleteRecordIfUnchanged`, as before. |
| +- **Leaks.** I found no reachable leak of an identifier or a secret. N3 is a latent one in a queue drain that nothing in production calls. |
| + |
| +## 1. Disposition table (round 15) |
| + |
| +| Finding | Status | Fix | Test; fails without the fix? | |
| +|---|---|---|---| |
| +| **Astra F1**: a sweep in flight publishes coverage after a retained teardown | **Closed.** `writeSidecar` re-reads the record before every write and returns `errSweepTornDown` on a teardown block (`O/sweep.go:411-413`). `RunSweep` then hands back `nil, nil` and lists no further page (`:305-307`); `markNotApplicable` does the same (`:375-377`). `scanReply` drops the sidecar for a teardown record (`C/apiscan.go:97-99`). That covers the status, grant, revoke, enforce and sweep doors, because each passes a record it has just read. Residual: one round trip between `writeSidecar`'s `Get` and `PutSidecar`. A write in that window carries only pages listed before the block landed, and the status door hides it anyway. | `sweep.go`, `apiscan.go` | `TestSweepInFlightDoesNotPublishAfterATeardownBegins` (`O/sweep_test.go:575`): **fails.** The stored sidecar would carry `Current.Objects=2`, and `calls` would exceed 1. Scanning is fully operational in this fixture: with antivirus off, `RunSweep` would error before listing and `calls` would be 0, so Astra's round-15 objection does not apply here. `TestScanStatusShowsNoCoverageForAStoreUnderTeardown` (`C/apiscan_test.go:761`): **fails.** | |
| +| **Astra F2 / Opus N1**: the storage pass sizes a retained bucket for the store that left it | **Closed for the reported shape.** A teardown record is not sized (`C/economicspass.go:211-219`). The hold it uses instead has the wrong scope: new N1. | `economicspass.go` | `TestStoragePassDoesNotSizeAStoreUnderTeardown` (`C/economicspass_test.go:1535`): **fails** (`sized==1`). It does not check which accounts the hold reaches. | |
| +| **Opus N2**: `Regrant` can mint a first store | **Closed.** `noTakeBack && rec.Provisioned == nil` returns `ErrNotProvisioned` on the record read under the lease, after the estate check and before any write (`O/walk.go:259-261`). The lease is released clean (`:1356`). | `walk.go` | `TestRegrantNeverMakesAFirstStore` (`O/ownership_test.go:3639`): **fails** both ways. Without the check, the provider gets users. Without the `walkEndedClean` change, the name stays reserved. | |
| +| **Opus N3**: the scan grant door checks for a teardown and enables scanning as two steps | **Accepted and left open** as 19f (`D/OUTSTANDING.md:69`). The code still matches what 19f says (`C/apiscan.go:367-371`). | — | none | |
| +| **Astra, noted**: `Put` would marshal `LegacyDelivering` | **Closed in code.** `Put` moves the mark to `Record.Delivering` (`O/objstore.go:276-279`). | `objstore.go` | The named reproduction **would pass without the fix** (§3). | |
| +| **Claims**: the `Delivering` doc, "for display", "an earlier build carries no mark", "resumed by running it again", the test comment, 19h | **Corrected.** `O/objstore.go:158-170`, `O/walk.go:1571`, `D/AGENT_HANDOFF.md` item 5, `C/apiscan_test.go:733-736`, `D/OUTSTANDING.md:68`. | — | — | |
| + |
| +## 2. New findings, most severe first |
| + |
| +### N1. P1 (stuck meter, not a wrong charge): the teardown hold makes storage incomplete for every account, permanently for that month |
| + |
| +**Where:** `C/economicspass.go:211-219` sets `StoreUnswept`. Any `StoreUnswept` entry on a day sets `storageOK = false` for the whole day (`:858-866`). `SourceDays[SourceStorage]` is one count for the whole deployment (`:918-920`). `accountSourceDays` adjusts only durability per account (`:1674-1683`). `MeterComplete` then fails `storage_gb_month` for every account (`usage/statement.go:176`). |
| + |
| +**Scenario:** |
| +1. A member destroys a large store. `storeTeardownTimeout` is 60 s (`C/apistore.go:438`), so the paginated `EmptyBucket` times out. The record keeps `Provisioned` and its teardown block (`O/walk.go` step order). The member sees "try again" and does not. |
| +2. Alternatively, an operator-adopted shared bucket is left to another account. Nothing ever re-runs that teardown (19a). |
| +3. Each daily storage pass writes that app as unswept. The storage day object is create-only, so the day can never be corrected. |
| +4. Every account's `storage_gb_month` is incomplete for that month. |
| +5. Storage incompleteness does not stop the month from closing; only store egress has `HoldClose`. The month closes, and `BillingSnapshot` refuses storage for every account for that month, with no rebuild door (item 18). |
| +6. A retained record repeats this every month, indefinitely. |
| + |
| +**Ordinary teardowns (the brief's question):** duration is not the trigger. Any teardown block present when the pass runs triggers the hold, even one that runs for seconds over that instant. Before this round, such a store was sized, and its bytes were still that member's. The exception was a bucket already deleted, which failed sizing and held fleet-wide even then. |
| + |
| +**Smallest fix:** make the storage hold per account, the way `DurabilityExcluded` works. Count the unswept days per account in `sweepUsage` and subtract them in `accountSourceDays`. Keep the fleet-wide failure only for an entry whose account cannot be resolved. Add a test: a teardown record for account A leaves account B's storage meter complete. |
| + |
| +### N2. P2: a live shared bucket is sized in full for each store on it |
| + |
| +**Where:** `C/economicspass.go:220-228` and `sizeProvisionedStore` (`:1867-1886`). The sizer lists the whole bucket with no prefix or owner split. |
| + |
| +**Scenario:** |
| +1. An operator adopts bucket X for A (account 1) and B (account 2), both live. |
| +2. Each daily pass writes the full size of X as A's `StoreBytes` and again as B's. |
| +3. Once storage is billed, both accounts are charged for all of X. |
| + |
| +This breaks rulings 2 and 4: an ambiguous attribution is measured rather than held. It predates this branch. Round 15 Astra F2 noted it only for the retained case. The new comment ("the bucket's bytes are that account's now", `:213-215`) implies the bytes were A's alone before the teardown, which they were not. |
| + |
| +**Smallest fix:** in `storagePass`, hold (per account, as in N1) any store whose bucket another store holds, by the ledger or by the records (the same holder inventory teardown uses). Size only a sole holder. Add a two-store test. |
| + |
| +### N3. P3 (latent): the queue drain does not re-check for a teardown before each item; my round 15 table said it did |
| + |
| +**Where:** `O/queue.go:276` checks `queuePermitted` once per batch. `drainOne` (`:417-476`) calls `sc.Status` and `sc.Scan` and returns `Drained` (key, version, ETag, size, verdict) for each group. It re-checks only on the result-object fallback (`:467`). |
| + |
| +**Scenario:** a batch is admitted, then a retained teardown completes. B overwrites a queued key. `drainOne` returns B's object's ETag, size and verdict for A's callback paths. |
| + |
| +**Why latent:** `DrainApp` has no production caller and nothing enqueues. My round 15 statement that "`drainOne` checks `queuePermitted` again before each item" was wrong. |
| + |
| +**Smallest fix:** call `queuePermitted` before each group's `Status`, and stop the batch, returning nothing, when it fails. Alternatively, add this to 19f so it is fixed before the queue is wired. |
| + |
| +### N4. P3: the `Put` migration has no test that exercises it |
| + |
| +`TestTheMarkIsReadFromWhereAnEarlierBuildWroteIt` (`O/ownership_test.go:3352`) reaches `Put` only through `Get`/`Inspect`, which have already cleared `LegacyDelivering`. The new branch at `O/objstore.go:276-279` is never reached, and the dispositions' mutation list does not include it. |
| + |
| +**Smallest fix:** add one case that calls `Put` with `Grant.LegacyDelivering=true`, then asserts that the raw JSON has a top-level `delivering` and none under `grant`. |
| + |
| +### Asked and not found |
| + |
| +- **Another reader of a retained bucket:** none reachable. |
| + - Settings and `storeReply` show only the record (bucket name, a revoked key ID). |
| + - The store egress fold uses ledger intervals, and A's interval is closed. |
| + - The economics view folds the held storage days. |
| + - The scan pass and `sweepNow` stop at `RunSweep`'s entry check. |
| + - The queue (N3) is unwired. |
| + - Up to one in-flight page (1000 objects) of B's objects may still be scanned and tagged after the block lands. The tags are merged, not replaced (`clamav.go:430-498`), nothing is published to A, and the same happens while both stores share the bucket live. |
| +- **A scanning door that `Regrant`'s new refusal leaves unable to converge a store:** none. |
| + - `regrantForScanning` already returns early when `Provisioned` is nil (`C/apiscan.go:574`). The new refusal applies only to a record destroyed and declared again between that read and the walk. |
| + - The scan grant door requires `Provisioned` (`:332`). |
| + |
| +## 3. Claims the code contradicts |
| + |
| +| Location | Claim | What the code supports | |
| +|---|---|---| |
| +| `S/CODEX-DISPOSITIONS…` round 15, Astra F2 row | "holds that day's storage for the account" | It holds that day's storage for every account in the deployment (N1). | |
| +| `D/OUTSTANDING.md:74` (19a) | "its storage is held each day, not measured" | It is held for every account, every month the record stands. And it is not only retained teardowns: any teardown block present at pass time holds the day (N1). | |
| +| `C/economicspass.go:216-217` (comment) | "the day is held for it" | Held for everyone. | |
| +| Dispositions round 15, Astra "noted" row | Reproduction is `TestTheMarkIsReadFromWhereAnEarlierBuildWroteIt` | That test passes without the `Put` change (N4). | |
| +| `D/AGENT_HANDOFF.md` item 5 | "a sweep already running does not publish after one begins" | One write can land after the block, carrying only what was listed before the block. Accurate in substance; "does not publish anything it listed after one begins" would be exact. | |
| +| `S/OPUS-REVIEW…round15.md` §1, N2 row (mine) | "`drainOne` checks `queuePermitted` again before each item" | It checks once per batch, plus on the fallback write (N3). | |
| + |
| +Nothing in `walklock.go`, `fence.go`, `aws.go`, the quantity worker's send lease, key fence or second switch, or `OUTSTANDING.md` items 6, 6a, 7, 9 and 19b to 19h is made worse by this round. 19a is understated (above). |
| + |
| +## 4. Verdict |
| + |
| +**Accept with fixes.** Nothing attests, the worker cannot reach Stripe with either switch unset, and nothing destroys data or leaks a secret. Every round 15 finding is closed. N1 (make the storage hold per account) should land before storage is billed, and N2 before any shared bucket is billed. N3, N4 and the claims are small. |
diff --git a/implementation/carloku-paid-launch/AGENT_HANDOFF.md b/implementation/carloku-paid-launch/AGENT_HANDOFF.md| index 04a612888..76ded750d 100644 |
| --- a/implementation/carloku-paid-launch/AGENT_HANDOFF.md |
| +++ b/implementation/carloku-paid-launch/AGENT_HANDOFF.md |
| @@ -32,7 +32,7 @@ When a review is approved, apply each string byte for byte: the site by replacin |
| 2. Console GET `/a/{sqid}/billing` (`billingpage.go`, `templates/billing.html`): current plan, owner-only chooser posting to the existing checkout door, `?country=` price preview computed from the plans terms, error and pending banners, portal button once a customer exists, read-only for members, byte-identical 404 for non-members, closed state when billing is off. `plans.Countries()` exposes the embedded country list. |
| 3. Durability ledger: the epoch read race is fixed (epoch and created-at are immutable after recover) with a `-race` regression. An unmetered replication fallback on a box whose ledger opened now files an escape (`DurabilityLedger.Escaped`): the box-day is incomplete and the owner is excluded for the run. A box whose ledger failed to open has no publisher and files nothing. The console fold asks each box's published day to prove the union of that box's durability, metrics and edge windows; a box with no ledger directory, no day file, or a ledger whose coverage starts after the box began serving holds the day for everyone. There is no tolerance. Durability days are counted per account: a box-day attested with an exclusion set is complete for every account the box did not exclude. The move worker's checkpoint is recorded against the route's owner. Nothing attests: attestation is false everywhere, in this code and in production. The scoped attestation (path A) was built, reviewed in round six, found unable to prove an exclusive writer, and removed from this branch. It is kept on `feat/carloku-scoped-attestation-20261001` with its findings open. Replication-write completeness is a launch blocker, not a waived requirement. |
| 4. `internal/awsbilling`: Cost and Usage Report reader (versioned deliveries only), newest-delivery selection, digests, charged S3 transfer classifier. It holds on a charged usage line it does not count and on any non-usage line (fee, refund, credit, discount) with a cost on a transfer usage type. Every transfer usage type the flagship account wrote from July to September 2026 is pinned in a test. |
| -5. `internal/objstore/ownership.go`: append-only bucket ownership ledger, opened at grant, closed at teardown. It resolves an owner only for a ledger that claims completeness, and only for hours that begin at or after the instant it began vouching, so a store's partial first hour never resolves (a ledger that began vouching exactly on the hour answers for that hour). Completeness needs a grant block recorded as minting, nothing delivered to the app yet, a probe showing the bucket does not exist, and an inventory showing no principal's policy names the bucket (`GranteeProber`), and is dated when both probes had answered. The AWS provisioner cannot list principals yet, so on AWS no ledger claims completeness, new stores included, and store transfer holds. A minting grant is refused while another app holds the name by ledger interval or by store record, and for a bucket that already exists and it never held. Grant, teardown and both rotation verbs are walks. A walk that spends backend authority takes a lease (`walklock.go`) on the bucket name, or on a hash of it when the name is one this build would refuse, or on a hash of the app when the record names no bucket: an object taken by compare-and-swap, bound to the invocation, thirty minutes long, standing until it runs out if the walk failed or any of its requests needed a second attempt. The record is read again under the lease and must still be the same estate, not only the same name: teardown and grant compare the grant instant, the key, how many grants have reached the provider, whether a teardown had begun, whether it noted the bucket deleted and how many take-backs it has seen; rotation compares the same estate and requires that no teardown has begun. Every grant is counted on the record before any call that changes anything at the provider, the user and its policy included (two read-only probes, of the bucket and of who can reach it, come first), so a healthy store granted again writes its record once. A grant marks the record before it changes the user's keys, and a rotation just before it delivers the new key (`Record.Delivering`); each takes the mark off when it records the result. A rotation that fails before delivery leaves no mark and is resumed by running it again, with the app's key kept (unless the write of the mark itself was lost in transit and landed, in which case the mark stands, which errs toward the repair). One whose delivery fails keeps the mark, because a failed write may still land. While the mark stands a grant run again walks in full and a rotation is refused (`ErrGrantUnfinished`). The mark does not save the key the app was holding: the repairing grant reaps and replaces it. A record from a build before the mark existed carries none, which means not recorded, not finished; the one earlier build of this branch that kept the mark inside the grant block is read. An older grant block that names no bucket is made to name it in the write that counts the walk. A record's bucket is the one it is provisioned on, else the one its grant block names: the arguments choose a bucket only for a store that has none, so a grant never moves a store to another name. A record whose grant block and provisioned block name different buckets (an earlier build could write one), or whose teardown note is about a bucket that is not its own, is refused by the record's one reader and one writer (`objstore.Get` and `objstore.Put`; the check is `objstore.Coherent`; `ErrTwoBuckets`, `ErrUnfinishedTeardown`), so no door, pass or walk can act on one or write one. The holder inventory reads such a record anyway (`objstore.Inspect`) and counts it as a holder of every bucket it names, whatever the ledger shows for that app. An operator repairs such a record by hand; there is no verb for it. A take-back over a bucket that a store of another account holds is refused unless an operator grants it; a store whose ledger interval is closed does not count as holding it; a scanning door never changes a store whose teardown has begun, its re-grant (`objstore.Regrant`) refuses one under the walk's own lease and never makes a first store, the scan pass does not sweep or queue one, a sweep already running does not publish after one begins, the scan status shows no coverage for one, and the daily storage pass holds it instead of sizing its bucket. A failed walk's lease stands until it runs out, with one exception: the same app re-running the same kind of walk may take it at once. A declaration-only teardown and the app purge take no lease; they delete the store record only as the generation they read. The user belongs to the app and the lease to the name, so a late request of a failed teardown can still remove the policy of a store granted afterwards under another name (round eight, Opus N2, open). Every request of a walk, to S3, to IAM and to the deployment bucket that holds the records and ledgers, is signed with the instant its lease was taken (`internal/sigfence`), so AWS refuses a request a paused process sends after its lease could be taken over. The record and its scan sidecar are deleted as the generation that was read. The signing-time rule was checked read-only against S3 and IAM with a list and two gets (`TestFenceAgainstAWS`); no delete was sent. A declaration-only teardown takes no lease (there is no bucket name) and relies on the conditional delete alone. A teardown writes a block on its own record before it touches anything and decides by compare-and-swap on the ledger (for a bucket name this build would refuse there is no ledger, and it decides from the store records alone): if nobody else holds the bucket it marks the bucket as being destroyed; otherwise it leaves the bucket and its scanner, revokes its own credentials, and records that the scanner is owed. A grant by the same app over its own unfinished teardown takes the store back, runs in full, and voids the teardown's note that the bucket was deleted. A teardown that leaves a bucket to another account keeps its record and returns `ErrBucketRetained` (discussion 191). It can finish once that bucket has been destroyed, when someone runs it again; nothing runs it again by itself. For a bucket name this build would refuse there is no ledger: a store whose teardown has begun is not counted as a holder there, so the last of two sharers destroys the bucket. `ErrBucketRetained` has no member copy yet; it needs Paul's review. A randomized simulation (`walk_sim_test.go`) checks the invariants over sequential walks with provider and store faults, including failures after the effect, and walks frozen past their lease while another takes the name. It does not generate a second bucket name, rotations, stores with no ledger interval or disagreeing clocks; those have direct tests. |
| +5. `internal/objstore/ownership.go`: append-only bucket ownership ledger, opened at grant, closed at teardown. It resolves an owner only for a ledger that claims completeness, and only for hours that begin at or after the instant it began vouching, so a store's partial first hour never resolves (a ledger that began vouching exactly on the hour answers for that hour). Completeness needs a grant block recorded as minting, nothing delivered to the app yet, a probe showing the bucket does not exist, and an inventory showing no principal's policy names the bucket (`GranteeProber`), and is dated when both probes had answered. The AWS provisioner cannot list principals yet, so on AWS no ledger claims completeness, new stores included, and store transfer holds. A minting grant is refused while another app holds the name by ledger interval or by store record, and for a bucket that already exists and it never held. Grant, teardown and both rotation verbs are walks. A walk that spends backend authority takes a lease (`walklock.go`) on the bucket name, or on a hash of it when the name is one this build would refuse, or on a hash of the app when the record names no bucket: an object taken by compare-and-swap, bound to the invocation, thirty minutes long, standing until it runs out if the walk failed or any of its requests needed a second attempt. The record is read again under the lease and must still be the same estate, not only the same name: teardown and grant compare the grant instant, the key, how many grants have reached the provider, whether a teardown had begun, whether it noted the bucket deleted and how many take-backs it has seen; rotation compares the same estate and requires that no teardown has begun. Every grant is counted on the record before any call that changes anything at the provider, the user and its policy included (two read-only probes, of the bucket and of who can reach it, come first), so a healthy store granted again writes its record once. A grant marks the record before it changes the user's keys, and a rotation just before it delivers the new key (`Record.Delivering`); each takes the mark off when it records the result. A rotation that fails before delivery leaves no mark and is resumed by running it again, with the app's key kept (unless the write of the mark itself was lost in transit and landed, in which case the mark stands, which errs toward the repair). One whose delivery fails keeps the mark, because a failed write may still land. While the mark stands a grant run again walks in full and a rotation is refused (`ErrGrantUnfinished`). The mark does not save the key the app was holding: the repairing grant reaps and replaces it. A record from a build before the mark existed carries none, which means not recorded, not finished; the one earlier build of this branch that kept the mark inside the grant block is read. An older grant block that names no bucket is made to name it in the write that counts the walk. A record's bucket is the one it is provisioned on, else the one its grant block names: the arguments choose a bucket only for a store that has none, so a grant never moves a store to another name. A record whose grant block and provisioned block name different buckets (an earlier build could write one), or whose teardown note is about a bucket that is not its own, is refused by the record's one reader and one writer (`objstore.Get` and `objstore.Put`; the check is `objstore.Coherent`; `ErrTwoBuckets`, `ErrUnfinishedTeardown`), so no door, pass or walk can act on one or write one. The holder inventory reads such a record anyway (`objstore.Inspect`) and counts it as a holder of every bucket it names, whatever the ledger shows for that app. An operator repairs such a record by hand; there is no verb for it. A take-back over a bucket that a store of another account holds is refused unless an operator grants it; a store whose ledger interval is closed does not count as holding it; a scanning door never changes a store whose teardown has begun, its re-grant (`objstore.Regrant`) refuses one under the walk's own lease and never makes a first store, the scan pass does not sweep or queue one, a sweep already running does not publish after one begins, the scan status shows no coverage for one, and the daily storage pass holds it, for its own account only, instead of sizing its bucket (a size taken across the start of a teardown is discarded). A failed walk's lease stands until it runs out, with one exception: the same app re-running the same kind of walk may take it at once. A declaration-only teardown and the app purge take no lease; they delete the store record only as the generation they read. The user belongs to the app and the lease to the name, so a late request of a failed teardown can still remove the policy of a store granted afterwards under another name (round eight, Opus N2, open). Every request of a walk, to S3, to IAM and to the deployment bucket that holds the records and ledgers, is signed with the instant its lease was taken (`internal/sigfence`), so AWS refuses a request a paused process sends after its lease could be taken over. The record and its scan sidecar are deleted as the generation that was read. The signing-time rule was checked read-only against S3 and IAM with a list and two gets (`TestFenceAgainstAWS`); no delete was sent. A declaration-only teardown takes no lease (there is no bucket name) and relies on the conditional delete alone. A teardown writes a block on its own record before it touches anything and decides by compare-and-swap on the ledger (for a bucket name this build would refuse there is no ledger, and it decides from the store records alone): if nobody else holds the bucket it marks the bucket as being destroyed; otherwise it leaves the bucket and its scanner, revokes its own credentials, and records that the scanner is owed. A grant by the same app over its own unfinished teardown takes the store back, runs in full, and voids the teardown's note that the bucket was deleted. A teardown that leaves a bucket to another account keeps its record and returns `ErrBucketRetained` (discussion 191). It can finish once that bucket has been destroyed, when someone runs it again; nothing runs it again by itself. For a bucket name this build would refuse there is no ledger: a store whose teardown has begun is not counted as a holder there, so the last of two sharers destroys the bucket. `ErrBucketRetained` has no member copy yet; it needs Paul's review. A randomized simulation (`walk_sim_test.go`) checks the invariants over sequential walks with provider and store faults, including failures after the effect, and walks frozen past their lease while another takes the name. It does not generate a second bucket name, rotations, stores with no ledger interval or disagreeing clocks; those have direct tests. |
| 6. Statement folds: `console/durabilityfold.go` (`durability_writes_1k`, policy in `usage.BillableReplicationWrites`) and `console/storeegressfold.go` (`store_egress_gb`). `usage.MeterSource` names a source for both meters. Month evidence pins ownership ledgers by their month projection and knows the durability box directory. Statements carry the five billed meters by UTC day (`Statement.Days`), summing to the monthly totals; the other meters are not recorded by day. With a transfer report configured, the statements catch-up walks a 36-month window and sweeps every month that has an open statement, is missing a statement an account is owed (the platform's included), or is a gap between metered months. An account's statement that will not parse, or that cannot be read three passes running, is reported once per process and the month is classified without it. The platform's own statement is different: while it cannot be read the month stays unknown, for as long as that lasts. A month seen with no statement is remembered for six hours, a closed statement for the life of the process. |
| 7. Operator API doors (`billingoperator.go`): list and resolve billing review holds, list and set the house marker. No CLI verb yet. |
| 8. Team quantity worker (`billingquantity.go`) behind two switches, both unset: `CONSOLE_BILLING_PRORATION` (`immediate` or `next_invoice`, Paul's decision) and `CONSOLE_BILLING_QUANTITY_FENCE=stripe-sandbox-verified` (set only after `TestSandboxQuantityFence` has passed against a Stripe test-mode account; it has not been run). With either unset it makes no provider call. An operation is recorded by a compare-and-set on the billing row's revision, subscription, period, quantity and status, on the target and newest pending addition recomputed from the database, and on its own instant lying inside the period and not before the addition it charges for. The request dates its proration at that instant (`proration_date`). Sending and retiring exclude each other through a send lease on the operation's row, owned by one sender's token; an operation dated ahead of the observing clock is left alone. Every key ends in the account's operation serial. An operation nobody can show ran is acknowledged only after its idempotency key has been tried at the provider with a metadata-only request (the fence), or after a person has resolved the review it was held with: a resolved review stands in for the fence, on that person's word and with no provider check. If the fence request took the key, the operation never ran and may be retired, on a provider read taken after the fence when the subscription is current. If the key is held by the operation's own request, that request began and its outcome is not known: the operation is recorded as landed if the provider shows its target, and otherwise held with a review (`billing_quantity_operation_unsettled`) and never replaced until a person resolves it. A fence the provider refuses is held the same way. The fence rests on Stripe's documented key rules, is modelled by a fake in the tests, is not verified against Stripe, and lasts only as long as Stripe remembers a key (documented as at least 24 hours): `docs/superpowers/specs/2026-10-01-stripe-idempotency-evidence.md`. A quantity that goes down inside a period files `billing_quantity_lowered`. An operation is confirmed only by the provider's success answer to its own request. A confirmed landing takes effect at the operation's instant, so a seat's charge and its allowance start together; a quantity that merely matches an unconfirmed operation's target is recorded as `landed_unverified`, dated when observed, with a review. If the provider refuses a request and names `proration_date`, the operation's key is fenced at once and it is retired on the next read; if it does not name it, the operation waits out the 22-hour replay limit. None of the provider's side is confirmed: test mode is not authorized. Only additions of the subscription's current period can raise it; older ones are closed uncharged with a review; additions and operations of an ended subscription are closed. It does not yet lower the quantity at renewal. |
diff --git a/implementation/carloku-paid-launch/OUTSTANDING.md b/implementation/carloku-paid-launch/OUTSTANDING.md| index ed4576b39..f0dc1461c 100644 |
| --- a/implementation/carloku-paid-launch/OUTSTANDING.md |
| +++ b/implementation/carloku-paid-launch/OUTSTANDING.md |
| @@ -63,15 +63,16 @@ Each is concrete. None is waived. |
| ### Process |
| -19. **No review round has passed.** Astra rejected rounds one to fifteen; Opus accepted each with fixes. Every round has closed the shapes the round before reported and found neighbouring ones, in the store walks since round seven. Some concern records no walk of this build writes; others need only this build's ordinary walks (a failed rotation, a retained teardown, a scanning door racing a teardown), and two were faults in the previous round's own fixes. Round eleven found one P1 and two P2 in the store walks (a grant could move an older record to another bucket name; a waiting rotation ignored a grant that ran; the user and its policy were written before the grant was counted). Its fixes are on the launch branch (`1395f7bf`). Round twelve read them: every round-eleven finding is closed for its reported shape, Astra rejected again (one P1, two P2) and Opus accepted with fixes (one P2, three P3). The new findings were about records no walk of this build writes (a teardown note for another bucket, a record naming two buckets) being acted on by teardown, rotation and the scanning doors; a rotation started after a grant that failed at its last write; and a take-back that returned access to a bucket another account holds, through the member's own door. Round thirteen read those fixes: closed for their reported shapes, Astra rejected again (two P1, four P2) and Opus accepted with fixes (one P2, five P3). The new findings were paths that read such a record and acted without the check (the holder inventory, the storage pass, the declare door), a rotation's own failed last write, and a retained record blocking another account's take-back. The check now sits in the record's reader and writer. Round fourteen read those: closed for their reported shapes; Astra rejected (two P1, two P2), Opus accepted with fixes (two P2, two P3). Round fifteen read those: all closed; Astra rejected (one P1, one P2), Opus accepted with fixes (one P2, two P3), all about readers of a bucket that a retained teardown left to another account (a sweep in flight, the scan status, the storage pass). Those fixes are on the launch branch and unconfirmed until the next round reads them. Each round's fixes are on the launch branch, gated locally, and unconfirmed until the next round reads them. |
| +19. **No review round has passed.** Astra rejected rounds one to sixteen; Opus accepted each with fixes. Every round has closed the shapes the round before reported and found neighbouring ones, in the store walks since round seven. Some concern records no walk of this build writes; others need only this build's ordinary walks (a failed rotation, a retained teardown, a scanning door racing a teardown), and two were faults in the previous round's own fixes. Round eleven found one P1 and two P2 in the store walks (a grant could move an older record to another bucket name; a waiting rotation ignored a grant that ran; the user and its policy were written before the grant was counted). Its fixes are on the launch branch (`1395f7bf`). Round twelve read them: every round-eleven finding is closed for its reported shape, Astra rejected again (one P1, two P2) and Opus accepted with fixes (one P2, three P3). The new findings were about records no walk of this build writes (a teardown note for another bucket, a record naming two buckets) being acted on by teardown, rotation and the scanning doors; a rotation started after a grant that failed at its last write; and a take-back that returned access to a bucket another account holds, through the member's own door. Round thirteen read those fixes: closed for their reported shapes, Astra rejected again (two P1, four P2) and Opus accepted with fixes (one P2, five P3). The new findings were paths that read such a record and acted without the check (the holder inventory, the storage pass, the declare door), a rotation's own failed last write, and a retained record blocking another account's take-back. The check now sits in the record's reader and writer. Round fourteen read those: closed for their reported shapes; Astra rejected (two P1, two P2), Opus accepted with fixes (two P2, two P3). Round fifteen read those: all closed; Astra rejected (one P1, one P2), Opus accepted with fixes (one P2, two P3), all about readers of a bucket that a retained teardown left to another account (a sweep in flight, the scan status, the storage pass). Round sixteen read those: all closed, and one of them had made a single store under teardown hold storage for every account. Astra rejected (one P1, two P2), Opus accepted with fixes (one P1, one P2, two P3). The storage hold is now per account; two findings are left open as 19f and 19i. Those fixes are on the launch branch and unconfirmed until the next round reads them. Each round's fixes are on the launch branch, gated locally, and unconfirmed until the next round reads them. |
| 19g. **Refusals whose doors say the wrong thing, and records only a person can repair.** A record that names two buckets (`ErrTwoBuckets`) or whose teardown note is about another bucket (`ErrUnfinishedTeardown`) is refused by the record's one reader and one writer, so nothing acts on it and nothing writes it; the settings page shows such a store as unreadable. No walk of this build writes either; whether an earlier build left such records in production has not been checked. There is no operator verb to repair one. The doors that meet these refusals answer with sentences written for other failures: the grant and provision doors say to run the grant again, the destroy door and the purge say to try again, the scanning doors say to try again or to re-run the grant. The same goes for a rotation refused because a grant must be run again first (`ErrGrantUnfinished`) and for a member's take-back refused because another account holds the bucket (`ErrBucketHeld`). Correct copy for each needs approval (A). |
| +19i. **A live bucket shared by two stores is sized in full for each of them.** The daily storage pass lists the whole bucket for every store record that names it, so two accounts on one operator-adopted bucket would each carry all of its bytes. It predates this branch. Storage on a shared bucket must hold, not bill, until the pass uses the holder inventory and sizes only a sole holder. Not built. |
| 19h. **A rotation whose delivery fails is repaired by a grant, which swaps the app's key with no overlap.** A failed delivery may still land, so the record stays marked, the next rotation is refused, and the grant run again replaces whichever key the app holds. Hosts hold a dead key until they adopt the new config. Before this branch such a rotation was resumed by running it again, which kept the overlap when the delivery had not landed and broke the app when it had. A repair that waits out the signing window and then reads which key was delivered would keep the overlap where the window is enforced (observed on AWS only, 19d); it is not built. |
| -19f. **A store that leaves a bucket with an unsupported name does not hand on its scanner debt.** There is no ledger for such a name. The last holder would delete the bucket and leave a scanning plan behind. Today's scanner (ClamAV) has no plan to leave, so this is latent. Also latent for the same reason: the scan grant door checks that no teardown has begun and then enables scanning as two steps, so a retained teardown landing between them would leave scanning enabled on the other account's bucket with nothing recording that it is owed. |
| +19f. **A store that leaves a bucket with an unsupported name does not hand on its scanner debt.** There is no ledger for such a name. The last holder would delete the bucket and leave a scanning plan behind. Today's scanner (ClamAV) has no plan to leave, so this is latent. Also latent for the same reason: the scan grant door checks that no teardown has begun and then enables scanning as two steps, so a retained teardown landing between them would leave scanning enabled on the other account's bucket with nothing recording that it is owed. And the scan queue's drain checks permission once per batch, so a drain admitted before a retained teardown could return the other account's object metadata; nothing calls the drain or enqueues today, and it must be fixed before the queue is wired. |
| 19e. **A purge can end with a store that was granted during it.** A declare and a grant admitted while the app was live can land between the purge's last check and the removal of its marker. The purge now reports that as an error and never deletes the record, but it does not prevent it: the doors check liveness when they admit a request, not when they write. And after that error the estate is named only by its record and a log line. The app's console row is still trashed, so the settings page and the store-destroy door answer 404 and a second purge answers "not trashed", until the console restarts and restores the row from the record. The flash the member sees says "Nothing was deleted", which is false in this case; correct copy needs approval (A). |
| 19b. **Two round-eight findings are accepted and not fixed.** A late request of a failed teardown can remove the policy of a store the same app is granted afterwards under another name (the store goes dark; nothing is lost or leaked). A console whose clock is far ahead reserves a bucket name until that time, with no operator verb to clear it. |
| 19c. **S3's conditional delete is unverified on a real backend.** The store walks and the app purge delete a record only as the generation they read. That has run against the directory store and an HTTP stub. No `If-Match` delete has been sent to AWS or to an S3-compatible store, and a backend that ignored the header would delete unconditionally. It needs one write to a scratch object, which is a production-adjacent write nobody has approved. |
| 19d. **The signing-time fence has been observed on AWS only.** An S3-compatible deployment or store endpoint has not been probed for its window. |
| -19a. **A retained teardown finishes only when someone runs it again.** Nothing runs it. A store that left a bucket to another account keeps its record until then. Until then its scanning shows no coverage and its storage is held each day, not measured: the bucket's bytes are the other account's. |
| +19a. **A retained teardown finishes only when someone runs it again.** Nothing runs it. A store that left a bucket to another account keeps its record until then. Until then its scanning shows no coverage and its storage is held each day, for its own account only, not measured: the bucket's bytes are the other account's. A day written as held stays held for that account; there is no rebuild door (item 18). |
| 20. **Amadan CI has passed on `1395f7bf`, `d0a21aa1`, `d5705a63` and `973a9f15`.** It sat queued on every tip all night and first ran at 07:34 Dublin on 2026-10-01. Any later code tip needs its own run; the local gate (build, vet, gofmt, `make test`, cross builds) is not CI and is not offered in its place. An agent account can neither see nor manage the runners. The hub's `carlos/platform` repository also answered 500 on and off through the night; bundles of every tip are in the run directory. |
| 21. **A scanner backend that calls a provider must be fenced like the walks.** Today's scanner (ClamAV) makes no provider call at teardown. A native one would, and its client would need the fenced signer. |
diff --git a/internal/console/economicspass.go b/internal/console/economicspass.go| index 1efed6578..33af140da 100644 |
| --- a/internal/console/economicspass.go |
| +++ b/internal/console/economicspass.go |
| @@ -211,10 +211,11 @@ func (s *Server) storagePass(ctx context.Context, date string) error { |
| } else if rec != nil && rec.Provisioned != nil && rec.Teardown != nil { |
| // A store whose teardown has begun. If that teardown left |
| // the bucket to another account's store, the record stays |
| - // and the bucket's bytes are that account's now: sizing it |
| - // here would put them on this app's storage meter. What |
| - // part, if any, is still this app's cannot be told, so the |
| - // day is held for it, not measured. |
| + // and this app can no longer reach the bucket: sizing it |
| + // here would put the other store's bytes on this app's |
| + // storage meter. What part, if any, is still this app's |
| + // cannot be told, so the day is held for this account |
| + // (and only this account), not measured. |
| st.StoreProvisioned = true |
| st.StoreUnswept = true |
| } else if rec != nil && rec.Provisioned != nil { |
| @@ -223,6 +224,15 @@ func (s *Server) storagePass(ctx context.Context, date string) error { |
| if serr != nil { |
| log.Printf("console: storage pass: store %s/%s: %v", acct, app, serr) |
| st.StoreUnswept = true |
| + } else if after, _, aerr := objstore.Get(ctx, s.cfg.Releases, acct, app); aerr != nil || after == nil || |
| + after.Teardown != nil || after.Provisioned == nil || after.Provisioned.Bucket != rec.Provisioned.Bucket { |
| + // Sizing lists the whole bucket and takes time. If the |
| + // store's teardown began meanwhile (or the record can |
| + // no longer be read as the same store), what was |
| + // listed may be another account's objects: the size is |
| + // discarded and the day held for this account. |
| + log.Printf("console: storage pass: store %s/%s changed while it was being sized; held", acct, app) |
| + st.StoreUnswept = true |
| } else { |
| st.StoreBytes = n |
| } |
| @@ -487,6 +497,13 @@ type usageSweep struct { |
| // that account out (durabilityfold.go). The account's own statement |
| // carries SourceDays[SourceDurability] less this number. |
| DurabilityExcluded map[string]int |
| + // StorageExcluded counts, per account, the days SourceDays counts for |
| + // usage.SourceStorage on which a store of that account could not be |
| + // measured (StoreUnswept: sizing failed, or the store was being torn |
| + // down). Its zero is not a measurement, so that account's own |
| + // statement carries SourceDays[SourceStorage] less this number. Every |
| + // other account's storage for the day was measured and stays complete. |
| + StorageExcluded map[string]int |
| // EgressMetered marks every box that has EVER written a usage/edge day |
| // (discovered via ListDirs on usage.EdgeBoxesPrefix, independent of |
| // [from, to]) — a box that reports metrics but has no edge component at |
| @@ -580,6 +597,7 @@ func (s *Server) sweepUsage(ctx context.Context, from, to time.Time, routes []re |
| BoxDays: map[string]int{}, |
| SourceDays: map[string]int{}, |
| DurabilityExcluded: map[string]int{}, |
| + StorageExcluded: map[string]int{}, |
| EgressMetered: map[string]bool{}, |
| Census: map[string]dayCensus{}, |
| Storage: map[string]dayStorage{}, |
| @@ -638,6 +656,7 @@ func (s *Server) sweepUsage(ctx context.Context, from, to time.Time, routes []re |
| owner := dayOwnerFunc(census, routes) |
| covered := false |
| storageOK, censusOK := false, false |
| + var storageHeld map[string]bool |
| // A per-box source is complete for the day only when every candidate |
| // expected that day proved it watched its whole expected span. It |
| // starts false with no candidates at all: an empty deployment, or one |
| @@ -855,15 +874,25 @@ func (s *Server) sweepUsage(ctx context.Context, from, to time.Time, routes []re |
| if ds.Decoded { |
| covered = true |
| storageOK = true |
| - for _, as := range ds.Day.Apps { |
| - if as.StoreUnswept { |
| - // A store the pass could not size contributes a |
| - // zero that reads as "no bytes". The day is |
| - // covered for modelling, never complete for |
| - // billing. |
| + for key, as := range ds.Day.Apps { |
| + if !as.StoreUnswept { |
| + continue |
| + } |
| + // A store the pass could not size contributes a zero |
| + // that reads as "no bytes". The day is covered for |
| + // modelling and never complete for billing, for the |
| + // account whose store it is. Every other account's |
| + // stores were measured. An entry whose account cannot |
| + // be told holds the day for everyone. |
| + acct, _, ok := strings.Cut(key, "/") |
| + if !ok || !release.ValidAccount(acct) { |
| storageOK = false |
| break |
| } |
| + if storageHeld == nil { |
| + storageHeld = map[string]bool{} |
| + } |
| + storageHeld[acct] = true |
| } |
| } |
| @@ -917,6 +946,9 @@ func (s *Server) sweepUsage(ctx context.Context, from, to time.Time, routes []re |
| } |
| if storageOK { |
| sweep.SourceDays[usage.SourceStorage]++ |
| + for acct := range storageHeld { |
| + sweep.StorageExcluded[acct]++ |
| + } |
| } |
| if censusOK { |
| sweep.SourceDays[usage.SourceCensus]++ |
| @@ -1646,7 +1678,7 @@ func (s *Server) statementsForMonth(ctx context.Context, month string, start, en |
| Month: month, |
| Meters: mm, |
| CoveredDays: coveredDays, |
| - SourceDays: accountSourceDays(sourceDays, sweep.DurabilityExcluded[acct]), |
| + SourceDays: accountSourceDays(sourceDays, sweep.DurabilityExcluded[acct], sweep.StorageExcluded[acct]), |
| Days: sweep.PerAccountDay[acct], |
| DaysInMonth: daysInMonth, |
| UpdatedAt: now, |
| @@ -1669,17 +1701,23 @@ func (s *Server) statementsForMonth(ctx context.Context, month string, start, en |
| // accountSourceDays is the month's source days as one account's statement |
| // carries them: the shared counts, with the durability source reduced by |
| -// the days a box's scoped attestation excluded this account. An account no |
| -// box excluded gets the shared map itself. |
| -func accountSourceDays(shared map[string]int, durabilityExcluded int) map[string]int { |
| - if durabilityExcluded <= 0 { |
| +// the days a box's scoped attestation excluded this account, and the |
| +// storage source by the days a store of this account could not be |
| +// measured. An account with neither gets the shared map itself. |
| +func accountSourceDays(shared map[string]int, durabilityExcluded, storageExcluded int) map[string]int { |
| + if durabilityExcluded <= 0 && storageExcluded <= 0 { |
| return shared |
| } |
| out := make(map[string]int, len(shared)) |
| for source, days := range shared { |
| out[source] = days |
| } |
| - out[usage.SourceDurability] = max(0, out[usage.SourceDurability]-durabilityExcluded) |
| + if durabilityExcluded > 0 { |
| + out[usage.SourceDurability] = max(0, out[usage.SourceDurability]-durabilityExcluded) |
| + } |
| + if storageExcluded > 0 { |
| + out[usage.SourceStorage] = max(0, out[usage.SourceStorage]-storageExcluded) |
| + } |
| return out |
| } |
diff --git a/internal/console/economicspass_test.go b/internal/console/economicspass_test.go| index debb0fa2b..35ae63487 100644 |
| --- a/internal/console/economicspass_test.go |
| +++ b/internal/console/economicspass_test.go |
| @@ -1565,3 +1565,98 @@ func TestStoragePassDoesNotSizeAStoreUnderTeardown(t *testing.T) { |
| t.Fatalf("sized %d time(s), day entry %+v; want no sizing and an unswept store", sized, app) |
| } |
| } |
| + |
| +// A store that could not be measured holds storage for its own account and |
| +// for nobody else: every other account's stores were measured that day. An |
| +// entry whose account cannot be told still holds the day for everyone. |
| +func TestUnsweptStorageHoldsOnlyItsOwnAccount(t *testing.T) { |
| + read := func(t *testing.T, fx *consoleFixture, acct string) usage.Statement { |
| + t.Helper() |
| + raw, _, err := fx.rel.Get(context.Background(), usage.StatementKey(acct, "2026-08")) |
| + if err != nil { |
| + t.Fatalf("statement %s: %v", acct, err) |
| + } |
| + var st usage.Statement |
| + if err := json.Unmarshal(raw, &st); err != nil { |
| + t.Fatal(err) |
| + } |
| + return st |
| + } |
| + now := time.Date(2026, 8, 1, 23, 0, 0, 0, time.UTC) |
| + const other = "bcd" |
| + |
| + fx := newConsoleFixture(t) |
| + putStoreJSON(t, fx.rel, usage.StorageDayKey("2026-08-01"), usage.StorageDay{ |
| + V: 1, Date: "2026-08-01", |
| + Apps: map[string]usage.AppStorage{ |
| + fixtureAcct + "/" + fixtureApp: {BucketBytes: 1000, StoreProvisioned: true, StoreUnswept: true}, |
| + other + "/shop": {BucketBytes: 2000, StoreProvisioned: true, StoreBytes: 5000}, |
| + }, |
| + }) |
| + if err := fx.srv.statementsPass(context.Background(), now); err != nil { |
| + t.Fatal(err) |
| + } |
| + if got := read(t, fx, fixtureAcct).SourceDays[usage.SourceStorage]; got != 0 { |
| + t.Errorf("the account with the unmeasured store: storage source days = %d, want 0", got) |
| + } |
| + if got := read(t, fx, other).SourceDays[usage.SourceStorage]; got != 1 { |
| + t.Errorf("another account, fully measured: storage source days = %d, want 1", got) |
| + } |
| + |
| + fx = newConsoleFixture(t) |
| + putStoreJSON(t, fx.rel, usage.StorageDayKey("2026-08-01"), usage.StorageDay{ |
| + V: 1, Date: "2026-08-01", |
| + Apps: map[string]usage.AppStorage{ |
| + "not an account": {StoreProvisioned: true, StoreUnswept: true}, |
| + other + "/shop": {BucketBytes: 2000, StoreProvisioned: true, StoreBytes: 5000}, |
| + }, |
| + }) |
| + if err := fx.srv.statementsPass(context.Background(), now); err != nil { |
| + t.Fatal(err) |
| + } |
| + if got := read(t, fx, other).SourceDays[usage.SourceStorage]; got != 0 { |
| + t.Errorf("an unmeasured store whose account cannot be told: another account's storage source days = %d, want 0", got) |
| + } |
| +} |
| + |
| +// Sizing a store lists its whole bucket and takes time. A teardown that |
| +// begins while the listing runs can leave the bucket to another account's |
| +// store. The size that comes back is discarded and the day held. |
| +func TestStoragePassDiscardsASizeTakenAcrossATeardown(t *testing.T) { |
| + fx := newConsoleFixture(t) |
| + ctx := context.Background() |
| + at := time.Date(2026, 8, 10, 9, 0, 0, 0, time.UTC) |
| + rec := &objstore.Record{ |
| + Grant: &objstore.GrantBlock{Backend: "aws", Bucket: "carlos-store-shared-x", Source: objstore.GrantSourceOperator, At: at, By: "op"}, |
| + Provisioned: &objstore.Provisioned{Bucket: "carlos-store-shared-x", Region: "eu-west-1", IAMUser: "u", KeyID: "K", At: at}, |
| + } |
| + if _, err := objstore.Put(ctx, fx.rel, fixtureAcct, fixtureApp, rec, ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + fx.srv.storeSizer = func(context.Context, *objstore.Provisioned) (int64, error) { |
| + cur, etag, err := objstore.Get(ctx, fx.rel, fixtureAcct, fixtureApp) |
| + if err != nil || cur == nil { |
| + t.Errorf("fixture: %+v %v", cur, err) |
| + return 0, nil |
| + } |
| + cur.Teardown = &objstore.TeardownBlock{StartedAt: at} |
| + if _, err := objstore.Put(ctx, fx.rel, fixtureAcct, fixtureApp, cur, etag); err != nil { |
| + t.Error(err) |
| + } |
| + return 31 << 30, nil |
| + } |
| + if err := fx.srv.storagePass(ctx, "2026-08-12"); err != nil { |
| + t.Fatal(err) |
| + } |
| + raw, _, err := fx.rel.Get(ctx, usage.StorageDayKey("2026-08-12")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + var day usage.StorageDay |
| + if err := json.Unmarshal(raw, &day); err != nil { |
| + t.Fatal(err) |
| + } |
| + if app := day.Apps[fixtureAcct+"/"+fixtureApp]; !app.StoreUnswept || app.StoreBytes != 0 { |
| + t.Fatalf("day entry %+v; want the size discarded and the store unswept", app) |
| + } |
| +} |
diff --git a/internal/objstore/ownership_test.go b/internal/objstore/ownership_test.go| index d1249af7f..217bb764f 100644 |
| --- a/internal/objstore/ownership_test.go |
| +++ b/internal/objstore/ownership_test.go |
| @@ -3396,6 +3396,27 @@ func TestTheMarkIsReadFromWhereAnEarlierBuildWroteIt(t *testing.T) { |
| if strings.Contains(string(raw), "delivering") { |
| t.Fatalf("the repaired record still carries a mark:\n%s", raw) |
| } |
| + // And the writer never puts the mark back in its earlier place, even |
| + // for a record handed to it with that field set. |
| + rec, etag, err := Get(ctx, s, "bab", "titogo") |
| + if err != nil || rec == nil { |
| + t.Fatal(err) |
| + } |
| + rec.Grant.LegacyDelivering = true |
| + if _, err := Put(ctx, s, "bab", "titogo", rec, etag); err != nil { |
| + t.Fatal(err) |
| + } |
| + raw, _, err = s.Get(ctx, Key("bab", "titogo")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + var written map[string]any |
| + if err := json.Unmarshal(raw, &written); err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, nested := written["grant"].(map[string]any)["delivering"]; nested || written["delivering"] != true { |
| + t.Fatalf("the writer left the mark in its earlier place:\n%s", raw) |
| + } |
| } |
| // An older grant block that names no bucket (the name used to be derived |