amadan
Nineteenth replicator build: a tenant's file in a platform directory refused in both layouts, the long-target fixture within the pathname budget, refusal messages quoted
Review eighteen (Opus accept; Astra reject on the deep items) named: the fixture's 3 KB link target (XFS refuses one over 1,024 bytes) and its fixed depth (a long root could not build it): the target is 999 bytes and directories are added within the budget. A tenant's file in a platform- owned directory was refused under the platform's tree and accepted under the tenant's own: the start check now refuses it in both. Refusal messages quote plain entries as render does. The witness test gains a host rename. The link-target comment says any component is followed and what a swapped directory can still do. OUTSTANDING item 1 corrected (Opus rejected three builds; no crossing onto another filesystem rather than "no mount"). Dispositions "Review eighteen" with both reports; two mutants killed; ops and registry pass under /var/tmp/ck and ~/.cache/tmp.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
11 files changed,
+403
−36
docs/superpowers/specs/CODEX-FINDINGS-metered-replicator-review18.md+192 −0docs/superpowers/specs/DISPOSITIONS-metered-replicator.md+24 −7docs/superpowers/specs/OPUS-REVIEW-metered-replicator-review18.md+94 −0implementation/carloku-paid-launch/OUTSTANDING.md+1 −1internal/ops/linktarget_unix.go+6 −2internal/ops/litestream_metered.go+20 −3internal/ops/litestream_metered_test.go+4 −1internal/ops/litestream_owners.go+15 −10internal/ops/litestream_owners_test.go+14 −12internal/ops/litestream_owners_unix_test.go+29 −0internal/registry/observe_ownership_test.go+4 −0
CI passed — run details
diff --git a/docs/superpowers/specs/CODEX-FINDINGS-metered-replicator-review18.md b/docs/superpowers/specs/CODEX-FINDINGS-metered-replicator-review18.md| new file mode 100644 |
| index 000000000..542c41d0b |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-FINDINGS-metered-replicator-review18.md |
| @@ -0,0 +1,192 @@ |
| +| Review-sixteen finding | Closed at `6249af63`? | Would its test pass without the fix? | |
| +|---|---|---| |
| +| **Opus P3-1 — fleet versus tenant namespace** | **Closed as a wording request.** Namespace divergence remains explicitly open. | Wording only; no behavioral test. | |
| +| **Opus T-1 — aliased tree root** | **Closed.** The fixture includes `state -> private/state`, then `cur -> ../d`. | **No** with lexical joining; **yes** with the preceding uncleaned concatenation, which correctly handles this case. | |
| +| **Opus T-1 — rendering coverage** | **Closed for the reported mutations.** Owner fields, `Platform`, each list and section boundaries now have assertions. | **No** when restoring the raw owner triple, omitting `Platform`, writing lists raw or removing section separators. | |
| +| **Opus claims** | **Mostly corrected.** Some disposition descriptions remain contradictory, listed below. | Prose changes have no behavioral test. | |
| +| **Astra 1 — Litestream pathname opens** | **Open.** | Existing tests permit the defect; none controls substitution at Litestream’s actual open. | |
| +| **Astra 2 — replaceable ancestry** | **Open.** | Existing endpoint tests do not establish protected ancestry. | |
| +| **Astra 3 — delayed admission escapes holds** | **Open.** | No test delays an admitted attempt beyond the margin across midnight. | |
| +| **Astra 4 — process exit loses uncertainty** | **Open.** | No durable-transition recovery test. | |
| +| **Astra 5 — concatenation exceeds `PATH_MAX`** | **Closed in the Unix implementation on stable paths.** The eighteenth build also fixes the short-root fixture failure. A longer-root fixture limitation remains below. | **No** when restoring concatenation, provided fixture construction succeeds. | |
| +| **Astra 6 — header-open device effects** | **Open.** The separate `statTarget` FIFO/device-open instance is fixed. | Header tests do not exercise ancestor substitution. The new FIFO test fails with the old directory open. | |
| +| **Astra 7 — hibernating aliases and skipped-file claims** | **Open.** | No distinguishing identity-based conflict fixture. | |
| +| **Astra 8 — required coverage disappears** | **Open.** | Missing configured console databases and external-inventory reconciliation remain uncovered. | |
| +| **Astra 9 — whole-fleet refusal and holds** | **Open and acknowledged.** | Existing tests expect broad refusal/holding; they do not establish tenant isolation. | |
| +| **Astra 10 — route identity versus filesystem authority** | **Open.** | The coherent foreign-tree fixture explicitly expects acceptance. | |
| +| **Astra 11 — weak tests** | **Partly closed.** Reported rendering gaps are closed; shutdown, watch, interleaving and timing gaps remain. | **No** for the repaired rendering mutations; **yes** for the surviving counterfactuals below. | |
| +| **Astra claims** | Witness wording, rendering and the Accept test’s own comment are corrected. Historical disposition claims remain partly stale. | Reported test/mutation runs were not independently executed. | |
| + |
| +All locations below refer to **`6249af63`**. This reviews `72b4f59a..6249af63` and `bffc76b5..6249af63`. Source was read through `git show`; no files were changed, tests run or network commands issued. Test counterfactuals are deductions from source. |
| + |
| +1. **P1 — Litestream still opens files outside the ownership checks.** |
| + |
| + **Locations:** `internal/lease/meteredstore.go:127`; `internal/ops/litestream_owners.go:382`; `third_party/litestream/db.go:1004`, `:1014`, `:1032`, `:1866`. |
| + |
| + A tenant passes inventory, substitutes another tenant’s database during initialization, then restores the original pathname before the next look. Litestream can retain the substituted handles while subsequent inventories—including Close’s final look—see the original inode. Initialization also enables WAL and writes bookkeeping tables. Later WAL opens independently follow mutable paths. |
| + |
| + The open item is therefore larger than a swap during the first open: it includes initialization effects, retained handles, retries and subsequent sidecar opens. The current owner-check comment correctly acknowledges this. |
| + |
| + **Smallest acceptable fix:** constrain the actual SQLite, database and sidecar opens through verified, pinned directory authority, including subsequent opens. A comparison after an unrestricted open cannot undo its effects. |
| + |
| +2. **P1 — Protected endpoints still have replaceable ancestors.** |
| + |
| + **Locations:** `internal/ops/litestream_owners.go:512`, `:587`, `:823`, `:851`; `internal/ops/litestream_metered.go:970`. |
| + |
| + T owns `/srv/T`; root owns `/srv/T/bridge`; A owns `bridge/A` and its database. A platform-made link to `bridge/A` passes the immediate-parent checks, although T can rename and replace `bridge` through `/srv/T`. |
| + |
| + Equivalent layouts work above `dataDir`, shadow state and the directory holding an approved platform link. A platform link whose target’s **immediate parent** is tenant-owned is refused; the surviving gap is higher ancestry. |
| + |
| + **Smallest acceptable fix:** establish and pin trusted ancestry for claim roots, approved platform links and shadow state. Checking endpoint ownership and modes does not establish that property. |
| + |
| +3. **P1 — An admitted request delayed beyond the margin escapes the inherited hold.** |
| + |
| + **Locations:** `internal/ops/litestream_metered.go:240`, `:325`, `:772`; `internal/lease/requestgate.go:58`. |
| + |
| + Concrete UTC sequence: |
| + |
| + - A admits an attempt on October 2 at **23:58:00**, then pauses before recording. |
| + - Ownership changes at **23:58:10**. A detects it and holds through **23:59:10**. |
| + - B is built for the new owner and closes unopened while A remains live. |
| + - A’s admitted attempt resumes, records and completes at **00:02 on October 3**. |
| + - A closes. Already stale, it skips the final checks and does not extend its window. |
| + - C replaces B. B and C inherit A’s window correctly, but **October 3 is absent from their ownership holds**. |
| + |
| + `goStaleFrom` correctly chooses the earlier lower bound. Unopened-fleet inheritance correctly preserves the supplied window. Neither supplies the missing upper bound. |
| + |
| + **Smallest acceptable fix:** track admitted attempts through recording/completion and include their doubtful dates before finalizing handover. Increasing a fixed margin is insufficient. |
| + |
| +4. **P1 — Ownership uncertainty disappears across process exit.** |
| + |
| + **Locations:** `internal/ops/litestream_metered.go:346`, `:379`, `:817`. |
| + |
| + An attempt passes admission, ownership changes, and the attempt completes before invalidation. The process exits. Its successor reads the new owner without a predecessor or durable uncertainty covering that completed attempt. |
| + |
| + Routine restarts also discard inheritance awaiting delivery to replacement recorders. An unfinished ledger row covers a different case. |
| + |
| + **Smallest acceptable fix:** persist the ownership/admission uncertainty before affected requests escape, and recover it before establishing successor coverage. |
| + |
| +5. **P2 — The directory-only open still permits link classification against a substituted directory.** |
| + |
| + **Locations:** `internal/ops/litestream_owners.go:654`, `:683`, `:702`, `:709`; `internal/ops/linktarget_unix.go:41`, `:47`. |
| + |
| + The walker lists `D/cache` and captures `current`’s relative target. Before `statTarget` opens `D/cache`, the tenant replaces it with another directory—or a link to another directory—where that target is absent. `Fstatat` returns `ENOENT`, and a non-database-named link that originally required reporting disappears from the inventory. A replacement containing a regular file instead of the original directory can change an unconditional problem into an accept-list entry. |
| + |
| + `O_DIRECTORY` correctly rejects a FIFO, device or ordinary file before opening it as such. It does **not** establish that the descriptor names the directory originally listed. Another directory, a followed directory link, or a bind mount visible in the fleet’s namespace can still change classification. |
| + |
| + **Smallest acceptable fix:** list, inspect entries, read links and classify captured targets through the same safely acquired directory descriptor. Handle the approved systemd root link explicitly when acquiring the root. |
| + |
| +6. **P2 — Header inspection still permits device-open effects before rejection.** |
| + |
| + **Location:** `internal/ops/sqliteopen_unix.go:26`. |
| + |
| + The walker observes regular file `T/d/watchdog`. T replaces `d` with a link to `/dev`; the privileged open reaches `/dev/watchdog` on a suitably equipped box. The subsequent regular-file and inode checks cannot undo open-side effects. |
| + |
| + `O_NOFOLLOW` protects the last component only. `O_NONBLOCK` and `O_NOCTTY` address particular effects, not ancestor confinement. |
| + |
| + **Smallest acceptable fix:** constrain traversal through pinned directory descriptors before opening the target. Preserve the refusing non-Unix fallback. |
| + |
| +7. **P2 — Hibernating aliases and skipped-file claims still hide competing ownership.** |
| + |
| + **Locations:** `internal/ops/litestream.go:381`, `:479`, `:540`, `:709`; `internal/ops/litestream_owners.go:166`, `:241`. |
| + |
| + A live unit claims `/var/lib/carlos-app/u`, reached through its platform state link. Another account’s hibernating exec route names the same database as `/var/lib/private/carlos-app/u/main.db`. The hibernating spelling supplies no candidate, and textual claim matching misses the shared inode. |
| + |
| + Two unit claims on a tree containing only skipped files, or competing claims on `u/.cache/app.db` or `u/main.sqlite`, also bypass listed-file ownership reconciliation. Two hibernating claims can bypass it altogether. |
| + |
| + **Smallest acceptable fix:** reconcile claims by file identity against listed **and skipped** files, retaining claimant information for skipped entries. |
| + |
| +8. **P2 — Required coverage can still disappear without an inventory entry.** |
| + |
| + **Locations:** `internal/ops/litestream.go:507`, `:525`; `internal/ops/litestream_owners.go:643`, `:665`, `:720`. |
| + |
| + Set `CONSOLE_DB` to a missing file outside the globs and route claims. It disappears like the optional console path on a box without a console. “Configured but missing” is not distinguished from “not required.” |
| + |
| + Empty tree-only files and entries disappearing before their first inspection remain deliberate omissions. A database known only to the external replicator, outside these trees and explicit live claims, reaches no list. |
| + |
| + **Smallest acceptable fix:** distinguish required platform databases from optional ones, and reconcile against the external replicator’s inventory before cutover. |
| + |
| +9. **P2 — Ordinary tenant changes can refuse replacement replication or hold every account.** |
| + |
| + **Locations:** `internal/ops/litestream_owners.go:707`, `:712`; `internal/ops/litestream_metered.go:660`, `:863`, `:889`. |
| + |
| + Installing a virtual environment or packages introduces ordinary file and directory links. File links require acceptance; directory links are unconditional problems. The current fleet reports drift, but its replacement refuses. A later ownership change can leave every account waiting for that tenant’s inventory to be resolved. |
| + |
| + A legitimate concurrent rename detected by either inode comparison can also remove an owned file’s facts and hold every recorder. |
| + |
| + **Smallest acceptable fix:** implement tenant-scoped failures and holds, or explicitly adopt the whole-box policy with an operational recovery procedure before wiring. |
| + |
| +10. **P3 — The resized long-target fixture still fails under sufficiently long valid temp roots.** |
| + |
| + **Locations:** `internal/ops/litestream_owners_test.go:760`, `:764`, `:766`. |
| + |
| + The target arithmetic fixes the reported short-root failure. However, the fixture first adds **1,005 bytes** of directories unconditionally. With a resolved `platformDir` path of 3,200 bytes, those directories require a 4,205-byte pathname; construction fails before the target sizing or production behavior is exercised. |
| + |
| + Linked temp roots are resolved correctly, but their resolved length remains relevant. The “under any TMPDIR” claim is too broad. |
| + |
| + **Smallest acceptable fix:** add only the depth needed for the fixture’s length relationship, sizing both directory and target against the available pathname budget before creating them. |
| + |
| +11. **P3 — Route identity remains unbound to independently established filesystem authority.** |
| + |
| + **Locations:** `internal/ops/litestream_owners.go:176`, `:435`, `:495`, `:563`; `internal/ops/litestream_owners_unix_test.go:381`. |
| + |
| + Route A naming B’s coherent directory and file attributes B’s file to A. The fixture explicitly expects acceptance. |
| + |
| + Root-run units and tenants sharing the platform UID can likewise claim otherwise-unclaimed qualifying platform files. They cannot claim files “from anywhere”: directory modes, link rules, filesystem boundaries, regular-file status, link count and sidecars still constrain acceptance. |
| + |
| + **Smallest acceptable fix:** bind route claims to independently established roots and identities, with explicit treatment of trusted platform workloads. Deployment approval H remains outside this review. |
| + |
| +12. **P3 — Several tests still permit removal of the behavior they describe.** |
| + |
| + **Locations:** `internal/ops/litestream_metered_test.go:689`, `:807`, `:1062`, `:1380`, `:1496`; `internal/ops/litestream_owners.go:194`, `:217`. |
| + |
| + Surviving counterfactuals: |
| + |
| + - Clearing the store pointer and refusing requests does not establish that retained handles and workers stopped. Those shutdown assertions can survive removing `store.Close`. |
| + - Removing the watch’s `lookAgain` call leaves the direct-call file tests intact. |
| + - Removing admission’s pre-witness under-lock stale check is not independently distinguished by the post-witness fixture. |
| + - Removing the latch’s under-lock recheck survives sequential tests; racing successors are outside the documented caller contract. |
| + - Removing `prepareShadowState`’s data-directory check survives the fixture refused earlier by inventory. |
| + - Neither inode comparison has a controlled distinguishing interleaving. |
| + - The 300 ms negative Close check can pass before the second goroutine runs. The clean-start test reconstructs time from Unix nanoseconds and depends on wall-clock stability. |
| + |
| + The opened-store test checks positive counts, but **the asymmetric direct-client assertions at `litestream_metered_test.go:334` and `:337` do catch swapped fleet recorders**. |
| + |
| + **Smallest acceptable fix:** add controlled shutdown, watch and interleaving assertions; synchronize the second Close caller’s entry; use a controlled clock for boundary assertions. |
| + |
| +The remaining requested checks hold with these qualifications: |
| + |
| +| Area | Result at `6249af63` | |
| +|---|---| |
| +| **Tenant-file chain — A/J/P/U/JJ2** | `litestream_owners.go:464`, `:495`, `:500`, `:512` reject foreign tenant starting directories, tenant-made links and foreign stopping parents on stable paths. Every accepting `tenantFile` path reaches the top check. Protected platform directories are an exception: `/srv/A/bridge/a.db`, with A owning `A` and the file and root owning protected `bridge`, can climb to A and pass. The final comparison is against the chosen chain root. | |
| +| **Platform links and file-valued `DBPath` links** | A platform link whose target has a tenant-owned immediate parent fails `:851`; higher ancestry remains finding 2. A `DBPath` that is itself a file link fails `:435` or `:534`. Root and the process UID are equivalent at `:563`; neither is equivalent to a distinct tenant UID. Root/platform files directly under that tenant’s directory are refused, as is the reverse mismatch against a platform chain root. | |
| +| **Root-run units** | The climb stops at platform ownership rather than continuing to `/`. Qualifying root/platform files can now pass. This is the authority limitation in finding 11, not unrestricted acceptance. | |
| +| **Admission — B/I** | `litestream_metered.go:226` and `:240` cover staleness while waiting for the lock and during the witness read. Staleness after the final load remains the already-admitted-request gap. An already-running witness retry sequence can continue reading after a file finding, but the post-read check refuses admission. | |
| +| **Witness retries** | `witnessUnchanged` at `:257` makes at most three reads, accepts a recovered error, stops immediately on a witnessed change and treats exhausted errors as uncertainty. Three five-second contexts and two delays do **not** bound mutex waiting or filesystem `Stat`. | |
| +| **Registry generation — C/II2** | `registry.go:569`, `:602`, `:612` cover all six currently consumed columns, insert/delete and rowid immutability. Older-build raw statements fire the triggers. Same-value updates, unrelated columns and Litestream bookkeeping do not move it. Database-less insertions/deletions and semantically irrelevant changes to those six columns still move it conservatively. The renamed test describes that property more accurately; its table has no host-only update case. | |
| +| **Replacement chain — D/I/J/M** | `chainClosed` at `litestream_metered.go:420` checks a latched predecessor’s own closure first. Window publication, chain detachment and latch publication share `staleMu`; published chain reads take that lock. Within the single-chain contract, a replacement cannot open beside an open/draining predecessor, and an unopened successor cannot latch through one. Inheritance is delivered once per fleet, including intermediate recorders that must outlive Close. | |
| +| **Close’s final look — M** | Order at `:746`, `:750`, `:773`, `:776`, `:782`, `:783` is store stopped → watch stopped → file look → witness → inheritance → closed. A persistent changed/missing owned file on a current fleet holds from its last clean look. Already-stale fleets skip the checks. Harmless changes after replication stops can over-hold; filesystem operations can still delay Close. `closeOnce` prevents a second caller from publishing closure during draining. | |
| +| **Lineage — E** | `ownedReplicaKey` at `:948` combines account and cleaned path. Different valid accounts do not share lineage or shadow keys, including after renaming. A→B→A at one path resumes A’s prior lineage; changing app within one account does not create a new key. External-prefix separation remains unenforced. | |
| +| **Strict walk — F/K/L/N** | Extensions are recognized case-insensitively; uppercase `.DB` is skipped rather than streamed by tree discovery. Every nonempty regular file not recognized by name receives a header read, including sidecar-shaped names. `sidecarOf` is gone, so a linked base cannot suppress inspection. Missing/relative unit claims, units without `DBPath`, empty explicit claims and discovered files failing their final look are reported. Findings 7–8 bound completeness. | |
| +| **Every claimant — N/P/Q/S** | `claimChecks` at `litestream_owners.go:278` retains every distinct root/kind, including two trees, tree-plus-file and unit-plus-exec claims. Normal DynamicUser links, protected operator-selected files and isolated-instance directories can pass. Explicit live exec claims outside the globs receive checks without changing external config generation. Stable acceptance does not depend on check order. | |
| +| **One inode and duplicate facts — W/Z** | The first successful check’s facts are retained at `:193`; later checks must agree. The pass at `:217` compares those kept facts. Stable distinct files are not confused. Legitimate concurrent replacements can conservatively refuse and hold everyone; this is not an atomic snapshot. | |
| +| **Leading `..` — K** | `walkWithoutLinks` at `:765` permits systemd’s leading `../private/...` shape and refuses ascents after named components. It does not clean away an intervening link. Stable accepted paths resolve consistently within its checked assumptions; replaceable ancestry remains separate. | |
| +| **Captured targets — R/BB2/CC2/EE2/GG2** | On stable paths, `Fstatat(fd, dest, …, 0)` follows middle links and resolves `..` physically from the opened directory, including an aliased tree root. Absolute targets use `Stat` unchanged. The second `Readlink` is gone. Non-`ENOENT` target failures report the original path and quoted captured target; ordinary dangling non-database links remain omitted. Finding 5 limits directory identity. | |
| +| **Non-Unix fallback** | `linktarget_other.go:17` retains concatenation, but `filefacts_other.go:16` refuses ownership and `sqliteopen_other.go:13` refuses header reads. It does not enable this fleet on unsupported platforms. | |
| +| **Link entries and Accept — T/V/Y/DD2** | `linkEntry` at `litestream_owners.go:896` distinguishes every absolute plain path and path/target pair. Relative targets retain literal `Readlink` spelling. Cleaning a plain absolute path cannot forge the prefixed entry. The flat impostor fixture distinguishes the old unprefixed encoding. Unchanged accepted pairs remain accepted while classification stays unchanged. Target identity is not pinned. | |
| +| **Rendering — AA2/FF2/II2** | `litestream_metered.go:679`, `:686` quote strings, include `Platform` and preserve sections. Reported content is unambiguous, with list order normalized. Private inode facts are intentionally excluded and checked separately. Production compares rendering at `:666` and `:939`; it neither parses it nor uses it for Accept. The new cross-section assertions fail if separators are removed. | |
| +| **FIFO regression — GG2** | `linktarget_unix_test.go:35` requires `ENOTDIR` in an `open` `PathError`. Restoring `os.Open` blocks until the timeout; removing only `O_DIRECTORY` returns a stat-stage refusal and also fails. It establishes the open’s refusal, not directory identity across listing and reopening. | |
| +| **Fixtures and CI — H/L/X/HH2** | `platformDir` resolves linked temp roots; relevant modes are explicitly chmodded, preserving meaning under umask `077`. Finding 10 bounds long-root portability. Root skips unreadable-directory/target cases and the root-versus-platform-user distinction. Fake UID tests still exercise distinct owners through `factsOf`. `.amadan/ci.d/10-gate:52` and `Makefile:18` do not establish CI’s UID. | |
| +| **Other open items and reports — G/O** | A single same-filesystem bind alias remains undetected; two inventoried aliases are rejected by identity. The mount must be visible in the fleet’s namespace. External-inventory comparison remains absent. Both reviewers’ review-eight, review-nine and review-seventeen reports are present. | |
| + |
| +**Claims the code contradicts** |
| + |
| +- **“Under any TMPDIR”** — `litestream_owners_test.go:757` and dispositions `:342` exceed the fixture’s actual portability: target sizing is adaptive, directory depth is not. Finding 10 gives the counterexample. |
| +- **The undecided ownership case is not stated consistently.** The revised comment at `litestream_owners.go:553` correctly compares against the chain root. Dispositions `:255` still broadly says a tenant file in a platform-owned directory is refused. The protected `bridge` example above is accepted. The earlier `tenantFile` description at `:351` also overstates every directory having the file owner’s UID. |
| +- **The Accept fixture description remains stale in one row.** Dispositions `:283` still says the distinguishing repoint is through `x/..`; `litestream_metered_test.go:1894` uses `sub/./live.sqlite`. The test’s own comment and the qualified history at dispositions `:268` are corrected. |
| +- **Review-seventeen numbering is mixed.** Dispositions `:337` identifies that review’s Astra 5 and 6 as the directory open and fixture defects; `:347` then calls Astra 6 an open pathname-walk problem using the earlier numbering. The fixture defect is fixed for short roots; the header-open defect is review-seventeen finding 7. |
| +- **“Opus accepts each with fixes”** — `implementation/carloku-paid-launch/OUTSTANDING.md:35` contradicts the recorded rejections in dispositions’ first, second and eighth reviews. |
| +- **Rendering versus display:** quoted rendering prevents filename/newline collisions in drift comparison. Refusal messages at `litestream_metered.go:864`, `:890`, `:893` still join raw plain entries, so a newline-bearing filename can mislead displayed diagnostics. That is not an Accept-key collision. |
| +- **Recorded verification:** dispositions `:333` attributes the reported gate/CI failure to `bffc76b5`; `:349` reports package runs and says the full gate is next. Neither establishes a green full gate at `6249af63`, and this source-only review cannot certify one. |
| + |
| +**Verdict: reject.** |
| \ No newline at end of file |
diff --git a/docs/superpowers/specs/DISPOSITIONS-metered-replicator.md b/docs/superpowers/specs/DISPOSITIONS-metered-replicator.md| index 8a6e0ab12..5c6189488 100644 |
| --- a/docs/superpowers/specs/DISPOSITIONS-metered-replicator.md |
| +++ b/docs/superpowers/specs/DISPOSITIONS-metered-replicator.md |
| @@ -252,7 +252,7 @@ Reports: `OPUS-REVIEW-metered-replicator-review11.md`, `CODEX-FINDINGS-metered-r |
| | Opus P2-1, Astra 9 (P2): a link to a file was skipped by its own name alone, so a re-pointed accepted link read the same; a `Stat` error other than ENOENT was treated as nothing; and every link tenant software makes becomes an unaccepted skip that refuses the whole box's next build. | Fixed for the first two: a link entry is named with its target, so a re-pointed link reads as a change; a link whose target cannot be looked at is a problem. [Review twelve: `Accept` cleaned the whole entry as a path, so a target that folds or pops under cleaning passed as the accepted pair; the thirteenth build quotes the target and compares a link entry as written.] The third is stated in `strictTreeDBs`'s comment and here as a decision before wiring: whether one tenant's problems and unaccepted skips refuse the whole box or that tenant's databases alone; today they refuse the box, through `Accept`. | `litestream_owners.go` `strictTreeDBs` | `TestStrictTreeNamesALinkToAFile` (the target named; re-pointed reads differently; a target that cannot be looked at is a problem); each fails with its case removed (mutation runs) | |
| | Astra 6 (P2): a hibernating exec route over a hibernating unit's file dropped the file (the unit was not counted a live claim). | Fixed: a unit route's database is this fleet's whether or not the unit hibernates, so only a hibernating exec route is no live claim. | `litestream.go` `liveClaim` | `TestExplicitClaimsOutsideTheGlobsAreInventoried` (a hibernating unit under a hibernating exec claim: a problem); fails with units skipped when hibernating (mutation run) | |
| | Opus P3-3, Astra 7 (P2): a unit tree above a platform file was silently platform-owned; a hibernating path spelled through an alias, or on a skipped file, is unseen. | Fixed for the first: `routeClaiming` uses the inventory's own claim predicate (a unit tree counts). The alias and the skipped-file cases are open: comparing hibernating paths by identity against listed and skipped files is not built. | `litestream_owners.go` `routeClaiming` | the same test (a unit route over the console's directory: unowned); fails with exact spellings alone (mutation run) | |
| -| Opus P3-2: root and the platform's user were two owners at the end of a tenant chain; `instances/*/*.db` rested on a premise isolated exec contradicts; a tenant's file in a platform-owned directory is refused. | Fixed for the first two: root and the platform's user are one owner there as in `platformOwned` (`platformUID`); `activatorFile` is `dataDir/*.db` alone, so an instance's database is checked by its chain. The third is stated as not decided (billable or not), and so refused until it is. | `litestream_owners.go` | `TestRootAndThePlatformUserAreOneOwner` (skipped under root, where the two owners are one and the test would say nothing); `TestAnInstancesDatabaseIsCheckedByItsChain` (an instance directory that is a tenant's link: refused); each fails with its fix removed (mutation runs, not as root) | |
| +| Opus P3-2: root and the platform's user were two owners at the end of a tenant chain; `instances/*/*.db` rested on a premise isolated exec contradicts; a tenant's file in a platform-owned directory is refused [from the nineteenth build in both layouts; before it the climb accepted such a file under a tenant's own tree: review eighteen]. | Fixed for the first two: root and the platform's user are one owner there as in `platformOwned` (`platformUID`); `activatorFile` is `dataDir/*.db` alone, so an instance's database is checked by its chain. The third is stated as not decided (billable or not), and so refused until it is. | `litestream_owners.go` | `TestRootAndThePlatformUserAreOneOwner` (skipped under root, where the two owners are one and the test would say nothing); `TestAnInstancesDatabaseIsCheckedByItsChain` (an instance directory that is a tenant's link: refused); each fails with its fix removed (mutation runs, not as root) | |
| | Opus P3-4, Astra 11: the "other user's directory" fixture keyed by base name was a foreign ancestor, not a coherent tree; the ranking untested. | Fixed: B's directory and file are named apart, and the case now shows what the code does (a coherent other-user tree named by a route is accepted: Astra 9, open with approval H; the test's introduction said "refused" until the thirteenth build); the every-claim test above. Still open as stated: retained handles and worker exit, the watch's own call, per-database attribution, the re-check under the lock, the 300 ms check, `withOwners` by base name, the admission's under-lock check, the shadow-state directory check. | tests | | |
| | Astra 1 to 4, 8, 10 (P1s, the device open, the uid binding): litestream's opens by path; tenant-replaceable ancestry; a delayed admitted request past the margin; the crash and restart gap; the header open through a swapped ancestor; a route naming another user's coherent tree. | Open, as before, stated where each applies; required before wiring, with approval H for the last. | | | |
| | Claims (both): the strict walk's "two things on purpose" and "never followed"; `activatorFile`'s premise; "strictest kind"; the hibernation and platform-conflict rows above; `Close` "asks the witness once more"; the close test's comment; `fleetInventory`'s comment. | Corrected in place, bar the leftovers review twelve found (the old ranking paragraph above the new one, "the two things", `strictestClaim` in `litestream.go`, the witness sentence at `inheritHold` and `Open`), corrected in the thirteenth build. | | | |
| @@ -280,7 +280,7 @@ Reports: `OPUS-REVIEW-metered-replicator-review13.md`, `CODEX-FINDINGS-metered-r |
| | Finding | Disposition | Where | Reproduction | |
| | --- | --- | --- | --- | |
| -| Opus P3-1, Astra 5 (P2): the link entry was not injective. | Fixed: a skipped link is `link "path" -> "target"`, both halves quoted behind a word no absolute path begins with, so no file name and no other pair renders the same; `Accept` compares such an entry as written. A plain file's name with a line break rendered as two lines in `render()`, which is the drift comparison, not display [review fourteen, both reviewers]; the fifteenth build quotes every entry in `render()`. A problem names its path first and its target quoted (nothing compares a problem with `Accept`). | `litestream_owners.go` `linkEntry`, `isLinkEntry` | `TestAcceptComparesALinkEntryAsWritten` (a relative target as tenant software makes them; the pair accepted builds the store; re-pointed through `x/..` it is refused naming the new pair; a database file named like the pair is refused as the plain file it is); fails with the unprefixed encoding (mutation run) | |
| +| Opus P3-1, Astra 5 (P2): the link entry was not injective. | Fixed: a skipped link is `link "path" -> "target"`, both halves quoted behind a word no absolute path begins with, so no file name and no other pair renders the same; `Accept` compares such an entry as written. A plain file's name with a line break rendered as two lines in `render()`, which is the drift comparison, not display [review fourteen, both reviewers]; the fifteenth build quotes every entry in `render()`. A problem names its path first and its target quoted (nothing compares a problem with `Accept`). | `litestream_owners.go` `linkEntry`, `isLinkEntry` | `TestAcceptComparesALinkEntryAsWritten` (a relative target as tenant software makes them; the pair accepted builds the store; re-pointed through ``sub/./live.sqlite` against `sub/live.sqlite` (the fifteenth build's fixture; the fourteenth's `x/..` case did not tell them apart)` it is refused naming the new pair; a database file named like the pair is refused as the plain file it is); fails with the unprefixed encoding (mutation run) | |
| | Opus P3-2: the one-inode rule did not reach the listing-time duplicate check. | Fixed: after the checks, two owned paths whose kept facts are one file are both unowned ("became this same file under the look"). No deterministic test (a rename between two files' checks within one inventory); stated. | `litestream_owners.go` `fleetInventory` | | |
| | Opus P3-3: the link's target was read, then the link classified by a second read. | Fixed in the fourteenth build by a second read, which review fourteen showed does not close a re-point-and-restore around the `Stat`; the fifteenth build classifies the spelling the entry names (`Stat` of the target as read, from the link's directory when relative), so the entry and what was looked at are one string whatever the link is re-pointed to meanwhile; a link that is no longer a link (EINVAL) is churn. No deterministic test; stated. | `litestream_owners.go` `strictTreeDBs` | | |
| | Opus P3-4: the `Accept` test asserted refusal by class alone. | Fixed: it asserts the entry named in the refusal, and that the accepted one is not. | tests | the same test | |
| @@ -334,16 +334,33 @@ The seventeenth build's claim here that its package tests passed under `/tmp` wa |
| ## Review seventeen (reviewed `bffc76b5`; Opus accepted with fixes, Astra rejected) |
| -Reports: `OPUS-REVIEW-metered-replicator-review17.md`, `CODEX-FINDINGS-metered-replicator-review17.md`. Both found review sixteen's findings closed, Astra 5 closed in behaviour and the render parts of Astra 11 closed, and the same two new defects in the fix for Astra 5 (Opus P2-1 and P2-2 are Astra's findings 5 and 6). Astra rejects on the deep items (its 1 to 4 at P1, 7 to 10 at P2, 11 at P3), open as before and listed as blocking wiring; its finding 12 (P3) names assertions that survive removing what they describe. |
| +Reports: `OPUS-REVIEW-metered-replicator-review17.md`, `CODEX-FINDINGS-metered-replicator-review17.md`. Both found review sixteen's findings closed, Astra 5 closed in behaviour and the render parts of Astra 11 closed, and the same two new defects in the fix for Astra 5 (Opus P2-1 and P2-2 are Astra's findings 5 and 6 of that report; Astra's 7 of that report is the header open through a swapped ancestor, numbered 6 in the earlier reports). Astra rejects on the deep items (its 1 to 4 at P1, 7 to 10 at P2, 11 at P3), open as before and listed as blocking wiring; its finding 12 (P3) names assertions that survive removing what they describe. |
| | Finding | Disposition | Where | Reproduction | |
| | --- | --- | --- | --- | |
| | Opus P2-1: `statTarget` opened the link's directory by path with Go's default flags: a FIFO a tenant swaps into the directory's place blocks the open until a writer appears (the watch goroutine hangs, Close waits for ever at `watching.Wait()`, `chainClosed` never lets the replacement open), and a device there would have had its driver's open run. | Fixed: the directory is opened with `O_RDONLY\|O_DIRECTORY\|O_NONBLOCK\|O_NOCTTY\|O_CLOEXEC`; anything but a directory is refused by the open itself with ENOTDIR, before any driver's open, and nothing can block on it or become its controlling terminal. `O_NOFOLLOW` is not added: the tree's own root is reached through systemd's state link. The descriptor is still a fresh open by path, not the one the walk listed from (EE2 stays open, stated in the code). | `linktarget_unix.go` `statTarget` | `TestStatTargetRefusesADirectorySwappedForAFIFO`: with Go's default flags the test fails after five seconds ("the open blocked"); with `O_DIRECTORY` alone dropped it fails because the refusal came from the stat through the handed-back descriptor, not the open (the test asserts the `PathError`'s op is `open`). | |
| -| Opus P2-2: the long-target fixture's guard held only under a long temp root; the gate under `/var/tmp/ck` and CI failed on it. | Fixed: the target is sized from the directory the run's temp root gives (`n := (4097-len(deep))/2 + 1`), so the concatenation outgrows the limit under any root while the target alone stays under 4,095 bytes; the guard checks both. | `litestream_owners_test.go` | With the fixed 1,500 repetitions put back, the test fails under `/var/tmp/ck` ("the concatenation is 4074 bytes"); with the fix it passes under `/var/tmp/ck`, `~/.cache/tmp` and the author's root. | |
| +| Opus P2-2: the long-target fixture's guard held only under a long temp root; the gate under `/var/tmp/ck` and CI failed on it. | Fixed: the target is sized from the directory the run's temp root gives (`n := (4097-len(deep))/2 + 1`), so the concatenation outgrows the limit while the target alone stays under 4,095 bytes; the guard checks both [the nineteenth build puts the length in the directories instead, the target under a kilobyte, and reports a root that leaves no budget: review eighteen, Opus P3-1 and Astra 10]. | `litestream_owners_test.go` | With the fixed 1,500 repetitions put back, the test fails under `/var/tmp/ck` ("the concatenation is 4074 bytes"); with the fix it passes under `/var/tmp/ck`, `~/.cache/tmp` and the author's root. | |
| | Opus P3-1, Astra claims: wording wider than the code ("a descriptor for that directory", "the directory the link is in"; "never a concatenation that could outgrow PATH_MAX"; this file's "pass under `/tmp`"). | Corrected: the comment says the directory is resolved again from its path at that moment and a link in its last component is followed; that the argument handed to the kernel is the target alone, the target itself bounded by what a link can hold; and the review-sixteen claim is marked false above. | `linktarget_unix.go`, `litestream_owners.go`, this file | | |
| -| Astra 12 (P3): assertions that survive removing what they describe (shutdown's handles and workers, the watch's `lookAgain`, admission's pre-witness check, the latch's recheck, the shadow state's data-directory check, the inode comparisons, render's section separators, the 300 ms negative Close check, the clean-start test's clock; no fixture for the directory-substitution window). | Two closed this build: the directory-substitution window has `TestStatTargetRefusesADirectorySwappedForAFIFO`, and `render` is compared across sections (the same entry in two different lists, and a list entry spelt like the separator, render differently). The rest stay open as named: each needs a controlled interleaving or a fake the package does not have yet, and none is closed by a claim here. | `litestream_metered_test.go` `TestRenderQuotesEveryEntry`, `linktarget_unix_test.go` | | |
| +| Astra 12 (P3): assertions that survive removing what they describe (shutdown's handles and workers, the watch's `lookAgain`, admission's pre-witness check, the latch's recheck, the shadow state's data-directory check, the inode comparisons, render's section separators, the 300 ms negative Close check, the clean-start test's clock; no fixture for the directory-substitution window). | Two closed this build: the open's refusal of anything but a directory has `TestStatTargetRefusesADirectorySwappedForAFIFO` (a direct call with a FIFO where the directory was; not an interleaving inside the walk), and `render` is compared across sections (the same entry in two different lists, and a list entry spelt like the separator, render differently). The rest stay open as named: each needs a controlled interleaving or a fake the package does not have yet, and none is closed by a claim here. | `litestream_metered_test.go` `TestRenderQuotesEveryEntry`, `linktarget_unix_test.go` | | |
| | Astra claims: the Accept fixture's comment still said "through a `..`" (the fixture is `sub/./live.sqlite`); this file's review-thirteen row described the sixteenth build's concatenation as current; the review-sixteen open-item list omitted Astra 7; `litestream_owners.go`'s "a tenant's file in a directory the platform owns is refused" was wider than the comparison (which is against the chain's root, not the file's immediate directory); `TestOwnershipWitnessMovesOnTheColumnsItReads` was named for a narrower property than it tests; `OUTSTANDING.md` 1 described a third build. | Corrected in place: the fixture comment, the row (bracketed with what the seventeenth and eighteenth builds do), the numbering, the owner-check comment (what it compares and the case it does not decide), the test's name (`TestOwnershipWitnessMovesOnTheColumnsItReads`, with its comment), and the outstanding item (on this branch and, in the same words, on the launch branch where it is maintained). | this file, `litestream_metered_test.go`, `litestream_owners.go`, `internal/registry/observe_ownership_test.go`, `OUTSTANDING.md` | | |
| | Claims the code contradicts: "a descriptor for the link's directory" (corrected as above); whether CI runs as root is not established by committed source (open, as before). | | | | |
| -| Astra 1 to 4, 6 to 10, approval H's uid binding, the box-wide refusal decision: open as before; Astra 6 has a second instance closed by P2-1's fix and its first (the walk by path) open. | | | | |
| +| Astra 1 to 4, 7 to 11 of this report (the earlier reports' 6 to 10), approval H's uid binding, the box-wide refusal decision: open as before; the header open through a swapped ancestor (this report's 7) has a second instance closed by P2-1's fix and its first (the walk by path) open. | | | | |
| -Package tests (the two link tests and the package) pass under `/var/tmp/ck` and `~/.cache/tmp` in the author's runs, not as root. The full gate on this tip is next; the eighteenth build has not been read by a reviewer. |
| +Package tests (the two link tests and the package) pass under `/var/tmp/ck` and `~/.cache/tmp` in the author's runs, not as root. The full gate on `6249af63` passed (exit 0, three cross builds, `gate-replicator-6249af63.txt`, 09:04Z) and Amadan CI passed on it. |
| + |
| +## Review eighteen (reviewed `6249af63`; Opus accepted, Astra rejected) |
| + |
| +Reports: `OPUS-REVIEW-metered-replicator-review18.md`, `CODEX-FINDINGS-metered-replicator-review18.md`. Opus found both of review seventeen's P2s closed with tests that fail without their fixes, no new P1 or P2, three P3s and stale wording. Astra found the same and rejected on the deep items, open as before (its 1 to 4 at P1, 5 to 9 at P2, 11 and 12 at P3), adding that the directory-only open does not establish the directory's identity (its 5, the walk by path again) and that the fixture's depth was still fixed (its 10). |
| + |
| +| Finding | Disposition | Where | Reproduction | |
| +| --- | --- | --- | --- | |
| +| Opus P3-1, Astra 10: the long-target fixture needed a link target of about 3 KB (XFS refuses one over 1,024 bytes), and added five directories unconditionally (a root of about 3,200 bytes could not build it). | Fixed: the target is 999 bytes; directories of 200 bytes are added until the concatenation outgrows the limit, within the pathname budget, and a root that leaves no budget is reported as the fixture's failure, not worked around. | `litestream_owners_test.go` | The guard checks the concatenation, the target's length and the file's own path. Passes under `/var/tmp/ck`, `~/.cache/tmp` and the author's root. | |
| +| Opus P3-2 (JJ2), Astra's claim: a tenant's file in a platform-owned directory was refused under the platform's tree and accepted under the tenant's own tree (the climb took the tenant's directory as the chain root). | Fixed: the start check refuses a file that is not the platform's in a directory that is; both layouts are now the undecided case, refused. | `litestream_owners.go` `checkOwnedFile` | `TestATenantsFileInAPlatformDirectoryIsRefusedInBothLayouts`; with the refusal removed, the test fails on the tenant's-tree layout. | |
| +| Opus P3-3 (GG2), Astra 5: what a swap of the link's directory still does (another directory, or a link to one, in its place between the listing and the open: the one link omitted or misread for that one look; never a database streamed or attributed through it). | Stated in the code as open under EE2, with what it can and cannot do; the walk by descriptor stays the fix and stays open (Astra 1, 5). | `linktarget_unix.go` | | |
| +| Astra's claim: refusal messages joined raw entries, so a name with a line break could mislead a displayed diagnostic. | Fixed: a plain entry is quoted in every refusal message; a link entry is already the quoted pair. | `litestream_metered.go` `quotedList` | With plain entries unquoted again, the impostor case of `TestAcceptComparesALinkEntryAsWritten` fails (it wants the quoted name). | |
| +| Opus II2: the witness test had no host-rename case. | Added: a raw `UPDATE routes SET host` moves the witness. | `observe_ownership_test.go` | | |
| +| Claims (both): "a link in its last component is followed" (any component is); `OUTSTANDING.md` "no mount" (no crossing onto another filesystem; a same-filesystem bind alias is not detected) and "Opus accepts each with fixes" (Opus rejected the first, second and eighth builds); this file's review-eight row, review-thirteen row (`x/..`), the review-seventeen numbering, "under any TMPDIR", the FIFO test's description, and the missing record of the green gate on `6249af63`. | Corrected in place. | this file, `linktarget_unix.go`, `OUTSTANDING.md` (this branch and the launch branch) | | |
| +| Astra 12 (P3): tests that survive removing what they describe. | Open as named; two were closed in the eighteenth build. | | | |
| +| Astra 1 to 4, 6 to 9, 11; approval H; the box-wide refusal decision. | Open as before; each blocks wiring. | | | |
| + |
| +Package tests pass under `/var/tmp/ck` and `~/.cache/tmp` in the author's runs, not as root. The full gate on this tip is next; the nineteenth build has not been read by a reviewer. |
diff --git a/docs/superpowers/specs/OPUS-REVIEW-metered-replicator-review18.md b/docs/superpowers/specs/OPUS-REVIEW-metered-replicator-review18.md| new file mode 100644 |
| index 000000000..c78163eb8 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/OPUS-REVIEW-metered-replicator-review18.md |
| @@ -0,0 +1,94 @@ |
| +# Review eighteen: feat/carloku-metered-replicator-20261001 at `6249af63` |
| + |
| +**Verdict: accept.** Both P2s from review seventeen are closed, and each has a test that fails without its fix. The eighteenth build adds no new P1 or P2. There are three P3s and some stale wording. Astra 1–4, approval H's uid binding and the decision on box-wide refusal still block wiring. |
| + |
| +I read source with `git show 6249af63:` only and ran nothing. |
| + |
| +**About the failed gate and CI.** The brief says they failed on `6249af63`. The committed dispositions record those failures on `bffc76b5` (`DISPOSITIONS…:333`) and call the gate on this tip "next" (`:349`). I cannot confirm a failure on `6249af63`. By the arithmetic below, the resized fixture holds under any temp-root length. The only remaining way I find for it to fail is P3-1: a filesystem that caps link targets. If CI did fail on this tip, check the runner's temp filesystem first. |
| + |
| +## Review seventeen's findings, one by one |
| + |
| +Review sixteen's findings stand as review seventeen found them, and nothing here reopens them. P2-2's fix makes Astra 5's test valid again under short roots. |
| + |
| +| Finding | At `6249af63` | Would its test pass without the fix? | |
| +|---|---|---| |
| +| **Opus P2-1 / Astra 5**: `statTarget` opened the directory with blocking default flags | **Closed.** See the details after this table. | **No.** With `os.Open` the test times out at 5 s. With `O_DIRECTORY` dropped, the error comes from the `stat` step, not the `open`, so the op assertion fails. `O_NONBLOCK` and `O_NOCTTY` are not pinned, but `O_DIRECTORY` makes both redundant. | |
| +| **Opus P2-2 / Astra 6**: the long-target fixture depended on the root's length | **Closed for length** (`litestream_owners_test.go:766`). See the arithmetic after this table. | **No.** The concatenation is always over 4,096 bytes, so the concatenating build gets `ENAMETOOLONG` and fails. A `filepath.Join` mutation is caught by the aliased-root case, not this one. | |
| +| **Opus P3-1**: wording ("a descriptor for that directory", PATH_MAX, "pass under `/tmp`") | **Closed**, with one remaining nit (claim 1 below). | Wording only. | |
| +| **Astra 1–4** (P1) | **Open**, unchanged and stated as blocking. | Yes, as before. | |
| +| **Astra 7–10** | **Open**, unchanged. | Yes, as before. | |
| +| **Astra 11** | **Open**, unchanged. | Yes, as before. | |
| +| **Astra 12** | **Two parts closed.** The section render pairs fail if the `--` separators are removed (pair 1 becomes identical). The FIFO test closes the open-flag part of the substitution window. The rest is open as named. | The rest: yes, as before. | |
| +| **Astra claims** | **Closed**, except "no mount" in `OUTSTANDING.md` (claim 2 below). | n/a | |
| + |
| +Details for P2-1: |
| +- `linktarget_unix.go:41` now opens with `O_RDONLY|O_DIRECTORY|O_NONBLOCK|O_NOCTTY|O_CLOEXEC`. |
| +- On Linux, `do_open` checks `LOOKUP_DIRECTORY` against `d_can_lookup` before `may_open` and `vfs_open`. So a FIFO, device or file in the directory's place gets `ENOTDIR` before any driver's open, and nothing can block. |
| + |
| +The arithmetic for P2-2, with L = `len(deep)` = root + 1,005: |
| +- The concatenation is always about 4,110–4,111 bytes. |
| +- `len(dest)` is at most 4,110 − L, which is about 3,100 or less. That stays under 4,095 for any root that can hold the fixture at all. |
| +- `platformDir` resolves the link first, so a linked root is sized after resolution. |
| + |
| +## Findings |
| + |
| +### P3-1. The long-target fixture needs about 3 KB link targets, which XFS refuses (HH2) |
| + |
| +**Where:** `internal/ops/litestream_owners_test.go:766-776`. |
| + |
| +**Scenario:** |
| +- `dest` is about 3,050–3,070 bytes under ordinary roots. |
| +- XFS caps a link target at 1,024 bytes (`XFS_SYMLINK_MAXLEN`). |
| +- With `TMPDIR` on XFS (the RHEL default, and common on CI volumes), `os.Symlink` at `:771` returns `ENAMETOOLONG` and the test fails. It fails with or without the fix, so it tests nothing there. |
| +- The seventeenth build's fixture had the same dependence. ext4, tmpfs and btrfs are fine. |
| + |
| +**Smallest fix:** shift the length into `deep` rather than `dest`. Add enough 200-byte components that `len(deep)` is at least about 3,100, then size `dest` from it as now. The target stays under 1,000 bytes and the concatenation still exceeds 4,096. Keep the guard. |
| + |
| +### P3-2. A tenant's file in a platform-owned directory: refused or accepted depending on the directory above it (JJ2) |
| + |
| +**Where:** `litestream_owners.go:475` (platform-owned start), `:495` (climb), `:563` (owner compare); comment at `:553-562`. |
| + |
| +The comment now describes the code exactly: |
| +- The comparison is with the chain's root. |
| +- A platform-owned directory is passed through only as the file's own directory. The climb at `:495` stops at any platform-owned parent, so "lower in a tenant's own chain" can only mean the bottom. |
| + |
| +But the case it calls undecided gets both answers: |
| +- **Accepted:** `/srv/T` (owned by T), `/srv/T/bridge` (root, 0755), `/srv/T/bridge/a.db` (owned by T). The start check at `:475` passes. The climb takes `/srv/T` as the chain root. `/srv` is the platform's. `a.db` matches T and is accepted, so it is billed to T. |
| +- **Refused:** the same `bridge/a.db` with `bridge` directly under a platform directory. |
| + |
| +Same file, same immediate directory, opposite answers. That is a decision taken by the layout, not the undecided case the comment says it is. |
| + |
| +**Smallest fix:** in `tenantFile`, refuse when the start directory is platform-owned and the file's owner is not `platformUID`. That makes both layouts "not decided". DynamicUser state directories are owned by the dynamic uid, so they are unaffected. |
| + |
| +### P3-3. GG2: what a swap of the link's directory still does is harmless, and stated only in part |
| + |
| +**Where:** `linktarget_unix.go:41`, `litestream_owners.go:687-713`. |
| + |
| +The tenant can still put another directory, or a link to one, in place of the listed directory between `ReadDir` and the open. The open follows links in every component of the path, not only the last. Effects: |
| +- **Target absent in the replacement:** a directory link or file link is omitted (`ENOENT`) for that look. |
| +- **Different file there:** a file link is misread as a directory problem, or a directory link as an accepted file pair. |
| +- **Bind mount:** not reachable without privilege in the fleet's namespace. |
| + |
| +None of this gives the tenant more than it already has. It can re-point or remove its own link around a look, and the walk's `open` of its directory tells it when one starts (inotify). Nothing is replicated or attributed through a link either way. So EE2 is now harmless and non-blocking, not a way to hide a database the fleet would otherwise stream. It stays inside Astra 1 and 6, the walk by path. |
| + |
| +Two notes: |
| +- `O_PATH|O_DIRECTORY` would also skip the read-permission check and the directory's own `open` op. It is not required. |
| +- The FIFO test calls `statTarget` with a FIFO path directly. It establishes the open's refusal (op `open`, `ENOTDIR`), not an interleaving in the walk. That is the right unit to test, but its name says "swapped". |
| + |
| +## Areas A–FF2 and GG2–JJ2 |
| + |
| +- **A–FF2: unchanged from review seventeen.** This build's only code change is the open flags in `statTarget`. `litestream_metered.go` and `registry` code are untouched, so their line numbers are as in review seventeen. In `litestream_owners.go`, lines after 550 move by +4, and the `strictTreeDBs` comment by +1. |
| +- **GG2:** see P3-3. `O_DIRECTORY` refuses a FIFO or device before the driver's open. A directory or a link to one can still be substituted: harmless and non-blocking, as stated there. The test establishes the open's refusal. |
| +- **HH2:** sized correctly for short, long and linked roots. The remaining dependence is the filesystem's link-length cap (P3-1). |
| +- **II2:** the cross-section pairs fail if the separators are removed, and the "--" entry pairs fail if separators go unquoted, so the render test establishes what its name says. |
| + - `TestOwnershipWitnessMovesOnTheColumnsItReads` tests five of the six columns. `host` has no move case: an `UPDATE … SET host` fires the trigger, but no fixture shows it. Small gap; adding a raw host rename to `moves` closes it. |
| +- **JJ2:** the comment matches the code. The policy is inconsistent (P3-2). |
| + |
| +## Claims the code contradicts |
| + |
| +1. **`linktarget_unix.go:22-23`, "a link in its last component is followed":** a link in any component is followed, since the directory is re-resolved from its whole path. |
| +2. **`OUTSTANDING.md:35`, "no mount":** the code refuses crossing onto another filesystem (a different `Dev`). A bind mount on the same filesystem is not detected (open item G). |
| +3. **`DISPOSITIONS…:344`, "the directory-substitution window has `TestStatTargetRefusesADirectorySwappedForAFIFO`":** the test exercises `statTarget` against a FIFO path, not a substitution during the walk. That is accurate for the flags, not for the window. |
| +4. **Whether CI runs as root:** still not established by committed source. |
| + |
| +**Verdict: accept.** P3-1 (the fixture's dependence on link length) should be fixed before CI is relied on with an XFS temp root. P3-2 and the claims are follow-ups. Astra 1–4, approval H and the box-wide-refusal decision still block wiring. |
diff --git a/implementation/carloku-paid-launch/OUTSTANDING.md b/implementation/carloku-paid-launch/OUTSTANDING.md| index d598f0288..fd34afd60 100644 |
| --- a/implementation/carloku-paid-launch/OUTSTANDING.md |
| +++ b/implementation/carloku-paid-launch/OUTSTANDING.md |
| @@ -32,7 +32,7 @@ Each is concrete. None is waived. |
| ### Meters (the complete metered model cannot invoice until both are proven) |
| -1. **Replication writes (`durability_writes_1k`) cannot be proven on any box.** The external replicator writes outside the ledger. Path A (scoped attestation) was built, reviewed in round six, and cannot prove an exclusive writer; it is off this branch and parked on `feat/carloku-scoped-attestation-20261001`. Path B (the platform's own replicator as the only writer) is **partial, unwired, and has not had a whole-branch pass from both reviewers** (eighteen builds; Opus accepts each with fixes, Astra rejects each on the deep items), on `feat/carloku-metered-replicator-20261001` (the forge has the tip; `READINESS.md` names it with its gate and CI state). What is built there: a request gate in `internal/lease` that stops an attempt before it is sent; an ownership generation in the registry moved by the six route columns the replicator reads (not by its own bookkeeping writes); a fleet that holds every recorder's days in doubt and watches without a request; one strict inventory with file checks (one name, ordinary siblings, a link named with its target as written and classified by that spelling through a directory-only descriptor, no mount, the chain root's owner, under a platform directory) and a second look on a timer; and shadow state made private before litestream can make it. Still open there, each named in its dispositions as blocking wiring: litestream's own opens by path after a look (needs litestream to open beneath a pinned root), replaceable ancestry above a protected directory, an admitted request delayed past the margin, a process dying between a change and the rebuild (the doubt is in memory only), device effects through an ancestor swap, hibernating aliases and claims on skipped files, required coverage that disappears without an entry, that any tenant can stop a rebuild for the box, route identity unbound to filesystem authority, and the uid binding under approval H. `DISPOSITIONS-metered-replicator.md` on that branch has every finding and what each build did about it. What is there: a litestream store whose every replica client is metered with no unmetered fallback, an owner for each database the box replicates, and a fleet store that refuses anything it cannot meter, each with tests. **Absent**: the agent wiring that binds those meters to the box's ledger, a restore path that does not shell out to the external binary, the unit change, moving the replica credentials so nothing else on the box holds them (the unit runs as the same user as the agent today), and any attestation. Until all of that exists and is approved for a box, this meter is incomplete every month and nothing is invoiced. |
| +1. **Replication writes (`durability_writes_1k`) cannot be proven on any box.** The external replicator writes outside the ledger. Path A (scoped attestation) was built, reviewed in round six, and cannot prove an exclusive writer; it is off this branch and parked on `feat/carloku-scoped-attestation-20261001`. Path B (the platform's own replicator as the only writer) is **partial, unwired, and has not had a whole-branch pass from both reviewers** (nineteen builds; Opus rejected the first, second and eighth and accepts the rest with fixes, Astra rejects each on the deep items), on `feat/carloku-metered-replicator-20261001` (the forge has the tip; `READINESS.md` names it with its gate and CI state). What is built there: a request gate in `internal/lease` that stops an attempt before it is sent; an ownership generation in the registry moved by the six route columns the replicator reads (not by its own bookkeeping writes); a fleet that holds every recorder's days in doubt and watches without a request; one strict inventory with file checks (one name, ordinary siblings, a link named with its target as written and classified by that spelling through a directory-only descriptor, no crossing onto another filesystem (a same-filesystem bind alias is not detected), the chain root's owner, under a platform directory) and a second look on a timer; and shadow state made private before litestream can make it. Still open there, each named in its dispositions as blocking wiring: litestream's own opens by path after a look (needs litestream to open beneath a pinned root), replaceable ancestry above a protected directory, an admitted request delayed past the margin, a process dying between a change and the rebuild (the doubt is in memory only), device effects through an ancestor swap, hibernating aliases and claims on skipped files, required coverage that disappears without an entry, that any tenant can stop a rebuild for the box, route identity unbound to filesystem authority, and the uid binding under approval H. `DISPOSITIONS-metered-replicator.md` on that branch has every finding and what each build did about it. What is there: a litestream store whose every replica client is metered with no unmetered fallback, an owner for each database the box replicates, and a fleet store that refuses anything it cannot meter, each with tests. **Absent**: the agent wiring that binds those meters to the box's ledger, a restore path that does not shell out to the external binary, the unit change, moving the replica credentials so nothing else on the box holds them (the unit runs as the same user as the agent today), and any attestation. Until all of that exists and is approved for a box, this meter is incomplete every month and nothing is invoiced. |
| 2. **Store transfer (`store_egress_gb`) has no source.** No Cost and Usage Report export exists on the account. It needs the scoped infrastructure approval, a first real delivery, an IAM principal inventory in the AWS provisioner (not built), and the bucket ownership attestation for stores that already exist (designed, not built). |
| 3. **A store's partial first hour** with a charged line holds the month, and nothing releases it. It holds by default and nobody is asked to waive it. What can release it is evidence: an operator door that records, for one bucket-hour, who is shown to have owned it and on what evidence, which the fold then honours. **Partial, unwired into this branch**, on `feat/carloku-bucket-hour-evidence-20261001` (`b50f29bc`; code at `58c1870c`): the record, the fold change and three operator API doors, with tests and mutation checks. It accepts only the hour a complete ledger began vouching in and only the owner that ledger names. The full local gate passes on it. First review: Astra rejected it, Opus accepted with fixes. Both confirmed that through the fold it releases nothing but that hour for that owner. Both also found that the evidence is free text that the same operator records and that stands at once, which in practice is a waiver: what counts as evidence, and whether a second person must approve it, needs Paul's ruling (approval J) before more is built. Nothing fixed yet (`DISPOSITIONS-bucket-hour-evidence.md` on that branch). No CI run, no CLI verb, and its operator-facing sentences are unreviewed. |
| 4. **A box's first minutes** before its ledger opened hold that day for everyone. Same as item 3: it holds by default, and the release is an operator evidence door, not a waiver. **Absent**. |
diff --git a/internal/ops/linktarget_unix.go b/internal/ops/linktarget_unix.go| index 02f0d22fa..d429b93c2 100644 |
| --- a/internal/ops/linktarget_unix.go |
| +++ b/internal/ops/linktarget_unix.go |
| @@ -19,8 +19,12 @@ type targetKind struct { |
| // process's mount namespace: an absolute target as it is; a relative one |
| // relative to the directory the link is in, through a descriptor opened by |
| // that directory's path at this moment (not the descriptor the walk listed |
| -// from: the directory is resolved again, and a link in its last component |
| -// is followed, as the tree's own root is reached through one), with |
| +// from: the directory is resolved again from its whole path, and a link in |
| +// any component of it is followed, as the tree's own root is reached |
| +// through one; a directory, or a link to one, put in its place between the |
| +// listing and this open is looked at instead, which can omit or misread |
| +// this one link for this one look and nothing more: stated open as EE2), |
| +// with |
| // fstatat, so the kernel resolves every ".." physically, from wherever the |
| // component before it led, and the argument handed to it is the target |
| // alone, so the link's directory adds nothing to the length the kernel |
diff --git a/internal/ops/litestream_metered.go b/internal/ops/litestream_metered.go| index c969313d5..f484b6a1c 100644 |
| --- a/internal/ops/litestream_metered.go |
| +++ b/internal/ops/litestream_metered.go |
| @@ -689,6 +689,23 @@ func (inv FleetInventory) render() string { |
| return b.String() |
| } |
| +// quotedList renders entries for a refusal message as render does for the |
| +// drift comparison: a plain entry quoted, so a name holding a line break or |
| +// a separator cannot read as two entries or hide one (review eighteen, |
| +// Astra's claim on the raw messages); a link entry as it is, being already |
| +// the quoted pair the operator names in Accept. |
| +func quotedList(entries []string) string { |
| + q := make([]string, len(entries)) |
| + for i, e := range entries { |
| + if isLinkEntry(e) { |
| + q[i] = e |
| + } else { |
| + q[i] = strconv.Quote(e) |
| + } |
| + } |
| + return strings.Join(q, ", ") |
| +} |
| + |
| // Databases is how many databases the store replicates (none once closed). |
| func (f *MeteredFleet) Databases() int { |
| f.mu.Lock() |
| @@ -861,7 +878,7 @@ func MeteredFleetStore(o MeteredFleetOptions) (fleet *MeteredFleet, err error) { |
| return nil, err |
| } |
| if len(inv.Problems) > 0 { |
| - return nil, fmt.Errorf("%w: places a database could be that cannot be vouched for: %s", ErrUnmeterable, strings.Join(inv.Problems, "; ")) |
| + return nil, fmt.Errorf("%w: places a database could be that cannot be vouched for: %s", ErrUnmeterable, quotedList(inv.Problems)) |
| } |
| // A plain path is accepted as cleaned; a link entry (link "path" -> |
| // "target") as written, so the spelled pair the operator accepted is |
| @@ -887,10 +904,10 @@ func MeteredFleetStore(o MeteredFleetOptions) (fleet *MeteredFleet, err error) { |
| } |
| } |
| if len(unseen) > 0 { |
| - return nil, fmt.Errorf("%w: files that look like databases, not replicated and not named as accepted: %s", ErrUnmeterable, strings.Join(unseen, ", ")) |
| + return nil, fmt.Errorf("%w: files that look like databases, not replicated and not named as accepted: %s", ErrUnmeterable, quotedList(unseen)) |
| } |
| if len(inv.Unowned) > 0 { |
| - return nil, fmt.Errorf("%w: %s", ErrUnmeterable, strings.Join(inv.Unowned, "; ")) |
| + return nil, fmt.Errorf("%w: %s", ErrUnmeterable, quotedList(inv.Unowned)) |
| } |
| if len(inv.Owned) == 0 { |
| return nil, errors.New("ops: no databases to replicate") |
diff --git a/internal/ops/litestream_metered_test.go b/internal/ops/litestream_metered_test.go| index c2f161b24..f117647a7 100644 |
| --- a/internal/ops/litestream_metered_test.go |
| +++ b/internal/ops/litestream_metered_test.go |
| @@ -1923,7 +1923,10 @@ func TestAcceptComparesALinkEntryAsWritten(t *testing.T) { |
| if err := os.WriteFile(impostor, append([]byte("SQLite format 3\x00"), make([]byte, 100)...), 0o644); err != nil { |
| t.Fatal(err) |
| } |
| - if _, err := MeteredFleetStore(o); !errors.Is(err, ErrUnmeterable) || !strings.Contains(err.Error(), impostor) || strings.Contains(err.Error(), "link \"") { |
| + // Named in the refusal as the quoted plain file it is (a plain entry |
| + // is quoted in a message, a link entry is already the quoted pair), so |
| + // it cannot read as a link entry there either. |
| + if _, err := MeteredFleetStore(o); !errors.Is(err, ErrUnmeterable) || !strings.Contains(err.Error(), strconv.Quote(impostor)) || strings.Contains(err.Error(), "link \"") { |
| t.Fatalf("a database named like the accepted link's entry: %v; want it refused as the plain file it is", err) |
| } |
| _ = db |
diff --git a/internal/ops/litestream_owners.go b/internal/ops/litestream_owners.go| index bb134b774..24b839a5c 100644 |
| --- a/internal/ops/litestream_owners.go |
| +++ b/internal/ops/litestream_owners.go |
| @@ -472,13 +472,21 @@ func checkOwnedFile(root, path string, kind claimKind) (fileFacts, string) { |
| if !start.IsDir() || start.Mode()&os.ModeSymlink != 0 { |
| return fileFacts{}, "its directory is not a directory" |
| } |
| - if sf.UID == 0 || int(sf.UID) == os.Geteuid() { |
| - // The platform's own directory (the file may be the platform's |
| - // too): writable by nobody else, or a local user could put a |
| - // sidecar link beside the file. |
| + if platformUID(sf.UID) { |
| + // The platform's own directory: writable by nobody else, or a |
| + // local user could put a sidecar link beside the file. And the |
| + // file in it is the platform's too: a tenant's file in a |
| + // directory the platform owns is not decided (whether it is |
| + // billable to the tenant), and until it is, it is refused |
| + // whatever the directory above (review eighteen, Opus P3-2: |
| + // the climb's root decided it one way under a tenant's tree |
| + // and the other under the platform's). |
| if start.Mode().Perm()&0o022 != 0 { |
| return fileFacts{}, "its directory is the platform's and can be written by its group or by anyone" |
| } |
| + if !platformUID(ownerFacts.UID) { |
| + return fileFacts{}, "its directory is the platform's and the file is not" |
| + } |
| } else if sf.UID != ownerFacts.UID { |
| return fileFacts{}, "its directory belongs to a user other than its own or the platform's" |
| } |
| @@ -554,12 +562,9 @@ func checkOwnedFile(root, path string, kind claimKind) (fileFacts, string) { |
| // the platform's user are one owner here as they are in platformOwned |
| // (a root-owned file in the platform user's directory, or the reverse, |
| // is the platform's either way). The comparison is with the chain's |
| - // root, which the climb chose, not with the file's immediate |
| - // directory: a tenant's file directly under a root the platform owns |
| - // is refused by this, while a platform-owned directory lower in a |
| - // tenant's own chain is passed through by the climb and this does not |
| - // see it. Whether a file in the first case is billable to the tenant |
| - // is not decided, and until it is, it is not. |
| + // root, which the climb chose. A tenant's file in a platform-owned |
| + // directory never reaches here: the start check above refuses it, |
| + // whatever is above the directory. |
| if (kind == tenantTree || kind == tenantFile) && facts.UID != rootFacts.UID && !(platformUID(facts.UID) && platformUID(rootFacts.UID)) { |
| return fileFacts{}, "it does not belong to the user its tenant's directory belongs to" |
| } |
diff --git a/internal/ops/litestream_owners_test.go b/internal/ops/litestream_owners_test.go| index 64da38df8..cc3ead9c5 100644 |
| --- a/internal/ops/litestream_owners_test.go |
| +++ b/internal/ops/litestream_owners_test.go |
| @@ -753,26 +753,28 @@ func TestStrictTreeNamesALinkToAFile(t *testing.T) { |
| // outgrow the kernel's path limit, though the link itself resolves: |
| // the target is looked at by itself, against the directory, and the |
| // link is named as the file link it is. (Review sixteen, Astra 5.) The |
| - // target is sized from the directory the temp root gives this run, so |
| - // the concatenation outgrows the limit under any TMPDIR while the |
| - // target alone stays under it (review seventeen, Opus P2-2: a fixed |
| - // size passed only under a long temp root). |
| + // length is put in the directories, not the target: the target stays |
| + // under a kilobyte (XFS refuses a longer link target: review eighteen, |
| + // Opus P3-1), and directories are added only until the concatenation |
| + // would outgrow the limit, within the pathname budget the temp root |
| + // leaves (review seventeen, Opus P2-2, and eighteen, Astra 10: a fixed |
| + // depth passed only under a root of the right length). A root that |
| + // leaves no budget for the fixture is reported, not worked around. |
| deep := platformDir(t) |
| - for i := 0; i < 5; i++ { |
| + dest := strings.Repeat("./", 494) + "live.sqlite" // 999 bytes, under XFS's limit |
| + for len(deep)+1+len(dest) <= 4096 { |
| + if len(deep)+1+200 >= 4000 { |
| + t.Fatalf("fixture: the temp root %q leaves no room for the fixture (%d bytes)", deep, len(deep)) |
| + } |
| deep = filepath.Join(deep, strings.Repeat("d", 200)) |
| } |
| mkdirs(t, deep) |
| long := writeFile(t, filepath.Join(deep, "live.sqlite")) |
| - n := (4097-len(deep))/2 + 1 |
| - if n < 1 { |
| - n = 1 |
| - } |
| - dest := strings.Repeat("./", n) + "live.sqlite" |
| if err := os.Symlink(dest, filepath.Join(deep, "current")); err != nil { |
| t.Fatal(err) |
| } |
| - if len(deep)+1+len(dest) <= 4096 || len(dest) >= 4095 { |
| - t.Fatalf("fixture: the concatenation is %d bytes and the target %d", len(deep)+1+len(dest), len(dest)) |
| + if len(deep)+1+len(dest) <= 4096 || len(dest) >= 1024 || len(deep)+1+len("live.sqlite") >= 4096 { |
| + t.Fatalf("fixture: the concatenation is %d bytes, the target %d, the directory %d", len(deep)+1+len(dest), len(dest), len(deep)) |
| } |
| dbs, skipped, problems = strictTreeDBs(deep) |
| if len(problems) != 0 || !contains(skipped, linkEntry(filepath.Join(deep, "current"), dest)) { |
diff --git a/internal/ops/litestream_owners_unix_test.go b/internal/ops/litestream_owners_unix_test.go| index 549182703..745a6444e 100644 |
| --- a/internal/ops/litestream_owners_unix_test.go |
| +++ b/internal/ops/litestream_owners_unix_test.go |
| @@ -514,3 +514,32 @@ func TestOpenRegularNoFollowRefusesALinkToTheSameFile(t *testing.T) { |
| t.Fatal("a link to the file that was looked at was opened through") |
| } |
| } |
| + |
| +// A tenant's file in a directory the platform owns is refused whatever is |
| +// above that directory: under the platform's own tree and under the |
| +// tenant's own tree alike. Before this the climb decided it one way in each |
| +// layout (review eighteen, Opus P3-2); the case itself (whether such a file |
| +// is billable to the tenant) is not decided, and until it is, it is not. |
| +func TestATenantsFileInAPlatformDirectoryIsRefusedInBothLayouts(t *testing.T) { |
| + const tenant = 44001 |
| + top := platformDir(t) |
| + withOwners(t, map[string]uint32{"srv": 0, "T": tenant, "bridge": 0, "a.db": tenant, "own.db": 0}) |
| + // Under the tenant's own tree: /srv/T/bridge/a.db, bridge the platform's. |
| + under := writeFile(t, filepath.Join(top, "srv", "T", "bridge", "a.db")) |
| + if _, why := checkOwnedFile(filepath.Dir(under), under, tenantFile); !strings.Contains(why, "its directory is the platform's and the file is not") { |
| + t.Fatalf("a tenant's file in a platform directory under the tenant's tree: %q", why) |
| + } |
| + // Under the platform's tree: /srv/bridge/a.db. |
| + above := writeFile(t, filepath.Join(top, "srv", "bridge", "a.db")) |
| + if _, why := checkOwnedFile(filepath.Dir(above), above, tenantFile); !strings.Contains(why, "its directory is the platform's and the file is not") { |
| + t.Fatalf("a tenant's file in a platform directory under the platform's tree: %q", why) |
| + } |
| + // The platform's own file in its own directory is still the platform's. |
| + own := writeFile(t, filepath.Join(top, "srv", "bridge", "own.db")) |
| + if err := os.Chmod(filepath.Join(top, "srv", "bridge"), 0o755); err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, why := checkOwnedFile(filepath.Dir(own), own, tenantFile); strings.Contains(why, "the file is not") { |
| + t.Fatalf("the platform's own file: %q", why) |
| + } |
| +} |
diff --git a/internal/registry/observe_ownership_test.go b/internal/registry/observe_ownership_test.go| index c7e1487c4..2903ef358 100644 |
| --- a/internal/registry/observe_ownership_test.go |
| +++ b/internal/registry/observe_ownership_test.go |
| @@ -29,6 +29,10 @@ func TestOwnershipWitnessMovesOnTheColumnsItReads(t *testing.T) { |
| "the route is deleted": func(t *testing.T, w *Registry) { |
| must(t, w.Delete(host)) |
| }, |
| + "the route's host is renamed (a raw statement; no verb renames one)": func(t *testing.T, w *Registry) { |
| + _, err := w.sql.Exec(`UPDATE routes SET host = 'renamed.example.com' WHERE host = ?`, host) |
| + must(t, err) |
| + }, |
| "the route is deleted and made again, the same": func(t *testing.T, w *Registry) { |
| r, err := w.Get(host) |
| must(t, err) |