amadan
Review five answered; inventory on retained descriptors, second slice: checkOwnedFile on the retained chain (the climb by ownership on Parent pointers, the directory above the top the platform's, every component acquired with O_NOFOLLOW, sidecars by Lookat), the root rule where the platform's authority ends (as many levels above a link's directory as its target ascends, the parent of the first non-platform component, an absolute platform-link target's parent pinned), the predicate's classes named (out of its platform root; a further link said before ownership), a directory changed under the look looked at again, a listing's per-entry errors, a pre-pin absent target as churn, resolvePlatformLinks, walkWithoutLinks, sqliteHeader and openRegularNoFollow retired, Facts.Nlink; inventory-level tests through systemd's shapes and each refusal, between the listing and the look, descriptor accounting; the model checks right after recovery, at-the-bound and first-change tests; design revision 12 (the root rule in Part A; the backstop limited to the witness; link shapes beside decisions 10 to 13); the plan's pre-wiring census
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
15 files changed,
+616
−413
DISPOSITIONS-replicator-pinned-root.md+19 −0docs/superpowers/plans/2026-10-03-replicator-authority-plan.md+3 −1docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md+4 −3docs/superpowers/specs/2026-10-03-replicator-pinned-roots-spikes.md+1 −1internal/authority/authority.go+6 −1internal/authority/authority_unix.go+13 −6internal/authority/authority_unix_test.go+3 −1internal/ops/admissionmodel/model.go+3 −3internal/ops/admissionmodel/model_test.go+43 −3internal/ops/authoritybridge.go+103 −17internal/ops/litestream_owners.go+177 −275internal/ops/litestream_owners_test.go+2 −6internal/ops/litestream_owners_unix_test.go+239 −38internal/ops/sqliteopen_other.go+0 −15internal/ops/sqliteopen_unix.go+0 −43
CI passed — run details
diff --git a/DISPOSITIONS-replicator-pinned-root.md b/DISPOSITIONS-replicator-pinned-root.md| index 5cc2170f8..c020524ab 100644 |
| --- a/DISPOSITIONS-replicator-pinned-root.md |
| +++ b/DISPOSITIONS-replicator-pinned-root.md |
| @@ -76,3 +76,22 @@ Opus: accept with fixes, no P1, one P2. Astra: accept with fixes, no P1, two P2. |
| | Astra: a later witness change after an undiscovered file change did not move the witness | fixed. | |
| | Astra 6: "closed once" in the spike note; the send test's comment | corrected. | |
| +## Review five (of `ccfd331a`: the fourth review answered, revision 11, the inventory's first slice) |
| + |
| +Opus: accept with fixes, no P1, two P2. Astra: accept with fixes, no P1, two P2. Reports: `logs/opus-replroot-review5.md`, `logs/astra-replroot-review5.md`. |
| + |
| +| Finding | Disposition | |
| +|---|---| |
| +| Astra 1, Opus (narrower predicate): the root rule refused legal platform links: an absolute target in another platform directory, and a leading ascent of more than one level | the root is as many levels above the link's directory as the target's leading ascent needs; an absolute platform-link target's parent is pinned by the same platform walk; a further link in an absolute target is said before ownership, as today; a target that leaves every pinned root is "a link in its path leads out of its platform root"; `fleetPins`'s comment says the roots are per call until the fleet holds them; inventory-level tests through the systemd shape (relative, two levels, absolute) and each refusal. The predicate is "today's rule, within the pinned roots", not "exactly", and the plan says so. | |
| +| Astra 2, Opus P2-1 and P3-4: the inventory tests did not distinguish the slice's properties | a test seam after each listing (`treeAfterList`) lets tests change the tree between the listing and what follows: a subdirectory swapped (looked at again, the replacement walked), a file replaced by a FIFO (cannot be read), the tenant root's `..` refused (`Confine`); descriptor accounting around `strictTreeDBs` and `checkOwnedFile`; a directory outside the tree, a non-platform component in an absolute target; the dangling link asserted by exact entry in both tests. | |
| +| Opus P2-2: common tenant layouts (a venv's links) become problems and the comments said otherwise | comments corrected; the consequence recorded in the design's parameters (beside decisions 10 to 13) and the plan's pre-wiring census; a dangling absolute link is churn (`Walk` answers Absent before a pin). | |
| +| Opus P3-2: a directory changed between `List` and `Child` was skipped silently | looked at again once and the replacement walked; a second change is "changed under the look"; the "on purpose" list says so. | |
| +| Opus P3-3: `List` errors changed meaning | `ENOENT` from a listing at depth is churn; one entry that cannot be looked at is its own problem (`Entry.Err`), the rest walked. | |
| +| Opus P3-5: a tree spelled below a tenant-owned directory with no link | the root rule's second clause (the parent of the first component that is not the platform's); `checkOwnedFile` on the retained chain (the second slice) uses it. | |
| +| Astra 4: the backstop statement too broad | limited to the witness observation. | |
| +| Astra 5: the driver skipped the check right after recovery | checked before anything else runs. | |
| +| Opus (model): `>=` at the bound; `WitnessChange` and the first change | tests added. | |
| +| Wording (both): "exactly", roots once per inventory, FIFO links passed over, a claimed second pin, witness-first ordering, the mutant count | corrected. | |
| + |
| +The second slice (`checkOwnedFile` on the retained chain, `resolvePlatformLinks`, `walkWithoutLinks`, `sqliteHeader` and `openRegularNoFollow` retired, `Facts.Nlink`) lands with these answers; Astra's note that pins live per call and Opus P3-6 stand until the fleet holds its chains. |
| + |
diff --git a/docs/superpowers/plans/2026-10-03-replicator-authority-plan.md b/docs/superpowers/plans/2026-10-03-replicator-authority-plan.md| index c0545298a..7aad067f9 100644 |
| --- a/docs/superpowers/plans/2026-10-03-replicator-authority-plan.md |
| +++ b/docs/superpowers/plans/2026-10-03-replicator-authority-plan.md |
| @@ -8,7 +8,7 @@ Date: 2026-10-03. For `docs/superpowers/specs/2026-10-03-replicator-pinned-roots |
| ## Task 2: the inventory on `Dir`s (Part A's second half) |
| -**Which roots a fleet pins.** The design says a root is built once per fleet from `/` and kept, and that a tenant root is reached from a platform root through the platform-link predicate; it does not list the roots. The rule for this task: for every spelled directory the inventory is asked about (a unit route's `db_path` tree, the activator's data directory, the registry's and console's directories), the pinned platform root is **the parent of the first link's directory** when the spelling contains a platform link (`/var/lib/carlos-app/<host>` is a link in `/var/lib/carlos-app`, so the root is `/var/lib`, and systemd's `../private/carlos-app/<host>` pops from `carlos-app` to the root and no further), and **the parent of the spelled directory** when it contains none (a plain tenant directory under the data directory pins the data directory). `OpenRoot(slash, root, platform)` requires every component from `/` to the root to pass the platform rule, so a tenant-owned or group-writable component above (Astra 2's layout) builds no root. Roots are keyed by path and shared across the fleet's spellings; `Pins` carries them all with the pinned `/`. |
| +**Which roots a fleet pins.** The design says a root is built once per fleet from `/` and kept, and that a tenant root is reached from a platform root through the platform-link predicate; it does not list the roots. The rule for this task, where the platform's authority ends: for every spelled directory the inventory is asked about (a unit route's `db_path` tree, the activator's data directory, the registry's and console's directories, a file claim's directory), the pinned platform root is **the parent of the first link's directory** when the spelling contains a platform link (`/var/lib/carlos-app/<host>` is a link in `/var/lib/carlos-app`, so the root is `/var/lib`, and systemd's `../private/carlos-app/<host>` pops from `carlos-app` to the root and no further), else **the parent of the first component that is not the platform's** (another user's, or writable by group or others: a tenant's tree top, so a file claim deep in a tenant's tree gets the tenant's parent as its root), else **the parent of the spelled directory**; and **the parent of an absolute platform-link target** is pinned too, so that walk enters it (today's predicate accepted an all-platform absolute target anywhere; the pin is established by the same platform walk). `OpenRoot(slash, root, platform)` requires every component from `/` to the root to pass the platform rule. A tenant-owned component above a platform-owned one (Astra 2's layout) puts the root above it; the directories beneath are the tenant's chain, retained and rechecked, so the swap that layout allows is a changed edge at the next look, never a steer, and the claim's own rules (the directory above a tree's root the platform's) decide what is owned. Roots are keyed by path and shared across the fleet's spellings; `Pins` carries them all with the pinned `/`; the slash itself is pinned deliberately (`SlashRoot`) when the rule names it. |
| **The platform-link predicate on descriptors.** `resolvePlatformLinks(dir string)` becomes `platformDir(pins, dir string) (*authority.Dir, string)`: from the root's `Dir`, each remaining component is `Child(name, nil)`; a component that is a link (`ErrLink`) is read with `Readlink` from its directory's `Dir` and must satisfy exactly today's predicate (the link's directory the platform's, the link itself the platform's by `Lookat`'s facts, the target walked with `Walk(target, pins)`: relative from the link's directory, absolute from the pinned `/`, with `..` only in the leading run, which `Walk` enforces, no further link, which `Walk`'s `Link` class refuses, ending at a directory whose parent is the platform's); the result's `Dir` is the next component's parent and the walk continues beneath it. A tenant root so reached is `Confine`d once by the inventory before anything walks from it. The predicate's restrictions do not change; its error strings stay, because the Accept list and the rendering compare them. |
| @@ -16,6 +16,8 @@ Date: 2026-10-03. For `docs/superpowers/specs/2026-10-03-replicator-pinned-roots |
| **`checkOwnedFile` on a chain.** The chain is acquired downward from the platform root as `Dir`s (the root, the predicate's result, each component by `Child`), never by `Lstat` of a joined path; each check it makes today (no link, one filesystem, a regular file with one name, the owner rules by kind, the `-wal`/`-shm`/`-journal` siblings regular with one name, by `Lookat` from the file's directory) is made on the retained chain; `fileFacts` gains the file's directory `Dir` (the design: one descriptor stack per owned file, counted in the fleet's rendering), kept for the fleet's life so every later look runs `Recheck` on it and `Lookat` of the file's name, and so litestream's opens (Part B, later) take it. `fleetInventory` closes the chains of files it does not keep. |
| +**Before wiring.** A census of the link classes on each box's live trees (the venv and `node_modules` shapes are problems under the new classes, where today they are skipped entries) and the owner's decision on them beside decisions 10 to 13; the live `DBPath`s checked against the root rule. |
| + |
| **The namespace condition.** A route whose instance runs in its own mount namespace is classified in the agent's namespace and rendered as such; until the support condition is declared it is a refusal rule (a problem naming the route), not a silence. |
| **Tests.** Every existing owners test keeps passing with its strings; added, from the design's Part A list at the inventory level: each link class rendered as written and none silent (a directory target, a database-named link, a regular file in the tree, a regular file under another pin, a regular file in a sibling tenant under the same pin, an absolute target inside the tenant's own tree (skipped, not out of the tree), a link to a link, a FIFO target, an absent target, an ascent past the tenant root, an absolute target outside every pin, a non-platform component in an absolute target); a tenant root reached through systemd's relative link shape and through an absolute one; Astra 2's layout refused at the root; a swap of the tenant directory between the listing and the classification changing nothing; the retained chain's `Recheck` at the next look reporting a moved or replaced directory; the descriptor count per fleet equal to the retained stacks' total. |
diff --git a/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md b/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md| index 731e669ad..1cd4774ac 100644 |
| --- a/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md |
| +++ b/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md |
| @@ -1,6 +1,6 @@ |
| # The replicator's authority: retained descriptors, a durable admission, claims by identity |
| -Date: 2026-10-03. Status: design, revision 11 (the fourth code review: the historical term's filter named `finished` and `abandoned` and so excluded `unrecorded`, a settlement whose transport did send; it now excludes only the empty terminals, `unissued` and `cancelled`; the look's order is stated as what safety needs rather than a sequence; the closing look agrees). Revision 10 was revision 9's correction of the lower bound refined after the third code review: the historical term's threshold is the earlier of the registry proof and the last clean look's start, since a witness change is known only to be after the proof; an empty terminal, unissued or cancelled, never sent and is not a settlement for that term; and every look reads the witness and rechecks every retained chain before it inspects files, so a clean look's start bounds every kind of change. Revision 9 was one mechanics correction to the uncertainty's lower bound, found by the second code review of the admission model: an admission admitted before the last clean look, still in flight across a change after it and settled clean before the change was noticed, was outside the bound; the bound now also takes the admission instant of every admission settled at or after the last clean look's start; the invariant I2 is unchanged and the model's named test and schedules cover it). Revision 8 was the accepted text: revision 7 was accepted with fixes by Astra (no P1; one P2: a negative read-back was not conclusive while the admission writer could still commit, and the pending set had three definitions; one P3: the restore wording made a salt mismatch imply a snapshot), both applied here. Revision 6 was rejected by Astra on one P1: a discovery whose write failed had no rule for the admissions that settled before its retry, so the replayed interval could lose a settlement; and three P2s: a helper dying before its recorder started left an admission nobody could finish; the transition table lacked the abort edges and the pending set's cleanup; the lineage record cited a durability discipline the shadow state does not have; each answered here. Revision 5 was rejected by Astra on two P1s: the admission's insertion had no ordering point against discovery, so a discovery between the proof and the commit counted nothing and a later send fell outside the closed uncertainty; construction did not backfill already-closed intervals for a successor's new owners before acknowledgement; and P2s: the transition table and the helper-death rule disagreed and the recorder participant was synthesised while its worker could still run; proof-only callers of admit would have created admissions with no terminal; a terminal `.` lost the directory requirement; conflicting summaries; each answered here. Revision 4 was rejected by Astra on four P1s: the helper has several connections of its own and needs SQLite's within-process lock bookkeeping between them; the producer fence did not cover a helper that outlives the agent with an admission in hand; an intermediate fleet closed before acknowledging broke transitive protection; a fleet could be marked closed with a discovery uncommitted; and four P2s: a dead helper's admissions had no terminal in the running agent, the sleeping binding persisted a reusable inode, the lineage predicate named header fields that identify nothing, the walk's grammar refused a `.` target the predicate accepts; each answered here. Revision 3 was rejected by Astra on five P1s: two SQLite VFSs in one process cannot coordinate through POSIX locks, since the library's own per-inode bookkeeping is in-process; a recorder that abandons early could settle an admission before its transport ended; an admission-only barrier does not cover a discovery whose uncertainty write failed after every admission settled; links at acknowledgement contradicted acknowledging only after settlement, and an inherited uncertainty vanished at the next successor; the ledger's lock does not prove the old process gone; and four P2s: closure at zero pending and the margin's extra day, sleeping claims with no binding, absolute and `.` spellings in the walk, `(dev, ino)` as a lineage; each answered here. Revision 2 was rejected by Astra on three P1s: the VFS delegated SQLite's Unix file methods, which open the SHM and check moved files by pathname themselves; every settled admission wrote a permanent hold, so ordinary replication never read complete; a late owner-scoped extension did not reach successors that had already acknowledged; and six P2s: orphan admissions never terminated, `..` and chain edges unspecified, `dup` sharing the directory offset, the platform-link predicate not preserved and terminal links unclassified, the uid producer without a lifecycle, one billing owner not one replicator; each answered here. Revision 1 was rejected by Astra on five P1s: SQLite canonicalises a `/proc` pathname before it opens anything and this driver passes no `nofollow`; `os.Root` confines but re-resolves every name, so a swap between a listing and an inspection still moves the answer; a `Start` row is not an admission, since the meter sends without one; the attempt-derived window lost a request completed before the change was noticed and the days between an admission and a finish; `closed_at` was not a handover; plus the directory open before `O_DIRECTORY`, per-owner holds the ledger cannot store, a uid with no producer, I3 against Part D, and tests). Branch `design-replicator-pinned-root`, off `feat/carloku-metered-replicator-20261001` at `e1280811`. It answers the eleven findings the twentieth review deferred (`DISPOSITIONS-metered-replicator.md`, review twenty) and keeps decisions 10 to 13 and approval H (`DECISIONS-FOR-PAUL.md`, section D, on the launch branch) as parameters. Nothing is built, wired or applied. |
| +Date: 2026-10-03. Status: design, revision 12 (Part A's root rule stated as the plan has it: the platform walk ends where the platform's authority ends, a tenant-owned component's parent, and the chain beneath is retained and rechecked rather than refused). Revision 11 was (the fourth code review: the historical term's filter named `finished` and `abandoned` and so excluded `unrecorded`, a settlement whose transport did send; it now excludes only the empty terminals, `unissued` and `cancelled`; the look's order is stated as what safety needs rather than a sequence; the closing look agrees). Revision 10 was revision 9's correction of the lower bound refined after the third code review: the historical term's threshold is the earlier of the registry proof and the last clean look's start, since a witness change is known only to be after the proof; an empty terminal, unissued or cancelled, never sent and is not a settlement for that term; and every look reads the witness and rechecks every retained chain before it inspects files, so a clean look's start bounds every kind of change. Revision 9 was one mechanics correction to the uncertainty's lower bound, found by the second code review of the admission model: an admission admitted before the last clean look, still in flight across a change after it and settled clean before the change was noticed, was outside the bound; the bound now also takes the admission instant of every admission settled at or after the last clean look's start; the invariant I2 is unchanged and the model's named test and schedules cover it). Revision 8 was the accepted text: revision 7 was accepted with fixes by Astra (no P1; one P2: a negative read-back was not conclusive while the admission writer could still commit, and the pending set had three definitions; one P3: the restore wording made a salt mismatch imply a snapshot), both applied here. Revision 6 was rejected by Astra on one P1: a discovery whose write failed had no rule for the admissions that settled before its retry, so the replayed interval could lose a settlement; and three P2s: a helper dying before its recorder started left an admission nobody could finish; the transition table lacked the abort edges and the pending set's cleanup; the lineage record cited a durability discipline the shadow state does not have; each answered here. Revision 5 was rejected by Astra on two P1s: the admission's insertion had no ordering point against discovery, so a discovery between the proof and the commit counted nothing and a later send fell outside the closed uncertainty; construction did not backfill already-closed intervals for a successor's new owners before acknowledgement; and P2s: the transition table and the helper-death rule disagreed and the recorder participant was synthesised while its worker could still run; proof-only callers of admit would have created admissions with no terminal; a terminal `.` lost the directory requirement; conflicting summaries; each answered here. Revision 4 was rejected by Astra on four P1s: the helper has several connections of its own and needs SQLite's within-process lock bookkeeping between them; the producer fence did not cover a helper that outlives the agent with an admission in hand; an intermediate fleet closed before acknowledging broke transitive protection; a fleet could be marked closed with a discovery uncommitted; and four P2s: a dead helper's admissions had no terminal in the running agent, the sleeping binding persisted a reusable inode, the lineage predicate named header fields that identify nothing, the walk's grammar refused a `.` target the predicate accepts; each answered here. Revision 3 was rejected by Astra on five P1s: two SQLite VFSs in one process cannot coordinate through POSIX locks, since the library's own per-inode bookkeeping is in-process; a recorder that abandons early could settle an admission before its transport ended; an admission-only barrier does not cover a discovery whose uncertainty write failed after every admission settled; links at acknowledgement contradicted acknowledging only after settlement, and an inherited uncertainty vanished at the next successor; the ledger's lock does not prove the old process gone; and four P2s: closure at zero pending and the margin's extra day, sleeping claims with no binding, absolute and `.` spellings in the walk, `(dev, ino)` as a lineage; each answered here. Revision 2 was rejected by Astra on three P1s: the VFS delegated SQLite's Unix file methods, which open the SHM and check moved files by pathname themselves; every settled admission wrote a permanent hold, so ordinary replication never read complete; a late owner-scoped extension did not reach successors that had already acknowledged; and six P2s: orphan admissions never terminated, `..` and chain edges unspecified, `dup` sharing the directory offset, the platform-link predicate not preserved and terminal links unclassified, the uid producer without a lifecycle, one billing owner not one replicator; each answered here. Revision 1 was rejected by Astra on five P1s: SQLite canonicalises a `/proc` pathname before it opens anything and this driver passes no `nofollow`; `os.Root` confines but re-resolves every name, so a swap between a listing and an inspection still moves the answer; a `Start` row is not an admission, since the meter sends without one; the attempt-derived window lost a request completed before the change was noticed and the days between an admission and a finish; `closed_at` was not a handover; plus the directory open before `O_DIRECTORY`, per-owner holds the ledger cannot store, a uid with no producer, I3 against Part D, and tests). Branch `design-replicator-pinned-root`, off `feat/carloku-metered-replicator-20261001` at `e1280811`. It answers the eleven findings the twentieth review deferred (`DISPOSITIONS-metered-replicator.md`, review twenty) and keeps decisions 10 to 13 and approval H (`DECISIONS-FOR-PAUL.md`, section D, on the launch branch) as parameters. Nothing is built, wired or applied. |
| ## What stands today |
| @@ -20,7 +20,7 @@ The replicator (`internal/ops/litestream_metered.go`, `litestream_owners.go`; pa |
| - `Dir`: a retained descriptor opened with `openat(parent, name, O_DIRECTORY|O_NOFOLLOW|O_RDONLY|O_CLOEXEC|O_NOCTTY)` (the kernel refuses a FIFO or device with `ENOTDIR` at lookup, before any open effect, which is the one existing directory-open test kept through the new shape), with its `(dev, ino, uid, mode)` read by `fstat` at acquisition and kept. Every method takes a basename (one component, not `.`, `..` or anything with a separator; anything else is refused before any syscall) and resolves it relative to this descriptor alone. `Child(name, expect) (*Dir, error)`: `openat` as above, then `fstat` compared with `expect` (the facts of the entry it is acquired from, when the caller has them); the child keeps a pointer to its parent `Dir` and its name, so the retained chain is a stack. `List() ([]Entry, error)`: a fresh open file description each time (`openat(fd, ".", O_DIRECTORY|O_RDONLY|O_CLOEXEC)`, `fstat`-compared with this descriptor, then `fdopendir` on it), never a `dup` (a `dup` shares the offset, so a second listing would start where the first ended); each entry's facts by `fstatat(fd, name, AT_SYMLINK_NOFOLLOW)`. `Readlink(name)`: `readlinkat`. `Walk(spelling, anchor)`: a captured link target, normalised first (repeated separators collapsed, `.` components dropped, a trailing separator or a terminal `.` component remembered as a requirement that the result be a directory, so `file/.` and `file/` are errors as today; a spelling that was empty before normalisation is refused, and one that is only `.` or only a run of `..` resolves to the anchor or the popped directory as a directory result, which is what today's predicate does with `.`), then walked by components from an anchor: a relative spelling from this `Dir`, an absolute one from the pinned `/` root with the platform walk's rules (every component platform-owned) until it enters a pinned platform root, and from there as a relative walk beneath that root; a leading run of `..` pops the retained stack (never `openat(fd, "..")`, whose answer follows a moved directory to its new parent) and is refused when the stack would pass the anchor's root; a `..` after a named component is refused; each named component is `Child`; the last component is `fstatat(..., AT_SYMLINK_NOFOLLOW)` and its kind is returned: directory, regular, link, other (a FIFO, a socket, a device), absent, or error; a spelling with a trailing separator whose last component is not a directory is an error (`regular-file/` does not become the file). The confinement boundary of a walk is the anchor's root: a relative walk cannot leave the tenant root it started in; an absolute walk must end beneath a pinned platform root or is refused. `OpenRegular(name, looked)`: `openat(fd, name, O_RDONLY|O_NOFOLLOW|O_NONBLOCK|O_NOCTTY|O_CLOEXEC)` then `fstat`, refused unless regular and equal to `looked`'s `(dev, ino)`. `Recheck()`: for this `Dir` and every ancestor on its stack, `fstatat(parent, name, AT_SYMLINK_NOFOLLOW)` compared with the retained child's facts; a difference anywhere is a changed edge. `Facts()`. |
| - `Root`: a `Dir` reached by a walk from `/` (`open("/")` then `Child` per component), each component required to be the platform's (root's or this process's user's), not writable by group or others; the walk's `(dev, ino)` list is the root's *ancestry*. A root is built once per fleet and kept; `lookNow` rebuilds the ancestry from `/` and compares it, and a difference is a change the fleet goes stale on. |
| -- Tenant roots: a unit route's tree is reached from a platform root through the platform-link predicate exactly as `resolvePlatformLinks` has it today (the link itself the platform's, its directory the platform's, its target's `..` only in a leading run, the target's parent the platform's, no further link in the target), evaluated on descriptors: the link read with `Readlink` from its directory's `Dir`, its target walked with `Walk` (anchored as the spelling says: relative from the link's directory, absolute from the pinned `/`), and the resulting `Dir` the tenant root; that predicate is one function, kept separate from the inventory's classification of links (below), and its restrictions do not change here. A file claim's chain top is found as today (upward by ownership from the file's directory) but every directory of the chain is then acquired downward from the platform root as `Dir`s, and the chain is the file's retained stack. Astra 2's layout (a tenant-owned component above a platform-owned one) fails the platform walk; no root is built over it. |
| +- Tenant roots: a unit route's tree is reached from a platform root through the platform-link predicate exactly as `resolvePlatformLinks` has it today (the link itself the platform's, its directory the platform's, its target's `..` only in a leading run, the target's parent the platform's, no further link in the target), evaluated on descriptors: the link read with `Readlink` from its directory's `Dir`, its target walked with `Walk` (anchored as the spelling says: relative from the link's directory, absolute from the pinned `/`), and the resulting `Dir` the tenant root; that predicate is one function, kept separate from the inventory's classification of links (below), and its restrictions do not change here. A file claim's chain top is found as today (upward by ownership from the file's directory) but every directory of the chain is then acquired downward from the platform root as `Dir`s, and the chain is the file's retained stack. Astra 2's layout (a tenant-owned component above a platform-owned one) is where the platform walk ends: the root is the tenant-owned component's parent, no root is built over the tenant's directory, and the directories beneath are the tenant's chain, retained and rechecked, so the swap that layout allows is a changed edge at the next look and never a steer (the plan's root rule, revision 12). |
| **Every inventory operation runs on `Dir`s**: `strictTreeDBs` takes a `*Dir`, lists with `List`, descends with `Child(name, entry facts)`, reads a header with `OpenRegular`, and classifies a link's captured spelling with `Walk` on the directory that listed it. The classification is explicit and conservative, and nothing disappears through an unnamed case: a target that is a directory, or a link whose name is a database name, is "a link that may lead to a database" (a problem, as today); a regular file is a skipped link entry (as today); a target that is itself a link is "a link to a link that may lead to a database" (a problem; today's classification followed it, and the only honest replacement is to refuse rather than follow); a target that is other than a file or directory is "a link to something that is neither a file nor a directory" (a problem); a target outside the root is "a link out of the tree" (a problem); absent is churn (as today); any error is "a link whose target cannot be looked at" (as today). The namespace question for an isolated instance (its own mount namespace may resolve a spelling elsewhere) is a declared support condition before wiring: a route whose instance runs in its own mount namespace is classified in the agent's namespace and rendered as such, and until the condition is declared it is a refusal rule, not a silence. |
| @@ -44,7 +44,7 @@ The VFS is the plan's first spike and it must prove, before anything else of Par |
| **Proof and admission are two things.** Fleet construction, `Open` and every watch tick ask the ownership *proof* (today's `admit`: the witness unchanged, the fleet not stale) and create nothing; only the request gate (`lease.RequestGate`, asked by the helper for each physical request) performs an *admission*, which is a proof followed by a row, so an idle healthy fleet has no admission and nothing bars its days. **The admission is a row, written before the send, with one ordering point against discovery.** Under a *discovery mutex* on the fleet (a new requirement: today `admit` and `lookAgain` do not share one, and the admission comments allow a discovery to run unlocked; this design serialises registration, the recheck and the start of a discovery on it) the gate checks the proof and, if the fleet is not stale and no discovery is in progress, registers the admission in the fleet's in-memory *pending set* with its `admitted_at`; it releases the mutex and commits the row in the durability ledger's own database (`admissions`: `token`, `fleet_id`, `owner account and app`, `admitted_at`, `transport_done_at`, `recorder_started_at`, `recorder_done_at`, `settled_at`, `how`), `synchronous=FULL`, within a bounded budget; after the commit it takes the mutex again and, if the fleet went stale or a discovery began since registration, marks the row `unissued` and refuses. Discovery, under the same mutex, sets the in-progress flag and counts the pending set, durable or not yet, as unsettled admissions when it computes its lower bound, and a registered admission that the discovery has counted and that then commits is inside the uncertainty (it was counted); one that commits after the discovery began is refused. So an admission is either counted by every discovery that follows its registration or refused, and Astra's 23:58:10 sequence (proof, discovery, commit, a send on the 4th) ends in a refusal at the recheck, not a send outside the uncertainty. The gate waits for the commit. A commit that does not return within the budget refuses the request (`ErrAdmissionNotDurable`; litestream sees a failed upload and does not advance) and the goroutine that awaited it stays to see the commit land and then marks the row `how = unissued, settled_at = now` in its own transaction, so a late commit never authorises anything and never blocks a day; if the process dies between, recovery classifies the row as below. The meter's `Start` row carries the token (`Start` may still fail or be late, as today; what is now mandatory is the admission, not the `Start`). An admission has two participants, the transport and the recorder, and is settled by whichever finishes last. The transport writes `transport_done_at` from the SDK middleware's finalize step, or `transport-lost` when its helper is proven gone (below). The recorder writes `recorder_done_at` with its outcome: `finished` (`Finish`), `abandoned` (`Abandon`), or `unrecorded` (the request path, when `Start` failed and made no attempt row, which today calls neither; it now writes this terminal); the recorder participant is never synthesised while its worker can still run: today's meter lets recording workers outlive the request path, and the admission stays unsettled (barring) until that worker finishes or abandons. The recorder runs in the agent, fed by the helper's messages over the fleet socket (`Start`, `Finish`, `Abandon` with the token), and the agent *registers* the recorder participant when it receives `Start` (`recorder_started_at`, under an agent-owned lifecycle guard per helper); a `Start` that the agent receives after it has begun the helper's termination handling is refused (the guard is closed first, then the socket drained, so registration and exit handling cannot interleave). The participant that finds the other already done writes `settled_at = max(transport_done_at, recorder_done_at)` and `how`, and performs, in that same transaction, the uncertainty extension or closure below. **The transition table is authoritative and is the plan's model:** states `registered` (in memory only), `admitted` (durable), `transport done`, `recorder done`, `settled`; edges `registered → admitted` (the commit), `admitted → unissued` (the recheck after the commit found the fleet stale or a discovery begun, or the gate's budget elapsed before the answer was returned: the row is rewritten as the terminal and the helper never receives the token), `registered → cancelled` (the admission writer has returned with a definite failure, or the writer's handle is closed and a fresh read then finds no row: the registration leaves the pending set), `admitted → transport done | recorder done`, `transport done → settled` (the recorder's completion), `recorder done → settled` (the transport's completion or `transport-lost`), and `admitted | transport done | recorder done → recovered-unknown` at process recovery only (all missing timestamps `now`); `transport-lost` is a transport completion, not a terminal, so an admission whose recorder is still running after its helper died stays `recorder pending`. **The pending set has one contract.** It holds every registration not yet resolved, and an entry leaves it in exactly two ways, both under the discovery mutex: (a) the admission row is durable, at which point the row carries the responsibility (the removal follows the commit, so a discovery sees the entry or the row and never neither); (b) the cancellation is *conclusive*, at which point the empty-settlement result is recorded for every discovery that counted it (the pending-discovery latch's counted set, or the open uncertainty's closure test, which ranges over rows and pending registrations alike), so an uncertainty that waited only on it closes then. The same empty-settlement rule applies to a counted admission that ends `unissued`. **A cancellation is conclusive only when the admission writer can no longer commit:** the writer is one goroutine owning the transaction, the gate's budget expiring refuses the request but does not resolve the registration, and the registration waits for the writer to return. A writer that returns having committed takes `admitted` (then `unissued`, since the request was refused); one that returns with a definite failure takes `cancelled`; one that cannot tell (the connection lost around the commit) is resolved by a read-back by `token` that begins only after the writer's handle is closed, on a read transaction opened after that close, so nothing it fails to find can be committed afterwards: a row found takes `admitted → unissued`; none found takes `cancelled`; the read is retried until it answers. There is no edge out of `cancelled`. Astra's sequence (the writer still running at the budget's expiry and the discovery's count, a read at 23:58:00.040, the commit at 23:58:00.050) cannot declare a cancellation, because the read waits for the writer; the writer's commit resolves it as `unissued`, and the discovery that counted it is told once. Every terminal write is keyed by `token` and idempotent, retried until it commits. Admissions are the ledger's pending state moved to before the send and given an owner. |
| -**Two kinds of hold.** A *barrier* is the ledger's completeness rule: a day is not published complete for an admission's owner while an admission of that owner admitted on or before that day is unsettled (box-wide while Part C2's owner scope is not in use); it is a query over `admissions`, writes nothing, and lifts at settlement. An *uncertainty* is a row (`uncertainties`: `id`, `fleet_id`, `from`, `to` null while open, `reason`) written when a fleet's authority is found changed (by the witness, by `Recheck`, by a changed file), in the transaction that also writes the holds for the days of `[from, now]` for every owner the uncertainty reaches: `bound = min(the registry proof, the last clean look's start)`; `from = min(bound, the earliest admitted_at among the fleet's admissions that are unsettled, or settled with settled_at >= bound and how not in {unissued, cancelled}) - margin`, the bound fixed before the rows are read and never the running minimum (revision 9: the first two terms alone missed an admission admitted before the look, in flight across the change, and settled clean before the discovery; such a settlement wrote nothing permanent, so its admission instant must bound the uncertainty; revision 10: the threshold is the bound itself, not the look alone; revision 11: every settlement whose transport ran counts, `unrecorded` included, and only the empty terminals, which never sent, are excluded; `transport-lost` is a transport completion, never a value of `how`; a `recovered-unknown` row has its own interval and need not be counted here). **What a look must do for its start to be the bound:** the look's start instant is stamped before its first observation, and every observation it makes (the witness, `Recheck` of every retained chain, the route's recorded invocation against systemd's, the files) sees any change that persists, so a look that finds nothing is clean against every kind of change from its start; the order of the observations within a look does not matter for that, and the earlier-of rule (the proof and the look) is kept as a backstop for a look built without one of them. If discovery finds no admission of the fleet unsettled, the uncertainty is opened and closed in that transaction (`to = discovery + margin`). While an uncertainty is open, settlements of that fleet's admissions extend it: each settlement, in its own transaction, writes the holds for every day of `[its admitted_at, its settled_at]` for every owner the uncertainty reaches, and the last settlement closes it with `to = max(discovery, that settlement) + margin` and materialises every day of the closed interval, the margin's extra day across midnight included; recovery uses the same helper and advances the publication watermark past the days it materialised. An admission that settles while no uncertainty of its fleet is open writes nothing permanent: the ordinary day of an unchanged fleet becomes complete when its barrier lifts. The multiple-midnights sequence (admitted on the 2nd, the change at 23:58:10, sent on the 3rd, finished on the 4th) holds the 2nd, 3rd and 4th through the settlement's interval, whichever participant finished last; the request completed before the change was noticed is inside the lower bound as today. **If the discovery transaction fails** the barrier is not enough (every admission may already be settled): discovery first latches a *pending discovery* in the ledger's in-memory failure overlay (`latchFailure`, extended to carry the lower bound and the owners reached), which the publisher's snapshot already overlays and detects across a drain (today's `snapshotOnce` and `drainFailures` behaviour, extended rather than replaced: a failed drain must not stop the overlay from making the snapshot incomplete), so every day from the bound reads incomplete for those owners until the uncertainty commits; the fleet refuses every request meanwhile and retries the write. The latch carries the lower bound, the owners reached and the tokens of the admissions the discovery counted; settlements of those admissions meanwhile find no uncertainty open and write no hold, but their rows record `settled_at` as always. **The replay is one transaction:** it opens the uncertainty with the original `from`; for each counted token whose row is settled it writes the holds for `[its admitted_at, its settled_at]` for every owner reached (and for a token that was cancelled or unissued, nothing); if any counted token is still unsettled the uncertainty stays open and the usual settlement rule closes it later; otherwise it closes with `to = replay instant + margin` (which is no earlier than the discovery and every counted settlement) and materialises every day of the closed interval; the overlay clears only after that commit, so no snapshot sees neither the latch nor the row. Astra's sequence (T counted at 23:58:10 on the 2nd, the write failing, T settling at 00:02 on the 4th, the replay at 00:03) writes the 2nd, 3rd and 4th through T's interval and closes at 00:04 on the 4th, and a successor constructed after the replay backfills that closed interval for its owners; the fleet cannot be marked `closed` while the discovery is uncommitted (above); a process that dies with the latch held leaves a fleet not `closed`, which recovery covers from the fleet's `opened_at`. |
| +**Two kinds of hold.** A *barrier* is the ledger's completeness rule: a day is not published complete for an admission's owner while an admission of that owner admitted on or before that day is unsettled (box-wide while Part C2's owner scope is not in use); it is a query over `admissions`, writes nothing, and lifts at settlement. An *uncertainty* is a row (`uncertainties`: `id`, `fleet_id`, `from`, `to` null while open, `reason`) written when a fleet's authority is found changed (by the witness, by `Recheck`, by a changed file), in the transaction that also writes the holds for the days of `[from, now]` for every owner the uncertainty reaches: `bound = min(the registry proof, the last clean look's start)`; `from = min(bound, the earliest admitted_at among the fleet's admissions that are unsettled, or settled with settled_at >= bound and how not in {unissued, cancelled}) - margin`, the bound fixed before the rows are read and never the running minimum (revision 9: the first two terms alone missed an admission admitted before the look, in flight across the change, and settled clean before the discovery; such a settlement wrote nothing permanent, so its admission instant must bound the uncertainty; revision 10: the threshold is the bound itself, not the look alone; revision 11: every settlement whose transport ran counts, `unrecorded` included, and only the empty terminals, which never sent, are excluded; `transport-lost` is a transport completion, never a value of `how`; a `recovered-unknown` row has its own interval and need not be counted here). **What a look must do for its start to be the bound:** the look's start instant is stamped before its first observation, and every observation it makes (the witness, `Recheck` of every retained chain, the route's recorded invocation against systemd's, the files) sees any change that persists, so a look that finds nothing is clean against every kind of change from its start; the order of the observations within a look does not matter for that; the earlier-of rule (the proof and the look) is kept as a backstop for a look built without the witness observation only, since a registry proof can stand in for a missed witness reading and for nothing else (the chains, the invocation and the files are mandatory observations). If discovery finds no admission of the fleet unsettled, the uncertainty is opened and closed in that transaction (`to = discovery + margin`). While an uncertainty is open, settlements of that fleet's admissions extend it: each settlement, in its own transaction, writes the holds for every day of `[its admitted_at, its settled_at]` for every owner the uncertainty reaches, and the last settlement closes it with `to = max(discovery, that settlement) + margin` and materialises every day of the closed interval, the margin's extra day across midnight included; recovery uses the same helper and advances the publication watermark past the days it materialised. An admission that settles while no uncertainty of its fleet is open writes nothing permanent: the ordinary day of an unchanged fleet becomes complete when its barrier lifts. The multiple-midnights sequence (admitted on the 2nd, the change at 23:58:10, sent on the 3rd, finished on the 4th) holds the 2nd, 3rd and 4th through the settlement's interval, whichever participant finished last; the request completed before the change was noticed is inside the lower bound as today. **If the discovery transaction fails** the barrier is not enough (every admission may already be settled): discovery first latches a *pending discovery* in the ledger's in-memory failure overlay (`latchFailure`, extended to carry the lower bound and the owners reached), which the publisher's snapshot already overlays and detects across a drain (today's `snapshotOnce` and `drainFailures` behaviour, extended rather than replaced: a failed drain must not stop the overlay from making the snapshot incomplete), so every day from the bound reads incomplete for those owners until the uncertainty commits; the fleet refuses every request meanwhile and retries the write. The latch carries the lower bound, the owners reached and the tokens of the admissions the discovery counted; settlements of those admissions meanwhile find no uncertainty open and write no hold, but their rows record `settled_at` as always. **The replay is one transaction:** it opens the uncertainty with the original `from`; for each counted token whose row is settled it writes the holds for `[its admitted_at, its settled_at]` for every owner reached (and for a token that was cancelled or unissued, nothing); if any counted token is still unsettled the uncertainty stays open and the usual settlement rule closes it later; otherwise it closes with `to = replay instant + margin` (which is no earlier than the discovery and every counted settlement) and materialises every day of the closed interval; the overlay clears only after that commit, so no snapshot sees neither the latch nor the row. Astra's sequence (T counted at 23:58:10 on the 2nd, the write failing, T settling at 00:02 on the 4th, the replay at 00:03) writes the 2nd, 3rd and 4th through T's interval and closes at 00:04 on the 4th, and a successor constructed after the replay backfills that closed interval for its owners; the fleet cannot be marked `closed` while the discovery is uncommitted (above); a process that dies with the latch held leaves a fleet not `closed`, which recovery covers from the fleet's `opened_at`. |
| **Who an uncertainty reaches, and what a fleet inherits.** Reach is computed over the chain, not stored per uncertainty: every fleet row names its predecessor, and a fleet's *unacknowledged ancestry* is the set of ancestors reached by following predecessors through fleets that have not been acknowledged (an acknowledged step ends the walk, because acknowledgement copies). An uncertainty, whenever it is opened, extended or closed, writes holds for the owners of its fleet and of every fleet whose unacknowledged ancestry contains that fleet (a query at write time, so an uncertainty discovered after a successor's construction reaches it, and so does one on a grandparent when the parent closed unopened without acknowledging); and **construction backfills**: the transaction that writes a new fleet's `open` row also writes, for the new fleet's owners, the holds of every interval already held by any uncertainty of its unacknowledged ancestry, open or closed (a successor built at 23:58:20 against a fleet whose uncertainty closed at 23:58:10 holds the 2nd for its owners from its first instant, before any acknowledgement), and the publisher's pending-discovery overlay reaches a fleet's owners through the same ancestry query; the barrier for an owner consults the unsettled admissions of its fleet and of its whole unacknowledged ancestry. Acknowledgement is terminal and **root-first**: a fleet may acknowledge its predecessor only when the predecessor is `closed` and has itself acknowledged its own predecessor (or has none), and it copies the predecessor's effective set (own and inherited, open and closed) into its inherited set (`inherited`: `fleet_id`, `uncertainty_id`) and writes holds for the closed ones its owners have not been held for, in one transaction; after that the chain walk stops there and the copy carries on. Astra's sequence: A has U open and an admission unsettled; B is built (its ancestry: A) and closes unopened without acknowledging; C is built against B (its ancestry: B, then A, since B has not acknowledged); on the 4th A's settlement extends U and the write reaches X, B's owners and C's owners through the ancestry; when A closes, B acknowledges A (root-first), then C acknowledges B and inherits U. Under owner scope the October sequence: Y, built at 23:58:20 with A in its ancestry, is barred on the 3rd by A's unsettled admission and held on the 4th by the settlement's write. |
| @@ -75,6 +75,7 @@ The VFS is the plan's first spike and it must prove, before anything else of Par |
| | Another user's directory inside a tenant's tree. | 12 | Today's tree behaviour (walked; its databases listed under the tenant's claim); refusing it is 12's change, not this design's; Part B's sidecar residual has two possible authors until then. | |
| | Which uids are the platform's workloads; a root-run unit's claims; the activator's own claims. | 13, approval H | `trustedUIDs` empty: refused. | |
| | The link shapes a tenant root may be reached through. | compatibility, not policy | Today's predicate, kept as one function, unchanged. | |
| +| Tenant link shapes the inventory refuses | Under Part A's classes a venv's `bin/python3 -> python -> /usr/bin/python3` is a link to a link and a file out of the tree, and a `node_modules/.bin` entry likewise; today these are skipped entries compared with the Accept list, and problems are never compared, so once wired they refuse the build unconditionally. Whether such links refuse the box, the tenant, or are named and passed over is the owner's, beside decisions 10 to 13; a census of link classes on each box's live trees is part of the pre-wiring check. A dangling link, relative or absolute, stays churn. | |
| | An isolated instance's own mount namespace; bind mounts and device nodes inside a tenant tree. | declared support conditions before wiring | A route outside the declared conditions is refused, not classified silently; the conditions are written in the inventory's rendering per route. | |
| | Handover of a database from the activator's hibernation path to the fleet. | not designed | Refused as a conflict of authorities, as today. | |
diff --git a/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-spikes.md b/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-spikes.md| index f34f93395..8df19ac29 100644 |
| --- a/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-spikes.md |
| +++ b/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-spikes.md |
| @@ -20,7 +20,7 @@ Plan step 0's second spike is code, kept in tree at `internal/ops/admissionmodel |
| - **The named sequences are tests**, one each: an unchanged fleet holds nothing; the three-midnight interval in both completion orders (closes at the settlement plus the margin); Astra's commit-after-discovery (unissued, the uncertainty closed by the empty settlement); review 7's budget expiry then commit; a definite failure cancels and an ambiguous commit is read back either way with the pending set empty after; zero-pending discovery closes at once and holds the margin's extra day across midnight; a failed discovery's latch bars and refuses until the replay, which with nothing pending closes at the replay instant plus the margin; review 6's replay with a counted settlement (the 2nd, 3rd and 4th held, closed at 00:04, a successor built after the replay backfilled); Astra's A, B, C construction and root-first acknowledgement (C refused before B acknowledged A; C inherits U); helper death before and after recorder registration, a `Start` refused after the guard closed; the fence outliving the agent (recovery refused until the helper is gone; recovered-unknown covers the transport that finished after the crash); a healthy chain contributing nothing; and the lower bound reaching a request completed before the change was noticed (the clean look minus the margin). |
| - **The boundaries are separate actions** (after the first code review): the writer's commit, the gate's answer (the recheck), a definite failure, an ambiguous return and its read-back, the send (the helper beginning the transport, which needs a durable issued row, recorded as a violation otherwise; a dead helper's send is a no-op, since a dead helper does nothing), the helper's proven exit (the guard closes) and the drain (transport-lost, unrecorded) are each their own step, so a schedule can put a discovery, a budget expiry or a crash between a commit and its answer, hold a token for a day before sending, or ask for a recorder's `Start` between the exit and the drain. The composites the named tests use (`WriterReturns`, `HelperDies`) are those steps in order. |
| -- **Randomised schedules**: every action above in random interleavings over about four days, 120 steps a schedule, one owner per fleet (owner scope; the box-wide default is a superset); after each step: the barrier, stated from the chain and not from the publisher (for every fleet and every owner of it, a durable unsettled admission of that fleet or of any fleet in its unacknowledged ancestry bars the owner from its day on, whether or not the token was ever handed out), the hazard invariant (once the ledger knows of a change, every day on which a transport overlapped the true change is incomplete for its owner; after a crash, every sent admission unsettled at the crash likewise; an issued token never sent has no transport and is covered by the barrier and recovery, not this oracle), a terminal state never changing, and a closed fleet having no latch or pending registration; at quiescence every fleet's owners hold every interval of every uncertainty on its whole chain, no uncertainty is open, and a day with no uncertainty reaching an owner is complete. `ADMISSION_MODEL_SEEDS=30000` passes in about six seconds; the default is 400. Publication is revisable by construction: a day published complete can be re-marked by a later hold (the design's materialisation and watermark), and the hazard invariant is satisfied by that retraction; the barrier's purpose is to make retraction rare, not to make it unnecessary. |
| +- **Randomised schedules**: every action above in random interleavings over about four days, 120 steps a schedule, one owner per fleet (owner scope; the box-wide default is a superset); after each step, and right after a recovery before anything else runs: the barrier, stated from the chain and not from the publisher (for every fleet and every owner of it, a durable unsettled admission of that fleet or of any fleet in its unacknowledged ancestry bars the owner from its day on, whether or not the token was ever handed out), the hazard invariant (once the ledger knows of a change, every day on which a transport overlapped the true change is incomplete for its owner; after a crash, every sent admission unsettled at the crash likewise; an issued token never sent has no transport and is covered by the barrier and recovery, not this oracle), a terminal state never changing, and a closed fleet having no latch or pending registration; at quiescence every fleet's owners hold every interval of every uncertainty on its whole chain, no uncertainty is open, and a day with no uncertainty reaching an owner is complete. `ADMISSION_MODEL_SEEDS=30000` passes in about six seconds; the default is 400. Publication is revisable by construction: a day published complete can be re-marked by a later hold (the design's materialisation and watermark), and the hazard invariant is satisfied by that retraction; the barrier's purpose is to make retraction rare, not to make it unnecessary. |
| - **What the runs found.** The first runs found the model's own bookkeeping wrong (an owner the fleet did not have; a crash-killed helper's transport end unset). After the second code review made the change its own action, discovered by a later look, Opus's sequence (an admission admitted before the last clean look, in flight across a change after it, settled clean before the discovery) showed the design's lower bound missing that admission's earlier day: **revision 9 of the design** adds the admission instant of every admission settled at or after the last clean look's start to the bound, and the mutant restoring revision 8's rule is killed by the named test. That is the one interleaving the design did not name that the model has found, within the actions it represents and the invariants it checks; the claim is no wider than that. |
| - **Mutants, in a tree copy**, each killed by a named test or a schedule. After the fourth code review: the historical term filtered to finished and abandoned (the unrecorded test), the cutoff as the running minimum (the fixed-bound test in both row orders), the historical term using the send instant, the historical term dropped, a later witness change ignored (the proof test). After the third code review: the threshold as the later of proof and look, as the look alone, the historical term including empty terminals, the proof not refusing a moved witness, the crash duplicating a landed row, `Close` without its unsettled loop, the latch overlay reaching the fleet's owners only. After the second code review: revision 8's bound (the in-flight test), the bound without the proof (the proof test), the bound without the clean look (the lower-bound test), a landed ambiguous row not durable until read-back, the open interval dropped from construction backfill (the per-step inheritance check), the replay's per-token writes dropped (the replay-while-open test), a read-back leaving the pending set entry. Removing the closed-implies-no-unsettled *check* removes an oracle; the mechanism mutant it exists for, `Close` without its unsettled-admissions loop, is killed by it (an unanswered durable row). Putting the send's terminal check before its gate is equivalent in the schedules tested (a settled, issued, unsent token exists after a clean stop, but no schedule sends it afterwards). After the first code review: the budget expiry a no-op (its own test, with no discovery); the barrier owner-only, or over issued rows only (the successor-barred test and the schedules); the pending set skipped at registration; the guard ignored, and the drain before the guard; the replay clearing the latch without the lower bound's holds (a named test: the schedules did not reach it in 3000 seeds); the send unchecked; a close without the replay (the closed-implies-no-latch check); acknowledgement copying the predecessor's own uncertainties only (not equivalent: a schedule found a fifth fleet's owner unheld for the first fleet's uncertainty, because the chain's copies are how a closed interval reaches a fleet built after its predecessor acknowledged); the answer ignoring the refusal; a read-back while the writer runs (the named test; the later commit would land on a cancelled registration, which the terminal check reports). Equivalent under this model: the unacknowledged-ancestry walk not stopping at an acknowledged step (over-reach writes the same holds). Not equivalent, though once claimed so here: the registry proof in the bound (Astra's schedule: a build, a clean look a day later, a discovery a day after that, with no admission: only the proof reaches the first day; and Opus's: a witness change known only to be after the proof; both are named tests). First: no construction backfill (reach); the pending set not counted (the commit-after-discovery tests); no recheck after commit (the same test, and a hazard at seed 264); the replay dropping counted settlements (hazard); the replay closing at the discovery instant (liveness and the latch tests); a settlement writing no interval (hazard); recovery ignoring the fence (the fence test); helper death without transport-lost (an uncertainty open at quiescence); the lower bound ignoring the clean look (the lower-bound test; the 30000-schedule sweep alone did not find it, which is why that sequence is a named test); the lower bound ignoring the earliest unsettled admission (liveness, because the model's proof then reached too far); unrecorded synthesised for a registered worker (the guard test); recovery skipping unclosed fleets (an uncertainty open at quiescence); acknowledgement not root-first and reach ignoring successors (reach). An epoch check made redundant by the stale state is the one equivalent mutant, noted in the model. |
| - **What the model does not cover**, by construction: the latch clearing after the commit as two instants (the replay is one step here; the rule stays with the ledger's tests), a snapshot between those instants, the recorder worker's lifetime and the socket drain as separate events, the fence as anything but helper liveness, retries of a failed terminal or replay write, the ledger's commit latency on a box's disk under the replicator's request rate (the spike's other half, which needs a box), Part C2's owner scope beyond the owner-scoped holds the model writes, and process kills inside a transaction (the model's transactions are atomic by fiat; the ledger's are SQLite's). |
diff --git a/internal/authority/authority.go b/internal/authority/authority.go| index 600aaae4e..b1aeb0b3d 100644 |
| --- a/internal/authority/authority.go |
| +++ b/internal/authority/authority.go |
| @@ -12,6 +12,7 @@ type Facts struct { |
| UID uint32 |
| Mode uint32 // the file type and permission bits as the kernel has them |
| Size int64 // for a regular file; not part of Same or Equal, since it changes as the file is written |
| + Nlink uint64 // the number of names; not part of Same or Equal |
| } |
| // Same reports whether two facts name the same object of the same type: |
| @@ -43,11 +44,15 @@ func (k Kind) String() string { |
| return [...]string{"absent", "directory", "regular", "link", "other"}[k] |
| } |
| -// Entry is one listed name with its facts. |
| +// Entry is one listed name with its facts, or, when the name could not be |
| +// looked at after the listing, the error; a listing never fails for one |
| +// entry's sake, so a caller sees every name and says what it could not |
| +// look at. |
| type Entry struct { |
| Name string |
| Kind Kind |
| Facts Facts |
| + Err error |
| } |
| // The refusals. Each is returned before any system call where the input |
diff --git a/internal/authority/authority_unix.go b/internal/authority/authority_unix.go| index c4139caf6..c4d8ce8ab 100644 |
| --- a/internal/authority/authority_unix.go |
| +++ b/internal/authority/authority_unix.go |
| @@ -28,7 +28,7 @@ type Dir struct { |
| } |
| func factsOf(st *unix.Stat_t) Facts { |
| - return Facts{Dev: uint64(st.Dev), Ino: uint64(st.Ino), UID: st.Uid, Mode: uint32(st.Mode), Size: st.Size} |
| + return Facts{Dev: uint64(st.Dev), Ino: uint64(st.Ino), UID: st.Uid, Mode: uint32(st.Mode), Size: st.Size, Nlink: uint64(st.Nlink)} |
| } |
| func kindOf(mode uint32) Kind { |
| @@ -163,7 +163,8 @@ func (d *Dir) List() ([]Entry, error) { |
| } |
| k, fs, err := d.Lookat(n) |
| if err != nil { |
| - return nil, err |
| + out = append(out, Entry{Name: n, Err: err}) |
| + continue |
| } |
| if k == Absent { |
| continue // gone between the listing and the look: churn |
| @@ -348,11 +349,17 @@ func (d *Dir) Walk(spelling string, pins *Pins) (Result, error) { |
| if !entered { |
| child, err := cur.Child(c, nil) |
| if err != nil { |
| - // absent, a file or a link where a directory is needed: the |
| - // spelling leads outside every pin; any other error (EACCES, |
| - // EMFILE, EIO) is itself, with its errno |
| + // nothing by that name: the target is absent (churn to an |
| + // inventory, as a dangling relative target is); a file or a |
| + // link where a directory is needed: the spelling leads |
| + // outside every pin; any other error (EACCES, EMFILE, EIO) |
| + // is itself, with its errno |
| var ee *EdgeError |
| - if errors.As(err, &ee) && (errors.Is(ee.Err, unix.ENOENT) || errors.Is(ee.Err, unix.ENOTDIR) || errors.Is(ee.Err, ErrLink)) { |
| + if errors.As(err, &ee) && errors.Is(ee.Err, unix.ENOENT) { |
| + res.Close() |
| + return Result{Kind: Absent}, nil |
| + } |
| + if errors.As(err, &ee) && (errors.Is(ee.Err, unix.ENOTDIR) || errors.Is(ee.Err, ErrLink)) { |
| return fail(&EdgeError{Name: c, Err: ErrOutside}) |
| } |
| return fail(err) |
diff --git a/internal/authority/authority_unix_test.go b/internal/authority/authority_unix_test.go| index 72a4e0a8d..1a4f09ef9 100644 |
| --- a/internal/authority/authority_unix_test.go |
| +++ b/internal/authority/authority_unix_test.go |
| @@ -439,7 +439,9 @@ func TestWalkClasses(t *testing.T) { |
| {root.Dir, "/other/f", Absent, ErrOutside}, |
| {root.Dir, "/other", Absent, ErrOutside}, |
| {root.Dir, "/w/x", Absent, ErrNotPlatform}, |
| - {root.Dir, "/missing/f", Absent, ErrOutside}, |
| + {root.Dir, "/missing/f", Absent, nil}, |
| + {root.Dir, "/r/f/x", Absent, unix.ENOTDIR}, |
| + {root.Dir, "/other/f/x", Absent, ErrOutside}, |
| {root.Dir, "/r/../other/f", Absent, ErrAscentAfterName}, |
| {root.Dir, "/../r/f", Absent, ErrAscentPastRoot}, |
| {inner, "/r/nothing", Absent, nil}, |
diff --git a/internal/ops/admissionmodel/model.go b/internal/ops/admissionmodel/model.go| index 2fc809157..2bbd9c9cc 100644 |
| --- a/internal/ops/admissionmodel/model.go |
| +++ b/internal/ops/admissionmodel/model.go |
| @@ -308,9 +308,9 @@ func (m *Model) Change(f *Fleet) { |
| } |
| // WitnessChange is the truth: the registry's ownership witness moves now, |
| -// unseen. The design has every look read the witness first, so a look |
| -// discovers it too; the proof at the next registration discovers it as |
| -// well. A model in which the look misses it (the mutant the third review |
| +// unseen. The design has every look observe the witness (among the chains, |
| +// the invocation and the files, in any order), so a look discovers it too; |
| +// the proof at the next registration discovers it as well. A model in which the look misses it (the mutant the third review |
| // named) leaves a request completed between the change and the look |
| // unheld unless the bound's threshold is the earlier of proof and look. |
| func (m *Model) WitnessChange(f *Fleet) { |
diff --git a/internal/ops/admissionmodel/model_test.go b/internal/ops/admissionmodel/model_test.go| index d40ea9377..1f3292b50 100644 |
| --- a/internal/ops/admissionmodel/model_test.go |
| +++ b/internal/ops/admissionmodel/model_test.go |
| @@ -664,8 +664,8 @@ func TestLowerBoundReachesInFlightAcrossCleanLook(t *testing.T) { |
| // Review 3 (Opus P2-1): a witness change between the last proof and a |
| // clean look; a request in flight across it settles before the look; the |
| // bound's threshold is the earlier of proof and look, so its day is held. |
| -// In the design every look reads the witness first and discovers at once; |
| -// here the look is told to miss it, which is the stronger case. |
| +// In the design every look observes the witness (in any order) and |
| +// discovers; here the look is told to miss it, which is the stronger case. |
| func TestLowerBoundThresholdIsTheEarlierOfProofAndLook(t *testing.T) { |
| m := New(1) |
| m.Now = at(1, 9, 0) |
| @@ -838,6 +838,42 @@ func TestLaterWitnessChangeRefusesTheProof(t *testing.T) { |
| } |
| } |
| +// Review 5 (Opus): a settlement exactly at the bound counts (>=, not >). |
| +func TestHistoricalTermAtTheBound(t *testing.T) { |
| + m := New(1) |
| + m.Now = at(1, 9, 0) |
| + f := m.Build(alice) |
| + m.Now = at(1, 23, 0) |
| + a := issue(t, m, f, alice) |
| + m.Now = at(2, 0, 30) |
| + m.Look(f) // the bound, if the proof is later |
| + finish(m, a) // settled at the bound's instant |
| + m.Now = at(2, 0, 35) |
| + p := m.Register(f, alice) |
| + m.WriterFails(p) |
| + m.Now = at(2, 0, 40) |
| + m.Change(f) |
| + m.Look(f) |
| + if u := f.Uncertainties[0]; u.From != at(1, 22, 59) { |
| + t.Fatalf("from %d, want %d: a settlement at the bound counts", u.From, at(1, 22, 59)) |
| + } |
| +} |
| + |
| +// Review 5 (Opus): a witness change after a file change keeps the first |
| +// change as the truth's instant. |
| +func TestWitnessChangeKeepsTheFirstChangeInstant(t *testing.T) { |
| + m := New(1) |
| + m.Now = at(1, 9, 0) |
| + f := m.Build(alice) |
| + m.Now = at(1, 10, 0) |
| + m.Change(f) |
| + m.Now = at(1, 11, 0) |
| + m.WitnessChange(f) |
| + if f.ChangedAt != at(1, 10, 0) { |
| + t.Fatalf("ChangedAt %d, want the first change %d", f.ChangedAt, at(1, 10, 0)) |
| + } |
| +} |
| + |
| // Review 4 (both): the cutoff is the bound, fixed before the rows are |
| // read, so a settlement before the bound never qualifies through another |
| // row's earlier admission, in either row order. |
| @@ -1109,7 +1145,11 @@ func runSchedule(seed int64, steps int) (err error) { |
| case 1: |
| m.SurvivingHelperDies(m.Fleets[m.rnd.Intn(len(m.Fleets))]) |
| default: |
| - m.Recover() |
| + if m.Recover() { |
| + if err := m.Check(); err != nil { // the invariants right after recovery, before anything repairs it |
| + return fmt.Errorf("step %d (after recovery): %v\n%s", i, err, dump(m)) |
| + } |
| + } |
| } |
| continue |
| } |
diff --git a/internal/ops/authoritybridge.go b/internal/ops/authoritybridge.go| index f60078ed1..5658db846 100644 |
| --- a/internal/ops/authoritybridge.go |
| +++ b/internal/ops/authoritybridge.go |
| @@ -22,12 +22,16 @@ var authFacts = func(name string, f authority.Facts) authority.Facts { return f |
| // platformFacts is the platform rule's uid half for the authority package. |
| func platformFacts(f authority.Facts) bool { return platformUID(f.UID) } |
| -// fleetPins is the pinned "/" and the platform roots a fleet's inventory |
| -// walks from, built once per inventory and closed with it. A root is the |
| -// parent of the first platform link's directory in a spelled path, or the |
| -// parent of the spelled directory when it has none (the plan's rule), so |
| -// systemd's "../private/..." pops from the link's directory to the root |
| -// and no further. |
| +// fleetPins is the pinned "/" and the platform roots an inventory walks |
| +// from, built for one tree walk or one claim check today (the fleet will |
| +// hold them for its life once owned files keep their chains) and closed |
| +// with it. A root is where the platform's authority ends in a spelled |
| +// path (the plan's rule): above the first link's directory by as many |
| +// levels as the link's target ascends (systemd's "../private/..." pops |
| +// from the link's directory to the root and no further; "../../x" needs |
| +// one more), the parent of the first component that is not the |
| +// platform's, or the parent of the spelled directory; and the parent of |
| +// an absolute platform-link target, so that walk enters a pin. |
| type fleetPins struct { |
| slash *authority.Dir |
| pins *authority.Pins |
| @@ -111,6 +115,40 @@ func (p *fleetPins) root(comps []string) (*authority.Root, string) { |
| return r, "" |
| } |
| +// linkIn descends the components from the slash with no rule but |
| +// O_NOFOLLOW and names the first that is a link, or "": the predicate's |
| +// "holds no further link" said before anything about ownership. |
| +func (p *fleetPins) linkIn(comps []string) string { |
| + cur := p.slash |
| + var opened []*authority.Dir |
| + defer func() { |
| + for i := len(opened) - 1; i >= 0; i-- { |
| + opened[i].Close() |
| + } |
| + }() |
| + at := filepath.Clean(slashPath) |
| + for i, c := range comps { |
| + at = filepath.Join(at, c) |
| + if i == len(comps)-1 { |
| + if k, _, err := cur.Lookat(c); err == nil && k == authority.Link { |
| + return at |
| + } |
| + return "" |
| + } |
| + child, err := cur.Child(c, nil) |
| + if err != nil { |
| + var ee *authority.EdgeError |
| + if errors.As(err, &ee) && errors.Is(ee.Err, authority.ErrLink) { |
| + return at |
| + } |
| + return "" |
| + } |
| + opened = append(opened, child) |
| + cur = child |
| + } |
| + return "" |
| +} |
| + |
| // walk is Walk with the pinned slash's spelling applied: on a box the |
| // slash is "/" and an absolute target is walked as spelled; under a test's |
| // pinned slash an absolute target beneath it is walked from there. |
| @@ -127,10 +165,12 @@ func (p *fleetPins) walk(d *authority.Dir, dest string) (authority.Result, error |
| // tenantRoot acquires the directory a path spells, through the platform |
| // walk to its root and then the platform-link predicate on descriptors |
| -// (exactly resolvePlatformLinks's rule: a link is followed when the link |
| +// (today's rule, within the pinned roots: a link is followed when the link |
| // and its directory are the platform's, its target holds no further link |
| -// and no ".." past its leading run, and the target's parent is the |
| -// platform's). The result is confined: a relative walk from it or beneath |
| +// and no ".." past its leading run, the target's directories from the |
| +// root down are the platform's, and the target's parent is the |
| +// platform's; a target that leaves every pinned root is refused where |
| +// today's pathname walk would have followed it). The result is confined: a relative walk from it or beneath |
| // it cannot leave it. The caller releases what was acquired with the |
| // returned func; the root stays with the pins. The reason, when the |
| // directory cannot be reached, is a string the inventory renders. |
| @@ -149,9 +189,16 @@ func (p *fleetPins) tenantRoot(dir string) (*authority.Dir, func(), string) { |
| owned[i].Close() |
| } |
| } |
| - // the first link in the spelling decides the root: the parent of the |
| - // link's directory, or the parent of the spelled directory |
| - linkAt := -1 |
| + // Where the platform's authority ends decides the root: the first |
| + // component that is a link puts the root at the parent of the link's |
| + // directory (so a relative target may pop to it and no further); the |
| + // first component that is not the platform's (another user's, or |
| + // writable by group or others) puts the root at its parent; a spelling |
| + // with neither puts it at the parent of the spelled directory. Every |
| + // component of the root is then required to be the platform's by |
| + // OpenRoot, and what lies beneath it is checked by the predicate and |
| + // the claim's own rules. |
| + rootN := len(comps) - 1 |
| { |
| cur := p.slash |
| var probe []*authority.Dir |
| @@ -160,21 +207,39 @@ func (p *fleetPins) tenantRoot(dir string) (*authority.Dir, func(), string) { |
| if err != nil { |
| var ee *authority.EdgeError |
| if errors.As(err, &ee) && errors.Is(ee.Err, authority.ErrLink) { |
| - linkAt = i |
| + // as many levels above the link's directory as the |
| + // target's leading ascent needs, one at least |
| + ups := 1 |
| + if target, rerr := cur.Readlink(c); rerr == nil && !filepath.IsAbs(target) { |
| + n := 0 |
| + for _, part := range strings.Split(target, "/") { |
| + if part == "" || part == "." { |
| + continue |
| + } |
| + if part != ".." { |
| + break |
| + } |
| + n++ |
| + } |
| + if n > ups { |
| + ups = n |
| + } |
| + } |
| + rootN = i - ups |
| } |
| break |
| } |
| probe = append(probe, child) |
| + if f := authFacts(c, child.Facts()); !platformUID(f.UID) || f.Mode&0o022 != 0 { |
| + rootN = i |
| + break |
| + } |
| cur = child |
| } |
| for i := len(probe) - 1; i >= 0; i-- { |
| probe[i].Close() |
| } |
| } |
| - rootN := len(comps) - 1 |
| - if linkAt >= 0 { |
| - rootN = linkAt - 1 |
| - } |
| if rootN < 0 { |
| rootN = 0 |
| } |
| @@ -215,11 +280,32 @@ func (p *fleetPins) tenantRoot(dir string) (*authority.Dir, func(), string) { |
| release() |
| return nil, nil, "a link in its path cannot be read" |
| } |
| + // An absolute platform-link target pins its parent as a root (the |
| + // platform walk from the slash requires every component to the |
| + // parent to be the platform's, as today's predicate did), so the |
| + // walk enters it; a relative one pops to this spelling's root. |
| + if filepath.IsAbs(target) { |
| + if tc, ok := p.comps(target); ok && len(tc) > 0 { |
| + // a further link anywhere in the target is said before its |
| + // directories' ownership, as today's predicate said it |
| + if at := p.linkIn(tc); at != "" { |
| + release() |
| + return nil, nil, "a link in its path leads through another link: " + at + " is a link" |
| + } |
| + if _, why := p.root(tc[:len(tc)-1]); why != "" { |
| + release() |
| + return nil, nil, "a link in its path leads to a directory that is not under the platform's: " + why |
| + } |
| + } |
| + } |
| res, werr := p.walk(cur, target) |
| switch { |
| case errors.Is(werr, authority.ErrAscentAfterName): |
| release() |
| return nil, nil, "a link in its path leads through another link: " + target + " steps up after a named component" |
| + case errors.Is(werr, authority.ErrAscentPastRoot), errors.Is(werr, authority.ErrOutside), errors.Is(werr, authority.ErrNotPlatform): |
| + release() |
| + return nil, nil, "a link in its path leads out of its platform root: " + target |
| case werr != nil: |
| release() |
| return nil, nil, "a link in its path leads through another link: " + target + " cannot be looked at" |
diff --git a/internal/ops/litestream_owners.go b/internal/ops/litestream_owners.go| index 399c57234..9f2711569 100644 |
| --- a/internal/ops/litestream_owners.go |
| +++ b/internal/ops/litestream_owners.go |
| @@ -3,9 +3,7 @@ package ops |
| import ( |
| "errors" |
| "fmt" |
| - "github.com/carlosframework/platform/internal/authority" |
| "io" |
| - "io/fs" |
| "os" |
| "path/filepath" |
| "sort" |
| @@ -13,6 +11,7 @@ import ( |
| "strings" |
| "syscall" |
| + "github.com/carlosframework/platform/internal/authority" |
| "github.com/carlosframework/platform/internal/registry" |
| "github.com/carlosframework/platform/internal/release" |
| ) |
| @@ -394,95 +393,85 @@ const ( |
| // RESOLVE_BENEATH and RESOLVE_NO_SYMLINKS) and to keep the opened |
| // identity, which is a change to litestream and is not made. |
| func checkOwnedFile(root, path string, kind claimKind) (fileFacts, string) { |
| - rootInfo, err := os.Stat(root) |
| - if err != nil || !rootInfo.IsDir() { |
| + root, path = filepath.Clean(root), filepath.Clean(path) |
| + rel, err := filepath.Rel(root, path) |
| + if err != nil || rel == "." || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) { |
| + return fileFacts{}, "it is not beneath the directory it is claimed from" |
| + } |
| + pins, perr := newFleetPins() |
| + if perr != nil { |
| return fileFacts{}, "the directory it is claimed from cannot be looked at" |
| } |
| - rootFacts, ok := factsOf(rootInfo) |
| - if !ok { |
| - return fileFacts{}, "file ownership cannot be read on this platform" |
| + defer pins.Close() |
| + // The spelled directory is acquired through the platform walk to its |
| + // root and the platform-link predicate on descriptors (systemd's |
| + // DynamicUser shape, /var/lib/carlos-app/<host> -> ../private/..., is |
| + // followed when the link and its directory are the platform's, its |
| + // target holds no further link and no ".." past its leading run, and |
| + // the target's parent is the platform's; a link anywhere else is |
| + // refused). The result is the resolved directory with its retained |
| + // chain up to the platform root, and nothing below is resolved by |
| + // pathname again. |
| + dir, release, why := pins.tenantRoot(root) |
| + if why != "" { |
| + if strings.HasPrefix(why, "a link in its path") { |
| + return fileFacts{}, why |
| + } |
| + return fileFacts{}, "the directory it is claimed from cannot be looked at" |
| + } |
| + defer release() |
| + facts := func(d *authority.Dir) authority.Facts { return authFacts(d.Name(), d.Facts()) } |
| + // platformOwnedDir is platformOwned on a retained directory. |
| + platformOwnedDir := func(d *authority.Dir, spelled string) string { |
| + if d == nil { |
| + return "" |
| + } |
| + f := facts(d) |
| + if !platformUID(f.UID) { |
| + return fmt.Sprintf("%s belongs to a user other than root or the platform's", spelled) |
| + } |
| + if f.Mode&0o022 != 0 { |
| + return fmt.Sprintf("%s can be written by its group or by anyone", spelled) |
| + } |
| + return "" |
| } |
| + rootFacts := facts(dir) |
| switch kind { |
| case platformDirectory: |
| - if why := platformOwned(root); why != "" { |
| + if why := platformOwnedDir(dir, root); why != "" { |
| return fileFacts{}, why |
| } |
| case tenantTree: |
| - // As spelled, and as resolved: a state directory may be reached |
| - // through a link the operator (or systemd) made, and both the |
| - // directory holding that link and the one holding the real tree |
| - // have to be out of the tenant's reach. |
| - if why := platformOwned(filepath.Dir(root)); why != "" { |
| + // The directory above the tenant's, as resolved: the spelled |
| + // parent of a link was checked by the predicate; the resolved |
| + // parent is the retained chain's. |
| + if why := platformOwnedDir(dir.Parent(), filepath.Dir(root)); why != "" { |
| return fileFacts{}, why |
| } |
| - if real, rerr := filepath.EvalSymlinks(root); rerr != nil { |
| - return fileFacts{}, "the directory it is claimed from cannot be resolved" |
| - } else if filepath.Clean(real) != filepath.Clean(root) { |
| - if why := platformOwned(filepath.Dir(real)); why != "" { |
| - return fileFacts{}, why |
| - } |
| - } |
| case tenantFile: |
| // The root is not the file's directory but the top of the tenant's |
| - // own chain: climb from the file's directory while the directory |
| - // above is the same user's, is not the platform's (root's or this |
| - // process's user's), and is not a link. What is above that must be |
| - // the platform's, and nothing from the top of the chain down may be |
| - // a link, the top included (an operator's link is allowed for a |
| - // tree claim's root, which the route names; this root is inferred). |
| - // Stopping at the platform's own directory matters where the file's |
| - // owner is the platform's user (tests, a single-user box): the |
| - // chain would otherwise climb to a directory nobody here owns. |
| - owner, lerr := os.Lstat(path) |
| - if lerr != nil || !owner.Mode().IsRegular() { |
| + // own chain: climb the retained chain from the file's directory |
| + // while the directory above is the same user's and not the |
| + // platform's (root's or this process's user's). What is above that |
| + // must be the platform's. Nothing in the chain is a link: every |
| + // directory of it was acquired with O_NOFOLLOW. Stopping at the |
| + // platform's own directory matters where the file's owner is the |
| + // platform's user (tests, a single-user box): the chain would |
| + // otherwise climb to a directory nobody here owns. |
| + k, of, lerr := dir.Lookat(filepath.Base(path)) |
| + if lerr != nil || k != authority.Regular { |
| return fileFacts{}, "it is not a regular file" |
| } |
| - ownerFacts, ok := factsOf(owner) |
| - if !ok { |
| - return fileFacts{}, "file ownership cannot be read on this platform" |
| - } |
| - // The spelled directory path is resolved first, one component at |
| - // a time: a link the platform made (systemd's DynamicUser shape: |
| - // /var/lib/carlos-app/<host> is a root-made link to |
| - // /var/lib/private/...) is followed when the link itself and its |
| - // parent are the platform's, its target (absolute, or relative to |
| - // the link's own directory) holds no further link, and the |
| - // target's parent is the platform's; the |
| - // chain is then the resolved one, and a file named beneath such a |
| - // link at any depth is owned under the path the route spells, |
| - // with the resolved file's facts. A link anywhere else in the |
| - // path, the tenant's or anyone's, is refused. |
| - resolved, why := resolvePlatformLinks(root) |
| - if why != "" { |
| - return fileFacts{}, why |
| - } |
| - if resolved != root { |
| - root, path = resolved, filepath.Join(resolved, filepath.Base(path)) |
| - } |
| - // The file's own directory next: it is the bottom of the chain and |
| - // has to be the owner's (or the platform's, and then writable by |
| - // nobody else) like every other directory in it. |
| - start, serr := os.Lstat(root) |
| - if serr != nil { |
| - return fileFacts{}, "its directory cannot be looked at" |
| - } |
| - sf, ok := factsOf(start) |
| - if !ok { |
| - return fileFacts{}, "file ownership cannot be read on this platform" |
| - } |
| - if !start.IsDir() || start.Mode()&os.ModeSymlink != 0 { |
| - return fileFacts{}, "its directory is not a directory" |
| - } |
| + ownerFacts := authFacts(filepath.Base(path), of) |
| + sf := rootFacts |
| if platformUID(sf.UID) { |
| // The platform's own directory: writable by nobody else, or a |
| // local user could put a sidecar link beside the file. And the |
| // file in it is the platform's too: a tenant's file in a |
| // directory the platform owns is not decided (whether it is |
| // billable to the tenant), and until it is, it is refused |
| - // whatever the directory above (review eighteen, Opus P3-2: |
| - // the climb's root decided it one way under a tenant's tree |
| - // and the other under the platform's). |
| - if start.Mode().Perm()&0o022 != 0 { |
| + // whatever the directory above (review eighteen, Opus P3-2). |
| + if sf.Mode&0o022 != 0 { |
| return fileFacts{}, "its directory is the platform's and can be written by its group or by anyone" |
| } |
| if !platformUID(ownerFacts.UID) { |
| @@ -491,76 +480,78 @@ func checkOwnedFile(root, path string, kind claimKind) (fileFacts, string) { |
| } else if sf.UID != ownerFacts.UID { |
| return fileFacts{}, "its directory belongs to a user other than its own or the platform's" |
| } |
| + top := dir |
| for { |
| - parent := filepath.Dir(root) |
| - if parent == root { |
| + parent := top.Parent() |
| + if parent == nil { |
| return fileFacts{}, "its directory chain reaches the root of the filesystem without leaving its owner" |
| } |
| - pfi, perr := os.Lstat(parent) |
| - if perr != nil { |
| - return fileFacts{}, "a directory above it cannot be looked at" |
| - } |
| - pf, ok := factsOf(pfi) |
| - if !ok || pfi.Mode()&os.ModeSymlink != 0 || !pfi.IsDir() || pf.UID != ownerFacts.UID || pf.UID == 0 || int(pf.UID) == os.Geteuid() { |
| + pf := facts(parent) |
| + if pf.UID != ownerFacts.UID || platformUID(pf.UID) { |
| break |
| } |
| - root = parent |
| - } |
| - if top, terr := os.Lstat(root); terr != nil || top.Mode()&os.ModeSymlink != 0 || !top.IsDir() { |
| - return fileFacts{}, "the top of its directory chain is a link or cannot be looked at" |
| - } |
| - // Nothing above the top of the chain is a link either: a link |
| - // higher in the spelled path was resolved above under the |
| - // platform's rules or refused there, so the resolved and spelled |
| - // paths agree. This is a second check against a link put in place |
| - // between the resolution and now; a tenant's link in a stable |
| - // path is refused by resolvePlatformLinks first. |
| - if real, rerr := filepath.EvalSymlinks(root); rerr != nil || filepath.Clean(real) != filepath.Clean(root) { |
| - return fileFacts{}, "a directory above its chain is a link" |
| + if pf.Dev != rootFacts.Dev { |
| + return fileFacts{}, "its path crosses onto another filesystem (" + parent.Name() + ")" |
| + } |
| + top = parent |
| } |
| - if why := platformOwned(filepath.Dir(root)); why != "" { |
| + if why := platformOwnedDir(top.Parent(), "the directory above the top of its chain ("+top.Parent().Name()+")"); why != "" { |
| return fileFacts{}, why |
| } |
| - if rootInfo, err = os.Stat(root); err != nil { |
| - return fileFacts{}, "the directory it is claimed from cannot be looked at" |
| - } |
| - if rootFacts, ok = factsOf(rootInfo); !ok { |
| - return fileFacts{}, "file ownership cannot be read on this platform" |
| - } |
| - } |
| - rel, err := filepath.Rel(root, path) |
| - if err != nil || rel == "." || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) { |
| - return fileFacts{}, "it is not beneath the directory it is claimed from" |
| + rootFacts = facts(top) |
| } |
| - at := root |
| - var last os.FileInfo |
| + // Every component from the claim root down to the file is what it is |
| + // named and not a link (acquired with O_NOFOLLOW), and none of them is |
| + // on another filesystem than the root. |
| + comps := strings.Split(rel, string(filepath.Separator)) |
| + cur := dir |
| + var opened []*authority.Dir |
| + defer func() { |
| + for i := len(opened) - 1; i >= 0; i-- { |
| + opened[i].Close() |
| + } |
| + }() |
| // dirUID is the owner of the file's immediate directory: the root's |
| // until the walk has passed a directory beneath it. |
| dirUID, lastUID := rootFacts.UID, rootFacts.UID |
| - for _, part := range strings.Split(rel, string(filepath.Separator)) { |
| - at = filepath.Join(at, part) |
| - fi, lerr := os.Lstat(at) |
| - if lerr != nil { |
| + at := root |
| + for _, c := range comps[:len(comps)-1] { |
| + at = filepath.Join(at, c) |
| + child, cerr := cur.Child(c, nil) |
| + if cerr != nil { |
| + var ee *authority.EdgeError |
| + if errors.As(cerr, &ee) && errors.Is(ee.Err, authority.ErrLink) { |
| + return fileFacts{}, "its path passes through a link (" + at + ")" |
| + } |
| return fileFacts{}, "part of its path cannot be looked at" |
| } |
| - if fi.Mode()&os.ModeSymlink != 0 { |
| - return fileFacts{}, "its path passes through a link (" + at + ")" |
| - } |
| - f, ok := factsOf(fi) |
| - if !ok { |
| - return fileFacts{}, "file ownership cannot be read on this platform" |
| - } |
| + opened = append(opened, child) |
| + f := facts(child) |
| if f.Dev != rootFacts.Dev { |
| return fileFacts{}, "its path crosses onto another filesystem (" + at + ")" |
| } |
| dirUID, lastUID = lastUID, f.UID |
| - last = fi |
| + cur = child |
| } |
| - if !last.Mode().IsRegular() { |
| + name := comps[len(comps)-1] |
| + at = filepath.Join(at, name) |
| + k, lf, lerr := cur.Lookat(name) |
| + if lerr != nil || k == authority.Absent { |
| + return fileFacts{}, "part of its path cannot be looked at" |
| + } |
| + if k == authority.Link { |
| + return fileFacts{}, "its path passes through a link (" + at + ")" |
| + } |
| + last := authFacts(name, lf) |
| + if last.Dev != rootFacts.Dev { |
| + return fileFacts{}, "its path crosses onto another filesystem (" + at + ")" |
| + } |
| + dirUID, lastUID = lastUID, last.UID |
| + _ = lastUID |
| + if k != authority.Regular { |
| return fileFacts{}, "it is not a regular file" |
| } |
| - facts, _ := factsOf(last) |
| - if facts.Nlink != 1 { |
| + if last.Nlink != 1 { |
| return fileFacts{}, "it has more than one name (a hard link)" |
| } |
| // The file's owner must be the chain's root's, except that root and |
| @@ -571,7 +562,7 @@ func checkOwnedFile(root, path string, kind claimKind) (fileFacts, string) { |
| // the platform's user counting as one there too), so this can fail |
| // there only when something changed between the climb and the walk; |
| // for a tree claim it is the real comparison. |
| - if (kind == tenantTree || kind == tenantFile) && facts.UID != rootFacts.UID && !(platformUID(facts.UID) && platformUID(rootFacts.UID)) { |
| + if (kind == tenantTree || kind == tenantFile) && last.UID != rootFacts.UID && !(platformUID(last.UID) && platformUID(rootFacts.UID)) { |
| return fileFacts{}, "it does not belong to the user its tenant's directory belongs to" |
| } |
| // A tenant's file in a directory the platform owns is the undecided |
| @@ -581,30 +572,25 @@ func checkOwnedFile(root, path string, kind claimKind) (fileFacts, string) { |
| // nineteen, Opus P3-1: the kind of claim decided it). The platform's |
| // own claims (a named file in the data directory, a platform |
| // directory) ask nothing of the file's owner and are not this case. |
| - // The two tenant claims still differ on purpose in what they ask of |
| - // the directories inside a tree (a tree claim asks nothing of their |
| - // owners or modes; review twelve): whether they should agree is a |
| - // decision, listed with the deferred ones, because every refusal here |
| - // refuses the whole box's build. |
| - if kind == tenantTree && platformUID(dirUID) && !platformUID(facts.UID) { |
| + if kind == tenantTree && platformUID(dirUID) && !platformUID(last.UID) { |
| return fileFacts{}, "its directory is the platform's and the file is not" |
| } |
| for _, suffix := range []string{"-wal", "-shm", "-journal"} { |
| - fi, lerr := os.Lstat(path + suffix) |
| - if errors.Is(lerr, fs.ErrNotExist) { |
| - continue |
| - } |
| - if lerr != nil { |
| + sk, sf, serr := cur.Lookat(name + suffix) |
| + if serr != nil { |
| return fileFacts{}, "its " + suffix + " file cannot be looked at" |
| } |
| - if !fi.Mode().IsRegular() { |
| + if sk == authority.Absent { |
| + continue |
| + } |
| + if sk != authority.Regular { |
| return fileFacts{}, "its " + suffix + " file is a link or not a regular file" |
| } |
| - if f, ok := factsOf(fi); !ok || f.Nlink != 1 { |
| + if sf.Nlink != 1 { |
| return fileFacts{}, "its " + suffix + " file has more than one name (a hard link)" |
| } |
| } |
| - return facts, "" |
| + return fileFacts{Dev: last.Dev, Ino: last.Ino, Nlink: last.Nlink, UID: last.UID}, "" |
| } |
| // platformOwned says why dir is not the platform's, or nothing: it must be |
| @@ -628,6 +614,11 @@ func platformOwned(dir string) string { |
| return "" |
| } |
| +// treeAfterList is a test seam: called after each directory's listing in |
| +// strictTreeDBs with the retained directory and its spelled path, so a |
| +// test can change the directory between the listing and what follows. |
| +var treeAfterList func(d *authority.Dir, dir string) |
| + |
| // strictTreeMaxDepth bounds strictTreeDBs. Past treeDBWalkDepth a database |
| // is reported and not replicated; past this the walk says it cannot vouch |
| // for what lies below. |
| @@ -669,7 +660,14 @@ const strictTreeMaxDepth = 32 |
| // live.sqlite) is named to the caller, never passed over, and a link |
| // re-pointed later reads as a change to the running fleet and to the |
| // Accept list at the next build alike (the target is quoted, and a |
| -// link entry is compared as written). The links tenant software makes in numbers (a venv's, a |
| +// link entry is compared as written). The links tenant software |
| +// makes in numbers (a venv's bin/python3 -> python -> /usr/bin/python3, |
| +// a node_modules' .bin) are now problems under these classes (a link |
| +// to a link; a file out of the tree), not skipped entries, and |
| +// problems are never compared with the Accept list: before this is |
| +// wired, the owner's decision on link shapes stands beside decisions |
| +// 10 to 13, and a census of the link classes on each box's live trees |
| +// is part of the pre-wiring check. The links tenant software makes in numbers (a venv's, a |
| // node_modules' .bin) are all named this way: whether one tenant's |
| // unaccepted links should refuse the whole box's next build, as they |
| // do today through Accept, or that tenant's databases alone, is to be |
| @@ -677,12 +675,17 @@ const strictTreeMaxDepth = 32 |
| // |
| // What it passes over on purpose: an entry that vanishes between being |
| // listed and being looked at, or that is a link when listed and something |
| -// else when read (ordinary churn: SQLite's journals come and go, deploys |
| -// swap a link for a directory; the next look sees what is there); an |
| -// empty regular file (it holds nothing; the fleet's periodic look notices |
| -// when it stops being empty); a link that leads nowhere (there is nothing |
| -// behind it); and FIFOs, sockets and devices, and links to them (not |
| -// files; a database cannot be one). |
| +// else when read, or a directory replaced once between the listing and |
| +// its acquisition (it is looked at again and the replacement walked; a |
| +// second replacement under the same look is a problem, "changed under the |
| +// look"), or a directory removed between its acquisition and its listing |
| +// (ordinary churn: SQLite's journals come and go, deploys swap a link for |
| +// a directory; the next look sees what is there); an empty regular file |
| +// (it holds nothing; the fleet's periodic look notices when it stops |
| +// being empty); a link that leads nowhere, relative or absolute (there is |
| +// nothing behind it); and FIFOs, sockets and devices met as entries (not |
| +// files; a database cannot be one). A link to one of those is a problem |
| +// (above), since a link is classified by where it leads. |
| func strictTreeDBs(root string) (dbs, skipped, problems []string) { |
| pins, err := newFleetPins() |
| if err != nil { |
| @@ -698,12 +701,23 @@ func strictTreeDBs(root string) (dbs, skipped, problems []string) { |
| walk = func(d *authority.Dir, dir string, depth int, hidden bool) { |
| entries, err := d.List() |
| if err != nil { |
| + var ee *authority.EdgeError |
| + if depth > 0 && errors.As(err, &ee) && errors.Is(ee.Err, syscall.ENOENT) { |
| + return // removed between its acquisition and its listing: churn |
| + } |
| problems = append(problems, dir+": cannot be read") |
| return |
| } |
| + if treeAfterList != nil { |
| + treeAfterList(d, dir) |
| + } |
| for _, e := range entries { |
| name := e.Name |
| p := filepath.Join(dir, name) |
| + if e.Err != nil { |
| + problems = append(problems, p+": cannot be looked at") |
| + continue |
| + } |
| switch e.Kind { |
| case authority.Link: |
| // Classified by where its captured spelling leads, walked |
| @@ -749,14 +763,31 @@ func strictTreeDBs(root string) (dbs, skipped, problems []string) { |
| problems = append(problems, p+": too deep to walk") |
| continue |
| } |
| + // with the listed facts: the identity rule; a single change |
| + // between the listing and this is looked at again below, so |
| + // what the facts add over no facts is the second change |
| child, cerr := d.Child(name, &e.Facts) |
| if cerr != nil { |
| var ee *authority.EdgeError |
| - if errors.As(cerr, &ee) && (errors.Is(ee.Err, authority.ErrChanged) || errors.Is(ee.Err, authority.ErrLink) || errors.Is(ee.Err, syscall.ENOENT) || errors.Is(ee.Err, syscall.ENOTDIR)) { |
| - continue // replaced under the look: churn, the next look sees what is there |
| + if errors.As(cerr, &ee) && errors.Is(ee.Err, authority.ErrChanged) { |
| + // replaced between the listing and the acquisition: |
| + // looked at again once, and the replacement walked if |
| + // it is a directory; replaced again is a problem |
| + k2, f2, lerr := d.Lookat(name) |
| + if lerr != nil || k2 != authority.Directory { |
| + continue // something else now: churn, as a vanished entry is |
| + } |
| + child, cerr = d.Child(name, &f2) |
| + if cerr != nil { |
| + problems = append(problems, p+": changed under the look") |
| + continue |
| + } |
| + } else if errors.As(cerr, &ee) && (errors.Is(ee.Err, authority.ErrLink) || errors.Is(ee.Err, syscall.ENOENT) || errors.Is(ee.Err, syscall.ENOTDIR)) { |
| + continue // gone or no longer a directory under the look: churn |
| + } else { |
| + problems = append(problems, p+": cannot be read") |
| + continue |
| } |
| - problems = append(problems, p+": cannot be read") |
| - continue |
| } |
| walk(child, p, depth+1, hidden || strings.HasPrefix(name, ".") || depth+1 > treeDBWalkDepth) |
| child.Close() |
| @@ -817,135 +848,6 @@ func sqliteName(name string) bool { |
| return false |
| } |
| -// walkWithoutLinks walks rel (an absolute or relative path, as spelled) |
| -// from start, one component at a time, looking at each with Lstat: a |
| -// component that is a link, or cannot be looked at, refuses. ".." is |
| -// allowed only in the target's leading run (systemd's ../private/...), |
| -// where it steps up from start, a directory already looked at and |
| -// link-free; a ".." after any named component refuses, because the |
| -// component before it may be a directory whose owner can later put a |
| -// link in its place, and every resolution after that goes where they |
| -// point. The result is the path the kernel would reach, with no cleaning |
| -// done before the walk, so a traversal through a link followed by ".." is |
| -// not folded away. |
| -func walkWithoutLinks(start, rel string) (string, string) { |
| - at := filepath.Clean(start) |
| - if filepath.IsAbs(rel) { |
| - at = string(filepath.Separator) |
| - } |
| - leading := true |
| - for _, part := range strings.Split(rel, string(filepath.Separator)) { |
| - if part == "" || part == "." { |
| - continue |
| - } |
| - if part == ".." { |
| - if !leading { |
| - return "", rel + " steps up after a named component" |
| - } |
| - at = filepath.Dir(at) |
| - continue |
| - } |
| - leading = false |
| - next := filepath.Join(at, part) |
| - fi, err := os.Lstat(next) |
| - if err != nil { |
| - return "", next + " cannot be looked at" |
| - } |
| - if fi.Mode()&os.ModeSymlink != 0 { |
| - return "", next + " is a link" |
| - } |
| - at = next |
| - } |
| - return at, "" |
| -} |
| - |
| -// resolvePlatformLinks resolves the links the platform made in a spelled |
| -// directory path, one component at a time, and refuses any other link. A |
| -// link is the platform's when the link itself and the directory holding |
| -// it are the platform's, its target (absolute, or relative to the link's |
| -// directory) holds no further link and no ".." past its leading run, and |
| -// the target's parent is the platform's. The result is a path with no |
| -// link in it, or the reason it could not be. |
| -func resolvePlatformLinks(dir string) (string, string) { |
| - dir = filepath.Clean(dir) |
| - if !filepath.IsAbs(dir) { |
| - return "", "its directory is not an absolute path" |
| - } |
| - parts := strings.Split(strings.TrimPrefix(dir, string(filepath.Separator)), string(filepath.Separator)) |
| - prefix := string(filepath.Separator) |
| - for i := 0; i < len(parts); i++ { |
| - if parts[i] == "" { |
| - continue |
| - } |
| - next := filepath.Join(prefix, parts[i]) |
| - fi, err := os.Lstat(next) |
| - if err != nil { |
| - return "", "a directory in its path cannot be looked at" |
| - } |
| - if fi.Mode()&os.ModeSymlink == 0 { |
| - prefix = next |
| - continue |
| - } |
| - if why := platformOwned(prefix); why != "" { |
| - return "", "a link in its path is in a directory that is not the platform's: " + why |
| - } |
| - if lf, ok := factsOf(fi); !ok || (lf.UID != 0 && int(lf.UID) != os.Geteuid()) { |
| - return "", "a link in its path is not the platform's" |
| - } |
| - target, rerr := os.Readlink(next) |
| - if rerr != nil { |
| - return "", "a link in its path cannot be read" |
| - } |
| - // systemd makes the state link relative (../private/carlos-app/<host>); |
| - // a relative target starts from the link's own directory, which is |
| - // link-free by now. The target is walked component by component, |
| - // never cleaned first, so a traversal through a link the tenant |
| - // controls (choice/../x) is seen for what it is, and ".." is taken |
| - // only in the leading run, before any named component. |
| - start := string(filepath.Separator) |
| - if !filepath.IsAbs(target) { |
| - start = prefix |
| - } |
| - resolvedTarget, why := walkWithoutLinks(start, target) |
| - if why != "" { |
| - return "", "a link in its path leads through another link: " + why |
| - } |
| - target = resolvedTarget |
| - if tfi, terr := os.Lstat(target); terr != nil || !tfi.IsDir() { |
| - return "", "a link in its path does not lead to a directory" |
| - } |
| - if why := platformOwned(filepath.Dir(target)); why != "" { |
| - return "", "a link in its path leads to a directory that is not under the platform's: " + why |
| - } |
| - prefix = target |
| - } |
| - return prefix, "" |
| -} |
| - |
| -// sqliteHeader reports whether the file begins with SQLite's header. A |
| -// file too short to hold one is not a database; a file that cannot be |
| -// opened or read is an error, which the caller treats as a place it could |
| -// not look. It opens without following a link and without blocking (a |
| -// FIFO put in the file's place would otherwise hold the walk, and with it |
| -// the watch and Close, for ever), and reads only the file that was looked |
| -// at: one that changed between the look and the open is an error. |
| -func sqliteHeader(path string, looked os.FileInfo) (bool, error) { |
| - f, err := openRegularNoFollow(path, looked) |
| - if err != nil { |
| - return false, err |
| - } |
| - defer f.Close() |
| - var head [16]byte |
| - n, err := io.ReadFull(f, head[:]) |
| - if err == io.EOF || err == io.ErrUnexpectedEOF { |
| - return false, nil |
| - } |
| - if err != nil { |
| - return false, err |
| - } |
| - return n == len(head) && string(head[:]) == "SQLite format 3\x00", nil |
| -} |
| - |
| // platformUID reports whether a uid is the platform's: root, or the user |
| // this process runs as. |
| func platformUID(uid uint32) bool { |
diff --git a/internal/ops/litestream_owners_test.go b/internal/ops/litestream_owners_test.go| index 499a5cd39..6dd00647f 100644 |
| --- a/internal/ops/litestream_owners_test.go |
| +++ b/internal/ops/litestream_owners_test.go |
| @@ -393,11 +393,7 @@ func TestFleetDBOwnersCheckTheFileAndItsSiblings(t *testing.T) { |
| if err != nil { |
| t.Fatal(err) |
| } |
| - // unowned with the reason, or, when the tree's own root fails |
| - // the platform walk (the directory above the tenant's), a |
| - // problem on the tree with the reason: either way the database |
| - // is not owned and the reason is said |
| - if _, ok := ownedByPath(inv)[f.db]; ok || !(named(inv.Unowned, f.db, tc.why) || named(inv.Problems, f.state, tc.why)) { |
| + if _, ok := ownedByPath(inv)[f.db]; ok || !named(inv.Unowned, f.db, tc.why) { |
| t.Fatalf("owned %+v, unowned %v, problems %v; want the database unowned, saying %q", inv.Owned, inv.Unowned, inv.Problems, tc.why) |
| } |
| }) |
| @@ -696,7 +692,7 @@ func TestStrictTreeNamesALinkToAFile(t *testing.T) { |
| t.Fatal(err) |
| } |
| dbs, skipped, problems := strictTreeDBs(root) |
| - if len(dbs) != 0 || !contains(skipped, linkEntry(filepath.Join(root, "current"), target)) || containsPrefix(skipped, filepath.Join(root, "dangling")) { |
| + if len(dbs) != 0 || !contains(skipped, linkEntry(filepath.Join(root, "current"), target)) || contains(skipped, linkEntry(filepath.Join(root, "dangling"), filepath.Join(root, "nowhere"))) || containsPrefix(problems, filepath.Join(root, "dangling")) { |
| t.Fatalf("dbs %v skipped %v problems %v; want the link to a file in the tree skipped with its target and the dangling one ignored", dbs, skipped, problems) |
| } |
| if len(problems) != 1 || !named(problems, filepath.Join(root, "away"), "a link out of the tree") || containsPrefix(skipped, filepath.Join(root, "away")) { |
diff --git a/internal/ops/litestream_owners_unix_test.go b/internal/ops/litestream_owners_unix_test.go| index 2389ae6a7..22f03889f 100644 |
| --- a/internal/ops/litestream_owners_unix_test.go |
| +++ b/internal/ops/litestream_owners_unix_test.go |
| @@ -3,6 +3,7 @@ |
| package ops |
| import ( |
| + "errors" |
| "os" |
| "path/filepath" |
| "strconv" |
| @@ -231,7 +232,7 @@ func TestStrictTreeDoesNotBlockOnAFIFO(t *testing.T) { |
| // A FIFO is not a regular file and is passed over by mode before |
| // any open; the point is that the walk returned. A regular file |
| // swapped for a FIFO between the look and the open is the open's |
| - // own refusal (openRegularNoFollow), exercised below. |
| + // own refusal (authority's OpenRegular), exercised below. |
| _ = problems |
| case <-time.After(10 * time.Second): |
| t.Fatal("the walk blocked on a FIFO") |
| @@ -240,7 +241,12 @@ func TestStrictTreeDoesNotBlockOnAFIFO(t *testing.T) { |
| if err := os.WriteFile(regular, []byte("not a database, long enough to read"), 0o644); err != nil { |
| t.Fatal(err) |
| } |
| - looked, err := os.Lstat(regular) |
| + rootDir, err := authority.Open(root) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + defer rootDir.Close() |
| + _, looked, err := rootDir.Lookat("swap.data") |
| if err != nil { |
| t.Fatal(err) |
| } |
| @@ -252,7 +258,7 @@ func TestStrictTreeDoesNotBlockOnAFIFO(t *testing.T) { |
| } |
| finished := make(chan error, 1) |
| go func() { |
| - _, err := sqliteHeader(regular, looked) |
| + _, err := sqliteHeaderAt(rootDir, "swap.data", looked) |
| finished <- err |
| }() |
| select { |
| @@ -274,7 +280,7 @@ func TestStrictTreeDoesNotBlockOnAFIFO(t *testing.T) { |
| if err := os.Symlink(target, regular); err != nil { |
| t.Fatal(err) |
| } |
| - if isDB, err := sqliteHeader(regular, looked); err == nil || isDB { |
| + if isDB, err := sqliteHeaderAt(rootDir, "swap.data", looked); err == nil || isDB { |
| t.Fatalf("a link in a file's place was followed: %v %v", isDB, err) |
| } |
| } |
| @@ -512,7 +518,12 @@ func TestRootAndThePlatformUserAreOneOwner(t *testing.T) { |
| func TestOpenRegularNoFollowRefusesALinkToTheSameFile(t *testing.T) { |
| dir := platformDir(t) |
| path := writeFile(t, filepath.Join(dir, "x")) |
| - looked, err := os.Lstat(path) |
| + d, err := authority.Open(dir) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + defer d.Close() |
| + _, looked, err := d.Lookat("x") |
| if err != nil { |
| t.Fatal(err) |
| } |
| @@ -522,7 +533,7 @@ func TestOpenRegularNoFollowRefusesALinkToTheSameFile(t *testing.T) { |
| if err := os.Symlink(filepath.Join(dir, "y"), path); err != nil { |
| t.Fatal(err) |
| } |
| - f, err := openRegularNoFollow(path, looked) |
| + f, err := d.OpenRegular("x", looked) |
| if err == nil { |
| f.Close() |
| t.Fatal("a link to the file that was looked at was opened through") |
| @@ -577,11 +588,12 @@ func TestATenantsFileInAPlatformDirectoryIsRefusedByBothTenantClaims(t *testing. |
| // Every link class the pinned-roots design names is rendered as written |
| // and none is silent: a directory target and a database-named link (may |
| // lead to a database), a regular file in the tree (skipped, with its |
| -// target), a regular file under another pinned root and an ascent past the |
| -// tenant's root and an absolute target outside every pin (out of the |
| -// tree), a link to a link, a FIFO target (neither a file nor a directory), |
| -// a target through a ".." after a name (cannot be looked at), and an |
| -// absent target (churn). |
| +// target, spelled relative or absolute), a regular file or a directory |
| +// outside the tree (a sibling tenant under the same root, another |
| +// directory outside every pin, an ascent past the tenant's root, a target |
| +// through a group-writable component), a link to a link, a FIFO target |
| +// (neither a file nor a directory), a target through a ".." after a name |
| +// (cannot be looked at), and an absent target (churn). |
| func TestStrictTreeNamesEveryLinkClass(t *testing.T) { |
| plat := platformDir(t) |
| root := filepath.Join(plat, "tenant") |
| @@ -593,18 +605,25 @@ func TestStrictTreeNamesEveryLinkClass(t *testing.T) { |
| if err := syscall.Mkfifo(filepath.Join(root, "fifo"), 0o644); err != nil { |
| t.Fatal(err) |
| } |
| + wdir := filepath.Join(filepath.Dir(plat), "w", "x") |
| + mkdirs(t, wdir) |
| + writeFile(t, filepath.Join(wdir, "w.db")) |
| + os.Chmod(filepath.Dir(wdir), 0o775) |
| links := map[string]string{ |
| - "to-dir": "d", |
| - "named.db": "store/live.sqlite", |
| - "current": "store/live.sqlite", |
| - "sibling": theirs, |
| - "ascent": "../other/theirs.sqlite", |
| - "far": elsewhere, |
| - "to-link": "current", |
| - "to-fifo": "fifo", |
| - "dotdot": "x/../store/live.sqlite", |
| - "dangling": "nothing", |
| - "abs-in-tree": inTree, |
| + "to-other-dir": "../other", |
| + "abs-other": other, |
| + "via-writable": filepath.Join(wdir, "w.db"), |
| + "to-dir": "d", |
| + "named.db": "store/live.sqlite", |
| + "current": "store/live.sqlite", |
| + "sibling": theirs, |
| + "ascent": "../other/theirs.sqlite", |
| + "far": elsewhere, |
| + "to-link": "current", |
| + "to-fifo": "fifo", |
| + "dotdot": "x/../store/live.sqlite", |
| + "dangling": "nothing", |
| + "abs-in-tree": inTree, |
| } |
| for name, target := range links { |
| if err := os.Symlink(target, filepath.Join(root, name)); err != nil { |
| @@ -617,14 +636,17 @@ func TestStrictTreeNamesEveryLinkClass(t *testing.T) { |
| } |
| at := func(name string) string { return filepath.Join(root, name) + " -> " + strconv.Quote(links[name]) } |
| for name, why := range map[string]string{ |
| - "to-dir": "a link that may lead to a database", |
| - "named.db": "a link that may lead to a database", |
| - "sibling": "a link out of the tree", |
| - "ascent": "a link out of the tree", |
| - "far": "a link out of the tree", |
| - "to-link": "a link to a link that may lead to a database", |
| - "to-fifo": "a link to something that is neither a file nor a directory", |
| - "dotdot": "a link whose target cannot be looked at", |
| + "to-dir": "a link that may lead to a database", |
| + "to-other-dir": "a link out of the tree", |
| + "abs-other": "a link out of the tree", |
| + "via-writable": "a link out of the tree", |
| + "named.db": "a link that may lead to a database", |
| + "sibling": "a link out of the tree", |
| + "ascent": "a link out of the tree", |
| + "far": "a link out of the tree", |
| + "to-link": "a link to a link that may lead to a database", |
| + "to-fifo": "a link to something that is neither a file nor a directory", |
| + "dotdot": "a link whose target cannot be looked at", |
| } { |
| if !named(problems, at(name), why) { |
| t.Errorf("%s: want %q among problems %v", name, why, problems) |
| @@ -635,11 +657,11 @@ func TestStrictTreeNamesEveryLinkClass(t *testing.T) { |
| t.Errorf("%s: want the link to a file in the tree skipped with its target: %v", name, skipped) |
| } |
| } |
| - if containsPrefix(problems, filepath.Join(root, "dangling")) || containsPrefix(skipped, filepath.Join(root, "dangling")) { |
| + if containsPrefix(problems, filepath.Join(root, "dangling")) || contains(skipped, linkEntry(filepath.Join(root, "dangling"), "nothing")) || containsPrefix(skipped, filepath.Join(root, "dangling")) { |
| t.Errorf("a dangling link is churn, not named: skipped %v problems %v", skipped, problems) |
| } |
| - if len(problems) != 8 { |
| - t.Errorf("%d problems, want 8: %v", len(problems), problems) |
| + if len(problems) != 11 { |
| + t.Errorf("%d problems, want 11: %v", len(problems), problems) |
| } |
| // the files themselves, named as today |
| for _, p := range []string{inTree} { |
| @@ -647,12 +669,191 @@ func TestStrictTreeNamesEveryLinkClass(t *testing.T) { |
| t.Errorf("%s not in skipped %v", p, skipped) |
| } |
| } |
| - // Astra 2's layout: a tenant-owned component above the platform's |
| - // refuses the walk at the root, no tree is built over it |
| + // Astra 2's layout: a tenant-owned component above a platform-owned |
| + // one. The platform walk ends above the tenant-owned component (the |
| + // root is its parent) and the directories beneath are the tenant's |
| + // chain, acquired as retained descriptors: the swap that layout allows |
| + // (the platform-owned directory replaced by a link) is a changed edge |
| + // at the next look, never a steer, so the tree is walked. |
| withOwners(t, map[string]uint32{"owned": 44001}) |
| mkdirs(t, filepath.Join(plat, "owned", "sub", "t")) |
| - _, _, problems = strictTreeDBs(filepath.Join(plat, "owned", "sub", "t")) |
| - if len(problems) != 1 || !named(problems, filepath.Join(plat, "owned", "sub", "t"), "not the platform's") { |
| - t.Fatalf("a tenant-owned component above the tree: %v", problems) |
| + writeFile(t, filepath.Join(plat, "owned", "sub", "t", "x.db")) |
| + dbs, _, problems = strictTreeDBs(filepath.Join(plat, "owned", "sub", "t")) |
| + if len(problems) != 0 || len(dbs) != 1 { |
| + t.Fatalf("a tenant-owned component above the tree: dbs %v problems %v", dbs, problems) |
| + } |
| + // and a tenant-owned component in the root's own chain, where the |
| + // spelled directory itself is the platform's, refuses: the root would |
| + // be the tenant-owned directory's parent, and the spelled directory is |
| + // then reached through the tenant's directory, which the platform |
| + // walk no longer vouches for; the claim's rules decide (tenantTree |
| + // asks the directory above the tenant's to be the platform's) |
| + withOwners(t, map[string]uint32{"mine": 44001, "shared": 0}) |
| + mkdirs(t, filepath.Join(plat, "mine", "shared")) |
| + if _, why := checkOwnedFile(filepath.Join(plat, "mine", "shared"), writeFile(t, filepath.Join(plat, "mine", "shared", "y.db")), tenantTree); !strings.Contains(why, "other than root or the platform's") { |
| + t.Fatalf("a platform directory under a tenant's, claimed as a tree: %q", why) |
| + } |
| +} |
| + |
| +// The root rule and the platform-link predicate at the inventory level: |
| +// a tree reached through systemd's relative link is walked; the same |
| +// through a two-level ascent and through an absolute target in another |
| +// platform directory; and each of the predicate's refusals renders. |
| +func TestStrictTreeThroughAPlatformLink(t *testing.T) { |
| + const tenant = 44001 |
| + top := platformDir(t) |
| + public := filepath.Join(top, "carlos-app") |
| + // the private tree's parent is named apart from the link's directory, |
| + // since the owners seam is by name |
| + private := filepath.Join(top, "private", "apps", "keymail.dev") |
| + mkdirs(t, public, private) |
| + writeFile(t, filepath.Join(private, "main.db")) |
| + link := filepath.Join(public, "keymail.dev") |
| + if err := os.Symlink(filepath.Join("..", "private", "apps", "keymail.dev"), link); err != nil { |
| + t.Fatal(err) |
| + } |
| + withOwners(t, map[string]uint32{"carlos-app": 0, "private": 0, "apps": 0, "keymail.dev": tenant, "main.db": tenant, "link:keymail.dev": 0}) |
| + dbs, _, problems := strictTreeDBs(link) |
| + if len(problems) != 0 || !contains(dbs, filepath.Join(link, "main.db")) { |
| + t.Fatalf("systemd's shape: dbs %v problems %v", dbs, problems) |
| + } |
| + // two levels up, and an absolute target in another platform directory |
| + deep := filepath.Join(top, "state", "two", "carlos-state", "h") |
| + mkdirs(t, filepath.Join(top, "state", "two", "carlos-app"), deep) |
| + writeFile(t, filepath.Join(deep, "two.db")) |
| + twoUp := filepath.Join(top, "state", "two", "carlos-app", "h") |
| + if err := os.Symlink("../../two/carlos-state/h", twoUp); err != nil { |
| + t.Fatal(err) |
| + } |
| + withOwners(t, map[string]uint32{"state": 0, "two": 0, "carlos-app": 0, "carlos-state": 0, "h": tenant, "two.db": tenant, "link:h": 0}) |
| + if dbs, _, problems := strictTreeDBs(twoUp); len(problems) != 0 || !contains(dbs, filepath.Join(twoUp, "two.db")) { |
| + t.Fatalf("a two-level ascent: dbs %v problems %v", dbs, problems) |
| + } |
| + elsewhere := filepath.Join(platformDir(t), "carlos-state", "abs") |
| + mkdirs(t, elsewhere) |
| + writeFile(t, filepath.Join(elsewhere, "abs.db")) |
| + absLink := filepath.Join(public, "abs") |
| + if err := os.Symlink(elsewhere, absLink); err != nil { |
| + t.Fatal(err) |
| + } |
| + withOwners(t, map[string]uint32{"carlos-app": 0, "carlos-state": 0, "abs": tenant, "abs.db": tenant, "link:abs": 0}) |
| + if dbs, _, problems := strictTreeDBs(absLink); len(problems) != 0 || !contains(dbs, filepath.Join(absLink, "abs.db")) { |
| + t.Fatalf("an absolute target in another platform directory: dbs %v problems %v", dbs, problems) |
| + } |
| + // the refusals |
| + for name, tc := range map[string]struct { |
| + owners map[string]uint32 |
| + want string |
| + }{ |
| + "the link is the tenant's": {map[string]uint32{"carlos-app": 0, "private": 0, "apps": 0, "link:keymail.dev": tenant}, "a link in its path is not the platform's"}, |
| + "the link's directory is the tenant's": {map[string]uint32{"carlos-app": tenant, "private": 0, "apps": 0, "link:keymail.dev": 0}, "in a directory that is not the platform's"}, |
| + "the target's parent is the tenant's": {map[string]uint32{"carlos-app": 0, "private": 0, "apps": tenant, "link:keymail.dev": 0}, "not under the platform's"}, |
| + } { |
| + withOwners(t, tc.owners) |
| + _, _, problems := strictTreeDBs(link) |
| + if len(problems) != 1 || !named(problems, link, tc.want) { |
| + t.Errorf("%s: problems %v, want %q", name, problems, tc.want) |
| + } |
| + } |
| + // a target through a further link, and one that leaves every platform root |
| + hop := filepath.Join(top, "hop") |
| + mkdirs(t, hop) |
| + if err := os.Symlink(private, filepath.Join(hop, "choice")); err != nil { |
| + t.Fatal(err) |
| + } |
| + via := filepath.Join(public, "via") |
| + if err := os.Symlink(filepath.Join(hop, "choice"), via); err != nil { |
| + t.Fatal(err) |
| + } |
| + withOwners(t, map[string]uint32{"carlos-app": 0, "private": 0, "apps": 0, "hop": 0, "link:via": 0, "link:choice": 0}) |
| + if _, _, problems := strictTreeDBs(via); len(problems) != 1 || !named(problems, via, "through another link") { |
| + t.Fatalf("a target through a further link: %v", problems) |
| + } |
| + out := filepath.Join(public, "out") |
| + if err := os.Symlink("../../../../../../../../../../nowhere-above", out); err != nil { |
| + t.Fatal(err) |
| + } |
| + withOwners(t, map[string]uint32{"carlos-app": 0, "link:out": 0}) |
| + if _, _, problems := strictTreeDBs(out); len(problems) != 1 || !named(problems, out, "leads out of its platform root") { |
| + t.Fatalf("a target above every root: %v", problems) |
| + } |
| +} |
| + |
| +// Between the listing and what follows: a subdirectory swapped is looked at |
| +// again and the replacement walked; a file replaced by a FIFO cannot be |
| +// read; the tenant root is confined (".." refused from it); and every |
| +// descriptor is released by the time the walk and the claim check return. |
| +func TestStrictTreeBetweenListingAndLook(t *testing.T) { |
| + root := platformDir(t) |
| + mkdirs(t, filepath.Join(root, "sub")) |
| + writeFile(t, filepath.Join(root, "sub", "old.db")) |
| + writeFile(t, filepath.Join(root, "plain.data")) |
| + if err := os.Symlink("sub/new.db", filepath.Join(root, "deep")); err != nil { // after the swap it resolves, and its walk opens sub, which the result owns |
| + t.Fatal(err) |
| + } |
| + swapped := false |
| + var confineErr error |
| + treeAfterList = func(d *authority.Dir, dir string) { |
| + if dir != root || swapped { |
| + return |
| + } |
| + swapped = true |
| + _, confineErr = d.Walk("..", nil) |
| + if err := os.Rename(filepath.Join(root, "sub"), filepath.Join(root, "gone")); err != nil { |
| + t.Fatal(err) |
| + } |
| + mkdirs(t, filepath.Join(root, "sub")) |
| + writeFile(t, filepath.Join(root, "sub", "new.db")) |
| + if err := os.Remove(filepath.Join(root, "plain.data")); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := syscall.Mkfifo(filepath.Join(root, "plain.data"), 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + t.Cleanup(func() { treeAfterList = nil }) |
| + before := openFDCount(t) |
| + done := make(chan struct{}) |
| + var dbs, problems []string |
| + go func() { |
| + dbs, _, problems = strictTreeDBs(root) |
| + close(done) |
| + }() |
| + select { |
| + case <-done: |
| + case <-time.After(10 * time.Second): |
| + t.Fatal("the walk blocked") |
| + } |
| + if !errors.Is(confineErr, authority.ErrAscentPastRoot) { |
| + t.Fatalf("the tenant root is not confined: %v", confineErr) |
| + } |
| + if !contains(dbs, filepath.Join(root, "sub", "new.db")) || contains(dbs, filepath.Join(root, "sub", "old.db")) { |
| + t.Fatalf("a swapped subdirectory: dbs %v (want the replacement, looked at again)", dbs) |
| + } |
| + if !named(problems, filepath.Join(root, "plain.data"), "cannot be read") { |
| + t.Fatalf("a FIFO in a file's place: %v", problems) |
| + } |
| + if n := openFDCount(t); n != before { |
| + t.Fatalf("descriptors after the walk: %d, before %d", n, before) |
| + } |
| + // the claim check releases its chain too, succeeding and failing |
| + db := writeFile(t, filepath.Join(root, "sub", "new.db")) |
| + if _, why := checkOwnedFile(filepath.Dir(db), db, tenantFile); why != "" { |
| + t.Fatal(why) |
| + } |
| + if _, why := checkOwnedFile(filepath.Dir(db), filepath.Join(root, "sub", "missing.db"), tenantFile); why == "" { |
| + t.Fatal("a missing file owned") |
| + } |
| + if n := openFDCount(t); n != before { |
| + t.Fatalf("descriptors after the claim checks: %d, before %d", n, before) |
| + } |
| +} |
| + |
| +func openFDCount(t *testing.T) int { |
| + t.Helper() |
| + ents, err := os.ReadDir("/proc/self/fd") |
| + if err != nil { |
| + t.Skip("no /proc/self/fd here") |
| } |
| + return len(ents) |
| } |
diff --git a/internal/ops/sqliteopen_other.go b/internal/ops/sqliteopen_other.go| deleted file mode 100644 |
| index a0ed9bb7d..000000000 |
| --- a/internal/ops/sqliteopen_other.go |
| +++ /dev/null |
| @@ -1,15 +0,0 @@ |
| -//go:build !unix |
| - |
| -package ops |
| - |
| -import ( |
| - "errors" |
| - "os" |
| -) |
| - |
| -// openRegularNoFollow: see sqliteopen_unix.go. The replicator never runs |
| -// where this is built, and nothing is owned here (factsOf answers false); |
| -// a header read is refused rather than done by an open that follows links. |
| -func openRegularNoFollow(path string, _ os.FileInfo) (*os.File, error) { |
| - return nil, &os.PathError{Op: "open", Path: path, Err: errors.New("header reads are not supported on this platform")} |
| -} |
diff --git a/internal/ops/sqliteopen_unix.go b/internal/ops/sqliteopen_unix.go| deleted file mode 100644 |
| index 367f09de1..000000000 |
| --- a/internal/ops/sqliteopen_unix.go |
| +++ /dev/null |
| @@ -1,43 +0,0 @@ |
| -//go:build unix |
| - |
| -package ops |
| - |
| -import ( |
| - "errors" |
| - "os" |
| - "syscall" |
| -) |
| - |
| -// openRegularNoFollow opens path for reading without following a link in |
| -// its last element and without blocking on a special file, and requires |
| -// the opened file to be the regular file that was looked at (same device |
| -// and inode). A tenant who swaps a FIFO, a device or a link into a file's |
| -// place between the look and the open gets an error and no read of |
| -// something else; the error comes after the open, though, and O_NOFOLLOW |
| -// covers the last element only: a directory swapped for a link higher in |
| -// the path steers the open itself, and a device reached that way has had |
| -// its open-side effects by the time it is refused. The walk by directory |
| -// descriptor that closes that is not built (dispositions). |
| -func openRegularNoFollow(path string, looked os.FileInfo) (*os.File, error) { |
| - // O_NOCTTY: a path steered to a terminal (a pty the tenant made) must |
| - // not become this process's controlling terminal, whose hangup would |
| - // end it; the fstat below refuses the file, but a controlling terminal |
| - // would stay. |
| - fd, err := syscall.Open(path, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC|syscall.O_NOCTTY, 0) |
| - if err != nil { |
| - return nil, &os.PathError{Op: "open", Path: path, Err: err} |
| - } |
| - f := os.NewFile(uintptr(fd), path) |
| - fi, err := f.Stat() |
| - if err != nil { |
| - f.Close() |
| - return nil, err |
| - } |
| - want, ok1 := factsOf(looked) |
| - got, ok2 := factsOf(fi) |
| - if !fi.Mode().IsRegular() || !ok1 || !ok2 || want.Dev != got.Dev || want.Ino != got.Ino { |
| - f.Close() |
| - return nil, &os.PathError{Op: "open", Path: path, Err: errors.New("not the regular file that was looked at")} |
| - } |
| - return f, nil |
| -} |