amadan
Design: the replicator's authority, revision 4 (the fork and its VFS in a helper process so the kernel coordinates locks; settlement by the last participant with a transition table; a pending-discovery latch for a failed uncertainty write; links at construction and transitive inheritance; a producer fence for recovery; closure at zero pending with the margin's day materialised; a sleeping route's binding and a coherent observation; a durable lineage record; walk anchors for absolute spellings)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
1 file changed,
+25
−25
docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md+25 −25
CI cancelled — superseded by a newer push — run details
diff --git a/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md b/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md| index 635030881..139cafda8 100644 |
| --- a/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md |
| +++ b/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md |
| @@ -1,28 +1,28 @@ |
| # The replicator's authority: retained descriptors, a durable admission, claims by identity |
| -Date: 2026-10-03. Status: design, revision 3, for review (revision 2 was rejected by Astra on three P1s: the VFS delegated SQLite's Unix file methods, which open the SHM and check moved files by pathname themselves; every settled admission wrote a permanent hold, so ordinary replication never read complete; a late owner-scoped extension did not reach successors that had already acknowledged; and six P2s: orphan admissions never terminated, `..` and chain edges unspecified, `dup` sharing the directory offset, the platform-link predicate not preserved and terminal links unclassified, the uid producer without a lifecycle, one billing owner not one replicator; each answered here. Revision 1 was rejected by Astra on five P1s: SQLite canonicalises a `/proc` pathname before it opens anything and this driver passes no `nofollow`; `os.Root` confines but re-resolves every name, so a swap between a listing and an inspection still moves the answer; a `Start` row is not an admission, since the meter sends without one; the attempt-derived window lost a request completed before the change was noticed and the days between an admission and a finish; `closed_at` was not a handover; plus the directory open before `O_DIRECTORY`, per-owner holds the ledger cannot store, a uid with no producer, I3 against Part D, and tests). Branch `design-replicator-pinned-root`, off `feat/carloku-metered-replicator-20261001` at `e1280811`. It answers the eleven findings the twentieth review deferred (`DISPOSITIONS-metered-replicator.md`, review twenty) and keeps decisions 10 to 13 and approval H (`DECISIONS-FOR-PAUL.md`, section D, on the launch branch) as parameters. Nothing is built, wired or applied. |
| +Date: 2026-10-03. Status: design, revision 4, for review (revision 3 was rejected by Astra on five P1s: two SQLite VFSs in one process cannot coordinate through POSIX locks, since the library's own per-inode bookkeeping is in-process; a recorder that abandons early could settle an admission before its transport ended; an admission-only barrier does not cover a discovery whose uncertainty write failed after every admission settled; links at acknowledgement contradicted acknowledging only after settlement, and an inherited uncertainty vanished at the next successor; the ledger's lock does not prove the old process gone; and four P2s: closure at zero pending and the margin's extra day, sleeping claims with no binding, absolute and `.` spellings in the walk, `(dev, ino)` as a lineage; each answered here. Revision 2 was rejected by Astra on three P1s: the VFS delegated SQLite's Unix file methods, which open the SHM and check moved files by pathname themselves; every settled admission wrote a permanent hold, so ordinary replication never read complete; a late owner-scoped extension did not reach successors that had already acknowledged; and six P2s: orphan admissions never terminated, `..` and chain edges unspecified, `dup` sharing the directory offset, the platform-link predicate not preserved and terminal links unclassified, the uid producer without a lifecycle, one billing owner not one replicator; each answered here. Revision 1 was rejected by Astra on five P1s: SQLite canonicalises a `/proc` pathname before it opens anything and this driver passes no `nofollow`; `os.Root` confines but re-resolves every name, so a swap between a listing and an inspection still moves the answer; a `Start` row is not an admission, since the meter sends without one; the attempt-derived window lost a request completed before the change was noticed and the days between an admission and a finish; `closed_at` was not a handover; plus the directory open before `O_DIRECTORY`, per-owner holds the ledger cannot store, a uid with no producer, I3 against Part D, and tests). Branch `design-replicator-pinned-root`, off `feat/carloku-metered-replicator-20261001` at `e1280811`. It answers the eleven findings the twentieth review deferred (`DISPOSITIONS-metered-replicator.md`, review twenty) and keeps decisions 10 to 13 and approval H (`DECISIONS-FOR-PAUL.md`, section D, on the launch branch) as parameters. Nothing is built, wired or applied. |
| ## What stands today |
| -The replicator (`internal/ops/litestream_metered.go`, `litestream_owners.go`; path B of `docs/superpowers/plans/2026-10-01-durability-attestation-path.md`) reads routes under the registry's ownership witness (`RouteObservation.OwnershipUnchanged`: a generation that moves when a route is added or removed or its host, account, app, database path, backing or hibernation changes; it witnesses nothing on the filesystem), lists each claimed tree by path (`strictTreeDBs`), checks each owned file's chain by path (`checkOwnedFile`; `resolvePlatformLinks` is an ownership and structure predicate on a link and its parents, not a list of layouts), reads a candidate's header by path with `O_NOFOLLOW|O_NONBLOCK|O_NOCTTY` and an inode check, and hands litestream paths. Every replica request asks the witness (`admit`) and then the meter writes a `Start` row with a bounded budget and sends whether or not that write succeeded in time (a late `Start` is abandoned while the transport may still run). A changed or missing owned file at a later look makes the fleet stale (an inventory-only change is drift, replication continues); stale holds the UTC days within `ownershipMargin` of the earlier of the last registry proof and the last clean look's start, through the change (`goStaleFrom`), and the next fleet inherits that window from an in-memory chain. The fork (`third_party/litestream`) opens the database, WAL, LTX and temporary files by path in `db.go` and `replica.go`, and its SQLite connection is opened by the driver by path; SQLite's `xFullPathname` canonicalises that path, links included, before the open. |
| +The replicator (`internal/ops/litestream_metered.go`, `litestream_owners.go`; path B of `docs/superpowers/plans/2026-10-01-durability-attestation-path.md`) reads routes under the registry's ownership witness (`RouteObservation.OwnershipUnchanged`: a generation that moves when a route is added or removed or its host, account, app, database path, backing or hibernation changes; it also checks that the registry's path still names the file it observes, and witnesses nothing about a tenant file's authority), lists each claimed tree by path (`strictTreeDBs`), checks each owned file's chain by path (`checkOwnedFile`; `resolvePlatformLinks` is an ownership and structure predicate on a link and its parents, not a list of layouts), reads a candidate's header by path with `O_NOFOLLOW|O_NONBLOCK|O_NOCTTY` and an inode check, and hands litestream paths. Every replica request asks the witness (`admit`) and then the meter writes a `Start` row with a bounded budget and sends whether or not that write succeeded in time (a late `Start` is abandoned while the transport may still run). A changed or missing owned file at a later look makes the fleet stale (an inventory-only change is drift, replication continues); stale holds the UTC days within `ownershipMargin` of the earlier of the last registry proof and the last clean look's start, through the change (`goStaleFrom`), and the next fleet inherits that window from an in-memory chain. The fork (`third_party/litestream`) opens the database, WAL, LTX and temporary files by path in `db.go` and `replica.go`, and its SQLite connection is opened by the driver by path; SQLite's `xFullPathname` canonicalises that path, links included, before the open. |
| ## Three invariants, made precise |
| -**I1, identity through retained descriptors.** (a) *Containment*: no operation of the replicator or of litestream on a tenant's files resolves a pathname above the directory the fleet accepted for that file; (b) *retention*: the descriptor that listed a directory is the descriptor against which its entries are inspected, its links read, their targets classified and its files opened, for that look, so nothing between a listing and an inspection can move the answer; (c) *acceptance*: an owned file is opened by the descriptor of its accepted directory with no link followed in the last component, and the opened file's `(dev, ino)` equals the accepted facts before any use. Retention is authority over directories and checked identities, not a frozen inventory: entries inside a retained directory can change, and a swap put back between two looks is not detected by looking; what (a) to (c) promise is that no operation is steered by such a change, and that every change a look does see is held for. Ascent never uses the kernel's `..`: it pops the retained stack. Each acquisition compares what it opened with the facts of the entry it was acquired from, and each later look compares every named edge (parent descriptor, name) with the retained child's facts, so a moved or replaced component is seen as a change at the next look. |
| +**I1, identity through retained descriptors.** (a) *Containment*: no operation of the replicator or of litestream on a tenant's files resolves a pathname above the directory the fleet accepted for that file; (b) *retention*: the descriptor that listed a directory is the descriptor against which its entries are inspected, its links read, their targets classified and its files opened, for that look, so nothing between a listing and an inspection can move the answer; (c) *acceptance*: an owned file is opened by the descriptor of its accepted directory with no link followed in the last component, and the opened file's `(dev, ino)` equals the accepted facts before any use. Retention is authority over directories and checked identities, not a frozen inventory, and it has two roots: the *accepted traversal root* (a platform root or a tenant root) through which acquisition and link classification may traverse retained ancestors, and the *accepted immediate directory* of an owned file, which every data open uses and nothing else. Ancestor replacement cannot redirect an operation, and an identity mismatch at acquisition refuses use; entries inside a retained directory can change, and a swap put back between two looks is not detected by looking; what (a) to (c) promise is that no operation is steered by such a change, and that every change a look does see is held for. Ascent never uses the kernel's `..`: it pops the retained stack. Each acquisition compares what it opened with the facts of the entry it was acquired from, and each later look compares every named edge (parent descriptor, name) with the retained child's facts, so a moved or replaced component is seen as a change at the next look. |
| -**I2, a durable admission, two kinds of hold.** No request is sent until a row naming it (fleet, owner, admission instant) is durable. A *barrier* is temporary: a day is not published complete for an admission's owner while that admission is unsettled, and lifts when it settles. An *uncertainty* is permanent: an interval written when a fleet's authority is found changed, from the earlier of today's lower bound (the registry proof, the last clean look's start) and the earliest admission of the fleet then unsettled, minus the margin, open until every admission of the fleet is settled, then closed at the latest settlement plus the margin; it holds its days for the fleet's owners and for every owner linked to it (below). An admission that settles with no uncertainty open on its fleet writes nothing permanent: the ordinary day of an unchanged fleet becomes complete. Discovery and its first hold are recorded in one transaction, and if that transaction fails the fleet refuses every request until it succeeds (the barrier, not the uncertainty, covers the gap). A successor acknowledges a predecessor only when the predecessor is terminal, and the acknowledgement links the successor's owners to every uncertainty of the predecessor still open, so a late extension reaches them. |
| +**I2, a durable admission, two kinds of hold.** No request is sent until a row naming it (fleet, owner, admission instant) is durable. A *barrier* is temporary: a day is not published complete for an admission's owner while that admission is unsettled, and lifts when it settles. An *uncertainty* is permanent: an interval written when a fleet's authority is found changed, from the earlier of today's lower bound (the registry proof, the last clean look's start) and the earliest admission of the fleet then unsettled, minus the margin, open until every admission of the fleet is settled, then closed at the latest settlement plus the margin; it holds its days for the fleet's owners and for every owner linked to it (below). An admission that settles with no uncertainty open on its fleet writes nothing permanent: the ordinary day of an unchanged fleet becomes complete. Discovery and its first hold are recorded in one transaction; if that transaction fails, a publication-visible *pending discovery* (the ledger's failure latch, extended with the lower bound) bars every day from that bound for every owner the fleet reaches until the write succeeds, and the fleet refuses every request meanwhile. Settlement is the last of the recorder's and the transport's completions, written by whichever lands last, with the upper bound no earlier than both. A fleet's effective uncertainties are its own and every one it inherited, and acknowledgement carries the whole set on. A successor acknowledges a predecessor only when the predecessor is terminal, and the acknowledgement links the successor's owners to every uncertainty of the predecessor still open, so a late extension reaches them. |
| -**I3, one billing owner and one replication authority by file identity.** A file is billed to one `(account, app)` and replicated by one authority (the external replicator, the activator's hibernation path, or this fleet; today the first two exist and the fleet is unwired); any number of routes of that one owner may claim it through any spelling; two owners on one `(dev, ino)`, or two replication authorities, whether the file is listed or skipped, whether a claimant is live or hibernating, make it unowned (or refused for this fleet) with every claimant named, until an explicit handover of authority exists; every claimant's own authority check must pass; a claim with no identity yet is recorded and reconciled when it resolves; identity grouping is a look, not a snapshot, and relaxes none of today's hard-link, sidecar or filesystem checks; a required platform database that cannot be reached is an incomplete box. |
| +**I3, one billing owner and one replication authority by file identity.** A file is billed to one `(account, app)` and replicated by one authority (the external replicator, the activator's hibernation path, or this fleet; today the first two exist and the fleet is unwired); any number of routes of that one owner may claim it through any spelling; two owners on one `(dev, ino)`, or two replication authorities, whether the file is listed or skipped, whether a claimant is live or hibernating, make it unowned (or refused for this fleet) with every claimant named, until an explicit handover of authority exists; every claimant's own authority check must pass; a claim with no identity yet is recorded and reconciled when it resolves; identity grouping is a look, not a snapshot, and relaxes none of today's hard-link, sidecar or filesystem checks; a database's lineage (which replica prefix continues) is a durable record in the shadow state, not an inode number; a required platform database that cannot be reached is an incomplete box. |
| ## Part A. Retained descriptors (Astra 2, 5, 6; Opus P2-2, P3-5) |
| **The primitive is a hand-walked, retained directory descriptor, not `os.Root`.** `os.Root` gives containment and re-resolves every name beneath its root; it follows links inside the root, `os.OpenRoot(path)` follows links in the path it is given, Go 1.26.5 walks per component with `openat` and there is no public constructor over a descriptor one has already verified. I1(b) and (c) need the descriptor itself. A small package, `internal/authority`: |
| -- `Dir`: a retained descriptor opened with `openat(parent, name, O_DIRECTORY|O_NOFOLLOW|O_RDONLY|O_CLOEXEC|O_NOCTTY)` (the kernel refuses a FIFO or device with `ENOTDIR` at lookup, before any open effect, which is the one existing directory-open test kept through the new shape), with its `(dev, ino, uid, mode)` read by `fstat` at acquisition and kept. Every method takes a basename (one component, not `.`, `..` or anything with a separator; anything else is refused before any syscall) and resolves it relative to this descriptor alone. `Child(name, expect) (*Dir, error)`: `openat` as above, then `fstat` compared with `expect` (the facts of the entry it is acquired from, when the caller has them); the child keeps a pointer to its parent `Dir` and its name, so the retained chain is a stack. `List() ([]Entry, error)`: a fresh open file description each time (`openat(fd, ".", O_DIRECTORY|O_RDONLY|O_CLOEXEC)`, `fstat`-compared with this descriptor, then `fdopendir` on it), never a `dup` (a `dup` shares the offset, so a second listing would start where the first ended); each entry's facts by `fstatat(fd, name, AT_SYMLINK_NOFOLLOW)`. `Readlink(name)`: `readlinkat`. `Walk(spelling)`: a captured link target's components, walked from this `Dir`: a leading run of `..` pops the retained stack (never `openat(fd, "..")`, whose answer follows a moved directory to its new parent) and is refused when the stack would pass the root; a `..` after a named component is refused; each named component is `Child`; the last component is `fstatat(..., AT_SYMLINK_NOFOLLOW)` and its kind is returned: directory, regular, link, other, absent, or error. `OpenRegular(name, looked)`: `openat(fd, name, O_RDONLY|O_NOFOLLOW|O_NONBLOCK|O_NOCTTY|O_CLOEXEC)` then `fstat`, refused unless regular and equal to `looked`'s `(dev, ino)`. `Recheck()`: for this `Dir` and every ancestor on its stack, `fstatat(parent, name, AT_SYMLINK_NOFOLLOW)` compared with the retained child's facts; a difference anywhere is a changed edge. `Facts()`. |
| +- `Dir`: a retained descriptor opened with `openat(parent, name, O_DIRECTORY|O_NOFOLLOW|O_RDONLY|O_CLOEXEC|O_NOCTTY)` (the kernel refuses a FIFO or device with `ENOTDIR` at lookup, before any open effect, which is the one existing directory-open test kept through the new shape), with its `(dev, ino, uid, mode)` read by `fstat` at acquisition and kept. Every method takes a basename (one component, not `.`, `..` or anything with a separator; anything else is refused before any syscall) and resolves it relative to this descriptor alone. `Child(name, expect) (*Dir, error)`: `openat` as above, then `fstat` compared with `expect` (the facts of the entry it is acquired from, when the caller has them); the child keeps a pointer to its parent `Dir` and its name, so the retained chain is a stack. `List() ([]Entry, error)`: a fresh open file description each time (`openat(fd, ".", O_DIRECTORY|O_RDONLY|O_CLOEXEC)`, `fstat`-compared with this descriptor, then `fdopendir` on it), never a `dup` (a `dup` shares the offset, so a second listing would start where the first ended); each entry's facts by `fstatat(fd, name, AT_SYMLINK_NOFOLLOW)`. `Readlink(name)`: `readlinkat`. `Walk(spelling, anchor)`: a captured link target, normalised first (repeated separators collapsed, `.` components dropped; a spelling that is empty after that is refused), then walked by components from an anchor: a relative spelling from this `Dir`, an absolute one from the pinned `/` root with the platform walk's rules (every component platform-owned) until it enters a pinned platform root, and from there as a relative walk beneath that root; a leading run of `..` pops the retained stack (never `openat(fd, "..")`, whose answer follows a moved directory to its new parent) and is refused when the stack would pass the anchor's root; a `..` after a named component is refused; each named component is `Child`; the last component is `fstatat(..., AT_SYMLINK_NOFOLLOW)` and its kind is returned: directory, regular, link, other (a FIFO, a socket, a device), absent, or error. The confinement boundary of a walk is the anchor's root: a relative walk cannot leave the tenant root it started in; an absolute walk must end beneath a pinned platform root or is refused. `OpenRegular(name, looked)`: `openat(fd, name, O_RDONLY|O_NOFOLLOW|O_NONBLOCK|O_NOCTTY|O_CLOEXEC)` then `fstat`, refused unless regular and equal to `looked`'s `(dev, ino)`. `Recheck()`: for this `Dir` and every ancestor on its stack, `fstatat(parent, name, AT_SYMLINK_NOFOLLOW)` compared with the retained child's facts; a difference anywhere is a changed edge. `Facts()`. |
| - `Root`: a `Dir` reached by a walk from `/` (`open("/")` then `Child` per component), each component required to be the platform's (root's or this process's user's), not writable by group or others; the walk's `(dev, ino)` list is the root's *ancestry*. A root is built once per fleet and kept; `lookNow` rebuilds the ancestry from `/` and compares it, and a difference is a change the fleet goes stale on. |
| -- Tenant roots: a unit route's tree is reached from a platform root through the platform-link predicate exactly as `resolvePlatformLinks` has it today (the link itself the platform's, its directory the platform's, its target's `..` only in a leading run, the target's parent the platform's, no further link in the target), evaluated on descriptors: the link read with `Readlink` from its directory's `Dir`, its target walked with `Walk` from that directory, and the resulting `Dir` the tenant root; that predicate is one function, kept separate from the inventory's classification of links (below), and its restrictions do not change here. A file claim's chain top is found as today (upward by ownership from the file's directory) but every directory of the chain is then acquired downward from the platform root as `Dir`s, and the chain is the file's retained stack. Astra 2's layout (a tenant-owned component above a platform-owned one) fails the platform walk; no root is built over it. |
| +- Tenant roots: a unit route's tree is reached from a platform root through the platform-link predicate exactly as `resolvePlatformLinks` has it today (the link itself the platform's, its directory the platform's, its target's `..` only in a leading run, the target's parent the platform's, no further link in the target), evaluated on descriptors: the link read with `Readlink` from its directory's `Dir`, its target walked with `Walk` (anchored as the spelling says: relative from the link's directory, absolute from the pinned `/`), and the resulting `Dir` the tenant root; that predicate is one function, kept separate from the inventory's classification of links (below), and its restrictions do not change here. A file claim's chain top is found as today (upward by ownership from the file's directory) but every directory of the chain is then acquired downward from the platform root as `Dir`s, and the chain is the file's retained stack. Astra 2's layout (a tenant-owned component above a platform-owned one) fails the platform walk; no root is built over it. |
| -**Every inventory operation runs on `Dir`s**: `strictTreeDBs` takes a `*Dir`, lists with `List`, descends with `Child(name, entry facts)`, reads a header with `OpenRegular`, and classifies a link's captured spelling with `Walk` on the directory that listed it. The classification is explicit and conservative, and nothing disappears through an unnamed case: a target that is a directory, or a link whose name is a database name, is "a link that may lead to a database" (a problem, as today); a regular file is a skipped link entry (as today); a target that is itself a link is "a link to a link that may lead to a database" (a problem; today's classification followed it, and the only honest replacement is to refuse rather than follow); a target outside the root is "a link out of the tree" (a problem); absent is churn (as today); any error is "a link whose target cannot be looked at" (as today). The namespace question for an isolated instance (its own mount namespace may resolve a spelling elsewhere) is a declared support condition before wiring: a route whose instance runs in its own mount namespace is classified in the agent's namespace and rendered as such, and until the condition is declared it is a refusal rule, not a silence. |
| +**Every inventory operation runs on `Dir`s**: `strictTreeDBs` takes a `*Dir`, lists with `List`, descends with `Child(name, entry facts)`, reads a header with `OpenRegular`, and classifies a link's captured spelling with `Walk` on the directory that listed it. The classification is explicit and conservative, and nothing disappears through an unnamed case: a target that is a directory, or a link whose name is a database name, is "a link that may lead to a database" (a problem, as today); a regular file is a skipped link entry (as today); a target that is itself a link is "a link to a link that may lead to a database" (a problem; today's classification followed it, and the only honest replacement is to refuse rather than follow); a target that is other than a file or directory is "a link to something that is neither a file nor a directory" (a problem); a target outside the root is "a link out of the tree" (a problem); absent is churn (as today); any error is "a link whose target cannot be looked at" (as today). The namespace question for an isolated instance (its own mount namespace may resolve a spelling elsewhere) is a declared support condition before wiring: a route whose instance runs in its own mount namespace is classified in the agent's namespace and rendered as such, and until the condition is declared it is a refusal rule, not a silence. |
| **Owned files keep their chain.** `fileFacts` gains the retained `Dir` of the file's own directory, which lives for the fleet's life (one descriptor stack per owned file; a box has tens of databases, not thousands, and the count of retained descriptors is part of the fleet's rendering and a test's assertion); every later look runs `Recheck` on it and `fstatat` of the file's name, and litestream's opens (Part B) take that directory descriptor. |
| @@ -34,23 +34,23 @@ Two layers. |
| 1. **The fork's own file operations** (`db.go`: the long-lived descriptor, WAL reads, shadow WAL, checkpoints, the `Stat`s of the database and its directory; `replica.go:186` and every LTX, temporary and cleanup path; enumerated in the plan's task by reading every call, not by sampling) take two retained directories, `SetDataDir(*authority.Dir, name)` for the database's directory and `SetMetaDir(*authority.Dir)` for the platform's shadow-state directory, and become `openat`/`fstatat`/`renameat`/`unlinkat` relative to them with `O_NOFOLLOW`; a sidecar is opened with `O_NONBLOCK` and refused unless `fstat` says regular (a FIFO put in a WAL's place is refused, not read). `db.f` is opened first through `OpenRegular` against the accepted facts and its identity is the database's for the life of the `DB`; before every reopen or sidecar open the fork compares `fstatat(dirfd, name)` with that identity and refuses on a difference (an error the fleet treats as a change). |
| -2. **SQLite's own opens, through a VFS this design owns whole.** The library opens the database, journal, WAL and SHM by pathname through its VFS after `xFullPathname` has canonicalised what it was given, and the default Unix VFS's *file* methods also act by pathname on their own: `xShmMap` opens the `-shm` file by deriving its name from the file's stored path and calling `open` directly, the SHM's removal calls the syscall table directly, the moved-file check stats the stored path, and the directory sync opens the directory by path. So delegating file methods to the Unix VFS is not confinement, and revision 2's delegation is withdrawn. The connection is given a **descriptor-rooted VFS** whose `sqlite3_file` object and every method are this design's: `xFullPathname` returns the name unchanged; `xOpen` is `openat(dirfd, name, flags|O_NOFOLLOW|O_CLOEXEC|O_NOCTTY)` for the main database and for each derived name (journal, WAL, SHM; `O_NONBLOCK` and a regular-file `fstat` for the sidecars), and for the main database compares the opened `(dev, ino)` with the accepted facts; `xAccess` and `xDelete` are `faccessat` and `unlinkat` on the descriptor; `xRead`, `xWrite`, `xTruncate`, `xSync`, `xFileSize` act on the descriptor; `xLock`, `xUnlock` and `xCheckReservedLock` are the POSIX advisory locks the Unix VFS uses, on the descriptor, with the same byte ranges, so the application's own connection and this one agree; `xShmMap`, `xShmLock`, `xShmBarrier` and `xShmUnmap` open the `-shm` through `openat` on the directory descriptor and `mmap` it, with the Unix VFS's lock byte layout, so the WAL index is shared correctly with the application; `xSync` of the directory is `fsync` of the retained directory descriptor; `xFileControl` answers the controls the library asks of a WAL-mode file (`SQLITE_FCNTL_PERSIST_WAL`, `SQLITE_FCNTL_POWERSAFE_OVERWRITE`, `SQLITE_FCNTL_SIZE_HINT`, `SQLITE_FCNTL_HAS_MOVED` answered from `fstatat(dirfd, name)` against the retained identity) and `SQLITE_NOTFOUND` for the rest; no method takes a pathname from the stored name after `xOpen`. The DSN is `file:<name>?vfs=carlos-dirfd-<id>` and the driver passes `vfs` through; the `id` names the retained descriptor the fork registered for that `DB`. |
| +2. **SQLite's own opens, through a VFS this design owns whole, in a process of its own.** POSIX record locks belong to a process, and SQLite coordinates connections within one process through its own in-memory bookkeeping (`unixInodeInfo`: a second VFS in the same process reproducing the byte ranges would take a lock the ordinary connection believes it holds). So the fork and its VFS run in a **helper process** (`carlos-replicate`, one per fleet), which is the only SQLite user in its process: the agent builds the inventory, holds the roots and the witness, and admits requests; it hands the helper, over a unix socket with `SCM_RIGHTS`, the accepted immediate directory descriptor and name of each database, the shadow-state directory descriptor, and a socket on which the helper asks for each admission (the gate crosses the boundary as a request and an answer; the admission row is the agent's to write) and reports each transport's end; the helper's SQLite connections then coordinate with the application's and the agent's through the kernel's locks, as two processes do today, and the fleet's close is the helper's exit (which also ends every descriptor the helper held). A helper that exits unexpectedly is a fleet that died: the agent sees the socket close and treats it as a change (stale, hold, rebuild). The library opens the database, journal, WAL and SHM by pathname through its VFS after `xFullPathname` has canonicalised what it was given, and the default Unix VFS's *file* methods also act by pathname on their own: `xShmMap` opens the `-shm` file by deriving its name from the file's stored path and calling `open` directly, the SHM's removal calls the syscall table directly, the moved-file check stats the stored path, and the directory sync opens the directory by path. So delegating file methods to the Unix VFS is not confinement, and revision 2's delegation is withdrawn. The connection is given a **descriptor-rooted VFS** whose `sqlite3_file` object and every method are this design's: `xFullPathname` returns the name unchanged; `xOpen` is `openat(dirfd, name, flags|O_NOFOLLOW|O_CLOEXEC|O_NOCTTY)` for the main database and for each derived name (journal, WAL, SHM; `O_NONBLOCK` and a regular-file `fstat` for the sidecars), and for the main database compares the opened `(dev, ino)` with the accepted facts; `xAccess` and `xDelete` are `faccessat` and `unlinkat` on the descriptor; `xRead`, `xWrite`, `xTruncate`, `xSync`, `xFileSize` act on the descriptor; `xLock`, `xUnlock` and `xCheckReservedLock` are the POSIX advisory locks the Unix VFS uses, on the descriptor, with the same byte ranges, so the application's own connection and this one agree; `xShmMap`, `xShmLock`, `xShmBarrier` and `xShmUnmap` open the `-shm` through `openat` on the directory descriptor and `mmap` it, with the Unix VFS's lock byte layout, so the WAL index is shared correctly with the application; `xSync` of the directory is `fsync` of the retained directory descriptor; `xFileControl` answers the controls the library asks of a WAL-mode file (`SQLITE_FCNTL_PERSIST_WAL`, `SQLITE_FCNTL_POWERSAFE_OVERWRITE`, `SQLITE_FCNTL_SIZE_HINT`, `SQLITE_FCNTL_HAS_MOVED` answered from `fstatat(dirfd, name)` against the retained identity) and `SQLITE_NOTFOUND` for the rest; no method takes a pathname from the stored name after `xOpen`. `xAccess` answers from `fstatat(dirfd, name, AT_SYMLINK_NOFOLLOW)` and reports a link as absent rather than escaping through it; `xOpen` of a temporary or null-name file (the library's transient files) is answered in the shadow-state directory descriptor, never the tenant's; permitted names are the main name and its four derived names, and any other name is refused; `xClose` closes the descriptor and releases the locks the object held; `xDeviceCharacteristics` reports the same as the Unix VFS for the filesystem; URI parameters are escaped as SQLite's URI grammar requires. The DSN is `file:<name>?vfs=carlos-dirfd-<id>` and the driver passes `vfs` through: that selects the VFS and confines nothing by itself; the confinement is the validated names and the descriptor-relative operations. |
| -What this closes: every open the library makes for the database and its sidecars, including reopens, checkpoints, the SHM and the first open's initialisation writes, happens beneath the accepted directory with no link followed and no pathname above it walked. What it leaves, stated: within its own directory a tenant can still replace `<name>` or a sidecar with another regular file it owns between two opens; the identity comparison at `xOpen` and before each of the fork's opens refuses the main database's replacement; a sidecar swapped under a live connection is a corruption of the replica, which litestream's own consistency checks are the place to catch and which the plan's task for the fork adds a test for. That residual is "the owner's own doing" only under decision 12 (a foreign user's directory inside a tenant's tree can supply sidecars today); until 12 is decided it is stated as a residual with two possible authors. |
| +What this closes: every open the library makes for the database and its sidecars, including reopens, checkpoints, the SHM and the first open's initialisation writes, happens beneath the accepted immediate directory with no link followed and no pathname above it walked, and the locks are coordinated with every other connection in the system by the kernel because no two SQLite instances share the helper's process. What it leaves, stated: within its own directory a tenant can still replace `<name>` or a sidecar with another regular file it owns between two opens; the identity comparison at `xOpen` and before each of the fork's opens refuses the main database's replacement; a sidecar swapped under a live connection is a corruption of the replica, which litestream's own consistency checks are the place to catch and which the plan's task for the fork adds a test for. That residual is "the owner's own doing" only under decision 12 (a foreign user's directory inside a tenant's tree can supply sidecars today); until 12 is decided it is stated as a residual with two possible authors. |
| -The VFS is the plan's first spike and it must prove, before anything else of Part B is written: registration from Go against this driver's generated API on amd64 and arm64; a database opened, written, checkpointed, closed and reopened through the VFS with a concurrent ordinary connection from the application side holding and releasing locks correctly; every file the library opened shown by `fstat` to be beneath the descriptor; SHM creation and removal through the VFS; the moved-file check; and the full list of the library's entry points that still take a pathname (expected: none for a connection opened with `SQLITE_OPEN_URI` and `vfs=`). The spike's result is a note, and if the spike fails on any of these the design's Part B is rewritten before the plan continues; Parts A, C and D do not wait on it. |
| +The VFS is the plan's first spike and it must prove, before anything else of Part B is written: registration from Go against this driver's generated API on amd64 and arm64; a database opened, written, checkpointed, closed and reopened through the VFS in one process while an ordinary connection in another process (the application's arrangement) and, as the negative case, an ordinary connection in the same process, write, read and checkpoint concurrently, with the locks observed to exclude correctly across processes and the same-process arrangement shown unsafe (which is why the helper exists); SHM creation and removal through the VFS; the moved-file check; closure releasing locks without disturbing the other process's; and a trace of every system call the library made (`strace` on the spike, not `fstat` alone, since identity does not show ancestry) with the list of pathname entry points that remain (expected: none for a connection opened with `SQLITE_OPEN_URI` and `vfs=`). The spike's result is a note, and if the spike fails on any of these the design's Part B is rewritten before the plan continues; Parts A, C and D do not wait on it. |
| ## Part C. The durable admission, two kinds of hold, and a handover on disk (Astra 3, 4) |
| -**The admission is a row, written before the send.** The gate (`lease.RequestGate`) returns an *admission*, `(fleet_id, owner, token, admitted_at)`, after committing a row in the durability ledger's own database (`admissions`: `token`, `fleet_id`, `owner account and app`, `admitted_at`, `transport_done_at`, `settled_at`, `how`), `synchronous=FULL`, within a bounded budget; the gate waits for the commit. A commit that does not return within the budget refuses the request (`ErrAdmissionNotDurable`; litestream sees a failed upload and does not advance) and the goroutine that awaited it stays to see the commit land and then marks the row `how = unissued, settled_at = now` in its own transaction, so a late commit never authorises anything and never blocks a day; if the process dies between, recovery classifies the row as below. The meter's `Start` row carries the token (`Start` may still fail or be late, as today; what is now mandatory is the admission, not the `Start`). The transport's end marks `transport_done_at` from the SDK middleware's finalize step, whether or not the recorder's workers still run; `Finish` and `Abandon` write `settled_at` and `how` in the same transaction as the attempt's outcome; an admission is *settled* when both are present. Admissions are the ledger's pending state moved to before the send and given an owner. |
| +**The admission is a row, written before the send.** The gate (`lease.RequestGate`) returns an *admission*, `(fleet_id, owner, token, admitted_at)`, after committing a row in the durability ledger's own database (`admissions`: `token`, `fleet_id`, `owner account and app`, `admitted_at`, `transport_done_at`, `settled_at`, `how`), `synchronous=FULL`, within a bounded budget; the gate waits for the commit. A commit that does not return within the budget refuses the request (`ErrAdmissionNotDurable`; litestream sees a failed upload and does not advance) and the goroutine that awaited it stays to see the commit land and then marks the row `how = unissued, settled_at = now` in its own transaction, so a late commit never authorises anything and never blocks a day; if the process dies between, recovery classifies the row as below. The meter's `Start` row carries the token (`Start` may still fail or be late, as today; what is now mandatory is the admission, not the `Start`). An admission has two participants, the transport and the recorder, and is settled by whichever finishes last. The transport writes `transport_done_at` from the SDK middleware's finalize step. The recorder writes `recorder_done_at` with its outcome: `finished` (`Finish`), `abandoned` (`Abandon`), or `unrecorded` (the request path, when `Start` failed and made no attempt row, which today calls neither; it now writes this terminal). The participant that finds the other already done writes `settled_at = max(transport_done_at, recorder_done_at)` and `how`, and performs, in that same transaction, the uncertainty extension or closure below. The terminals that have no participants (`unissued`, `recovered-unknown`) write all three timestamps at once. The transition table is the plan's model: `admitted` → (`transport done` | `recorder done`) → `settled`, with `unissued` and `recovered-unknown` as terminals from `admitted`, and no other edge. Admissions are the ledger's pending state moved to before the send and given an owner. |
| -**Two kinds of hold.** A *barrier* is the ledger's completeness rule: a day is not published complete for an admission's owner while an admission of that owner admitted on or before that day is unsettled (box-wide while Part C2's owner scope is not in use); it is a query over `admissions`, writes nothing, and lifts at settlement. An *uncertainty* is a row (`uncertainties`: `id`, `fleet_id`, `from`, `to` null while open, `reason`) written when a fleet's authority is found changed (by the witness, by `Recheck`, by a changed file), in the transaction that also writes the holds for the days of `[from, now]` for every owner the uncertainty reaches: `from = min(the registry proof, the last clean look's start, the earliest admitted_at of the fleet's admissions then unsettled) - margin`. While an uncertainty is open, settlements of that fleet's admissions extend it: each settlement, in its own transaction, writes the holds for the days of `[its admitted_at, its settled_at]` for every owner the uncertainty reaches, and the last settlement closes it (`to = that settlement + margin`). An admission that settles while no uncertainty of its fleet is open writes nothing permanent: the ordinary day of an unchanged fleet becomes complete when its barrier lifts. The multiple-midnights sequence (admitted on the 2nd, the change at 23:58:10, sent on the 3rd, finished on the 4th) holds the 2nd, 3rd and 4th through the settlement's interval; the request completed before the change was noticed is inside the lower bound as today. If the discovery transaction fails, the fleet refuses every request (`admit` answers stale) until it succeeds, and the barrier covers the gap. |
| +**Two kinds of hold.** A *barrier* is the ledger's completeness rule: a day is not published complete for an admission's owner while an admission of that owner admitted on or before that day is unsettled (box-wide while Part C2's owner scope is not in use); it is a query over `admissions`, writes nothing, and lifts at settlement. An *uncertainty* is a row (`uncertainties`: `id`, `fleet_id`, `from`, `to` null while open, `reason`) written when a fleet's authority is found changed (by the witness, by `Recheck`, by a changed file), in the transaction that also writes the holds for the days of `[from, now]` for every owner the uncertainty reaches: `from = min(the registry proof, the last clean look's start, the earliest admitted_at of the fleet's admissions then unsettled) - margin`. If discovery finds no admission of the fleet unsettled, the uncertainty is opened and closed in that transaction (`to = discovery + margin`). While an uncertainty is open, settlements of that fleet's admissions extend it: each settlement, in its own transaction, writes the holds for every day of `[its admitted_at, its settled_at]` for every owner the uncertainty reaches, and the last settlement closes it with `to = max(discovery, that settlement) + margin` and materialises every day of the closed interval, the margin's extra day across midnight included; recovery uses the same helper and advances the publication watermark past the days it materialised. An admission that settles while no uncertainty of its fleet is open writes nothing permanent: the ordinary day of an unchanged fleet becomes complete when its barrier lifts. The multiple-midnights sequence (admitted on the 2nd, the change at 23:58:10, sent on the 3rd, finished on the 4th) holds the 2nd, 3rd and 4th through the settlement's interval, whichever participant finished last; the request completed before the change was noticed is inside the lower bound as today. **If the discovery transaction fails** the barrier is not enough (every admission may already be settled): discovery first latches a *pending discovery* in the ledger's existing failure latch (`latchFailure`, extended to carry the lower bound and the owners reached), which the publisher consults (`drainFailures` before any snapshot) and which bars every day from the bound for those owners until the uncertainty commits; the fleet refuses every request meanwhile and retries the write; a process that dies with the latch held leaves no durable row, and recovery's uncertainty for a fleet not `closed` (below) covers it from the fleet's `opened_at`. |
| -**Who an uncertainty reaches.** Each uncertainty reaches its fleet's owners and every owner *linked* to it. A successor fleet, when it acknowledges a predecessor (below), links its own owners to every open uncertainty of that predecessor (`uncertainty_links`: `uncertainty_id`, `fleet_id`), in the acknowledgement's transaction; a later extension of that uncertainty writes holds for the linked fleet's owners too, and the barrier for a linked owner consults the predecessor's unsettled admissions as well as its own. That is Astra's October sequence under owner scope: Y, built at 23:58:20, links to A's open uncertainty; on the 3rd A's admission is unsettled and Y's day is barred through the link; on the 4th the settlement writes the 2nd to 4th for X and for Y. |
| +**Who an uncertainty reaches, and what a fleet inherits.** A fleet's *effective* uncertainties are its own and every one it inherited. Links are made at **construction**, not at acknowledgement: a successor, in the transaction that writes its own `open` row, links itself (`uncertainty_links`: `uncertainty_id`, `fleet_id`) to every open uncertainty of its predecessor's effective set, and to its predecessor's unsettled admissions as a barrier source (`barrier_links`: `predecessor fleet_id`, `fleet_id`), so from its first instant a late extension of the predecessor's uncertainty writes holds for the successor's owners and the predecessor's unsettled admissions bar the successor's days. Acknowledgement is the later, terminal step (below) and carries the predecessor's whole effective set, closed intervals included, into the successor's inherited set (`inherited`: `fleet_id`, `uncertainty_id`), so the next successor inherits it in turn: A's U reaches B at B's acknowledgement of A and C at C's acknowledgement of B, as `staleWindow` passes an inherited window on today. Astra's October sequence under owner scope: Y, built at 23:58:20, links at construction to A's open uncertainty and to A's unsettled admission; on the 3rd Y's day is barred through the barrier link; on the 4th the settlement writes the 2nd to 4th for X and, through the link, for Y. |
| -**The fleet journal and the handover.** One row per fleet (`fleets`: `fleet_id`, `box`, `witness_generation`, `owners`, `opened_at`, `state`, `predecessor`, `acknowledged_by`). States and the transaction each is written in: `open` at build, before any admission; `stale` when an uncertainty is opened; `closing` when litestream has stopped and the fleet has released its witness; `closed` when every admission of the fleet is settled (the recorder exposes a wait on settlement; `closeNow` waits, bounded, and past the bound the row stays `closing`, with its admissions still barring and its uncertainties still open: a late worker is a late settlement, and nothing about it is lost). A successor acknowledges a predecessor only when the predecessor is `closed`, or when the predecessor's process is gone (recovery, below); acknowledgement writes `acknowledged_by`, the links to the predecessor's open uncertainties, and holds for the predecessor's closed uncertainties the successor's owners have not yet been held for, in one transaction, before the successor admits anything. A `closed` predecessor with no uncertainty contributes nothing: routine replacement of a healthy fleet holds no day. A predecessor still `open` or `closing` when the successor is built is not acknowledged yet; the successor's barrier consults it (its unsettled admissions bar the successor's owners' days as linked), and acknowledgement happens when it closes, as `inheritHold` latches today, now on disk. |
| +**The fleet journal and the handover.** One row per fleet (`fleets`: `fleet_id`, `box`, `witness_generation`, `owners`, `opened_at`, `state`, `predecessor`, `acknowledged_by`). States and the transaction each is written in: `open` at build, before any admission, with the construction links above; `stale` when an uncertainty is opened; `closing` when the helper has exited and the fleet has released its witness; `closed` when every admission of the fleet is settled (the recorder exposes a wait on settlement; `closeNow` waits, bounded, and past the bound the row stays `closing`, with its admissions still barring and its uncertainties still open: a late worker is a late settlement, and nothing about it is lost). A successor acknowledges a predecessor only when the predecessor is `closed`, or when recovery has closed it; acknowledgement writes `acknowledged_by`, copies the predecessor's effective uncertainties (its own and inherited, open and closed) into the successor's inherited set, and writes holds for the closed ones the successor's owners have not yet been held for, in one transaction. A `closed` predecessor whose effective set is empty contributes nothing: routine replacement of a healthy chain holds no day. An unopened intermediate fleet (built and closed without replicating) has its construction links and is acknowledged like any other, so the chain's barriers and inheritance pass through it. |
| -**Recovery at process start.** The ledger's writer lock (`flock`, already the ledger's) proves the previous process is gone. Then, in one transaction before any fleet is built: every admission of this box that is unsettled is classified `how = recovered-unknown, settled_at = now` and, since its fleet cannot settle it any other way, an uncertainty is opened and closed for it at once over `[admitted_at - margin, now + margin]` for its owner (box-wide while owner scope is not in use), so the day it could have touched is held and later days are not barred for ever; every fleet not `closed` is marked `closed` with an uncertainty over `[min(opened_at, its earliest admission) - margin, now + margin]` if it had none open, or its open uncertainty closed at `now + margin`; every fleet with no `acknowledged_by` is acknowledged by the first fleet the new process builds, with the links and holds above. Orphans therefore terminate, conservatively, and nothing blocks beyond its recovery. |
| +**The producer fence, and recovery at process start.** The ledger's own lock does not prove the old process gone: `DurabilityLedger.Close` releases it when its bounded wait ends with work in flight, and a request admitted before could still be sent after. So dispatch is fenced separately: a *producer fence*, an `flock` on `admissions.lock` in the ledger's directory, taken by the agent process before its first admission and held for the process's life (never released by a timeout; released only by the kernel at exit), and every admission's commit requires it held (checked under the ledger's mutex before `BEGIN`); the helper process cannot dispatch without an admission, and the admission cannot be written without the fence, so a process that has lost the fence cannot cause a send. Recovery takes the fence first and, holding it, in one transaction before any fleet is built: every admission of this box that is unsettled is classified `how = recovered-unknown` with all three timestamps `now`, an uncertainty opened and closed for it at once over `[admitted_at - margin, now + margin]` for its owner (box-wide while owner scope is not in use), its days materialised; every fleet not `closed` is marked `closed` with an uncertainty over `[min(opened_at, its earliest admission) - margin, now + margin]` if it had none open, or its open uncertainty closed at `now + margin` and materialised; every fleet with no `acknowledged_by` is acknowledged by the first fleet the new process builds, with the inheritance and holds above. Orphans therefore terminate, conservatively, and nothing blocks beyond its recovery; a producer that is somehow still alive cannot send, because the fence is no longer its. |
| **Part C2, who is held.** The ledger's failure rows and `Incomplete` are box-wide today (`DurabilityClient.Incomplete` discards the client's identity; `CompletenessFor(account, …)` fails on any box failure). Owner-scoped holds are a ledger change: a `holds` table `(day, account, app or empty for the box, uncertainty_id, written_at)`, the publisher's snapshot carrying them, the reader treating a box-wide row as failing every owner and an owner row as failing that owner alone, and older readers of a published day seeing a box-wide incompleteness marker whenever any hold exists (so an older console never reads a held day as complete). The design specifies the table, the reader and the marker and keeps **box-wide the default**: every hold this design writes names the box until decision 10 is taken. Decision 10's implementation boundary is also stated, so it is not mistaken for a switch: per-tenant refusal means per-owner fleets (a fleet per `(account, app)`, each with its own witness view, uncertainties and holds, and a tenant's unacceptable file refusing only its fleet), which is a change to `MeteredFleetStore`'s shape beyond the holds table. |
| @@ -60,11 +60,11 @@ The VFS is the plan's first spike and it must prove, before anything else of Par |
| **Replication authority.** Separately from the owner, each identity has at most one replication authority: the external replicator (the box today), the activator's hibernation path (which replicates a hibernating exec route's database and holds its own stream), or this fleet. The fleet replicates an identity only when no other authority has it: a hibernating exec route whose database resolves to the same identity as a live unit tree's listed file is a conflict of authorities, refused for this fleet with both named, exactly as today's live/hibernating conflict is, until an explicit handover (a route's hibernation path releasing the database to the fleet) exists, which is not designed here. Deduplicating two spellings of one identity within the fleet is allowed only when both claimants are routes this fleet would replicate. |
| -**Lineage.** A replica key today is `(account, path)`. Under identity grouping the key is bound to `(owner, identity)` with the accepted spelling recorded beside it: another accepted spelling of the same identity reuses the key (one stream, as the identity is one file); a different identity under a previously accepted spelling is a new database and a new lineage (a new prefix) after the hold the change causes, never a continuation; the shadow state's key follows the same rule. |
| +**Lineage.** A replica key today is `(account, path)`, and an inode number is an observation, not a database: a deleted and recreated file can reuse it across a restart. The lineage is therefore a durable record in the shadow state (platform-owned): per replica key, `lineage_id` (random), `owner`, the accepted spelling, `(dev, ino)`, the file's birth time (`statx` `btime`, where the filesystem has it) and the SQLite header's database id fields read at acceptance. Within one fleet's life, another accepted spelling of the same `(dev, ino)` reuses the key (one stream, one file). Across a restart, a key is resumed only when `(dev, ino)` and birth time match the record (and the header's id fields do not contradict it); otherwise the file is a new database and a new lineage (a new prefix) after the hold the change causes. Existing path-based prefixes are adopted as the first lineage of the file found under that path at the first build after this design, recorded then with its identity and birth time; a later mismatch rotates as above. |
| **Required coverage.** The platform's required databases are a list: the registry this process has open, the console database when the agent's role configuration says the box is a console box, `CONSOLE_DB` when set. A required database unreachable through its root is a problem that makes the inventory incomplete and is written as a durable box-wide hold for the day before and independent of whether a fleet is built. Before any cutover, a read-only operator command lists this inventory beside the external replicator's configuration. |
| -**Route identity (Astra 10, approval H): a producer and a lifecycle, not a number.** The current whole-app backend starts a unit and checks its health; it reads no uid or invocation, units restart on their own (`Restart=always`, socket activation), and a stale pair would survive an autonomous restart. The design names the mechanism: the identity of a route's running workload is read from systemd, not inferred: the unit observer the activator already runs for isolated exec is extended to every unit route and reads, at every start, stop and restart it is told of (D-Bus `PropertiesChanged` on the unit, with a poll of `systemctl show -p InvocationID,MainPID,ActiveState` as the fallback), the `InvocationID` and the main process's uid from `/proc/<MainPID>/status`; it writes `route_uid` and `invocation_id` on the route row, under the ownership witness (the generation covers the columns), as a conditional write keyed on the previously recorded invocation (a write that finds another invocation recorded clears both and lets the next observation set them); it clears both at stop. A claim is honoured only when the route's recorded invocation is current (the observer's last reading agrees with systemd now) and the claim root's owning uid equals `route_uid`; unknown identity refuses the claim. Hibernation capability is a route property; a hibernating route that is asleep has no invocation and claims only through the platform root's private spelling; one that is awake has one and claims as a live unit. An exec or named-file claim of a non-unit route (the activator's own processes) is claimed under the activator's identity, which is a trusted workload and so a parameter. Which uids are the platform's own workloads, and whether a root-run unit may claim, are `trustedUIDs` (empty) and approval H. |
| +**Route identity (Astra 10, approval H): a producer and a lifecycle, not a number.** The current whole-app backend starts a unit and checks its health; it reads no uid or invocation, units restart on their own (`Restart=always`, socket activation), and a stale pair would survive an autonomous restart. The design names the mechanism: the identity of a route's running workload is read from systemd, not inferred: the unit observer the activator already runs for isolated exec is extended to every unit route and reads, at every start, stop and restart it is told of (D-Bus `PropertiesChanged` on the unit, with a poll of `systemctl show -p InvocationID,MainPID,ActiveState` as the fallback), the `InvocationID` and the main process's uid from `/proc/<MainPID>/status`; it writes `route_uid` and `invocation_id` on the route row, under the ownership witness (the generation covers the columns), as a conditional write keyed on the previously recorded invocation (a write that finds another invocation recorded clears both and lets the next observation set them); it clears both at stop. A claim is honoured only when the route's recorded invocation is current (the observer's last reading agrees with systemd now) and the claim root's owning uid equals `route_uid`; unknown identity refuses the claim. Hibernation capability is a route property; a hibernating route that is awake has an invocation and claims as a live unit; one that is asleep has none, and reaching its private directory through the platform root proves which directory was reached, not that this route owns it, so a sleeping route claims only against a **binding** the observer recorded while it was awake: `state_dir_identity`, the `(dev, ino)` of the state directory the live workload's uid owned at the observer's last coherent reading, written on the route row under the witness beside `route_uid` and `invocation_id` and kept across sleep; a sleeping route with no binding, or whose private spelling resolves to another identity, is refused. A coherent reading is: `InvocationID` and `MainPID` from systemd, the uid from `/proc/<MainPID>/status`, then `InvocationID` and `MainPID` from systemd again, equal to the first; a reading that is not coherent is retried and, failing, leaves the route unknown. Freshness is checked where claims are made: at the inventory's build and at every look (`lookNow`), by comparing the recorded invocation with systemd's current one; a route whose invocation changed between is a change the fleet goes stale on, since the registry row changed under the witness. An exec or named-file claim of a non-unit route (the activator's own processes) is claimed under the activator's identity, which is a trusted workload and so a parameter. Which uids are the platform's own workloads, and whether a root-run unit may claim, are `trustedUIDs` (empty) and approval H. |
| ## The parameters this design keeps open |
| @@ -81,21 +81,21 @@ The VFS is the plan's first spike and it must prove, before anything else of Par |
| ## Tests, each naming the production event it observes and the mutation it kills (Astra 11) |
| - **The watch looks.** A hook at the entry of `lookAgain` (a counter the fleet exposes to tests) with a controlled clock advanced through one watch interval: the counter moves. Kills: the watch's `lookAgain` call removed. |
| -- **The clean look's start.** A controlled clock pinned at a named instant for the look's start and a later instant for the change; `staleWindow().from` equals the start minus the margin. Kills: the start replaced by the end of the look. |
| +- **The clean look's start.** A controlled clock advanced *during* the look (a hook inside it moves the clock), so the look's start and end differ, with the registry proof later than both; `staleWindow().from` equals the start minus the margin. Kills: the start replaced by the end of the look or by the proof. |
| - **Both inode comparisons.** A barrier at each of the two sites by name, the cross-claim comparison (`litestream_owners.go:194` today: a second claim check reading another file) and the kept-facts grouping (`:217`: two owned paths that became one file), with a rename performed between the look and the comparison; each refuses with its own message. Kills: either comparison removed. |
| - **Shadow state's root.** The shadow-state directory's platform walk refused on a tenant-owned, group-writable or linked component, in a test that touches no data directory. Kills: the walk skipped for the shadow root. |
| -- **Shutdown.** The recorder's settlement wait returns, the fork's `db.f` and every retained `Dir` are closed (asserted by identity: the fleet exposes its retained descriptors' `(dev, ino)` and the test confirms each is gone from `/proc/self/fd` by identity, not by count), and a request after `Close` is refused. Kills: a retained descriptor leaked; a worker not awaited. |
| +- **Shutdown.** The recorder's settlement wait returns; every descriptor the fleet owns is closed, asserted through the fleet's own closer (each retained `Dir` and the helper's socket register with it, and the test reads the register, not `/proc/self/fd`, since the registry and shared ancestors are legitimately open elsewhere in the process); the helper process has exited; a request after `Close` is refused. Kills: a retained descriptor leaked; a worker not awaited; the helper left running. |
| - **Interleavings.** `admit` against `lookAgain`, and `latch` against `inheritHold`, each on a two-goroutine barrier; a synchronised concurrent `Close`. Kills: the lock dropped at either site. |
| - **Admission before send.** The middleware order makes an admission-less dispatch impossible: a fake gate that refuses sees no request on the wire; a gate whose commit is held past the budget refuses, the request is not sent, and the late commit's row is marked unissued. (A `Start`-less send remains possible, as today, and is not claimed otherwise.) |
| - **Part A.** A platform ancestry with a tenant-owned, group-writable or linked component refused; a directory swapped between `List` and `Walk` changes nothing; a listed subdirectory replaced by a FIFO fails `Child` with `ENOTDIR` and nothing blocks; a `..` past the root, and a `..` after a named component, refused; a retained child moved under another parent is a changed edge at `Recheck`; `List` twice on one `Dir` gives the same entries; `Child(name, expect)` refuses when the entry changed between the listing and the acquisition; each link class classified as written and none silent. |
| -- **Part B.** The spike's own tests, then the fork's suite under the roots; a WAL replaced by a link or a FIFO refused; a database replaced within its directory refused at reopen; a concurrent application connection's locks honoured; the SQLite identity asserted by a write through the connection, a checkpoint, and a read through `db.f`. |
| -- **Part C.** A settled admission on an unchanged fleet writes no permanent hold and its day becomes complete; a change with an unsettled admission opens an uncertainty from the earlier bound; the three-midnight interval; a linked successor held by a late extension; the fleet rows across build, stale, closing with a late settlement, closed, and acknowledgement, with a kill at each transition and the ledger reopened by a second process, orphans classified and later days completing; a `closed` healthy predecessor contributing nothing. |
| -- **Part D.** The identity cases, the authority conflict (live tree and hibernating exec route on one identity), the lineage rule, a required database missing writing the hold without a fleet, a claim refused while the route's invocation is unknown or stale. |
| +- **Part B.** The spike's own tests, then the fork's suite under the roots in the helper process; a WAL replaced by a link or a FIFO refused; a database replaced within its directory refused at reopen; locks honoured against an ordinary connection in another process, and the same-process arrangement shown unsafe; a helper that dies seen as a change; the SQLite identity asserted by a write through the connection, a checkpoint, and a read through `db.f`. |
| +- **Part C, as a model with the transition table and then against the ledger.** A settled admission on an unchanged fleet writes no permanent hold and its day becomes complete; a change with an unsettled admission opens an uncertainty from the earlier bound; the three-midnight interval in both completion orders (transport last; recorder last, including `abandoned` first and `unrecorded`); discovery with every admission already settled closes at once and materialises the margin's extra day across midnight; a discovery whose write fails with no admission pending bars the day through the pending-discovery latch until the retry commits; a successor linked at construction barred by the predecessor's unsettled admission and held by its late extension; A opens and closes U, healthy B acknowledges A, healthy C acknowledges B, and C's inherited set holds U; the fleet rows across build, stale, closing with a late settlement, closed, and acknowledgement, with a kill at each transition and the ledger reopened by a second process, orphans classified with all three timestamps and later days completing; the fence: a process that lost the ledger's lock but not the fence cannot be recovered over, and one without the fence cannot admit; a `closed` healthy chain contributing nothing. |
| +- **Part D.** The identity cases; the authority conflict (live tree and hibernating exec route on one identity); the lineage record (a file deleted and recreated with a reused inode across a restart gets a new prefix; the same file resumes; an existing path-based prefix adopted once); a required database missing writing the hold without a fleet; a claim refused while the route's invocation is unknown or stale; a sleeping route with no binding, or whose private spelling resolves to another identity, refused; an observation that is not coherent (the pid changing between the two reads) leaving the route unknown. |
| - **Mutants** in a tree copy with the killing test named, per part, after the assertions above exist. |
| ## Plan |
| -0. **Two spikes, results kept as notes and thrown-away code:** the descriptor-rooted VFS on `modernc.org/sqlite` v1.55.0 proving registration on amd64 and arm64, a database and WAL opened, written, checkpointed, closed and reopened through it beside an ordinary application connection with correct locking, SHM created and removed through it, the moved-file check, every file `fstat`-shown beneath the descriptor, and the list of pathname entry points that remain; and the admission protocol's state transitions as a model (an ordinary complete day; a change with an unsettled admission; a late settlement after acknowledgement; a kill at each boundary) with the ledger's commit latency on a box's disk under the replicator's request rate. The VFS spike gates Part B only; the admission spike gates Part C. |
| +0. **Two spikes, results kept as notes and thrown-away code:** the descriptor-rooted VFS on `modernc.org/sqlite` v1.55.0 in a helper process, proving registration on amd64 and arm64, a database and WAL opened, written, checkpointed, closed and reopened through it while an ordinary connection in another process writes, reads and checkpoints with the locks excluding correctly, the same-process arrangement shown unsafe, SHM created and removed through it, the moved-file check, closure releasing only its own locks, a system-call trace of the library's operations, and the list of pathname entry points that remain; and the admission protocol as an executable model of the transition table (both completion orders; `unissued`, `unrecorded` and `recovered-unknown`; discovery with zero pending; a failed discovery write; construction links and transitive inheritance across A, B and C; the fence at shutdown timeout; a kill at each boundary) with the ledger's commit latency on a box's disk under the replicator's request rate. The VFS spike gates Part B only; the admission spike gates Part C; required coverage (Part D) proceeds independently of the uid lifecycle. |
| 1. `internal/authority` (`Dir`, `Root`, the platform walk, the platform-link predicate on descriptors, `Walk`'s classes, the tests above) and the inventory on it, with the header read through `OpenRegular` (Part A). |
| 2. The admission, the two kinds of hold, the fleet journal, acknowledgement with links, recovery, and the holds table and reader with the older-reader marker (Part C and C2, box-wide by default). |
| 3. The uid lifecycle (the unit observer's columns under the witness) and the replication-authority and lineage rules (Part D's contracts), then claims by identity and required coverage on them. |
| @@ -106,4 +106,4 @@ Each task is its own reviewed commit with the full `internal/ops`, `internal/lea |
| ## Questions for the reviewer |
| -A. The `Dir` contract: is popping the retained stack for a leading `..`, refusing `..` after a name, `Child(name, expect)` and `Recheck` on named edges enough for I1 as now worded, or is there a re-resolution left? B. The VFS: is the method set complete for WAL mode beside an application connection on Linux, and is anything in the library's open sequence still a pathname outside it? C. Part C: the barrier, the uncertainty, the links and the recovery: name a kill instant or an owner-scoped sequence that leaves a day unheld or barred for ever. D. Part D: the replication-authority rule and the lineage rule: a case misread. E. The uid lifecycle: is the observer's conditional write and "unknown refuses" enough across autonomous restarts, or is there a window between a restart and the observation in which a claim is honoured under a stale pair? F. Anything in the parameters table that is mechanics, or the reverse. G. The order. |
| +A. The `Dir` contract with `Walk`'s anchors: a re-resolution left, or an absolute spelling the platform predicate accepts today that the anchored walk refuses? B. The helper process as the lock boundary and the VFS contract: a same-process SQLite user left in the helper, or a method the library still reaches by pathname? C. Part C's transition table, construction links, transitive inheritance, the pending-discovery latch and the fence: a kill instant or an owner-scoped sequence that leaves a day unheld or barred for ever, or a producer that can still send after recovery. D. The lineage record and the sleeping binding: a case misread, or a continuity the record cannot tell. E. The coherent observation and freshness points: a window in which a claim is honoured under a stale pair. F. Anything in the parameters table that is mechanics, or the reverse. G. The order. |